Beowulf Moves from Underground to Trading Floor

Beowulf–the NASA project created in 1994 that used clustering technology to link a large number of commodity machines running Linux into a massive supercomputer–has usually met with skepticism on Wall Street. Despite the potential cost savings from being able to parcel out a complex calculation to 1,000 idle servers connected over a virtual LAN or the Internet,the benefits have been outweighed by concerns over reliability, failover and support issues. Elsewhere, so-called Beowulf technology has gained a strong following in research labs and universities with teams posting their results with Linux clusters on Web sites such as The Beowulf Underground (https://beowulf-underground.org).

However, to judge by a recent announcement from J.P. Morgan, clustering technology is now moving out of the lab and into the securities industry. The investment bank, which has been evaluating the technology for the past year in its London office, has recently announced the deployment of vendor TurboLinux's EnFuzion software as part of its global risk management system for fixed-income derivatives.

Despite the vendor's name, EnFuzion is not simply another version of Beowulf and so does not require a dedicated cluster of Linux servers. Instead it is happy to combine machines running a mixture of operating systems, including Windows NT, Solaris, HP UX, AIX, Linux and Tru64. This catholic approach fits in particularly well with the computing landscape of most securities firms, which typically contain a mixture of hardware and operating systems. More importantly, this technology frequently spends at least 50% of its time sitting idle when local markets are closed. According to Michael Liberman, head of global swaps and derivatives technology at J.P. Morgan, this was one of the principal reasons that J.P. Morgan decided to use EnFuzion as part of its derivatives risk management system. "Our demands for computing power in this field are virtually limitless," he says. "Deploying EnFuzion allowed us to harness the power of hundreds of powerful desktop workstations during times when they would otherwise be sitting idle. We continue to move more of our critical calculations and processes to this architecture."

At present the bank is using EnFuzion to manage a cluster of some fifty Sun Microsystems Enterprise Server CPUs running Solaris and more than two hundred Dell workstations running Microsoft Windows NT. However, it is currently extending its use of EnFuzion to more than a thousand cluster nodes, including an installation at J.P. Morgan's New York headquarters. Apart from the use of otherwise idle machines, a number of other factors swayed the bank's decision to adopt the technology. These included the ease with which cluster nodes could be added and maintained and its superior performance on Windows NT. In view of the number of NT workstations in the securities industry that currently sit around doing nothing at night, that could well mean that EnFuzion starts appearing on a few more CIOs' wish lists.