Managing institutional money, some of it in alpha strategies, makes significant demands on real-time market data and fast analytics. So Bridgewater Associates, a Westport, CT-based firm that managed $150 billion in global investments for a wide array of institutional clients, used stream processing software from StreamBase to build some key components in its investment infrastructure – only a few of which it will discuss publicly.
Bridgewater uses StreamBase to process market data from multiple providers and check for errors, all in real-time, said Ed Thieberger, who leads the firm’s trading technology group. The firm has also built a trading decision support application using the StreamBase technology.
“StreamBase is good for any situation where you have real-time data in large volumes and you have to process it quickly to make decisions,” explained Thieberger. “In financial services applications that sort of need is pretty pervasive.”
StreamBase provides a platform that enables companies in many industries, including financial services, to build applications for fast analysis very quickly using the Microsoft .NET Framework. The company was co-founded by Michael Stonebraker, who was the main architect of the INGRES relational DBMS.
“To achieve low latency, a system must be able to perform message processing without having a costly storage operation in the critical processing path,” Stonebraker explained in a white paper which is on the company’s site. “Active systems avoid this overhead by incorporating event/data-driven processing capabilities. In streaming applications, some querying mechanism must be used to issue arbitrary queries on moving data and compute real-time analytics.”
The latest release, StreamBase 3.0, which came out in April, can process more than one million messages per second with near-zero latency. It extends the benefits of streaming applications across the enterprise through integration with the TIBCO Enterprise Message Services and the Microsoft .NET Framework through a software development kit. StreamBase can use multiple processors and its StreamOptimizer enables developers to synchronize and combine sequential high-speed operations to reduce latency.
Bridgewater had built some real-time applications, said Thiebeger, and then it had some applications that it wanted to build but the company didn’t have the code it needed. So it turned to StreamBase. Like other financial firms, it found that building on the StreamBase platform cut development time from months to weeks, and the StreamBase development environment allows non-technical users to see the flows of data and analytics so they can build applications or look at applications and understand what the technology is doing. And because the core platform was already developed and tested, building on StreamBase reduced the time needed for quality assurance.
“StreamBase cut our development cycle, and allowed us to end up with a much better environment from a support standpoint. It also gives our end users visibility into the logic behind the software,” said Thieberger.
StreamBase has a facility to build its data flows graphically, Thieberger said.
“You can describe the flows and how the data flows are used to make decisions. The result is that a non-technical user can use the interface to put together business logic. As a result, we can offload some of the development work and reduce the amount of time-to-market for applications that we want to build because our business analysts can be involved in graphically building the applications which we can then run without writing new code,” he said.
“It provides a way for our end users to take a part in building an application, and we don’t need programming for the core pieces,” he said.
William Hobbib, vice president of marketing at StreamBase, said one client built a market data monitor on the system and found a vendor’s system had five times the latency advertised. Another firm built a compliance engine in eight days, after failing to build one in Java in more than eight months.
“It runs very fast on a Windows Server,” Hobbib said. “You can process 100,000 messages per second on a $2,000 Dell.”
At Wall Street on Demand, a Boulder, CO-based company that develops investor Web sites for many of the nation’s largest brokerage firms, StreamBase helps handle all the information coming in from nearly 1,000 market data feeds. When clients want to add new features to their retail investor sites, they want the innovations up and running fast, said John Leslie, chief technology officer at the company. In the past, adding a new market data stream could take between eight weeks and 32 weeks.
His team is currently working on a project for several clients to offer real-time end-user portfolio evaluation. This is retail, so the system has to meet the needs of one million active users out of a total customer base of 13 million investors.
The firm evaluated StreamBase as a tool for two and a half weeks. By the end of the trial period, it had an engine that could evaluate 250,000 portfolios on a single server.
“It would have taken us months of engineering optimization to achieve that efficiency,” he said. “This lets my engineers focus on the business problems instead of on the pipes and sewers.” StreamBase has created an abstraction layer that lets developers work on the logical layers and the user interface. Wall Street on Demand had a model up and running in four or five days and then worked on maximizing the number of portfolios it could run.
“I now have a near production-ready solution that we did in the course of the evaluation,” added Leslie.
“We send out half a million alerts a day,” Leslie said. The company runs entirely on Microsoft with two data centers housing about 500 servers running a mix of Microsoft Windows 2000 and Windows 2003 Server and SQL Server.
www.bwater.com
www.streambase.com
www.wallst.com