StreamBase Systems (www.streambase.com), a leader in the fast-growing complex event processing (CEP) market, has pioneered a remarkable new approach to processing and analyzing real-time streaming data. Founded by data management experts who have been driving industry innovation for decades and backed by top-tier investor firms, StreamBase has developed a new class of stream processing software. Global organizations are witnessing its high performance coupled with its intuitive graphical development environment that eliminates the need for costly, inflexible custom-coding.

We recently spoke with Barry Morris, Chairman and CEO of StreamBase Systems, Inc. about high-performance CEP and the trends that are driving its adoption in financial markets.

WFS:What is high-performance complex event processing (CEP)?

BM: High-performance CEP enables instant reaction to complex combinations of real-time events represented by incoming live data. Often the live events data also needs to be continuously integrated with historic data in a relational database – and CEP can do this. While ideally suited for making investment decisions, CEP is easily applied to other business challenges as well.

WFS: What kinds of other problems?

BM: Some of our clients are federal agencies that must be able to ‘sense and respond’ to patterns of events that indicate pending threats. Others are very big Web sites that need to track their click streams. Believe it or not, some are multi-user online gaming companies. We can help anyone who needs to process real world events at very high speed, however, more than 50% of our clients are financial firms.

WFS: What do you mean by ‘events?’

BM: Very simply put, events are digital representations of something that happened in the real world, like a market price changing, a person logging into a computer, or a Web link being clicked on. Complex event processing refers to filtering, correlating, aggregating, and otherwise computing with a combination of these real world events. These computations typically produce other events that can be further processed, or that may issue alerts, or may trigger automated actions.

WFS: Sounds great for algorithmic traders.

BM: It is, but StreamBase fits other financial applications such as cleaning up errors in data as they occur; order execution optimization; transaction cost analysis; real-time risk management, compliance; and even market surveillance. All of these things need to be done at very high speeds. We have demonstrated that StreamBase can operate at 10 times the speed of any alternative commercial software solution.

WFS:How?

BM: StreamBase searches events streams using technologies derived from SQL databases. Extraordinarily high performance is enabled by specifying the application problem in SQL, the standard computer language for database systems. The core idea came from Dr. Michael Stonebreaker – he founded StreamBase to commercialize the cutting-edge technology conceived as part of a university project spanning M.I.T., Brown University, and Brandeis University. If you Google him, you will find that he is a global ‘rock star’ of computer technology. StreamBase is able to analyze the SQL code that clients have written and execute optimized query strategies very, very fast.

WFS: Does this require specialized computer equipment?

BM: Absolutely not. We run on standard commodity hardware platforms like Intel and Windows, and StreamBase is a Windows partner. Just give us lots of small machines and we will go very, very fast.

WFS: What is unique about StreamBase?

BM: Doing queries using SQL. For example, one client produces real-time data for many Wall Street firms. They wanted to maintain 500,000 e-brokerage portfolios with an equity feed which runs at about 20,000 messages a second. At peak loads, it will go up to 60,000 to 80,000 per second. The problem was how to update 500,000 portfolios at 1/20,000th of a second before the next message came along. They had planned to deliver a solution in six months using 12 people. Because SQL is a very high-level language, we built their applications very rapidly, delivering them in less than 10 days with two people.

WFS: Were they happy?

BM: They were, and that is why this is such an exciting space to be in because it really changes the economics for people.

WFS: Does StreamBase have any competitors?

BM: The major players have not yet delivered anything in this space. IBM, for example, does not have a project to build a technology like this. We are IBM’s solution in this space.

WFS: What key trends in financial markets are driving high-performance event processing?

BM: Data rates are doubling every year, so you have firehoses that are delivering too much to digest with conventional technologies. Another is the demanding regulatory requirements both in the US and Europe that are essentially looking for much greater transparency. Yet another is that there is now a kind of an arms race in algorithmic trading, where everyone needs to be faster than his competitors while getting the best prices.

WFS: What do these trends mean?

BM: Performance requirements of database systems are growing exponentially, to the point where you really need the kind of infrastructure that only StreamBase can offer just to try to stay ahead of your competition, and these trends are not going away.