ENGINEERINGNET.EU -- The researchers combined a streaming engine and a database system, thus exploiting and merging both technologies.
Terabytes of data are generated on a daily basis in fields as finance, science and information technology. Numerous applications, whether stock market trading, scientific discovery or online advertising are dependent on analysis of this big data streams.
Performing real-time analysis on this data, rather than storing it and analyzing it later, could be very profitable. Traders could for instance have different stocks monitored and be informed if a certain correlation occurs.
Current database systems however do not support continuous processing, while data stream systems are not built to scale up to big data.
Researcher Erietta Liarou and her colleagues innovatively built streaming functionality into MonetDB, the database system targeted at big data developed at CWI, integrating storage and analysis in the same engine.
The resulting MonetDB/DataCell system is optimized for analyzing big streaming data, and outperforms commercial systems when stream data increases.
Big Data research is part of CWI's research theme Information. This line of research is aimed at developing methods and technologies to extract meaningful information from large amounts of data.
(GL)
