SMARTER · A Scalable and Elastic Platform for Near-Realtime Analytics for The Graph of Everything

A Scalable and Elastic Platform for Near-Realtime Analytics for The Graph of Everything

The SMARTER project carried out the research on how to derive actionable information from enormous amount of data generated by the Internet of Everything to leverage data-driven strategies to innovate, compete, and capture value from deep web and real-time information. The project targeted innovative research outcomes by addressing Big Dynamic Data Analytic requirements from three relevant aspects: variety and velocity and volume. The project introduced the concept, “Graph of Everything” (GoT), to deal with the issue of data variety in data analytics for Internet of Things (IoT) data. The Graph of Everything extends RDF Data model, that has been widely used for representing deep web data, to connect dynamic data from data streams generated from IoT, e.g. sensor readings, with any knowledgebase to create a single graph as an integrated database serving any analytical queries on a set of nodes/edges of the graph, so called, analytical lens of everything. The dynamic data represented as RDF data stream, or Semantic Stream in general, may contain valuable, but perishable insights which are only valuable if it can be detected to act on them right at the right time. Moreover, to derive such insights, the dynamic data needs to be correlated with various large datasets. Therefore, SMARTER has to deal both the velocity requirements together volume requirements of analysing GoT. For the dealing with fast update streams in conjunction with massive volume of data, the project investigated on how to create a native and adaptive solution for storing and processing data of the Graph of Everything on the cloud infrastructure. Whereas conventional relational database infrastructures are crumbling under the volume of this data with the tight constraints on schemata, the SMARTER relied on a novel graph-based computing paradigm to a massively parallel processing architecture power by relying on elastic computing platforms, e.g, Apache Flink. Furthermore, with the RDF-based model with places no conditions on the structure of the data that can be processed, the project's solution supports the pre-computing of slow-updated data for continuous queries over highly-update data from streams which proved a huge performance gain in the extensive experiments on the standalone environment . Velocity reflects the need for “just-in-time” processing of dynamic stream data i.e. incoming stream data flows into processing pipelines that must be matched to patterns in the static data.

Текст от CORDIS, на английски · Данни: CORDIS, © Европейски съюз