INNOVATE · INtelligeNt ApplicatiOns oVer Large ScAle DaTa StrEams
Horizon 2020 — Marie Skłodowska-Curie Actions
- Duration
- 2018-04-01 → 2020-03-31
- EU contribution
- €195,455
- Participants
- 1
- Scheme
- MSCA-IF-EF-ST
Lines connect the coordinator with its partners.
Results in brief
INtelligeNt ApplicatiOns oVer Large ScAle DaTa StrEams
Predictive analytics is the key research subject for query-driven applications. Analytics offer the necessary basis for intelligent decision making. Due to the huge volumes of data, analytics could be executed on top of data partitions. Each partition contains only a piece of data and a dedicated processor handles the incoming queries. Continuous queries over multiple data partitions require intelligent mechanisms to (1) massively assign queries to distributed data nodes and (2) efficiently aggregate the multipart final query response in limited time with maximum performance (i.e., the Quality of Result - QoR). Query Controllers (QCs) serve the incoming queries realizing the connection of large-scale data systems with the real world. Query Processors (QPs) are placed in front of data partitions realizing a ‘response mechanism’. QCs can have access to multiple QPs and a QP can be ‘connected’ with multiple QCs (i.e., ‘grid’ - ecosystem). INNOVATE introduces an intelligent decision making mechanism in three axes: (i) top-down, by realizing an intelligent mechanism to assign queries to QPs; (ii) bottom-up, by realizing a decision making mechanism to provide an efficient management of the collected data and aggregate responses to applications over partial results; (iii) horizontal, by realizing queries optimization schemes for a ‘swarm’ of QCs. The objectives of this programme are: O1. Design & implement Query and QP Models. O2. Design & implement Learners. O3. Create a Pool of Learners and Implement an Ensemble Learning Scheme. O4. Design & implement the Queries Assignment Process. O5. Design & implement the Multiple Controllers Management Plane. O6. Develop a holistic approach to research training and career evolvement of the Fellow. O7. Disseminate and Exploit INNOVATE outcomes.
Data: CORDIS, © European Union
Project objective
Large scale data analytics is the key research domain for future data driven applications as numerous of devices produce huge volumes of data in the form of streams. Analytics services can offer the necessary basis for building intelligent decision making mechanisms to support novel applications. Due to the huge volumes of data, analytics should be based on efficient schemes for querying large scale data partitions. Partitions contain only a piece of data and a dedicated processor manages the incoming queries. The management of continuous queries over data streams is a challenging research issue requiring intelligent methods to derive the final outcome (i.e., query response) in limited time with maximum performance. The management process of continuous queries involves their assignment to specific processors and the processing of the derived responses. We focus on a group of query controllers serving the incoming queries and, thus, becoming the connection of big data systems with the real world. INNOVATE proposes solutions for the management of the controllers behavior. We propose an intelligent decision making process for each controller in three axes: (i) top-down, by realizing a mechanism that assigns queries to the underlying processors; (ii) bottom-up, by proposing decision making mechanisms for returning responses to users/applications on top of early results; (iii) horizontal, by proposing optimization schemes for queries management. We adopt a pool of learning schemes and an ensemble learning model dealing with how and on which processors each query should be assigned. We also propose specific schemes for combining processors responses. Intelligent and optimization techniques are adopted for the controllers group management. Machine learning, Computational Intelligence and optimization are the key adopted technologies that, when combined, provide efficient solutions to a challenging problem like the support of intelligent analytics over big data streams.
Original text from CORDIS.
Participants
- UNIVERSITY OF GLASGOW · GlasgowCoordinatorUnited Kingdom
Links
Data: CORDIS, © European Union