Expert system scalability using Apache Spark
This thesis will be conducted in close work with startup www.inspirient.com.
Inspirient is a Berlin-based startup run by former graduates of FU Berlin’s Computer Science department that is developing an Artificial Intelligence (AI) to automate business analytics. The AI is implemented as an expert system and continually learns usage behavior through a machine learning feedback loop.
As one might expect, automatically mining data for interesting insights is computationally expensive, and in the current state, the algorithm is compiled to an execution model that can be run on a single machine, which limits the amount of data that can be analyzed. In order to enable the AI to scale to Big Data, it is necessary to distribute data and computations over that data. Thus, the objective of this thesis is to use the Apache Spark framework to store and process data on its cluster computing framework.
A master thesis will thus comprise the following items:
- Review of rule-based expert system design principles, architectures, and current approaches
- Design, adaptation, and implementation of algorithms and data structures to suitably execute an expert system on the Apache Spark framework
- Quantitative evaluation of speed-up and parallelism with real-world workloads
If you are interested in this thesis and would like to know more, please reach out to Dr. Guillaume Aimetti at email@example.com, or visit us at www.inspirient.com. From the FU side, please contact Dr. Matthias Wählisch.