STUDENT INFORMATION MANAGEMENT USING CLOUDERA FRAMEWORK

Abstract

In recent years, facing a dramatic increase in the number and variety of data types. The problem of storing and processing huge amounts of data is making it is difficult for organizations in general; training centers and educational institutions in particular. As a result, this thesis will present the process of applying the technologies of the Cloudera framework to address such a problem. To be more specific, we use Cloudera components to restructure the data of a training center, which is in different formats, into a unified structure data warehouse. After that, we create a website that exploits the data warehouse for managing the student information of the center. Specifically, we use Sqoop to process structured data from DBMSs, e.g., SQL Server and MySQL, while Solr is utilized to process unstructured and semistructured data such as word and excel files. The processed data is then stored and aggregated on HDFS in a unified form. Finally, the student management website is developed using the Flask framework.