ABOUT NORCOM
Big data reporting on vehicle data
The task
Before a vehicle model can go into production, a very large number of test drives must be completed. The data-driven evaluation of these test drives has enormous potential for reducing the costs of the test phase. We are therefore looking for an analytics framework for the automated evaluation and visualization of sensor measurement data from test drives.
The challenge
Robust analyzes require statistics on extremely large amounts of data, analyzes should be customizable and ad hoc.
our solution
In the first phase, the reporting framework was implemented programmatically in a big data script. For this purpose, the user can define per page which data should be analyzed and in which way visualized. Several options are available for visualization, including line plots, histograms and 2D histograms. The framework offers a great deal of flexibility in terms of the number of pages, the selection of data and the specifications of the plots. The reporting framework was then implemented in an app, which offers a very good user experience via a GUI.
The customer benefit
The added value of an automatic reporting framework in the development process of a vehicle lies in the quick assessment of the visualized data in the report. With our solution, the customer does not receive a report, but a framework to create any number of reports. Thanks to the clear data selection and input mask, this happens even without programming knowledge or big data experience. Even complex evaluations are made possible by a larger user community.
Project-
Characteristics
Our role
Customer support from data scientists, data engineers and software developers
Our activities
Creation of a framework for the aggregation of data and calculation of visualizations on large data
Creation of templates for the reporting framework
The reporting framework went live as an app
Technologies & methods
Applications: DaSense
Databases: MF4, Parquet
Languages / Frameworks: Python (Anaconda Stack), Javascript, AngularJS, Hadoop, Spark, Yarn, Oozie, Docker
Methods: time series analysis