top of page


Background detection with deep learning on image and time series data

The task

The customer is developing an autonomously driving robot that is to be equipped with artificial intelligence for optimal navigation. With the help of deep learning algorithms, the first step is to identify different types of subsurface (e.g. concrete, grass, gravel, earth, etc.). The robot prototype has ten different sensors that collect image and time series data.


The challenge

The energy consumption of the algorithms should be dimensioned for the largest possible movement radius of the robot, at the same time a high, practical detection rate must be ensured.

our solution

The first step was to provide the robot's sensor data for the training of algorithms. For this purpose, we have provided workflows for loading, extracting and analyzing the data in a big data environment. In this environment, we then trained a selection of the latest deep learning models for the detection of several underground classes from image and time series data. Then the best models were optimized for resource consumption and achieved high accuracy with the lowest possible energy consumption.

The customer benefit

The resource-saving deep learning models enable use directly on the robot, which accelerates development cycles and significantly more development ideas can be tested in a shorter time. The close cooperation enables the customer to train further models in self-service and to use them on the prototype.



Our role

Support of the customer by data scientists and data engineers

Our activities

  • Provision of a big data development environment on the customer cluster

  • Creation of big data workflows for the provision and processing of the data

  • Selection, training, application and evaluation of various deep learning methods for the classification of time series and image data

  • Selection of the best models for further optimization with TensorFlow-Lite and export for use on embedded systems

  • Evaluation of the results and advice for further development

Technologies & methods

  • Applications: DaSense, Nifi, Docker

  • Data / databases: ROSbag

  • Languages / frameworks: Python, Jupyter, Spark, TensorFlow, Scikit-Learn, Hadoop / Hortonworks

  • Methods: deep learning, machine learning, image analysis, time series analysis

bottom of page