From climate change to stock market prices

- EN - DE
In the interdisciplinary ,,Deep Turb’ project, neural networks are being u
In the interdisciplinary ,,Deep Turb’ project, neural networks are being used to better predict how flows behave. (Photo: Nasa)

Scientists at TU Ilmenau have succeeded in improving the accuracy of data evaluations for forecasting weather events, among other things, by up to 80 percent. In the scientific publication ,,Flipped Classroom - Effective Teaching for Time Series Forecasting", which was published in October, Prof. Patrick Mäder, head of the department Data-intensive Systems and Visualization, and Philipp Teutsch, research associate at the same department, present their research results on the training of recurrent neural networks. They combined different training methods of machine learning to minimize errors in the evaluation of large data sets.

Artificial neural networks have become an indispensable part of our everyday lives: In voice assistants, in translation or simulation software, machine learning technology processes data in a multitude of applications. The principle behind artificial intelligence (AI)-based information evaluation is based on neurons that are interconnected in a network like nodes. They record, process and evaluate information. Artificial networks thus follow the model of the human brain, but can evaluate correlations that would often not be recognizable to humans.

To enable neural networks to evaluate data, they must first be trained. In the plant identification app developed by Mäder, Flora Incognita , for example, millions of plant images are fed into the app with the corresponding assignment. This is followed by a long and complex optimization process in which the system learns to recognize the plants on its own and correct errors. However, data such as series of numbers, such as consecutive days in the weather forecast, place a higher demand on the AI. Scientists are working in this context with so-called recurrent neural networks (RNNs) - a special form of neural network in which the neurons are fed back. Thus, when processing series of numbers, contextual information of previous data points is available to the network at any point in time. Prof. Patrick Mäder explains:

If we look at the weather forecast as an example of a long series of numbers, we see that its forecast is very prone to error. If the RNN calculate a temperature value for the next day, this value is immediately used as a basis for the following day. However, if the first calculation is not correct, the entire forecast is wrong.

The scientists’ training methods are therefore designed to minimize the risk of incorrect calculation. For this purpose, they use a combination of two training methods - free running tranining and teacher forcing - in so-called curriculum learning strategies. In the first variant, the network is completely exposed to its own errors from the very first training step and must learn to compensate for them on its own. In Teacher Forcing, on the other hand, a teacher intervenes at each training step and immediately corrects the net’s errors. In this way, any prediction errors that arise are suppressed before they can become entrenched and amplified in the time series.

In their publication, Prof. Patrick Mäder and Philipp Teutsch consider a new class of the aforementioned curriculum learning strategies, which significantly stabilizes the training behavior of the networks and largely avoids premature termination of the learning process. The scientists investigated which combination best combines the advantages of both training methods, as Professor Mäder explains:

With our novel combination of both training methods, we have succeeded for the first time in improving the previous results of forecasts by up to 80 percent. This enables us to forecast climate changes, plant flowering times or stock prices earlier and more accurately.

The basic research of Patrick Mäder and Philipp Teutsch is already being put to practical use in interdisciplinary research projects at the TU Ilmenau. In cooperation with Andreas Möckel and the Department of Small Machines, their novel method is being used to analyze engines. It provides information on how long an engine can be expected to remain operational. In addition, the Department of Data-intensive Systems and Visualization is part of a consortium of four departments - including Fluid Mechanics , Optimization-based Control and Technical Thermodynamics : In the research project ,,Deep Turb - Deep Learning in and of Turbulence", funded by the Carl Zeiss Foundation, RNNs are used to better predict how flows behave.

The technical article appeared in ,,Transactions on Machine Learning Research."