Self-learning, intelligence forecasting solution – it is possible!

The most used datawarehouse solutions such as SAP BI, Cognos, etc, does not offer usually all these parameters what really affects to forecasting.  One of that kind of parameter is the weather. Weather affect really much what products (articles) are sold. This article offers an new approach how we could forecast selling more automatic, self-learning and intelligence. The method is the same what has been used widely in stockmarket, identify the cancer-cells, used in plant maintenance services to identify when some parts of the engines most probably are going to break and we all know Chess. The accuracy of the forecasting is getting even better during the time goes on and what is even greater – you have to run the daily data once through the neural network and you’re done!

Scientist are admitting, the neural networks are the right way to create forecasting models since it is based on how human brains work. Neural networks (specially SOM-net) are usually used to solve problems what can not be solved with mathematic since there is too much input parameters.

What are the neural networks?

Researchers have studied neural networks strongly during the last five years in Finland and abroad, but it started on 70’s. The results have been significant. Companies have developed major technological solutions in manufacturing, physic and gambling sector. Teuvo Kohonen is one of the greatest researcher who has invent the SOM-network (self-learning map). It is one of the most known neural network model.

Self Organizing Map is a network what is used when you can not know the exact answer. The SOM itself create the output and it learns from every runtime sequence done. In backpropagation network teaching are done by comparing the result to the known results and if the results are the same, then the network is teached correctly.

SOM network will learn by contesting where the most activated neurons are weighted even more and other neurons are lighten.  In this way the network consist different kind of areas what are the main part in classification

Idea and how to build the solution?

The most important part during the production is modelling the input parameters. There can be several hundreds of input parameters but you have to remember, to bigger the network is more memory it will consume. Many smaller network is better than just one big even they do the same functionality.

Input parameters could be temperature, season, price, product category, sales channel, etc. The professionals can say what parameters have to use. The data must collect day by day if we want to forecast selling by date.

When parameters are agreed, create the network. There are many neural network softwares in the world and you just have to find out yours.

SOM is learning by contest and it classifies data to different areas and the most important part is to identify what those areas means.

Interfaces

To get a weatherdata into the system it is easy to get. Just make a contract with the company who is producing that data and you propably get also the API-interfaces.

I do not know the SAP Business Intelligence solution well enough but unlike in BI reports the rawdata has to feed to network with all the parameters modelled. If some of the parameters are missing, bigger the error ratio is. You could fetch the data straight from database or use APIs if there are any.

User interface can be browser-based solution and it can be a portlet or like SAP would say, the iView. Put that iView to your portal.

Costs?

Event the whole thing might sounds a hype – trust me, it can be done. Just make sure you have a people, wh understand how the SOM-network should be done and other people can focus on doing the UI and servlets.This kind of project is still a one of a kind so you propably would get economical help from some research funding organisation. In Finland this could be about 50 -80% of all costs. Costs depends on how large project you are doing, but I would say that the network, UI, APIs itself can done less than 500 000 euros and if you get 50-80% off that costs it is even better!

Benefits?

Using the SOM network is the only true way to make forecasting. Others are just a traditional data-analyses and most certainly those are not intelligence forecasting!

Just imagine, how many companies are stuggling with forecasting problems and specially how many of those include the weather etc. factors to theis model? I know no one! Just ask from retailers how big factor the weather is in their business?