Intermittent Demand Forecasting With Machine Learning

1INTRODUCTIONDemand forecasting is the process of predicting the demand for thefuture using historical sales values. Demand forecasting can help indeveloping an estimate of the expected forecast for industries mainlyin the supply chain section. There are several statistical forecastingtechniques such as Croston’s[9], ARIMA[7] and Exponential[7] areused for the Demand forecasting for long which are of great success over the years. But such successful models still fail to give lesserprediction errors to some of the difficult problems like Intermittentdemand forecasting and bull-whip effect prediction. In the researchtopic, we will be mainly focusing on Intermittent data. Intermittentspare parts are the parts that have a large no of zero demand for alonger period in the sales data(an example shown in figure ?). After a demand occurrence in time series, the next occurrence will bepreceded by a longer period of zeroes or there is a gap in time between demand occurrence, such kinds of demand data are called Intermittent demand data. Volvo has been witnessing the intermittentFigure 1: An example for Intermittent data. This is a sales data representation of part number 755 over the years of 2015 to 2020. In this wecan see them in 2015 there was only 2 demand period for the product in February and September. The next demand for the sameproduct occurs in the next year 2016 November, which is a gap ofmore than one year. These kinds of gap spare parts are called intermittent spare parts. The X-axis shows Part no 755( JPRTN ) andyear(JSTYY). The legend of the graph shows the different monthsstarting from January (JQS19) to December(JQS12)1[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

2introductiondemand nature in their sales data from all sections including logistics, spare parts, Volvo Penta and Truck sales. Since if we look intodata there is not much difference in demand levels, they wanted asystem that can predict the demand, in such a way it let them plantheir inventory from CDC(Central Distribution Center) to RDC( Regional Distribution Center) and from RDC to Dealer2. The poor forecast and inventory planning in the Warehouse during the past yearshas resulted in stock accumulation and cost for a long. According tostatistics Volvo has nearly 60 million SEK worth of Dead Stock in RDCout of which 39 million are non-refundable stock. Volvo has a reactiveinventory management system, where they try to predict their futuresales from their previous demand. And since they cannot properlypredict intermittent demand there is a huge inventory stock of suchkind of spare parts in Volvo truck’s and Volvo Penta’s RDC, whichstill they could not identify as obsolete or dead stock.Figure 2: The basic layout of the Supply chain in VolvoIn this thesis work, we are dealing with intermittent data fromVolvo Trucks and they wanted to investigate how Machine learningmodels would work on such data. In recent years forecasting withMachine learning and deep learning techniques are used widely dueto the availability of a large amount of data, resources, lesser processing time and better accuracy. Volvo also has been investigating Machine learning models for not only intermittent demand forecastingbut also for other problems such as obsolesce detection. Right nowwe have done a comprehensive evaluation on certain models fromMachine learning as well as deep learning by considering the problem as a regression and Classification model. Also, we compared themodels with already implemented Croston’s model to understand theeffects of Machine learning on intermittent data and how statisticalmodels differentiate. Then we have also formulated a Novel Hybridmodelling with an ensemble to improve the accuracy of prediction[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

1.1 literature review3especially demand occurrence and also reduce the loss margin fromthe business point of view.Also, we focused on bringing the uncertainty factor into predictionin the form of confidence interval which let us in deciding how confident we are about our predictions. We were able to bring more than a95 per cent confidence level into our prediction. which shows us thatthere is a 95 per cent chance that the predictions will fall in the prediction interval generated with our equation and also it makes surethat the interval is narrow enough to not give stock accumulationson each prediction. This also addresses the problem of unnecessarystock accumulations in the warehouses, as the only required amountwill be sent to RDCs.1.1literature reviewForecasting Time series for sales, finance, stocks, and in many otherfields has been done for so long for the inventory management, production, to increase the profit, to detect the non-demand periods,high demand periods and also to detect obsolesces in certain products. There are several methods for demand forecasting in statisticslike the Exponential smoothing[9] but the major problem with sucha statistical model was the forecasting results were not reliable especially for the intermittent data, where there is a large number ofnon-demand periods or zero demand periods. The data appears tobe sparse or lumpy.But in 1972 Croston[9] comes with a forecasting method for intermittent demand forecasting in which Croston separated non-zero demand and non-zero demand interval as two separate time series andperformed an Exponential smoothing on them to predict the averagefuture demand with a smoothing parameter alpha, and the demandforecasting will be completely depended on the smoothing parameter alpha. Later Syntetos and Boylan in 2001[21] comes with themajor drawback of Croston’s model and pointed out that the Croston’ model is biased to positive demands. And in 2005 introduce anew method for better accuracy known as the Syntetos and Boylanmethod(SBA method) but later it was also found biased for low intermittent data by Teunter and Sani (2009b)[22]and Wallström andSegerstedt (2010)[23]Later many of them extended research on the intermittent demandforecasting on Croston and new improvements were made on theCroston’s model and new ones were suggested including Shale, Boylan, and Johnston (2006),[19] and Teunter, Syntetos, and Babai (2011)[4].Recently several other models have also come into the intermittentdemand forecasting like the Bootstrapping methods(Willemain et al.,2004;[24] Porras and Dekker, 2008[17]) and an improved bootstrapping method was also introduced by the M. Hasina,â, M.S. Aguirb,[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

4introductionM.Z. Babaic, Z. Jemaia in 2019[5]. Several papers were published onaggregating the data rather than a new approach. In 2014 MAPA wasintroduced by Kourentzes, Petropoulos Trapero[13] and improvements on aggregation also came in like the greedy aggregation by[14]Chongshou Liâ, Andrew Lim in 2018.There are several research papers on Machine learning like the Neural network[25] and Support vector regression(SVR)[25]. Several othermachine learning methods were also introduced like the XGBoost,[25]Random Forest[25] which perform similar to the bagging techniquethat was previously mentioned. After the availability of a large amountof data Deep learning also came into forecasting and became moresuccessful than the statistical approaches. Some of the machine learning techniques that were used are LSTM which is a type of RNN andwas used to predict time series data. In the past year(2020) SushilPunia, Konstantinos Nikolopoulos, Surya Prakash Singh, Jitendra K.Madaana and Konstantia Litsiouc[18] came with a model that is acombination of LSTM and RF that can train on the temporal and explanatory variable, which performed with better accuracy that manystatistical approaches Right now researchers are mainly focusing onthe hybrid models where they can combine Machine learning/ deeplearning models with the Statistical model as well. This can help inimprovising accuracy as well as learning the important variables thatcan affect the demand or sales(can help in learning linear as wellas the non-linear relationship in the data). Nikolas Kourentzes together with Devon Barrow and Sven Crone gave a talk about hybrid forecasting at the QR62 conference were combining both statistical or machine learning models for better accuracy and forecasting. Several papers are talking about hybrid modelling, ZhongshengHua, Bin Zhang[10] talk about a hybrid support vector machine andlogistic regression approach for forecasting intermittent demand ofspare parts where they introduce hybrid machine learning models.In another paper by Kourentzes[12], he introduces a Neural networkmodel that uses the median of prediction of neural networks anduses it as an ensemble input for predicting demand accurately. Inanother paper by Suhartono, Rahayu, S.P., Prastyo, D.D., Wijayanti,D.G.P., Juliyanto[20] introduces a hybrid model of Neural networkand ARIMAX, where inputs are fed into the ARIMAX model andthe residuals are fitted with the neural network, with final forecastsbeing the sum of forecast values of NN and ARIMAX. And we areintroducing a novel approach where we use the prediction from different machine learning and statistical model along with the inputdata as metadata to fit it with another machine learning model forbetter generalisation and optimised results. The network work in theformat of an ensemble staking architecture.Forecasting cannot be always precise and there can be several factors affecting the same like climate or pandemic which may affect[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

1.1 literature review5the prediction, so it is very wise to incorporate an uncertainty factorinto our prediction. There are several methods to incorporate uncertainty in prediction likes probability prediction[15], differential prediction[reference] and quantile predictions[6]. Uber technologies incorporated this idea of uncertainty for their prediction on-demandusing Bayesian LSTM.[26]. The Bayesian uncertainty or prior distribution is also incorporated into all machine learning models including ensembles. Ensembles can be predicted with uncertainty usingthe Bayesian mean averaging(BMA)[8]. There are several other techniques incorporated for the uncertainty or probability prediction likeGaussian averaging[[3], negative log loss[8], binomial distribution[16]etc.[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

2OBJECTIVEIntermittent demand forecasting has taken a lot of attention betweenforecast researchers in recent years due to its random nature and difficulty in the problem of prediction. Now let us look from the industrial perspective, forecasting intermittent demand can be useful forthe industry to manage the inventory, deadstock and also to mitigateeffects such as bull-whip effects in the supply chain. As per discussion with Volvo analytic manager they are getting a lot of sales datawhich has intermittent demand nature for over years from differentregions around the world and predicting occurrence alone accuratelyto some extent can affect the profit margin and stock monitoring. Themain reason for Volvo to invest in intermittent demand forecastingis to solve the problem of unnecessary stock accumulation in warehouses, accurate safety stock management and also reduce the additional cost caused due to incorrect predictions. Right now they havea basic structure of Supply chain function shown in figure 2 whichis the series implementation of demand from dealers will be met byRDC(Regional distribution centre) and the collective demand of eachRDC met by the CDC(central distribution centre), which in the problem of intermittent data can lead to demand accumulation. For thisfirst, they wanted to know if Machine learning can be considered asa solution to the data that they have and also they need to identify ifthere is any relationship between variables or attributes in the salesof intermittent spare parts.In time series forecasting we know that the demand is presentin all the time points. But in this scenario, only 20 per cent of thetime points have demand and the rest of them are simply null orzero demand (straight line ). Several statistical models like Croston[9],ARIMA[7] and exponential smoothing[7] has been utilised for Intermittent demand forecasting over a long period and researchers arestill working on improving the accuracy of prediction and reducingthe error. The occurrence of a large number of zero values affects theforecasting accuracy and error to a larger extent[22]. But still, Machine learning models like SVM, RF and Gradient boosting are someof the techniques that are widely used for time series forecasting aswell as intermittent demand forecasting.Our Aim or objective is to find a way to predict the demand of intermittent demand using machine learning models and statistical models(hybrid modelling). And then to evaluate is hybrid models a viablesolution for problems like Intermittent demand forecasting. For thatwe evaluate different models from considering the problem as both7[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

8objectiveregression and classification, then bring together the models that havethe best performance from both statistical as well as machine learningthen form a hybrid model[17] and experiment if Hybrid model canbe considered as a better model comparing to individual models inprediction. We will also be investigating the approaches based on theadditional features related to the spare parts such as the price andhow it can affect the prediction. We take two paths for it consideringthe problem as occurrence forecasting and also considering the problem as demand forecasting. For occurrence forecasting, we follow theprocedure of classification and for demand forecasting, we followedregression as a solution. With both the solution we implemented hybrid modelling to analyse demand forecasting and how the error oraccuracy value varies. During our experiment with classification, wefound out that the occurrence forecasting is completely dependent onthe trend and seasonality factors which we failed to identify duringour data preparations made the reliability of classification or occurrence forecasting questionable with the data that we have. But eventhen we have included the observations in the report for references inthe future.We would also focus on the evaluation metric, during our literatureresearch we found out that evaluation metrics such as MAE, RMSE,accuracy for classification cannot be considered a good evaluationmetric in the case of intermittent demand forecasting due to the influence of a large number of zeroes on the prediction[12]. So we aim tocome up with a metric that can reduce the influence of zero to someextent and then give a considerable evaluation for the entire hybridarchitecture. And during the thesis, we came up with a new evaluation strategy called Average per year error in which we consideredthe prediction over a yearly basis and evaluated the error rather thangoing for a monthly error evaluation which helped us to get rid ofthe influence of zero to a larger extent in the evaluation and gave usa more continuous series.Apart from this, we would also like to predict with a confidencevalue or prediction interval rather than giving a point prediction. Thedata that we are dealing with are of spare parts. So instead of forecasting the next month’s sales for part number 755 as 10 units we willbe forecasting that there can be a demand during a certain period (as5-15 units or may just predicting demand) say for next one year it canbe between 5-15 units on a mean of 10 units with a confidence of 95per cent. This gives us the confidence that the demand can be in between this range, at the same time it shows that the demand does notfall outside the range. so much a prediction can help us to pre-planthe stock management and safety stock within the warehouses. Theproducts that show a larger deviation in the range can be productsthat have a very large chance of changing the demand range in an actual sense as well. And the products that give narrow range has more[ November 30, 2021 at 15:45 – Intermittent demand forecasting with Machine learning version 6.1 ]

objective9chance that they can fall under the range. Such forecasting can reducethe huge impact in the supply chain due to the deviation from actualsales and the predicted value. This kind of prediction can avoid stockaccumulation in warehouses, safety stock management and improvetheir supply chain cycle. We can utilise the deviation of prediction orthe deviation from the distribution of error and depending on thatwe can calculate the confidence of the prediction. Thus we will beputting forward an approach that can formulate the confidence ofour prediction or uncertainty of prediction from calculating the deviation of forecast from actual sales. At the end of the experiments, wecould come up with a 95 per cent confidence.In the experiments below we can see that we have chosen Regression as one of the viable solutions for the intermittent demand forecasting problem. This was mainly because regression models can helpus in confidence interval predictions and is more useful for Volvo asthe data is mainly time series. But we also observed that the classification models could open up the possibility to identify the occurrenceof demand in the test data and train data compared to the regressionmodel but we also doubt that occurrence prediction will be more dependent on the trend and seasonality of prediction. In the attempt toidentify trend and seasonality we couldn’t come up with a patternin the sales for the products that belonged in categories, functionalgroups as well as in geographic location. This made the reliability ofthe classification questionable.The more evaluation and experimentalvalues are explained in section 5 with image 5.The results from ourexperiments showed us how classification failed in identifying thedemand class as demand. But also it opens the idea of occurrenceforecasting with data that shows a better pattern in sales.Volvo has an existing system called Forecast with cumulative quantity(FCQ). Where they take the cumulative quantity as forecast anda better FCQ value result in the reduced safety stock and bad FCQleads to increased Safety stock. The FCQ also depends on a modelwith lesser error(optimised model), that gives a better prediction andconsiders cost as a Function. With an increase in FCQ, there is anincrease in total forecast-ed value. This increased the average stockvalue due to larger order quantities. The increase in average stockvalue shou

Figure 36 The validation curve for NN 53 Figure 37 The validation curve for LSTM 54 [November 30, 2021at 15:45- Intermittent demand forecasting with Machine learning version 6.1 ] LIST OF TABLES Table 1 The MAE of each model after training with the . future using historical sales values. Demand forecasting can help in