A Hybrid Prediction Method For Stock Price Using LSTM And . - Hindawi

Complexityexperiment data preprocessing, and modeling processing. InSection 5, we describe the experimental results of ourproposed hybrid LSTM-EEMD method and analyze theresults of simulation experiment of our proposed hybridLSTM-EEMD method for prediction. Finally, the conclusionof this paper and some future works are described in Section8.2. Related WorksOver the years, many studies in the financial field havefocused on the problems of stock price prediction. Thesestudies mainly focus on three important research directions:(1) based on the machine learning method; (2) based on thetime-series analysis method; and (3) based on the hybridmethod. Below, we first briefly introduce related terminology. Then, we introduce the LSTM and EEMD related tothis study.2.1. Stock Price Predicting. Stock prices are a highly volatiletime series. Stock prices are affected by various factors suchas national policies, interest rates, exchange rates, industrynews, inflation, monetary policies, temporary events, investor sentiment, and human intervention. Predicting stockprices on the surface requires the establishment of a model ofthe relationship between stock prices and these factors.Although these factors will temporarily change the stockprice, in essence, these factors will be reflected in the stockprice and will not change the long-term trend of the stockprice. Therefore, stock prices can be predicted simply withhistorical data.This paper believes that there are many studies using asingle analysis method to predict stock market trends, butthe results are not good. Need to consider a variety of factorsor use a variety of techniques to build a hybrid model tofurther explore the prediction of stock prices. In-depthsystematic research is required to answer the followingresearch questions:RQ1 Which factors or combinations of factors mostaffect the trend of the stock?RQ2 What kind of analysis technology combination ismost suitable for stock trend prediction?RQ3 Do we need to use deep learning methods to minedata in order to better discover the internal relationshipbetween the stock market and influencing factors?RQ4 Whether the predictability of the analysis modeldepends on specific stock company characteristics,such as the domain, shareholder background, andpolicies?RQ5 In the context of stock market forecasting, have wedeveloped some effective forecasting methods?RQ6 We should focus on the analysis of the specificnature of the stock price itself, rather than solvinggeneral relationship problems. Whether the priceanalysis driven by influencing factors can be moreeffective?32.2. LSTM. The long-short term memory neural network isgenerally called LSTM. The LSTM was proposed byHochreiter and Schmidhuber [27] in 1997. The LSTM is aspecial type of recurrent neural network (RNN). The biggestfeature of the RNN which was improved and promoted byAlex Graves is that long-term dependent information of datacan be obtained. LSTM has been widely used in many fieldsand has achieved considerable success in many problems.Since LSTM can remember the long-term information ofdata, the design of LSTM can avoid the problem of longterm dependence. Currently, the LSTM is a very populartime-series forecasting model. Below, we first introduce theRNN network, followed by the LSTM neural network.3. Preliminaries3.1. Recurrent Neural Network. When we deal with problemsrelated to the timeline of events, such as speech recognition,sequence data, machine translation, and natural languageprocessing, traditional neural networks are powerless. RNNis specifically proposed to solve these problems. Because thecorrelation between the contexts of the text needs to beconsidered in the word processing, the weather conditions ofconsecutive days and the relationship between the weatherconditions of the day and the past days need to be consideredwhen predicting the weather.The RNN has a chain form of repeating neural networkmodules. In the standard RNN, this repeated structuralmodule has only a very simple structure, such as a tanh layer.The simple structure of the recurrent neural network isshown in Figure 1.The design intent of RNN is to solve nonlinear problemswith timelines. The way of internal connection of the recurrent neural network generally only feeds forward thedata, but in the bidirectional recurrent neural network, itallows the forward and backward directions to feedback thedata. The RNN has designed a feedback mechanism, so theRNN can easily update the weight or residual value of theprevious step. The design of the feedback mechanism is verysuitable for time-series forecasting. The RNN can extractrules from historical data and then use the rules to predicttime series. Figure 1 shows the simple structure of the RNN,and Figure 2 shows the expanded diagram of the basicstructure of the RNN. The left side of the arrow with unfoldlabel in Figure 2 is the basic structure of the recurrent neuralnetwork. The right side of the arrow with unfold label inFigure 2 is a continuous 3-level expansion of the basicstructure of the recurrent neural network. An input data xt isinput into module h of the RNN. The yt is an output ofmodule h of the RNN values at time t. Like other neuralnetworks, the recurrent neural network shares all parametersof each layer, such as Whx, Whh, and Wyh in Figure 2. Asshown in Figure 2, the RNN shares two input parametersWhx and Whh, and one output parameter Wyh. As we allknow, the number of parameters in each layer of a generalmultilayer neural network is different. Looking at Figure 2,we feel that the operation of each step of the recurrent neuralnetwork is the same on the surface. In fact, the output yt andthe input xt are different. The number of parameters of the

4Complexityht–1htAht 1tanhXt–1AXtXt 1Figure 1: The simple structure of the RNN.yt–1yWyhWhhytWyhhWhxht–1UnfoldWhxxXt–1yt 1WyhWhhWyhhtWhhWhxht 1WhxXtXt 1Figure 2: The expanded diagram of the basic structure of the RNN.recurrent neural network will be significantly reducedduring training. After multilevel expansion, the recurrentneural network becomes a multilayer neural network.Looking closely at Figure 2, we find that Whh between layerht-1 and ht is the same as Whh between layer ht and ht 1 inform. In value and meaning, Whh between layer ht-1 and ht isdifferent as Whh between layer ht and ht 1. Similarly, Whxand Wyh have similar situations.Although each layer of the RNN neural network has outputand input modules, the output and input modules of somelayers can be omitted in specific application scenarios. Forexample, in language translation, we only need the overalllanguage symbol output after the last language symbol is inputand do not need to know the language symbol output after eachlanguage symbol is input. The main feature of RNN is selfexpanding, with multiple hidden layers.As we all know, during network training, the recurrentneural network models are prone to disappearing gradients.Once the gradient of the model disappears completely, thealgorithm enters an endless loop, and the network trainingwill not end, eventually leading to RNN paralysis. Therefore,simple RNNs are prone to gradient disappearance problemsand are not suitable for long-term predicting.The purpose of designing LSTM is to avoid or reduce theappearance of the problem of vanishing gradient whiledealing with long-term correlation time series by simpleRNN. Based on the simple RNN, the LSTM adds the outputgates, the input gates, and the forget gates. In Figure 3, allthree gates are replaced by σ, which can effectively preventthe gradient from being eliminated. Therefore, LSTM cansolve long-term dependence problems. The purpose of designing memory neurons is to store some LSTM importantinformation for state information. In addition, in general,each gate has an activation function. This function performsnonlinear transformations or trade-offs on data. Generally,the forget gate ft can filter some status information. Equations (1)–(6)fd6 associated with the LSTM neural network isshown below. They are for the forget gate, input gate, inverseof the memory cell, memory cell, output gate, and output,respectively: ft σ Wf · ht 1 xt bf ,(1) it σ Wi · ht 1 xt bi , t tanh WC · ht 1 xt bC ,C(2)(3)

Complexity5ytCt–1Ct tanhftitσ Ctσ ottanhσht–1htXtFigure 3: The basic structure of the LSTM neural network. t,Ct ft Ct 1 it C(4) ot σ Wo · ht 1 xt bo ,(5)1ht ot tanh Ct .(6)0IMF 1; iteration 02–13.2. EMD. The empirical mode decomposition (EMD)proposed by Huang et al. in 1998 [42] is a widely usedadaptive time-frequency analysis method. Empirical modedecomposition is an effective decomposition method fortime-series data. Due to the common local features of timeseries data, the EMD method has extracted the required datafrom them and obtained very good results in applicationsfields. Hence, many scholars apply the EMD method inmany fields successfully. The prerequisite for EMD decomposition is the existence of the following three assumptions [43]:The following briefly introduces the decompositionprocess:(1) Suppose there is a signal s(i) with the black line inFigure 4. The extreme value of the signal is shown inred and blue dots. Form the upper wrapping linethrough all the blue dots and form the lowerwrapping line through all the red dots.(2) Calculate the average value of the lower and upperwrapping lines to form an average purple-red linem(i). Here, define the discrepancy line d(i) asd(i) s(i) m(i).(7)(3) Judge whether the discrepancy line d(i) is an IMFaccording to IMF judgment rules. If the discrepancyline d(i) comply with IMF judgment rules, the discrepancy line d(i) is the ith IMF f(i). Otherwise, thediscrepancy line d(i) is considered the signal s(i) and–2102030405060708090100 110 120Figure 4: The decomposition process chart of sequences.repeat two steps above until d(i) complies with IMFjudgment rules. After this, the IMF f(t) is defined asf(t) d(t),t 1, 2, 3, . . . , n 1.(8)(4) Calculate and get the IMF f(t) byr(t) s(i) c(t) ,(9)where r(t) is considered the residual signal.(5) Repeat execution the four steps above N times untilrunning status meets stop conditions. Obtain the NIMFs which meet withr 1 c 2 r2 , rN 1 cN rN .(10)(1) Next level signal has to contain more than two extreme values: one is the minimum value and theother is the maximum value

6Complexity(2) Determine the time scale of signal characteristicbased on the time difference between two extremevalues(3) If the data has only an inflection point and no extremum, more times judgments are needed to revealthe extremumFinally, the following equation (11) expresses the composition of the original signal s(i):Ns(i) fj (t) rj (t) .(11)j 13.3. EEMD. A classical EMD has mode mixing problemswhen decomposing complex vibration signals. To solve theabove problem, Wu and Huang [44] proposed the EEMDmethod in 2009. The EEMD method is short for ensembleempirical mode decomposition method. EEMD is commonly used for nonstationary signal decomposition.However, the EEMD method is significantly different fromWT transform and FFT transform. Here, WT is the wavelettransform and FFT is the fast Fourier transform. Without theneed for basis functions, the EEMD method can decomposeany complex signal. At the same time, the EEMD methodcan decompose any signal into many IMFs. Here, IMF is theintrinsic modal function. The decomposed IMF componentscontain local different feature signals. EEMD can decomposenonstationary data into multiple stable subdata and then useHilbert transform to get the time spectrum, which hasimportant physical significance. Comparative analysis withFFT and WT decomposition, the EEMD has the characteristics of intuitive, direct, posterior, and adaptive. TheEEMD method has adaptive characteristics because of thelocal features of the time-series signal. The following brieflyintroduces the process of EEMD decomposing data.Assume that the EEMD will decompose the sequence X.According to the steps of EEMD decomposition, n subsequences will be obtained after decomposition. These nsubsequences include n 1 IMFs and one remaining subsequence Rn. Here, these n 1 IMFs are n 1 componentsubsequence Ci(i 1, 2, ., n 1) of the original sequence X.These n 1 IMFs are named IMFi(i 1, 2, ., n 1). Theremaining subsequence Rn is sometimes named residualsubsequence. The detailed steps of using EEMD to decompose the sequence are introduced as follows:(1) Suppose there is a signal s(i) with the black line inFigure 4. The extreme value of the signal is shown inred and blue dots. Form the upper wrapping linethrough all the blue dots and form the lowerwrapping line through all the red dots. The lowerwrapping line is shown in the red line in Figure 4.And the upper wrapping line is the blue line inFigure 4.(2) Calculate the average value of the lower and upperwrapping lines to form a mean line m(i) which isshown in purple-red line in Figure 4.(3) Obtain the first component IMF1 of the signal s(i).The IMF1 is obtained by calculation formula d(i) s(i) m(i). This formula means the difference of theoriginal signal s(i) minus the mean line m(i) of thelower and upper wrapping lines.(4) Take the first component IMF1 d(i) as a new signals(i) and repeat execution the three steps above untilrunning status meets stop conditions. The followingdescribes the stop condition in detail:(a) The average m(i) is approximately 0(b) The number of signal lines d(i) passing throughzero points is greater than or equal to the numberof extreme points(c) The number of iterations reaches the setmaximum(5) Take subsequence d(i) as the ith IMF fi (i 1, 2, . . .,n 1). Obtain the residual R by calculating theformula r(i) s(i) d(i).(6) Take the residual r(i) as the new signal s(i) to calculate the (i 1)th IMF, and repeat execution the fivesteps above until running status meets stop conditions. The following describes the stop condition indetail:(a) The signal s(i) has been completely decomposedand has obtained all IMF(b) The number of decomposition level reaches theset maximumFinally, the following equation (12) expresses the composition of the original sequences and n subsequencesdecomposed by the EEMD:n 1x(t) Ci Rn ,i 1, 2, . . . , n 1,(12)i 1where the number n of subsequences depends on thecomplexity of the original sequences. Figure 5 shows the sintime series represented by equation (13) and the IMF diagram of it which is decomposed by the EEMD:x(t) sin(20πt) 2 sin(200πt) 3t,(13)where t 0, f, 2f, . . . , 1000f, f 0.001 .4. MethodologyThe principle of our hybrid prediction method LSTMEEMD for stock price based on LSTM and EEMD is introduced in detail in this section. These theories are thetheoretical formation of our forecasting methods. The following first introduces the flowchart, the basic structure, andthe process of the LSTM-EEMD hybrid stock index prediction method based on the ensemble empirical modedecomposition and the long-short term memory neuralnetwork. Our proposed hybrid LSTM-EEMD predictionmethod first uses the EEMD to decompose the stock indexsequences into a few simple stable subsequences. Then, thepredict result of each subsequence is predicted by the LSTM

Complexity7EEMD decomposition of sin .0IMF 120–2IMF 210–13R.210Time (s)Figure 5: The sequences of sin signal decomposed by the EEMD.method. Finally, the LSTM-EEMD obtains the final prediction result of the original stock index sequence by fusingall LSTM prediction results of several stock indexsubsequences.Figure 6 shows the structure and process of the LSTMEEMD method. The basic structure and process of theLSTM-EEMD predict method include three modules. Thethree modules are the EEMD decomposition module, LSTMprediction module, and fusion module. Our proposedLSTM-EEMD prediction method includes three stages andeight steps. Figure 7 shows three stages and eight steps of ourproposed method. The three stages of the proposed hybridLSTM-EEMD prediction method are input data, modelpredict, and evaluate model. The model evaluation stage andthe data input stage each include 4 steps, and the modelprediction stage includes 3 steps. The hybrid LSTM-EEMDprediction method is introduced in detail as follows:(1) The simulation data is generated. And the real-worldstock index time-series data are collected. Then, theoriginal stock index time-series data are preprocessed to make the data format of stock indextime series satisfy the format requirements for decomposition of the EEMD. Finally, the input data Xof the LSTM-EEMD hybrid prediction method isformed.(2) The input data X is decomposed into a few sequencesby the EEMD method. If n subsequences are obtained, then there are one residual subsequence Rnand n 1 subsequences. These n subsequences areStock index sequence input g LSTM predict for IMFi and RnSubP1SubPn-1SubP2SubPnFusionPrediction results totalPFigure 6: The structure and process of the LSTM-EEMD method.expressed as Rn, IMF1, IMF2, IMF3, . . ., IMFn 1,respectively.(3) The prediction process of any subsequence is notaffected by the prediction process of other subsequences. A LSTM model is established and trainedfor each subsequence. Hence, we need to build and

8ComplexityR2 to evaluate the LSTM-EEMD hybrid predictionmethod. According to these evaluation values, thepros and cons of the method can be judged.(1) Collection data.Data input(2) Preprocess data.(3) EEMD decompose data.Model predict(4) Obtain IMFi, Rn of n subsequenceIMF1IMF2 IMFn–1RnLSTM1LSTM2 LSTMn–1LSTMnSubP1SubP2 SubPn–1SubPnModel evaluationn (k SubP )(5) Fusion: totalP i 1ii(6) Obtain prediction results totalP.(7) Obtain RMSE, MAE, R2 of totalP(8) Model evaluation results.Figure 7: The data flowchart of our proposed hybrid method.train n LSTM for n independent subsequences. Thesen independent LSTM models are namedLSTMk(k 1, 2, . . ., n 1, n), respectively. We use then LSTM models to predict these n independentsubsequences and get n prediction values of the stockindex time series. These n prediction values arenamed SubPk (k 1, 2, . . ., n 1, n), respectively.(4) Fusion function is the core of hybrid method. Atpresent, there are many fusion functions, such assum, weighted sum, weighted product, and so on.The function of these fusion functions is to mergeseveral results into the final result. In this paper, theproposed hybrid stock prediction method selectedthe weighted sum as the fusion function. Theweighted results of all subsequences are accumulatedto form the final prediction result for the originalstock index data. The weight here can be presetaccording to the actual application. In this paper, weuse the same weight of each subsequence and theweight of each subsequence is 1.(5) Finally, we compare the predicted values with theactual value of stock index time-series data sequenceand calculate the values of RMSE, MAE, and R2. Weuse three evaluation criteria of the RMSE, MAE, andFigure 7 shows the predict progress and data flowchart ofthe proposed LSTM-EEMD method in this paper. Thepredict progress in Figure 7 can be introduced in 3 stages.The three stages are input data, model predict, and evaluatemodel. The stage of input data is divided into 4 steps. Thefour steps are collect data, preprocess data, decompose databy the EEMD, and generate n sequence. There are n LSTMmodel in the stage of model predict. The input data of theLSTM model is n sequences genera

the relationship between stock prices and these factors. Although these factors will temporarily change the stock price, in essence, these factors will be reflected in the stock price and will not change the long-term trend of the stock price. erefore, stock prices can be predicted simply with historical data.