LSTM Based Stock Prediction Modeling And Analysis

Advances in Economics, Business and Management Research, volume 648Proceedings of the 2022 7th International Conference on Financial Innovation and Economic Development (ICFIED 2022)LSTM-based Stock Prediction Modeling and AnalysisRuobing Zhang1,*1Beijing University of Technology, BJUT*Ruobing Zhang.Email:1262385605@qq.comABSTRACTThe stock market plays an important role in the economy of a country in terms of spending and investment. Predictingstock prices has been a difficult task for many researchers and analysts. Research in recent years has shown that LongShort-Term Memory (LSTM) network models perform well in stock price prediction, and it is considered one of themost precise prediction techniques, especially when it is applied to longer prediction ranges. In this paper, we set theprediction range of the LSTM network model to 1 to 10 days, push the data into the built LSTM network model afterpre-processing operations such as normalization of data, and set the optimal values of epochs, batch size, dropout,optimizer and other parameters through training and testing. By comparing with Linear Regression, eXtreme gradientboosting (XGBoost), Last Value and Moving Average, the results show that the LSTM network model does notperform better than other models when applied to a short forecasting horizon.Keywords: Long Short-Term Memory, Stock Market, forecasting, prediction.1.INTRODUCTIONWith the continuous application and development ofartificial intelligence technology and big datatechnology, along with the further improvement of thefinancial market and the strong demand of the financialservice industry, stock market forecasting has attractedwide attention from the industry and academia[1].Machine learning algorithms such as decision trees,genetic algorithms, support vector machines, logisticregression and deep learning network models have beenapplied to stock forecasting in research. In recent years,LSTM network models have become a hot researchtopic for researchers. The LSTM neural network is atype of realizable recurrent neural network model withselective memory and intra-temporal influence, which isvery suitable for the stochastic non-stationary series ofstock price time series. LSTM network is considered tobe one of the most accurate forecasting techniques.A large number of studies is currently active on thesubject of LSTM neural network used in finance. Somestudies used a various set of parameters with a differentnumber of epochs to measure the RMSE of the LSTMneural network model so as to improve the accuracy ofthe model [2], others used ML algorithm based onLSTM RNN to do predictions and tried to find the bestsets for bout data length and the number of trainingepochs that better suit the assets and maximize theaccuracy of the predictions[3]. Another paper comparedthe evaluation indicators with the prediction results tofind the appropriate number of LSTM layers and hiddenneurons[4]. And some studies explored whether LSTMneural networks can be applied to the price trendprediction of individual stocks[5]. Also, somestudiesemployed tree-based models and neural networks (ANN,RNN, and LSTM) to correctly forecast the values offour stock market groups as a regression problem and tosee which models are more precise[6]. Another paperdid a comparison of ARIMA, ANN and LSTM for stockprice prediction[7]. Many existing literatures study theapplication of LSTM neural network models to stockmarket forecasting, but most of them focus on applyingLSTM to forecast a longer forecast range and improvingthe accuracy of the predictions. The performance ofLSTM neural network models in this regard doesoutperform other machine learning methods. In additionto improving the prediction accuracy of LSTM neuralnetwork models by optimizing parameter settings andprocessing data, some studies have proposed morecomplex models based on LSTM models to improve theprediction accuracy. Some studies processed stock datathrough a wavelet transform and used an attention-basedLSTM neural network to predict the stock[8]. Othersproposed a multi-value associated network model ofLSTM-based deep-recurrent neural network (AssociatedNet) to predict multiple prices of stocksimultaneously[9].Copyright 2022 The Authors. Published by Atlantis Press International B.V.This is an open access article distributed under the CC BY-NC 4.0 license 37

Advances in Economics, Business and Management Research, volume 648However, there is a lack of research applying LSTMneural network models to predict the stock market in ashort-term range. In this paper, we apply five models,LSTM, Linear Regression, XGboost, Moving Average,and Last Value, respectively, by setting the predictionscope and using the historical data of Vanguard TotalStock Market Index Fund ETF Shares (VTI) from10/5/2018 to 10/4/2021, as the original data forshort-term prediction. The forecasting performance ofthe models is evaluated using root mean square error(RMSE) and mean absolute percentage error (MAPE).The main objective of this paper is to investigate theprediction accuracy of LSTM neural network modelsapplied to short-term prediction ranges and to seewhether the LSTM model shows certain advantages,compared to other machine learning algorithms.2.LONG SHORT-TERM MEMORY (LSTM)Long Short - Term Memory (LSTM) is a long andshort-term memory network, which is a time-recursiveneural network suitable for processing and predictingimportant events with relatively long intervals anddelays in time series. The LSTM algorithm was firstproposed by Sepp Hochreiter and Jurgen Schmidhuberin 1997 as a specific form of Recurrent neural network(RNN).Long Short-Term Memory (LSTM) is one of manytypes of Recurrent Neural Network RNN, it’s alsocapable of catching data from past stages and using itfor future predictions [10]. In general, an ArtificialNeural Network (ANN) consists of three layers: consistsof three layers:1) Input layer,2) Hidden layers,3) Output layer.In LSTM, the early stages can be remembered bygating and joined along the memory line. The followingFigure 1 depicts the composition of LSTM nodes.one cell to another. Finally, the sigmoidal neuralnetwork layers consisting of gates drive the cells to anoptimal value by disposing or letting the data pass. Eachsigmoid layer has a binary value (0 or 1), where 0 means"doesn't let anything through"; and 1 means "letseverything through". The goal here is to control the stateof each cell.The gate is controlled in the following way:- The Forget Gate outputs a number between 0 and 1,where 1 means "keep this completely"; and 0 means"ignore this completely".- The Memory Gate selects which new data will bestored in the cell. First, a sigmoid layer "input gatelayer" selects which values will be changed. Next, atanh layer makes a vector of new candidate values thatcan be added to the state.- The Output Gate determines what the output ofeach cell will be. The output value will be based on thestate of the cell and the filtered freshest data[3].3.METHODOLOGY AND DATAOur system can be divided into 4 parts. These areraw data acquisition, date pre-processing, featureextraction and training of Neural Network.3.1. Raw DataThe data used in this article is from Yahoo Financeand includes 7 features: Date, Open Price, High Price,Low Price, Close Price, Adjusted Closing Price andVolume. The data structure is shown in Table 1. For VIT,our data series cover the period going from 10/5/2018 to10/4/2021.3.2. Data Pre-processingThe pre-processing stage involves:a) Data discretization: Part of data reduction but withparticular importance, especially for numerical datab) Data transformation: Normalization.c) Data cleaning: Fill in missing values.Figure 1. The internal structure of an LSTM [11].The ability to memorize data sequences makesLSTMs a special kind of RNNs. Each LSTM node ismostly composed of a set of cells responsible for storingthe passed data streams. The upstream connection modelin each cell acts as a transport line to pass data from thepast to the present, and the independence of the cellshelps the model to dispose of filters that add values fromAfter the dataset is transformed into a clean dataset,the dataset is divided into training, testing andcross-validation set so as to evaluate. In order to buildour model, we are going to use the LSTM RNN. Ourmodel uses 60% of data for training, 20% of data fortesting and 20% of data for cross-validation.3.3. Feature ExtractionIn this stage, only the features which are to be fed tothe neural network are chosen. We will choose the featurefrom date, open, high, low, close, adj close and volume.2538

Advances in Economics, Business and Management Research, volume 648In this paper, the feature adj close is selected and itsline graph is shown in Figure 2. It is clear that adjustedclosing price rose considerably between 2018 and 2021.However, at the beginning of 2020, the entire economicmarket has been hit hard by the outbreak of theCOVID-19 and there is a sudden decrease can be seen inthis year, with the price of the stock dropped by roughly 60.Figure 2. Adj closeand trained for prediction. For training, we use meansquared error to optimize our model and determine theoptimal parameter settings:3.4. Training Neural NetworkIn this stage, the data is fed to the neural networkTable 1.The structure of raw 9996143.800003141.720001adj 12003386400399040031483003783800epochs 50dropout 0.2batch size 8The structure of our model is shown in the Table 2.lstm units 128Table 2. the LSTM model summeryLayer(type)Output ShapeParameterslstm 2(LSTM)(None,9,50)10400dropout 2 (Dropout)(None, 9, 50)0lstm 3 (LSTM)(None, 50)20200dropout 3 (Dropout)(None, 50)0dense 1 (Dense)(None, 1)51Total params: 30,651Trainable params: 30,651Non-trainable params: 02539

Advances in Economics, Business and Management Research, volume 6484. RESULT AND DISCUSSIONIn this experiment, four other methods were alsoused to make predictions in this paper separately andcompared with the prediction results of applying theLSTM neural network model. The prediction results areshown in the following figures:The result of using Last Value to predict is shown inthe Figure 3. This is the most cost effective forecastingmodel. We can find that the forecast for each day (redcross) is simply the value of the previous day (greencross). Last Value is often used as a benchmark forcomparing more complex models.The result of using linear regression to predict isshown in the Figure 4. It can be found that this methoddoes not capture changes in stock prices up or downwell.The prediction results of using Moving Average,XGBoost and LSTM can be shown in Figure 5, Figure 6and Figure 7 respectively. Using the method ofXGBoost is an iterative process of transforming weaklearners into strong learners. In the moving averagemethod, the predicted value will be the average of thefirst N values. As can be seen from the figures, there isnot much difference between the prediction results ofthese three methods, but the prediction results obtainedby using the LSTM and the XGBoost respectively areslightly better than using the Moving Average, withrelatively small errors between the test and predictedvalues.Figure 4. Forecast results of Linear RegressionFigure 5. Forecast results of Moving AverageFigure 6.Forecast results of XGBoostFigure 3. Forecast results of Last Value2540