Trend Prediction with LSTM RNNs using Keras (Tensorflow) in 3 Steps

Understanding the up or downward trend in statistical data holds vital importance. It helps in estimation, prediction, and forecasting things ahead of time. Its potential application is predicting stock markets, prediction of faults and estimation of remaining useful life of systems, forecasting weather, etc.

This article covers the implementation of LSTM Recurrent Neural Networks to predict the trend in the data.

We have been using different machine learning algorithms for classification, clustering, detection, and estimation purposes. Presently Convolutional Neural Networks (CNNs) is considered as the best machine learning algorithm that implements Deep Learning concept, it learns more like the way, human learns. But even CNNs don’t perform well on the data that are time-dependent or hold sequential information.

Recurrent Neural Networks (RNNs) is a vital candidate for solving that kind of problem. RNNs have shown excellent performance in different problems such as machine translations, image captioning, speech recognition, character and word prediction in a sentence, etc. But still knowledge in sequential and time series problems is something hard to keep track of. Conventional RNNs fall short of memory while holding the information back in time.

In this article, onwards RNNs and LSTMs will be used interchangeably. I will not go in detail of explaining how LSTMs work and solve the issue of vanishing gradient, because there are so many blogs available over the internet about that. The thing, which lacks is a candid programmatic implementation of LSTMs for prediction of a trend in the data. In this article, I will try to cover the implementation of LSTMs for sequence prediction purposes.

I am hopeful; you will understand it and find it very helpful. Steps involved in the process starting from data preparation to classifier training and making predictions are given as follows:

1. Data preparation

Convert an array of values into a dataset matrix
 def dataset_generate(data, step_size=1):
    dataX, dataY = [], []
    for i in range(len(data)- step_size -1):
       a = data[i:(i+ step_size), 0]
       dataX.append(a)
       dataY.append(data[i + step_size, 0])
    return numpy.array(dataX), numpy.array(dataY)
 
 Fix random seed for reproducibility
 numpy.random.seed(7)
 
 Load the dataset
 dataframe = pandas.read_csv('sample_var.csv', usecols=[0], engine='python', skipfooter=3)
 dataset = dataframe.values
 dataset = dataset.astype('float32')
 
 Normalize the dataset
 scaler = MinMaxScaler(feature_range=(0, 1))
 dataset = scaler.fit_transform(dataset)
 
 Split into train and test sets
 train_size = int(len(dataset) * 0.75)
 test_size = len(dataset) - train_size
 train, test = dataset[0:train_size,:], dataset[train_size:len(dataset),:]
 
 Reshape into X=t and Y=t+1
 step_size = 1
 trainX, trainY = dataset_generate(train, step_size)
 testX, testY = dataset_generate(test, step_size)
 
 Reshape input to be [samples, time steps, features]
 trainX = numpy.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
 testX = numpy.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

2. Training LSTMs Network

Create a fit the LSTM network:

model = Sequential()
 model.add(LSTM(4, input_dim= step_size))
 model.add(Dense(1))
 model.compile(loss='mean_squared_error', optimizer='adam')
 model.fit(trainX, trainY, nb_epoch=1000, batch_size=1, verbose=2)

3. Testing LSTMs Network

trainPredict = model.predict(trainX)
 
 testPredict = []
 
 input = trainY[-1]
 temp = [[input]]
 predX = [temp]
 
 a = model.predict(numpy.array(predX))
 b = a.tolist ()
 predX = [b]
 testPredict.append(b[0])

Make predictions with the updated models:

for i in range(test_size - 3):
     print ('Iteration %d: Done' % i)
     trainX = numpy.concatenate([trainX, [b]])
     trainY = numpy.concatenate ([trainY, b[0]])
     model.fit (trainX, trainY, nb_epoch=1000, batch_size=1, verbose=2)
 
     a = model.predict (numpy.array ([b]))
     b = a.tolist ()
     testPredict.append (b[0])
 
     print ('Prediction %d:'%i, b)
 testPredict = numpy.array(testPredict)

LSTM RNNs are implemented in order to estimate the future sequence and predict the trend in the data. It does predict unseen data really well within the range of training data. But outside the boundaries of training data, it does not make the estimation as expected. You will notice it after implementing the given code. Further improvement needs to be incurred to predict the trend accurately similar to regression techniques.