2.1.

LSTM network

LSTM network is an improved RNN¹³, which can learn the long-term dependence information of sequences and overcome the gradient disappearance and explosion problems of traditional RNN. The structure of LSTM cell is shown in Figure 1. t is current time. f_t, i_t, o_t, g_t and c_t represent forgetting gate, input gate, output gate, the candidate value of cell state and cell state at time t respectively. x_t is current input. h_t-1 represents the output at time 0074-1. σ is Sigmoid function. ® represents the product of the corresponding elements of two vectors.

Figure 1.

The structure of LSTM cell.

2.2.

Construction of forecasting model based on LSTM

For LSTM forecasting model, its input and output data should be set first. Assuming that the IES load (electric, cooling and heat load) at time t on day d is predicted, the input data are composed of the 16-dimensional features of time t on the previous 7 days (d-7, …, d-1). The features include IES load, meteorological information and calendar information, as shown in Table 1. DNI, DHI and GHI are abbreviation for direct normal irradiance, diffuse horizontal irradiance and global horizontal irradiance. Therefore, the input sequence length of LSTM network is 7 and the feature dimension of each time is 16. For the working day type in Table 1, the working day is 1, otherwise it is 0. For the holiday type, the holiday is taken as 1, otherwise it is taken as 0.

Table 1.

Feature set of LSTM.

The feature values of different dimensions vary dramatically. Therefore, the normalization operation is carried out as follows:

where x_i and represent the values before and after normalization. and are the maximum and minimum values of the ith feature respectively.

LSTM network structure can be described as follows: the normalized input sequence data are sent to the input layer, then cascade multiple hidden layers, and finally the electric, cooling and heat load are predicted through the fully connected layer.

3.1.

CNN model

CNN is a special feedforward neural network¹⁴. Its structure is similar to that of multi-layer perceptron (MLP). It has the advantages of equivalent representation, sparse interaction and parameter sharing. CNN usually consists of three parts: convolution layer, pooling layer and full connection layer. Among them, the convolution layer generally convolves with input data with multiple kernels, and then adopts nonlinear activation function to extract features. The dimension of the pooling layer is reduced by down sampling. Then the features are flattened into one-dimensional vector and sent to the fully connected layer for prediction.

3.2.

Construction of forecasting model based on CNN

The input structure of CNN model is slightly different from that of LSTM network. The 16-dimensional features of LSTM network input data are arranged into 4×4 matrix, taking the input sequence length as the number of channels. The output is the same as that of LSTM network.

4.1.

LightGBM model

LightGBM is an efficient GBDT model based on histogram¹¹. It adopts gradient based single-sided sampling and leaf growth strategy with depth limit, makes it have better training speed, less memory consumption and higher accuracy. LightGBM constructs one regression tree at a time by fitting the residual of the previous regression tree. LightGBM combines weak learners into a single strong learner in the iterative process by minimizing the loss function, as shown in equation (2).

where x is the input feature vector, f (x) denotes the final predicted value and f_i (x) accounts for the output of the ith regression tree.

4.2.

Construction of forecasting model based on LightGBM

6.1.

Experimental data and platform

The user level IES multi-energy load data is from Tempe campus of Arizona State University¹⁶. Meteorological data are from the National Renewable Energy Laboratory of the United States¹⁷.

The data from May 25, 2017 to August 31, 2019 are selected as the experimental data, and the sampling interval is 1 hour. The data from August 25 to August 31, 2019 constitute the test set, and the other data are randomly divided into training set and validation set according to 4:1.

The experimental hardware is configured with Intel Core i5-4200 CPU and 8G memory. Python language is used for programming, and each model is implemented by calling PyTorch and scikit-learn libraries respectively.

6.2.

Evaluation indices

In order to comprehensively evaluate the performance of MELF methods, mean absolute percentage error (MAPE) and RMSE indicators are selected to evaluate the effect of single load and integrated load forecasting. MAPE is defined as:

where m is the number of samples in the test set. denote MAPE for class j load. W_MAPE is the integrated MAPE of IES loads, which represents the overall performance of MELF. α_j is the importance ratio of class j load (electric, cooling or heat load).

6.3.

Model parameter setting

The parameter setting has a great impact on MELF performance of each model. In this paper, the grid search method is used to obtain the optimal hyper parameters of LSTM network and CNN model.

The LSTM network is provided with a hidden layer with 16 neurons. Two convolution layers are set in CNN model, the number of convolution kernel is 8 and 16 respectively, and the size of convolution kernel is 2×2. There is no pooling layer, two fully connected layers are set, the number of neurons is 10 and 3 respectively, and the activation function is ReLU. Both LSTM network and CNN model are optimized by Adam algorithm, the learning rate is 0.01, the training batch size is set to 24, and epochs are 100.

In order to evaluate the performance of the combined method (denoted as IRMSE-LCL) in MELF, it is compared with SVR and MLP. The hyper parameters of LightGBM, SVR and MLP are optimized by HS algorithm. The relevant parameter settings of HS are shown in Table 2.

Table 2.

Parameter setting of HS.

The inputs of SVR and MLP are the same as that of LightGBM model. Similar to the LightGBM model, the output of the SVR model is one-dimensional, so it is necessary to train the SVR model for electric, cooling and heat loads respectively. The output of MLP model is 3-dimensional, so only one MLP model needs to be trained to predict the electric, cooling and heat loads at the same time. The SVR model adopts Gaussian kernel function, and the hyper parameters optimized by HS include insensitive loss parameter, penalty coefficient and kernel function parameter. For the MLP model, the hyper parameters optimized by HS include the number of hidden layer neurons and L2 norm penalty parameter. The activation function also selects ReLU. The optimization algorithm is Adam, with adaptive learning rate and 500 iterations.

6.4.

Result analysis

IRMSE-LCL model is compared with single LSTM network, CNN, LightGBM, SVR and MLP, and with the simple average combination model of LSTM, CNN and LightGBM (represented by Avg-LCL). The MAPE, RMSE and integrated MAPE of day ahead MELF on the test set are shown in Tables 3 and 4. When calculating the integrated MAPE, the importance ratios of electric, cooling and heat load are set to 0.4, 0.4 and 0.2 respectively³.

Table 3.

MAPE comparison of different models.

Table 4.

RMSE comparison of different models.

From Tables 3 and 4, we can conclude that:

The proposed method combines LSTM network, CNN and LightGBM, which can effectively combine the respective advantages of each model and learn from each other. It can not only enhance the model’s perception of timing information, but also fully mine the characteristic information of discontinuous data. Giving more weight to the model with higher prediction accuracy can effectively improve the overall prediction performance.