Multistep Ahead Prediction of Electric Power Systems Using Multiple Gaussian Process Models

Multistep Ahead Prediction of Electric Power Systems Using Multiple Gaussian Process Models Tomohiro Hachino, Hitoshi Takata, Seiji Fukushima, Yasutaka Igarashi, and Keiji Naritomi Kagoshima University, Kagoshima, Japan Email: {hachino, takata, fukushima, igarashi}@eee.kagoshima-u.ac.jp, k0519282@kadai.jp Abstract—This paper focuses on the problem of multistep ahead prediction of electric power systems using the Gaussian process models. The Gaussian process model is a nonparametric model and the output of the model has Gaussian distribution with mean and variance. The multistep ahead prediction for the phase angle in transient state of the electric power system is accomplished by using multiple Gaussian process models as every step ahead predictors in accordance with the direct approach. The proposed prediction method gives the predictive values of the phase angle and the uncertainty of the predictive values as well. Simulation results for a simplified electric power system are shown to illustrate the effectiveness of the proposed prediction method. Index Terms—multistep ahead prediction, Gaussian process model, direct method, electric power system I. INTRODUCTION In recent years, model predictive control (MPC) has received much attention in both process control and servo control [1]-[5]. The performance of MPC greatly depends on the accuracy of the model used for prediction. Therefore, to improve the performance of MPC, it is urgent to develop an accurate predictor. The Gaussian process (GP) model is one of the attractive models for multistep ahead prediction. The GP model is a nonparametric model and fits naturally into Bayesian framework [6]-[8]. This model has recently attracted much attention for system identification [9], [10], time series forecasting [11]-[13], and predictive control [3], [14], [15]. Since the GP model gives us not only the mean value but also the variance of the conditionally expected value of the output, it is useful for MPC considering the uncertainty of model. Moreover, the GP model has far fewer parameters to describe the nonlinearity than the parametric models such as radial basis function (RBF) model, neural network model, and fuzzy model. There are two approaches to multistep ahead prediction. One is the direct method that makes multistep ahead prediction directly by using a specific step ahead predictor. The other is the iterated method that repeats one-step ahead prediction up to the desired step. The Manuscript received June 3, 2014; revised September 18, 2014. iterated multistep ahead predictions with propagation of the prediction uncertainty based on the GP model were presented in [11], [12]. Although the computational burden of this approach is not so heavy during the training phase, unacceptable prediction errors are gradually accumulated as the prediction horizon increases especially in the presence of measurement noise. Therefore, with the aim of MPC, this paper proposes the direct method for multistep ahead prediction of the electric power systems in the GP framework. Multistep ahead prediction for the phase angle in transient state of the electric power system is directly performed by using the multiple trained GP models as every step ahead predictor. The proposed direct method uses not only one- step ahead predictor but also all-step ahead predictors. Therefore, although each step ahead predictor has a systematic error, the prediction errors are not accumulated so much as the prediction horizon increases. The proposed direct method gives the predictive values of the phase angle and uncertainty of the predictive values as well. The uncertainty of the predictive values is usually not obtained by the non GP-based direct methods such as the RBF-based direct method. This paper is organized as follows. In section II, the problem of multistep ahead prediction is formulated for an electric power system. In section III, the multiple GP prior models are derived for every step ahead predictors and the training method of the GP prior models is briefly described. In section IV, the direct multistep ahead prediction is carried out using the GP posterior distribution. In section V, simulation results are shown to illustrate the effectiveness of the proposed prediction method. Finally, conclusions are given in section VI. II. STATEMENT OF THE PROBLEM Consider a single machine power system described by { ̃ ̈( ) ̃ ̇( ) ( ( )) ( ) ( ) ( ) ( ) (1) where ( ) phase angle， ( ): phase angle corrupted by the measurement noise ( ), ( ): increment of 316©2015 Engineering and Technology Publishing Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015 doi: 10.12720/joace.3.4.316-321 excitation voltage, ̃ : inertia coefficient, ̃ : damping coefficient, : generator output power, : turbine output power, : excitation voltage, : infinite bus voltage, and : system impedance. The measurement noise ( ) is zero mean white Gaussian noise with variance It is assumed that the input ( ) ( ) and the noisy measurement of the output ( ) ( ) at are available when the multistep ahead predictors are trained, where is the sampling period. The problem of multistep ahead prediction is usually to estimate the future outputs given the past input and output data. The optimal predictor can be written as ̂( ) [ ( ) ( )] (2) where [ ] is the expectation operator, and ( ) [ ( ) ( ) ( ) ( ) ( ) ( )] (3) Which is the state vector consisting of the past outputs and inputs up to the prespecified lags and . Actually, with the GP framework, not only estimates ̂( ) but also its uncertainty, i.e., the variance ̂ ( ) are estimated. Therefore, the problem of this paper is to construct the following probability distributions for the multistep ahead prediction ( ) ( ) ( ̂( ) ̂ ( )) ( ) (4) And to carry out multistep ahead prediction up to step based on these distributions, by using the GP framework. III. GP PRIOR MODEL A. Derivation of GP Prior Models Consider a -step ahead predictor as ( ) ( ( )) ( ) ( ) (5) where ( ) is a function which is assumed to be stationary and smooth. ( ) is zero mean Gaussian noise with unknown variance . In this paper, this predictor is constructed in the GP framework. Putting on (5) yields (6) where [ ( ) ( ) ( )] [ ( ) ( ) ( )] [ ( ) ( ) ( )] [ ] [ ( ) ( ) ( )] (7) and are the vector of model outputs and the vector of function values for the j-step ahead predictor, respectively. is the model input matrix and is common for every step ahead predictors. { } is the training input and output data for the j-step ahead predictor. A GP is a Gaussian random function and is completely described by its mean function and covariance function. We can regard it as a collection of random variables which has joint multivariable Gaussian distribution. Therefore, the vector of function values can be represented by the GP as ( ( ) ( )) (8) where ( ) is the N-dimensional mean function vector and ( ) is the N-dimensional covariance matrix evaluated at all pairs of the training input data. Equation (8) means that has a Gaussian distribution with the mean function vector ( ) and the covariance matrix ( ). The mean function is often represented by a polynomial regression [8]. In this paper, the mean function vector ( ) is expressed by the first order polynomial, i.e., a linear combination of the model input: ( ) [ ( ) ( ) ( )] ̃ (9) where ̃ [ ] and [ ] is the N- dimensional vector consisting of ones, and [ ( )] is the unknown weighting parameter vector of the mean function to be trained. The determination of will be discussed in the next subsection. The covariance matrix ( ) is constructed as ( ) [ ( ) ( ) ( ) ( ) ] (10) where the element ( ) ( ( ) ( )) ( ) is a function of and Under the assumption that the process is stationary and smooth, the following Gaussian kernel is utilized for ( ) : ( ) ( ) ( ‖ ‖ ) (11) where is the signal variance, is the length scale, and denotes the Euclidean norm. The free parameters and of (11) and the noise standard deviation are called hyperparameters and construct the hyperparameter vector [ ] can control the overall variance of the random function ( ) and determines the magnitude of the function ( ) can change the characteristic length scale so that the axis about the model input changes. If is set to be smaller, the function ( ) becomes more oscillatory. Therefore, the hyperparameter vector should be appropriately determined according to the training data for precise 317©2015 Engineering and Technology Publishing Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015 prediction. This parameter selection will be also presented in the next subsection. Since is noisy observation, we have the following GP model for j-step ahead prediction from (6) and (8) as ( ( ) ( )) (12) where ( ) ( ) (13) In the following, ( ) and ( ) are written as and , respectively. B. Training of GP Prior Models To perform multistep ahead prediction, the proposed direct approach needs 1 to step ahead prediction models as shown in Fig. 1. The accuracy of prediction greatly depends on the unknown parameter vector [ ] and therefore has to be optimized. This training is carried out by minimizing the negative log marginal likelihood of the training data: ( ) ( | ) | | ( ( )) ( ( )) ( ) | | ( ̃ ) ( ̃ ) ( ) (14) Since the cost function ( ) generally has multiple local minima, this training problem becomes a nonlinear optimization one. However, we can separate the linear optimization part and the nonlinear optimization part for this optimization problem. The partial derivative of (14) with respect to the weighting parameter vector of the mean function is as follows: ( ) ̃ ̃ ̃ (15) Note that if the hyperparameter vector of the covariance function is given, then the weighting parameter can be estimated by the linear least-squares method putting ( ) ⁄ : ( ̃ ̃) ̃ (16) However, even if the weighting parameter vector is known, the optimization with respect to hyperparameter vector is a complicated nonlinear problem and might suffer from the local minima problem. Therefore, the unknown parameter vector is determined by the separable least-squares (LS) approach combining the linear LS method and the genetic algorithm (GA) [16], as [ ] [ [ ] [ ] ] [ [ ] [ ] [ ] [ ]] Figure 1. The proposed multistep ahead prediction scheme. IV. MULTISTEP AHEAD PREDICTION BY GP POSTERIOR In section III, we have already obtained the GP prior models for ( ) step ahead predictors. In the proposed direct approach, multistep ahead prediction up to step is carried out directly using every GP prior models as shown in Fig. 1. For a new given test input ( ) [ ( ) ( ) ( ) ( ) ( ) ( )] And corresponding test output ( ) ( ) , we have the following the joint Gaussian distribution: [ ( ) ] ([ ( ) ( ) ] [ ( ) ( ) ( ) [ ] ]) ( ) (17) where is the starting step for prediction, and ( ) ( ) is the N-dimensional covariance vector evaluated at all pairs of the training and test data. ( ) is the variance of the test data. ( ) and ( ) are calculated by the trained covariance function. From the formula for conditioning a joint Gaussian distribution [17], the posterior distribution for a specific test data is ( ) ( ̂ ( ) ̂ ( )) ( ) (18) where ̂ ( ) ( ) ( ) ( ( )) (19) 318©2015 Engineering and Technology Publishing Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015 ̂ ( ) ( ) ( ) ( ) [ ] Which are the predictive mean and the predictive variance at the j-step ahead, respectively. It is noted that the nonlinearity of the predictive mean can be expressed by the trained hyperparameters even if the prior mean function is set to be a linear combination of the model input as (9). V. NUMERICAL SIMULATIONS Consider a simplified electric power system [18] described by { ̃ ̈( ) ̃ ̇( ) ( ( )) ( ) ( ) ( ) ( ) (20) where ̃ ̃ and These are all per unit values. The training data are sampled with sampling period as ( ) ( ) and ( ) ( ) at The measurement noise ( ) is zero mean Gaussian noise with standard deviation (noise to signal ratio (NSR): 1%), (NSR: 3%), or (NSR: 5%). The lags for the state vector (3) are chosen as and in the case of and in the case of and and in the case of respectively. The number of the training input and output data is taken to be for training each ( ) step ahead predictor. To validate the results of training, the prediction results for 1, 10 and 20 step ahead predictors in the case of are shown in Figs. 2-4. In these figures, the circles with lines show the predictive mean ̂ ( ), the crosses show the measurements (test output) ( ), and the shaded areas give the double standard deviation confidence interval (95.5% confidence region). From these figures, we can confirm that the error between the test data and the predictive mean is quite small for every step ahead predictors and it does not become so large as the prediction horizon increases. After training, the multistep ahead prediction up to step is carried out, where the starting step is set to be as an example. Figs. 5-7 show the results of the multistep ahead prediction by the proposed method. In these figures, the dotted lines show the true output ( ). The predictive means ̂ ( ) are quite close to the true output ( ) for all noise levels. Moreover, the probability that the true measurements ( ) are included in the double standard deviation confidence interval is totally 96.7%, which is very close to the expected value 95.5%. This indicates that the proposed prediction method gives the reasonable uncertainty (predictive variance). Therefore, we can say that the proposed multistep ahead prediction can be carried out successfully even in the presence of measurement noise. Figure 2. Prediction result for 1 step ahead predictor ( ). Figure 3. Prediction result for 10 step ahead predictor ( ). Figure 4. Prediction result for 20 step ahead predictor ( ). Figure 5. The result of multistep ahead prediction ( ). 319©2015 Engineering and Technology Publishing Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015 Figure 6. The result of multistep ahead prediction ( ). Figure 7. The result of multistep ahead prediction ( ). VI. CONCLUSIONS In this paper, we have proposed the multistep ahead prediction of electric power systems by using multiple Gaussian process models. The multistep ahead prediction has been carried out directly by using multiple Gaussian process models as every step ahead predictors. Through the numerical simulations for the simplified electric power system, it has been experimentally demonstrated that the proposed direct method is very accurate even in the presence of measurement noise. Therefore, the proposed prediction method has high potential for MPC. Developing MPC algorithm based on this prediction model is one of the future works. REFERENCES [1] H. Seki, M. Ogawa, S. Ooyama, K. Akamatsu, M. Ohshima, and W. Yang, “Industrial application of a nonlinear model predictive control to polymerization reactors,” Control Engineering Practice, vol. 9, no. 8, pp. 819-828, 2001. [2] Z. K. Nagy, B. Mahn, R. Franke, and F. Allgöwer, “Evaluation study of an efficient output feedback nonlinear model predictive control for temperature tracking in an industrial batch reactor,” Control Engineering Practice, vol. 15, no. 7, pp. 839-850, 2007. [3] B. Likar and J. Kocijan, “Predictive control of a gas-liquid separation plant based on a Gaussian process model,” Computers and Chemical Engineering, vol. 31, no. 3, pp. 142-152, 2007. [4] T. Geyer, “Generalized model predictive direct torque control: Long prediction horizons and minimization of switching losses,” in Proc. of IEEE Conference on Decision and Control, 2009, pp. 6799-6804. [5] J. Scoltock, T. Geyer, and U. K. Madawala, “A Comparison of model predictive control schemes for MV induction motor drives,” IEEE Trans. Industrial Informatics, pp. 909-919, 2013. [6] A. O’Hagan, “Curve fitting and optimal design for prediction (with discussion),” J. Royal Statistical Society B, vol. 40, pp. 1-42, 1978. [7] C. K. I. Williams and C. E. Rasmussen, “Gaussian processes for regression,” Advances in Neural Information Processing Systems, vol. 8, pp. 514-520, MIT Press, 1996. [8] C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, MIT Press, 2006. [9] J. Kocijan, A. Girard, B. Banko, and R. Murray-Smith, “Dynamic systems identification with Gaussian processes,” Mathematical and Computer Modelling of Dynamical Systems, vol. 11, no. 4, pp. 411-424, 2005. [10] T. Hachino and H. Takata, “Identification of continuous-time nonlinear systems by using a Gaussian process model,” IEEJ Trans. Electrical and Electronic Engineering, vol. 3, no. 6, pp. 620-628, 2008. [11] A. Girard, C. E. Rasmussen, J. Q. Candela, and R. Murray-Smith, “Gaussian process priors with uncertain inputs-application to multiple-step ahead time series forecasting,” Advances in Neural Information Processing Systems, vol. 15, pp. 542-552, MIT Press, 2003. [12] J. Q. Candela, A. Girard, J. Larsen, and C. E. Rasmussen, “Propagation of uncertainty in Bayesian kernel models-application to multiple-step ahead forecasting,” in Proc. IEEE Int. Conference on Acoustics, Speech, and Signal Processing, 2003, pp. 701-704. [13] T. Hachino and V. Kadirkamanathan, “Multiple Gaussian process models for direct time series forecasting,” IEEJ Trans. Electrical and Electronic Engineering, vol. 6, no. 3, pp. 245-252, 2011. [14] J. Kocijan, R. Murray-Smith, C. E. Rasmussen, and B. Likar, “Predictive control with Gaussian process models,” in Proc. IEEE Eurocon 2003: Int. Conference on Computer as a Tool, 2003, pp. 352-356. [15] R. Murray-Smith, D. Sbarbaro, C. E. Rasmussen, and A. Girard, “Adaptive, cautions, predictive control with Gaussian process priors,” in Proc. 13th IFAC Symposium on System Identification, 2003, pp. 1195-1200. [16] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989. [17] R. V. Mises, Mathematical Theory of Probability and Statistics, Academic Press, 1964. [18] H. Takata, “An automatic choosing control for nonlinear systems,” in Proc. IEEE Conference on Decision and Control, 1996, pp. 3453-3458. Tomohiro Hachino received the B.S., M.S. and Dr. Eng. degrees in electrical engineering from Kyushu Institute of Technology in 1991, 1993, and 1996, respectively. He is currently an Associate Professor at the Department of Electrical and Electronics Engineering, Kagoshima University. His research interests include nonlinear control and identification, signal processing, and evolutionary computation. Dr. Hachino is a member of IEEJ, SICE, and ISCIE. Keiji Naritomi received the B.S. degree in electrical and electronics engineering from Kagoshima University in 2014. He is currently a master’s student at the Department of Electrical and Electronics Engineering, Kagoshima University. His research interests include nonlinear control and identification. Hitoshi Takata received the B.S. degree in electrical engineering from Kyushu Institute of Technology in 1968 and the M.S. and Dr. Eng. degrees in electrical engineering from Kyushu University in 1970 and 1974, respectively. He is currently a Professor Emeritus and a part-time lecturer at Kagoshima University. His research interests include the control, linearization, and 320©2015 Engineering and Technology Publishing Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015 identification of nonlinear systems. Dr. Takata is a member of IEEJ and RISP. Seiji Fukushima received the B.S., M.S., and Ph.D. degrees in electrical engineering from Kyushu University in 1984, 1986, and 1993, respectively. He is currently a Professor at the Department of Electrical and Electronics Engineering, Kagoshima University. His research interests include photonics/radio hybrid communication systems and their related devices. Dr. Fukushima is a member of IEICE, IEEE/Photonics Society, Japan Society of Applied Physics, Japanese Liquid Crystal Society, and Optical Society of America. Yasutaka Igarashi received the B.E., M.E., and Ph.D. degrees in information and computer sciences from Saitama University in 2000, 2002, and 2005, respectively. He is currently an Assistant Professor at the Department of Electrical and Electronics Engineering, Kagoshima University. His research interests include optical CDMA communication systems and the cryptanalysis of symmetric-key cryptography. Dr. Igarashi is a member of IEICE and RISP. 321©2015 Engineering and Technology Publishing Journal of Automation and Control Engineering Vol. 3, No. 4, August 2015