2009 First Asian Conference on Intelligent Information and Database Systems

Short-term Load Forecasting using Support Vector Regression Based on Pattern-base

YING-CHUN GUO 1,2, DONG-XIAO NIU 1
1

School of Business Administration, North China Electric Power University 2 College of Mathematics and Computer Science, Hebei University Baoding, Hebei Province, China e-mail: guoyc@hbu.cn the previous week and on the day with the same denomination in the previous year [3]. Furthermore, it is difficult to model the relationships between the loads and the variables that influence the loads, such as weather or seasonal variations, holiday activities, etc. These are the major factors that make the modeling process complicated. Another difficulty lies in estimating and adjusting the model parameters, which are estimated from historical data that may be outmoded or may not reveal short-term load pattern changes [4]. Over the last decade, a great deal of attention has been devoted to the use of artificial neural networks (ANNs) to model load [5-6]. There is also a great deal of researches concentrating on applying regression SVM to short-term electricity load forecasting [7-9] and the forecasting accuracy outperforms other forecasting models. The forecast error is greatly influenced by load fluctuation and rapid change in temperature. Recently, several methods based on similarity have been reported for the purpose of load forecasting [10], according to which, load curve is forecasted by using information of the days being similar to weather condition of the forecast day. These methods have an advantage of dealing not only with the non-linear part of load, but also with the weekend and special days. In general, the load based on several selected similar days is averaged to improve the accuracy of load forecasting. But for the lack of adequate cognition of the influencing mechanism of the load, the map and weight can not reflect the actual function of the factors, thus influences the selections of the samples. To solve the problems, we put forward a new idea that preprocessing should be the key to improving the precision of daily load forecasting. And in this paper we presented a new method of daily load forecasting using SVR based on pattern-base. This method firstly, uses CART to find out the important factors that influence the daily load, and classify the training data into different patterns; secondly, it sets up a pattern-base which is indexed by daily load pattern, each sub- pattern-base is composed of daily load data sequence with highly similar features; thirdly, according to the weather forecast and date type, it recognizes the pattern of the forecasting day and establishes SVR forecasting model based on the pattern-base matching to the forecasting day. This paper presents a feasible method for STLF using similar days based on pattern-base. Especially it shows the different patterns of daily load. The paper is organized as

AbstractA new idea is proposed that preprocessing is the key to improving the precision of short-term load forecasting (STLF). This paper presents a new model of STLF which is using support vector regression (SVR) based on pattern-base. Our model can be described as follows: firstly, it recognizes the different patterns of daily load according such features as weather and date type by means of data mining technology of classification and regression tree (CART); secondly, it sets up pattern-bases which are composed of daily load data sequence with highly similar features; thirdly, it establishes SVR forecasting model based on the pattern-base which matches to the forecasting day. Since the patterns of daily load are treated beforehand, the rule of the historical data sequence is more obvious. The model has many advantages: first, since the training data has similar pattern to the forecasting day, the model reflects the rule of daily load accurately and improves forecasting precision accordingly; second, as the pattern variables need not to be input into model, the mapping of the categorical variables is solved; third, as inputs are reduced, the model is simplified and the runtime is lessened. The simulation indicates that the new method is feasible and the forecasting precision is greatly improved. Keywords- short-term load forecasting (STLF); classification and regression tree (CART); pattern-base; support vector regression (SVR)

I.

INTRODUCTION

Short-term load forecast (STLF) is aimed at predicting system load over a short time interval of one day or one week and plays an important role in the operation of power systems from which basic operating functions such as energy transactions, unit commitment, security analysis, economic dispatch, fuel scheduling and unit maintenance have all benefited. Several approaches to the load forecast model have been reported in the last decades, including conventional smoothing techniques, regression methods, statistical analysis, time series analysis techniques, autoregressive moving average model and expert systems [12]. Although these techniques and models are reliable, they are unable to adapt to unusual weather conditions and varied holiday activities, which form a highly non-linear relationship with the daily load. Hence, their load predictions are not as satisfactory as desired. Load forecasting is a difficult task as the load at a given day is dependent not only on the load at the previous day, but also on the same day in
978-0-7695-3580-7/09 $25.00 2009 IEEE DOI 10.1109/ACIIDS.2009.52 336 342

Authorized licensed use limited to: Khulna Univ of Engineering and Technology. Downloaded on May 24,2010 at 13:30:29 UTC from IEEE Xplore. Restrictions apply.

follows. Section 2 gives an overview of the classification and regression tree. Section 3 discusses the method of setting up pattern-base of daily load. Section 4 proposes support vector regression forecasting based on pattern-base, simulation results are covered in Section 5. Conclusions are drawn in Section 6. Our analysis in this paper is based on actual load data of HeBei Electric Power Company, China. II. CLASSIFICATION AND REGRESSION TREE Classification and regression tree (CART) is a technique of data mining, including classification tree and regression tree [11-12]. The classification tree deals with categorical output and the regression tree deals with continual output. CART can abstract rules to describe the relationship between input and output. The CART structure which is showed as Fig. 1 includes split nodes and terminal nodes. The top node is called root node. The terminal nodes give the result of classification. The input data belongs to the left or to the right node determined by the comparison with the partition condition of the split node. The result of the classification comes out by repeating the process mentioned above. From the root node to terminal node indicates a rule between input and output. There are two steps to build a CART: growth and pruning. The growth of the tree splits the history data into subsets gradually by searching the splitting point which has the minimum classification error. Repeat the splitting on the two child node until one of the stopping rules is triggered. As a complicated tree may over fit to the training data, so replace some splitting nodes with terminal node, this is called pruning. As the electric load is numeric, so take the regression tree as example to explain the growth and pruning of the tree. A. CART growth CART works by choosing a split at each node so that each child node created by the split is purer than its parent node. Here purity refers to similarity of values of the target field. CART measures the impurity of a split at a node by defining an impurity measure. There are different impurity measures depending on the type of the target field. Since the electric load is continuous targets, we selected the leastsquared deviation (LSD) method.

The LSD measure R(t) is the weighted within-node variance for node t, and it is equal to the risk estimate of the node. It is defined as:

R (t ) =

2 1 ( yi y (t )) N (t ) it

(1)

where N (t ) is the number of records in node t,

yi is the

value of the target field, and y (t ) is the (weighted) mean for node t:

y (t ) =
as

1 yi N (t ) it

(2)

The LSD criterion function for split s at node t is defined

( s, t ) = R (t ) pL R (tL ) pR R (t R )

(3)

where pL is the proportion of records in t sent to the left child node, and pR is the proportion sent to the right child node. The split s is chosen to maximize the value of ( s , t ) . The following steps are used to build a CART (starting with the root node containing all records): 1) Find each predictor's best split. For each predictor field, find the best possible split for that field. If the field is numeric, then sort the field values for records in the node from smallest to largest, choose each point in turn as a split point, and compute the impurity for the resulting child nodes of the split; if the field is categorical, then examine each possible combination of values as two subsets, calculate the impurity of the child nodes for the split based on that combination. Select the best split point for the field as the one that yields the largest decrease in impurity relative to the impurity of the node being split. 2) Find the best split for the node. Identify the field whose best split gives the greatest decrease in impurity for the node, and select that field's best split as the best overall split for the node. Apply the split to creating two child nodes and apply the algorithm again to each child node until one of the stopping rules, for example R (t ) <0.01, is triggered.

B.

CART pruning Pruning refers to the process of examining a fully-grown tree and removing bottom-level splits that do not contribute significantly to the accuracy of the tree. It tries to create the smallest tree whose misclassification risk is not much greater than that of the largest tree possible. It uses an index that measures both the misclassification risk and the complexity of the tree. This cost-complexity measure is defined as:

R (T ) = R (T ) + T

(4)

Figure 1. Structure of CART

where R (T ) is the misclassification risk of tree T and T is the number of terminal nodes for tree T. The term represents the complexity cost per terminal node for the tree and it is calculated by the algorithm during pruning. The pruning algorithm follows these steps:

337 343

Authorized licensed use limited to: Khulna Univ of Engineering and Technology. Downloaded on May 24,2010 at 13:30:29 UTC from IEEE Xplore. Restrictions apply.

1) To each child node of the tree calculate complexity parameter t

t =

R (T ) R (Tt ) Tt 1

(5)

2) Calculate complexity of the child tree t; 3) Compare complexity of child tree t with its parent node, if the difference less than the standard value, then substitute the parent node as terminal node; 4) Repeat the previous step until only the root node is left; The smallest tree Topt is the best tree of all substitutions.

The number of sample data in each sub-pattern-base is determined by the circumstance of the area, not less than 20 normally. The selections of similar days are based on the same season. The limits on the selection of similar days corresponding to forecast day are shown in Fig. 2. The 60 days before a forecast day, and 60 days before and after the forecast day in the previous year are considered for the selection of similar days. If the forecast day is changed, similar days are selected in the same manner. That is to say, we select the days with the same pattern in the same season of the year. Moreover, the sample data in the pattern-base must be updated with time to adapt to the latest change of the daily load. SVR DAILY LOAD FORECASTING MODEL BASED ON PATTERN-BASE Support vector machine (SVM) was introduced by Vapnik the late 1960s. It was developed by Vapnik and his coworkers in 1995 [13], and it is based on the structure risk minimization (SRM) principle that seeks to minimize an upper bound of the generalization error consisting of the sum of the training error and a confidence interval. There are some remarkable characteristics of SVM, such as good generalization performance, the absence of local minima and sparse representation of solution. SVM nonlinearly maps inner product of original space to the feature space via a kernel. Training SVM is equivalent to solving a linearly constrained quadratic programming, the solution of SVM is unique global, and it is only dependent on a small subset of training data points which are referred to as support vectors. So SVM is capable of learning in highdimensional spaces with a small number of training examples and has high generalization ability [14-15]. Short term load forecasting is a multivariable forecasting problem. It can be treated as a function regression problem. There is also a great deal of researches concentrating on applying regression SVR to short-term electricity load forecasting [7-9]. The load of next day is output of regression model and the corresponding load influencing factors such as history load data, temperature information and meteorological information are the input data of regression model. The training data is supplied by history database. The final target is to find a mapping function from influencing factors to future load with a good generalization capability. In this paper, we proposed SVR daily load forecasting model based on pattern-base to forecast the 24h load of the forecast day. The history data is divided into two data sets. One is the training data and the other is the testing data. The training data is used to train the SVR, and the testing data is used to evaluate the trained SVR, while the data should be selected from the pattern-base according to the pattern of the forecast day. The Gaussian kernel
2 K X i , X j = exp - (1 2 ) X i - X j

C.

Create set of rules The reason why the decision tree has such an attraction, to a great degree is that the decision tree represents rules. The rules can be expressed by IF-THEN mode very easily and easy to be understood. Another important reason is that the rules can be expressed by the language of Access to index records in the particular category. From root to leaf of the CART, each path establishes a rule and all the rules make up the set of rules. The feasible rules are saved to database after refined. Moreover, we can find out the importance of influencing factors to the electric power load according to structure of the tree and we can get rid of the factors from the model whose influence is insignificant, so the efficiency of the model is improved.
III. SET UP PATTERN-BASE After creating set of rules, we can set up pattern-base of daily load accordingly. The method is: input the daily attributes of the samples data in the original database from the recent day to farther one by one to the CART which has been already trained. At the same time the CART classifies each sample, it will send the samples into different subpattern-base according to the pattern of the day. There are two ways to set up the sub-pattern-base, one is to creating new database and appending the samples which have the same pattern gradually, the other is appending a pattern attribute to each sample, in order to search samples according to this attribute in the future. The second method is suggested, as it saves memory and is easy to search related information.

IV.

(6)

Figure 2.

Limits on the selection of similar days

is employed as kernel function here.

338 344

Authorized licensed use limited to: Khulna Univ of Engineering and Technology. Downloaded on May 24,2010 at 13:30:29 UTC from IEEE Xplore. Restrictions apply.

In this way, we establish the SVR model to forecast daily load based on pattern-base. As the proposed method greatly intensifies the rules of the historical load sequence, the forecasting result is highly alike in pattern using the trained SVR to forecast the daily load. Furthermore, as the pattern characters no longer need to be input, the studying and training become much easier. As the pattern has been treated, the forecasting result gained from the model need not be corrected artificially. So the method not only can improve the forecasting precision, but also can reduce the complexity of forecasting work. SVR must be retrained to obtain different function according to the pattern of the day. V. SIMULATION RESULTS AND DISCUSSION The performance of the proposed method for STLF has been tested using the actual hourly load and weather data (for the year 2006-2007) of HeBei Electric Power Company, China. CART is implemented by Clementine and SVR is programmed by Matlab. Load forecasting is done for the year 2007. To verify the predictive ability of the proposed method, we performed simulations for different seasons.

B. Set up pattern-base according to CART rules After creating set of rules, we can set up pattern-base of daily load accordingly. There are 30 sub-pattern-bases according to the result of CART. Input the daily attributes of the samples data in the original database from the recent day to farther one by one to the CART which has been already trained. The selections of the same pattern days are based on the same season. The limits on the selection of similar days corresponding to forecast day are shown in Fig. 2. At the same time the CART classifies each sample, it will send the samples into different sub-pattern-bases according to the pattern of the day. We choose the method that appends a pattern attribute to each sample, so that it is easy to search training samples and related information according to this attribute. The number of sample data in each sub-patternbase is determined by the circumstance of the area, normally no less than 20. C. Forecast daily load using SVR based on pattern-base This paper adopts the improved SMO algorithm train SVR [16]. The Gaussian kernel is selected as the kernel function. Many experiments with different SVR parameters were conducted in order to identify the parameters that give the best results. In the investigation, the value of the mean absolute percent error (MAPE), shown as (7), serves as the criterion for identifying suitable parameters in the SVR model.
MAPE =
Where

A. Discover pattern rules by CART Usually, the shape of the load curve presents periodicity with day and week. But small random appears as influenced by the weather condition and the activity of people. The load curve of weekends is different from workdays. In addition, the load curve of the holidays obviously differs from general workdays and weekends. So the load curve is influenced by the type of the day. Respectively, we divide the days into 4 categories: workdays, weekends, holidays and days close to holidays. As the load curve is also influenced by weather, the other factors need to be input into CART are: daily highest temperature (Th), daily lowest temperature (Tl), daily average temperature (Ta), degree of humidity (Hu), and daily rainfall (Rf), air pressure (Ap), total quantity of the cloud (Cl), power of wind (Wp), direction of wind (Wd) and time of sunshine (St), etc. The target variable of the samples is 24h daily load. The input variables are mentioned above. The sample set is divided into training set and testing set. After build and pruning CART, we gained the optimized tree. The tree not only got ride of the input variables whose influence is insignificant to the load, but also gained a sequence of factors ordered by importance. That is: date type, daily highest temperature, daily lowest temperature, daily rainfall, degree of humidity and air pressure. Some factors do not appear in CATR, that is to say, these factors are not very important to daily load. The output of CART are pattern rules expressed by structure of IFTHEN. Take the rule of number i for example: IF factor 1 is Ai AND factor 2 is Bi AND AND factor n is Ni, THEN the Mode is Yi. We must take notice of the fact that the rules may be different according to areas.

1 n di yi 100% n i=1 di

(7)

n is the number of forecasting periods; di is the actual load value of testing data i ; and yi is the forecasting
load value. The three parameters C , and of SVR model are adjusted by checking errors improvement occurs. The range constrains of the there parameters are set as C [0.01, 1000], [0.01, 10], and [0.01, 0.9]. Then, the adjusted parameters with minimum testing error are selected as the most appropriate parameters. Finally, a SVR model is built to forecast electricity load of next day. Train the SVR based on the pattern-base which matches to the forecast day until the MAPE less than 2%.

D.

Test result and error analysis The performance of the proposed SVR model for 24 h load forecasting has been tested using one year (2007) of load and weather data. Firstly, input CART the pattern of the forecast day which consists of weather and date attributes, find out the pattern-base matching to the forecast day according to the output of CART; secondly, input the load of the previous three days which in the pattern-base matching to the forecast day, run the trained SVR which matches to the forecast day. The SVR should be updated if the pattern-base is updated. Take may 1, 2007 as example, the comparison of the forecast and actual load is shown as Fig. 3.

339 345

Authorized licensed use limited to: Khulna Univ of Engineering and Technology. Downloaded on May 24,2010 at 13:30:29 UTC from IEEE Xplore. Restrictions apply.

1600

1400

1200 load/(104MWh)

1000

forecasting. As the attributes of pattern need not to be input into SVR, the forecasting is simplified greatly. To verify the predictive ability of the proposed method, we performed simulations and compared the proposed model with some other models. From the results we can see that the model is feasible and can greatly improve forecasting precision. The method achieved a good performance in special regions where daily load is changeable. ACKNOWLEDGMENT
actual PBSVM SVM

800

600

400

The authors would like to thank the National Science Council of the Republic of China, for financially supporting under National Natural Science Foundation No. 70671039.
20 25

200

10 hour/h

15

REFERENCE
[1] Charytoniuk W, Chen MS, Olinda PV, Nonparametric regression based short-term load forecasting, IEEE Trans Power Syst, 13(3), 1998, pp.72530. Huang SJ, Shih KR, Short-term load forecasting via ARMA model identification including non-gaussian process considerations, IEEE TransPower Syst, 18(2), 2003, pp.6739. Hippert HS, Pedreira CE, Souza RC, Neural networks for short-term load forecasting: a review and evaluation, IEEE Trans Power Syst, 16(1), 2001, pp.4455. Rahman S, Hazim O., A generalized knowledge-based short term load forecasting technique, IEEE Trans Power Syst, 8(2), 1993, pp.50814. Taylor JW, Buizza R. Neural network load forecasting with weather ensemble predictions, IEEE Trans Power Syst, 17(3), 2002, pp.626 32. Senjyu T, Takara H, Uezato K, Funabashi T., One-hour-ahead load forecasting using neural network, IEEE Trans Power Syst, 2002, 17(1), pp.1138. X.M. Li, D. Gong, L. Li, and C.Y. Sun, Next day load forecasting using SVM, in Proc. ISNN05, Lecture Notes in Computer Science, Springer, Berlin, 3498, 2005, pp. 634-639. J.F. Yang and H.Z. Cheng, Application of SVM to power system short term load forecasting, Electric Power Automation Equipment, vol.24, 2004, pp.30-32. Y.C. Li, T.J. Fang, and G.X. Zhang, Wavelet support vector machine for short-term load forecasting, Journal of university of science and technology of China, vol. 3, 2003, pp. 726-732. Kim K, Youn HS, Kang YC., Short-term load forecasting for special days in anomalous load conditions using neural networks and fuzzy inference method, IEEE Trans Power Syst, 15(2), 2000, pp.55965. Breiman, L., Friedman, J., Olshen, R., & Stone, C. J. Classification and regression Trees, Wadsworth, Belmont, CA, 1984. Chipman, H. A., George, E. I., & McCulloch, R. E. Bayesian CART model search, Journal of the American Statistical Association, 93(443), 1998, pp.935960. V.N. Vapnik, The Nature of Statistical Learning Theory, SpringerVerlag, NY, USA, 1995. L. Cao, Q. Gu. Dynamic support vector machines for non-stationary time series forecasting, Intell Data Anal, vol. 6, 2002, pp. 67-83. Tay FEH, L. Cao. Modified support vector machines in financial time series forecasting. Neurocomputing, vol. 48, 2002, pp. 847-861. H.T. Lin, C.J. Lin, A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods, Technical Report, Department of Computer Science and Information Engineering, National Taiwan University, available at: http://www.csie.ntu.edu.tw/cjlin/papers/tanh.pdf, 2003.

Figure 3.

Comparison of forecasting and actual load [2]

E.

The comparison of forecasting results with a few models The daily load of South Net Hebei Province is obviously affected by weather, such as temperature, rainfall, date type and season, etc. As a result, different load pattern must be taken into account in forecasting. We compare the forecasting result of the SVR model based on pattern-base (PBSVR) with the ANN and common SVR model and we can see the result in Tab.1. By comparison and examination, we can see that using the proposed model forecast daily load, the model is simplified and the precision is greatly improved.
TABLE I. THE MAPE OF SEP 1, 2007 TO SEP 7, 2007 (%)

[3]

[4]

[5]

[6]

[7]

DATE 1/9 2/9 3/9 4/9 5/9 6/9 7/9

ANN 4.21 4.40 4.56 4.50 3.64 4.56 4.33 VI.

SVR 3.54 4.62 3.99 2.72 3.85 4.19 3.42 CONCLUSIONS

PBSVR 2.23 2.85 3.18 2.58 1.73 2.51 2.21

[8]

[9]

[10]

[11] [12]

We put forward a new idea that preprocessing is the key to improving the precision of daily load forecasting, especially in the area where daily load is obviously affected by weather and date factors. This paper presented a new daily load forecasting model. The model uses data mining technology of CART to recognize the pattern of the day and according to CART rules, set up pattern-base which is indexed by daily load pattern, each sub-pattern-base is composed of daily load data sequence with highly similar features which can intensify rules and weaken disturbance. Based on pattern-base, SVR is used to forecast daily load subsequently. That is to say, CART deals with attributes of pattern such as weather and date factors before SVR

[13] [14] [15] [16]

340 346

Authorized licensed use limited to: Khulna Univ of Engineering and Technology. Downloaded on May 24,2010 at 13:30:29 UTC from IEEE Xplore. Restrictions apply.