A PAPER PRESENTATION ON NEURAL NETWORKS BASED FORECASTING OF ELECTRICITY MARKETS

SUBMITTED BY,
G.Mahendranath & E.Kranthikumar,final B-TECH (E.E.E) (mahinath_230@yahoo.co.in, kranthi kumar_et@yahoo.co.in)

Narayana Engineering college,Nellore. Contact address: G.Mahendranath, S/O:G.Parandhamulu,D/NO:18-1-308, Acharistreet,Nellore.

Contact:No: 9866906616
9949369270 0861-2324639

Abstract:
This paper deals with load forecasting of : the short and medium-term load forecasting. This Load forecasting was usually performed using statistical time series and regression methods.Among these methods, the ARMA and ARIMA models (Autoregressive (Integrated Moving Average) were very popular in the near past. The researches and

results in the last 2 or 3 years show a significant change of the interest in load forecasting from traditional to artificial intelligence based methods. Thus, scientists have proved that an Artificial Neural Network (ANN) produces the same or best results compared to an ARIMA model. This finding demonstrated that, in fact, statistical models could be viewed as special cases of ANNs, and hence an ANN is a more powerful and more flexible tool to forecast load behavior.

1. Introduction It is a common place that load forecasting plays an important role in the planning and operation of power systems. At present, as electricity markets appear all over the world, problems like long and short term electricity contracts, market prices and energy trading in general involve more and more outputs from load forecasters. Typically, load forecasting can be long term, medium term, short term or very short term. Long term forecasting refers to intervals of years in advance and it is applied primarily to system development planning. For shorter intervals (from several months to one year), the medium term load forecasting deals with problems like fuel supply scheduling or maintenance planning. Load prediction for one day to one week ahead is the field of short term load forecasting and refers to the balance between electricity consumption and generation as an efficient way to control the operation of the system and the electricity market. Finally, the very short term load forecasting problem deals with the balance between consumption and generation during shorter time intervals (e. g. 10 minutes) . . 2. Artificial Neural Networks: The main advantage of using ANNs to forecast system load lies in their ability to learn the dependencies between the exogenous parameters and the forecasted load. The exhaustive description of the ANNs matter is not the subject of this paper. However, for completeness this section presents the basics of ANNs architectures and learning algorithms. 2.1. The Multilayered Perceptron An MLP is a collection of simple processing units highly interconnected, which process the information fed to the input units (see Figure1.a). These units, called neurons, are organized by layers. Typically, the neurons in the input layer serve only for transferring the input pattern to the rest of the network, without any processing. The information is processed by the hidden and output neurons. Figure 1.b shows the internal structure of a neuron. Each neuron has a certain number of inputs, but only one output. For a certain neuron the inputs could originate from external stimuli or could represent the output of the other neurons. Signals travel from one neuron to others through special links or connections. The importance of a connection between neurons i and j is described by its weight wij. The MLP is operated in two steps: the training stage and the generalization stage. In both stages the first operation to be performed is the forward propagation. An input pattern is fed to the input layer and transferred to the hidden units. The output of each neuron xj is multiplied by the weights of the outgoing connections wij and then fed out to

the input of the neurons in the next layer. The inputs of a neuron are summed up to form the net input neti, and the activation function fi produces the output of the neuron oi = fi (neti). This operation is repeated for all neurons in the network until the outputs of all output neurons have been computed. This is a forward propagation step. The MLPs most popular learning rule is the error back propagation algorithm. At the beginning of the learning stage all weights in the network are initialized to small random values. The algorithm uses a learning set, which consists of input desired output pattern pairs. Each input output pair is obtained by the off line processing of historical data. These pairs are used to adjust the weights in the network to minimize the Summed Squared Error (SSE) which measures the difference between the real and the desired values over all output neurons and all learning patterns. After computing SSE, the error back propagation step computes the corrections to be applied to the weights. 2.2. Self Organizing Feature Maps: Another type of ANN is the Kohonen network, also known as the Self Organizing Feature Map(SOFM). SOFMs were inspired by the way human sensorial segments are mapped into the cortex: spatial relationships among stimuli correspond to spatial relationships among neurons. Unlike MLP, which uses unsupervised learning, SOFMs were especially conceived for unsupervised learning, also called self organization. A self organizing neural network learns by itself, without any information about the right answer or the right value(s) it must produce at the output. The information concerning these characteristic features is created during the learning process and is stored in the weights of synaptic connections. These weights form a set of prototypes. The Kohonen network uses a grid of neurons. Each neuron from this grid is associated to a class or category, hence the name of class neuron. Each class neuron can be described by its position in the grid and a prototype. The prototype is specified by the weights of synaptic connections between the class neuron and the input neurons (see Figure 2). The self organizing process occurs during the learning stage. First, the prototypes of all class neurons are initialized, for example to small, random values. Then, all patterns from the learning set are applied one by one to the input layer. Each pattern xp is compared to the prototypes wu, and the class neuron with the smallest distance ||wuxp|| is chosen to be the wining neuron U*. Then the prototype of the wining neuron is modified to become closer to the current pattern, using a learning rate. The force of the self organizing algorithm proposed by Kohonen consists in modifying the prototype not only for the wining neuron U*, but for all neurons inside a certain neighborhood of the wining neuron NU*.

During the generalization stage, the Kohonen network is used to identify or recognize patterns. The unknown pattern is applied to the input, and compared to all prototypes in the network. The nearest prototype is chosen and used thereafter in accordance with any specific request of the application. 3. General load characteristics Any model for the short or medium term load forecasting is greatly influenced by then characteristics of the load to be predicted. The time series that describe the daily load profiles in a system are non stationary processes, affected by weather conditions (especially temperature, but other parameters that could be considered are humidity or wind speed), special events (strikes, sport events) and the randomness of the industrial consumption. As a rule, the most stable group of consumers, whose load changes following a common pattern that can be assessed, is the group of residential loads. As for considering special events and / or the randomness of industrial loads in the forecasting model, these are very difficult tasks and, as far as the author knows, there are no references to mention them. The shape of the load profiles describes usually a daily and weekly periodicity. However, the load profile for tomorrow or the next week is not just a simple copy of the load profile from today or this week. Instead, the load profile is slightly modified from day to day and week to week, to reflect changes in consumers behavior or weather conditions. Typically, daily load profiles are classified as week days (from Monday to Friday)and weekend days (Saturday and Sunday). Some authors consider separate analysis for each weekend day, while others deals with separate analysis for 3 types of week days, i. e. (a) Monday,(b) Tuesday to Thursday and (c) Friday. In the last case the shapes of the load profiles are similar for all week days except the morning of and the evening of Friday as shown in Figure 3. A special type of day is the holiday. Some authers group the holidays with the weekend days, and assimilate them with Sundays . However, some holidays (e. g. the Easter, or the Christmas) have completely different load profiles and must be considered separately, as in reference. During weekend days and holidays the load lowers with 6 to 11 % as compared to the load in week days, and the shape of the load profiles changes

dramatically. For usual length of the data history (2 to 3 years), the number of weekend days and holidays is much smaller than the number of normal week days, hence the need of a special approach to forecast special days load. The basic approach for special days forecasting consist in preparing an initial forecast using the normal day model, followed by a correction action, to take into account the presence of a weekend day or holiday. Until now there is no consistent methodology to determine the relevant parameters to be used as independent variables in short or medium term load forecasting. In most cases these variables are chosen using a linear correlation analysis with respect to the load values. However, considering that subsequently these variables shall be used as input to an essentially nonlinear model (namely the ANN), the linear correlation criterion raises some validity questions. As a rule, the main parameters considered as independent forecast variables are: (a) general variables: year, month, hour, day of the year, day of week, holiday; (b) weather conditions: temperature, wind speed, humidity, etc; (c) load variables: load values at different moments in the load history (e. g. load values with 1, 2, 24 or 168 hours before the forecasting moment); (d) special, functional variables: different types of functions (exponential, logarithmic, trigonometric, etc) having as arguments any variable in (a), (b) or (c). 4. Short term load forecasting: Short term load forecasting provides daily load profiles with hourly or half hourly load values for one day to one week in advance. At present, the most commonly used forecast methods are those based on ANNs. Some of these methods will be described briefly hereafter. There is a reach literature providing references about short term load forecasting with ANN. Among many structure of ANNs, two architectures are commonly used: the MLP and the Kohonen network. Among different types of MLP used as forecasting models, three architecture could be considered as representative: (a) networks for 1 hour forecast; (b) networks for 24 hours forecast and (c) modular networks for 24 hours forecast.

4.1. Networks for 1-hour forecast The basic architecture of the MLP for this type of networks is shown in Figure 4. The output layer comprises only one neuron, which produces the forecast for one hour. Using the network in Figure 4 sequentially, it can generate the forecast for the entire day (24 or 48 values). The number of input neurons varies from one model to another. For example, Table 1 shows the input values used for three different models of MLP. For this type of architecture, the forecasting procedure uses real data for the first hour only (h = 1). For the rest of the hours (h = 2, 3, ) at least one of the input values is a forecasted one. Hence an error propagation process appears, which could generate important errors for hours late in the day. The test shows however that, due to positive and negative errors compensation, this propagation process is not so harmful as feared. 4.2. Networks for 24-hours forecast: For this type of network the architecture is similar to that in Figure 4, except that the output layer contains not only one, but 24 neurons, which forecast 24 hourly values for a daily load profile. The structure of the input layer is similar to the one in the case of 1-hour forecast, as shown in Table 1, for two references. References report a maximum absolute mean error of 4 %. 4.3. Modular networks for 24-hours forecast: This approach uses two or more modular, interconnected MLPs, which generate a 24 hours forecast. Each module is responsible for the load forecast for a certain moment of the day, using certain input variables. An example of such a modular approach is the 3 generations forecaster model described in references. The input values for the first and last generations are described in Tables 2. The last generation architecture consists of two ANNs with 24 outputs each. The first ANN produces a basic forecast; the second ANN generates a set of 24 correction values, which added to the load profile values of the previous day determine the load profile forecast for the next day. These values and the basic forecast produced by the first ANN are processed by a RLS (Recursive Least Squares) Combiner to produce the final forecasted load profile. The maximum mean error reported in reference is 2.99 %. Another modular forecasting model was described in which uses 4 modules, with the following functionality. Module I, or the Basic ANN generates a 24 hours load forecast using hourly values for the load and the temperature in the previous day , a forecast for the temperature in the current day and a special code for the day of week. The second module is used to predict the peak and valley load values used then in the third module as input data to produce a second 24 hours forecast for the current day. This forecast and the one produced by the first module are processed inside Module IV (the Adaptive Combiner) to produce the final forecast. No error values are reported in this reference. 4.4. Self-organizing architectures:

A special case of load forecasting using ANN is the one described in reference , which uses a self-organizing Kohonen network to produce a basic forecast. The SOFM comprises a grid of 20 * 20 = 400 class neurons with 48 hourly load values: 2 load profiles for the previous day and the forecasting day respectively. The basic forecast produced by the SOFM is updated using a fuzzy model to take into consideration weather conditions. The maximum absolute mean error reported is less then 1%. 5. Medium-term load forecasting: Almost all short-term forecasting techniques use as independent variables certain weather condition information such as temperature, humidity or wind speed. As for the medium-term load forecasts, it is almost impossible to take into consideration such influence factors, as the forecasting period is much wider, and detailed and accurate weather predictions might not be available. As a consequence, in reference the author has analyzed the possibility to take into consideration some macroeconomic indicators (MEc) such as the Consumer Price Index (CPI) or the Inflation Rate, the Average Salary Earning (ASE) and the Currency Exchange Rate (CER), available from official organisms. One major component of an electricity market is the bulk market, which is a medium term market, where trading arrangements between generators and suppliers are based on bilateral contracts. Typically, a bilateral contract is concluded between the two parts for a period of one year. The New Electricity Trading Arrangements (NETA), already in place in UK, offer forward and futures contracts which can shorten this period to few months, weeks, days, or even hours. If we consider the long-term alternative, contracts set fixed energy quantities, which generators sell to suppliers at fixed prices. Even for the NETA whose prices vary on the market the offer of a new contract is the same thing as concluding a usual bilateral contract. Thus, electricity quantities are expressed as mean hourly (or half hourly) values for a number of days. For example, one can use a classification with 4 types of day for each month, or 48 characteristic days in one year. These 4 types of day are: [Mo + Fr], [Tu + We + Th], [Sa] and [Su]. The medium-term forecasting model uses the selforganization process in Kohonen networks or SOFMs. Twelve SOFMs have been used, one for each month of the year. Each SOFM has four class-neurons, corresponding to the four characteristic days in a month, and 50 input neurons for load and MEc data. The input layer has the following structure (see Figure 5): neurons 1-24 (the load profile with 24 hourly values for day D, year Y-1); neuron 25 (the value of the MEc indicator for day D, year Y-1); neurons 26-49 (the load profile with 24 hourly values for day D, year Y); and neuron 50 (the value of the MEc indicator for day D, year Y). The weights of the connections between each class-unit and the input units describe the prototype associated to that class. The learning algorithm used with the SOFM is the winner-take-all strategy combined with a fuzzy procedure. This approach avoids the sensitivity of the traditional algorithm with respect to the order the learning patterns are presented to the network. In fact, it is a sort of batch learning strategy.

6. Experience with short and medium-term load forecasting in the Romanian system: The Romanian National Energy Regulating Authority (ANRE) has developed a portfolio of decisions and recommendations and put into place the energy market in Romania. Today, the Romanian electricity market offers regulated access to about 45 eligible consumers. In order to become eligible for electricity supply, and thereafter to choose the supplier, the electricity consumption of a company must be greater than 40 GWh. Though several new suppliers have recently obtained the official agreement from ANRE, the main electricity supplier in the Romanian system is Electrica S.A. and its subsidiaries. 6.1. Experience with short-term load forecasting: The ANN forecasting model was developed using data from several electricity subsidiaries in Romania. The basic model is a MLP whose architecture is depicted in Fig. 6. Based on a correlation analysis we have chosen to use the following input variables: load data: 1, 2 weather data: (a) maximum daily temperature: Tmax(d); (b) minimum daily temperature: Tmin(d); (c) mean daily temperature: Tmean(d) (a) D = 0.5 weekend days; (b) D = 1.0 week days. (a) hourly load values: P(h k) k= 1, , 5; (b) daily load values: P(h 24 k) k = 0, 1, 2; (c) weekly load values:P(h 168 k) k = 0,

day of week code:

hour code: H=00001; H=00010; H=00011; H=11000;

Compared to the MLP architectures described in section 4, the architecture from Figure 6 is a 1-hour forecast type network. Some results obtained using this model will be described using load data provided by one Electrical subsidiary.Table 3 shows the correlation coefficients for this subsidiary. The values are common values for the rest of subsidiaries. These values show a relatively weak cross correlation of the load and a even weaker correlation of the load and the maximum temperature. As to the load cross correlation, we expected a greater value of the correlation coefficient for the daily and weekly historical data, as compared to the recorded values (0.764 and 0.778). These values are in accordance with the behavior of the load illustrated by the load profiles in Figure 7. Figures 7 (a) and 7 (b) show several load profiles for week days and weekend days respectively. As one can see load profiles differ significantly for the two day types (week or weekend days). Also, for the same type of day there are still differences between load profiles, which is not a desirable behavior. On the other hand, Figure 7 (c) shows the load profiles for 5 consecutive Wednesdays. As one can see, with few exceptions, the load has an almost periodical behavior.

Due to the activation function used in the output layer, which takes values between 0 and1, the load values had to be pre scaled to the same interval. On the other hand, the learning data set was prepared using a scaling interval between 0.15 and 0.85. The second scaling procedure was applied for two reasons: (a) to avoid saturation of sigmoid functions in the neighborhood of 0 and 1, and (b) to allow forecasted load to take values greater than the maximum historical values (which form the learning data set). The tests were carried out for a week in the mid December 1999. The results are depicted in Figure

8 and Table 4. This forecasting model, together with a special interface was implemented as a software tool, which is applied today in 9 electricity subsidiaries in Romania

Figure 7 Load profiles for: (a) 3 week days (Wednesday, Thursday and Friday); (b) 2 weekend days (Saturday and Sunday); (c) 5 consecutive Wednesdays.

Figure 8 Real and forecasted load profiles for one week in the mid December 1999 6.2. Experience with Medium-term load forecasting model: The fuzzy SOFMs model described in section 5 has been used to perform medium-term load forecasts for the load profiles of the 48 characteristic days used in bilateral contracts between generators and suppliers. The database consists of hourly load profiles for the contour of an Electrica SA subsidiary and values for the MEc indicators from 1st April 1998 to 13th April 2000. The learning data set was built using the input structure described in the previous section and load and Mec data between 1st April 1998 and December 1999.The data for year 2000 will be used to test the SOFMs performances after training.

For the learning stage of the SOFMs the patterns were arranged using the following model: for each month M the learning data set contains data from the whole month M and other two weeks, one week from month M-1, and one week from month M+1. This model aims to take into consideration the influence of the load and MEc data in a month on the load level in the next month. The load profiles for each day of the week have been chosen from this period, and regrouped according to the four characteristic days. They resulted in six load profiles for each day of the week, that is 42 learning patterns for each month. These patterns were then used to train 12 SOFMs for each month of the year. After the learning stage the forecasting data set was built as follows. Each day of the forecasting month generates an input pattern, whose structure is: neurons 1-24 (the load profile with 24 hourly values for day D, year 1999); neuron 25 (the value of the MEc indicator for day D, year 1999); and neuron 50 (the forecasted value of the MEc indicator for day D, year 2000). This pattern is presented to the SOFM network and the winning class-neuron is established. The winning class-neuron is determined on the basis of the distances between the input pattern and the prototypes of each class neuron, using only inputs 1-25 and 50. The prototype associated to the input neurons 26-49 of the winning class-neuron will be considered as the forecasted load profile for that day. 7. Conclusion: Short and medium term load forecasting are important present day problems for both power system and electricity market operation. For the short term load forecasting the results in the literature and the authors experience recommend ANNs as feasible solutions. As to the medium term load forecasting, our experience found that self organization and Kohonen networks could be a valuable and reliable approach. This neural network based forecasting of electric markets or say, electric stations can fetch us many advantages like, saving the time, man power, sudden break downs, there by increasing the efficiency of the stations to provide efficient power supply, and power supply with added advantages.