Available online at www.sciencedirect.

com

Environmental Modelling & Software 23 (2008) 396e403 www.elsevier.com/locate/envsoft

Combining principal component regression and articial neural networks for more accurate predictions of ground-level ozone
Saleh M. Al-Alawi a,*, Sabah A. Abdul-Wahab b, Charles S. Bakheit c
a b

Department of Electrical and Computer Engineering, Sultan Qaboos University, P.O. Box 33, Al-Khod, Postal Code 123, Muscat, Oman Department of Mechanical and Industrial Engineering, Sultan Qaboos University, P.O. Box 33, Al-Khod, Postal Code 123, Muscat, Oman c Department of Mathematics and Statistics, Sultan Qaboos University, P.O. Box 36, Al-Khod, Postal Code 123, Muscat, Oman Received 29 June 2005; received in revised form 13 June 2006; accepted 25 August 2006 Available online 26 October 2007

Abstract This work encompasses ozone modeling in the lower atmosphere. Data on seven environmental pollutant concentrations (CH4, NMHC, CO, CO2, NO, NO2, and SO2) and ve meteorological variables (wind speed, wind direction, air temperature, relative humidity, and solar radiation) were used to develop models to predict the concentration of ozone in Kuwaits lower atmosphere. The models were developed by using summer air quality and meteorological data from a typical urban site when ozone concentration levels were the highest. The site was selected to represent a typical residential area with high trafc inuences. The combined method, which is based on using both multiple regression combined with principal component analysis (PCR) and articial neural network (ANN) modeling, was used to predict ozone concentration levels in the lower atmosphere. This combined approach was used to improve the prediction accuracy of ozone. The predictions of the models were found to be consistent with observed values. The R2 values were 0.965, 0.986, and 0.995 for PCR, ANN, and the combined model prediction, respectively. It was found that combining the predictions from the PCR and ANN models reduced the root mean square errors (RMSE) of ozone concentrations. It is clear that combining predictions generated by different methods could improve the accuracy and provide a prediction that is superior to a single model prediction. 2006 Elsevier Ltd. All rights reserved.
Keywords: Tropospheric ozone concentration; Combining forecast; Principal component regression; Articial neural network

1. Introduction Ozone is an important secondary air pollutant in the atmosphere that has received extensive attention in the literature et al., 2005; Koop and Tole, 2006; Zunckel et al., (San Jose 2006; Agirre-Basurko et al., 2006; Arvanitis and Moussiopoulos, 2006; Schlink et al., 2006; Sokhi et al., 2006; Volta and lez Cabrera et al., 2006). It has been reFinzi, 2006; Gonza ported that ozone is a major oxidant, and it is the most important index substance of photochemical smog which has been recognized as one of the key pollutants degrading the air quality (Seinfeld, 1989; Seinfeld and Pandis, 1998; Van Eijkeren

* Corresponding author. Tel.: 968 24 415 314; fax: 968 24 413 416. E-mail address: saleh@squ.edu.om (S.M. Al-Alawi). 1364-8152/$ - see front matter 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.envsoft.2006.08.007

et al., 2002; Xu and Zhu, 1994). Epidemiological eld studies indicated large increases in mortality rates during summer smog situations, that is, during episodes of high ozone concentrations (Hoek et al., 1997; Schlink et al., 2002). In addition, ozone levels play an important role in damage to plant species and it can cause harmful effects in vegetation during the growing season. Ozone is unique among pollutants because it is not emitted directly into the air. This is the main reason why ozone is such a serious environmental problem that is difcult to predict and control. Ozone results from complex chemical reactions in the atmosphere (Abdul-Wahab and Al-Alawi, 2002). It is produced when the primary pollutants, nitrogen oxides (NOx) and non-methane hydrocarbons (NMHC), interact under the action of sunlight. High ozone concentrations are strongly related to meteorological conditions and usually occur during

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403

397

sunny days, when primary pollutants (NOx and NMHC) interact photochemically, supported by strong solar radiation and et al., 2005). Therefore, meteorohigh temperatures (San Jose logical conditions strongly inuence the efciency of photochemical processes leading to ozone formation and destruction (Lengyel et al., 2004). To track and predict ozone, one must create an understanding of not only ozone itself, but also the conditions that contribute to its formation (Abdul-Wahab and Al-Alawi, 2002). Emission rates of NOx and NMHC can be estimated to some extent, since they are closely related to the nature of industrial and urban activities. It is, therefore, useful if some relationship can be found between these primary pollutants and meteorological factors, which can then be used to determine ozone levels (Elkamel et al., 2001). One idealistic approach would be to make a deterministic model that depends on chemical processes and atmospheric movements. Although photochemical kinetic mechanisms that describe these processes are being formulated to estimate the amount and type of emission reduction needed to limit the formation of urban ozone, their complexities, coupled with the uncertainties in assigning values to parameters, restrict their use (Zannetti, 1994). It is still not feasible to explicitly represent the chemistry of the hundreds of organic species present (Elkamel et al., 2001). Models that depend on the statistical analysis of air quality data have been an alternative approach for expressing the ozone-hydrocarbons-nitrogen oxides relationship (Kuntasal and Chang, 1987; Abdul-Wahab et al., 1996; Lavecchia et al., 1996; Peton et al., 2000). A variety of statistical methods for ozone forecasting was proposed in the literature over the last decade. A critical review of these methods (regression, extreme value, and space-time methods) can be found in Thompson et al. (2001) and Schlink et al. (2003). The multiple linear regression method is a commonly used technique to obtain a linear input-output model for a given data set. If the functional relationship is non-linear, then one or more other inputs can be transformed rst into a non-linear form (Abdul-Wahab et al., 1996). On the other hand, the multiple regression approach can face serious difculties when the independent variables are correlated with each other (multicollinearity). One method of removing such multicollinearity from the independent variables is to use the principal component analysis (PCA). Principal component analysis can be used for ltering the data so that only the signicant independent variables responsible for the ozone levels observed can be determined. The selected variables with high loadings generated from the PCA become ideal to use as predictors in a regression equation (PCR) since they optimize spatial patterns and remove possible complications caused by multicollinearity. However, as ozone in the lower atmosphere is a complex non-linear process, principal component regression cannot adequately model the non-linear relationships. Hence, it is expected that PCR alone is inadequate in forecasting or predicting ozone observations. Although there are now several non-linear multivariate statistical methods that are able to

approximate any non-linear relationship, the assumption of functional dependency is a serious drawback of such procedures (Utojo and Bakshi, 1995). An alternative approach would be the use of articial neural networks for extracting information, without any assumptions of the nature of the non-linearity. Therefore, the neural network is a well-suited method for modeling ozone since it allows for non-linear relationships between variables. Nevertheless, there are many inconsistent reports in the literature regarding ANN performance in forecasting time series data (Zhang et al., 1998). Ozone in the lower atmosphere, as mentioned earlier, is a complex process. It is, therefore, difcult to understand whether the time-related observations under study are generated from linear or non-linear underlying processes. Hence, selecting a specic linear or a non-linear model to predict ozone concentration might not be enough to capture all the relationships. Principal component regression, for instance, cannot deal with non-linear relationships while the neural network models alone are not able to handle both linear and non-linear patterns equally well (Zhang et al., 1998; Zhang, 2003). It is clear that no single method is best in every situation. Hence, by combining forecasts from PCR and ANN, forecasting accuracy is higher than individual forecasts. In this paper, combined approach modeling was used to model ozone concentration levels in the lower atmosphere. The idea is to use the method of principal components to select explanatory variables for multiple regression analysis, followed by applying an articial neural network (ANN) to the resulting residuals. The particular aim is to relate the lower ozone concentration to both meteorological variables as well as to concentrations of primary pollutants. 2. Forecasting techniques 2.1. Multiple regressions & principal component analysis (principal component regression) Multiple linear regression analysis (MLR) is one of the most widely used methodologies for expressing the dependence of a response variable on several independent variables. In spite of its success in many applications, the regression approach can face serious difculties when the independent variables are correlated with each other (McAdams et al., 2000). Multicollinearity, or high correlation among the independent variables in a regression equation, can make it difcult to correctly identify the most important contributors to a physical process. One method of removing such multicollinearity and redundant information from the independent variables is to use multivariate data analysis (MDA) techniques. MDA techniques have been used for analyzing voluminous environmental data to identify trends and relationships (Buhr et al., 1992, 1995; Chang et al., 1988; Sanchez et al., 1986; Hair et al., 1987; Statheropoulos et al., 1998). In addition, MDA techniques can be used for data reduction which can simplify possible models describing the data set. One of the MDA methods is principal component analysis (PCA), which has been employed in air-quality studies to

398

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403

separate interrelationships into statistically independent basic components (Maenhaut et al., 1989; Statheropoulos et al., 1998; Shi and Harrison, 1997; Tian et al., 1989; Vaidya et al., 2000). Several investigations used PCA to classify the variations of ozone and to identify the important factors inuencing ozone concentrations (Paterson et al., 1999; Klaus et al., 2001; Lengyel et al., 2004). PCA is equally useful in regression analysis for mitigating the problem of multicollinearity and in exploring the relationships among the independent variables, particularly if it is not obvious which of the variables should be the predictors. The new variables from the PCA become ideal to use as predictors in a regression equation (PCR) since they optimize spatial patterns and remove possible complications caused by multicollinearity. Detailed procedures for the application of PCR relating to the ozone data set used can be found elsewhere (Abdul-Wahab et al., 2005). Moreover, the complex and sometimes non-linear relationships of multiple variables can make statistical models awkward and complicated (Comrie, 1997). It is, therefore, expected that they will underperform when used to model the relationship between ozone and other variables that may relate to ozone in an extremely non-linear way. Bloomeld et al. (1996) and Davis et al. (1998) argue correctly that statistical linear models have difculty capturing the complex relationships between the meteorological variables and ozone. Thompson et al. (2001) stressed that regression models are useful for modeling average behavior. Complex, stratied, non-linear regression models are needed to approximate the true underlying relationships. 2.2. Articial neural network Articial neural network (ANN) models can handle multivariable problems. They have the potential to describe highly non-linear relationships such as those controlling ozone production. Therefore, the application of articial neural networks to ozone modeling has recently become available to capture those non-linear features of the relationship that a conventional statistical technique (e.g., regression model) might rez et al., 1995). Heravi et al. (2004) indioverlook (Ruiz-Sua cated that ANN provides a general framework, which can approximate any type of non-linearity in the data. Cybenko (1989) showed that ANN could approximate the non-linear relationship between ozone and predictors with any degree of accuracy. Procedures for the application of ANN to the ozone data set used can be found in Abdul-Wahab (2001) and AbdulWahab and Al-Alawi (2002). Despite these great features and the successful performance of ANN models, there are many inconsistent reports in the literature regarding ANN performance in forecasting time series data. Zhang et al. (1998) indicated that the inconsistencies in ANN performance may be due to the large number of factors including network structure, training methods, and sample data. These factors may affect the forecasting ability of networks. Another factor cited in the literature is that the ANN model is a black-box method. There is no explicit form to explain and analyze the relationship between input and output

variables. This causes difculties in interpreting results from the network. In addition, there are overtting problems associated with ANN models. 2.3. Combining principal component regression (PCR) and articial neural network (ANN) Bates and Granger (1969) were the rst to introduce combining forecasts as an alternative to using one single forecast. Techniques such as bootstrapping, gagging, stacking, and boosting are well documented approaches that are based on combining methods that lead to better performance. The literature indicates that work on time series forecasting demonstrated that performance increases through combining forecasts (Clemen, 1989; Makridakis et al., 1982). It was found that selecting among combinations, on the average, leads to signicantly better performance than that of a selected individual forecast. The idea of combining forecasts is to use each models unique features to capture different patterns or features in the data set. Hence, by combining forecasts from different models, forecasting accuracy can often be improved over the individual forecast (Makridakis et al., 1982, 1993; Clemen, 1989; Zhang, 2003).
3. Materials and methods 3.1. Areas description
The state of Kuwait covers an area of approximately 17,818 km2. It is situated at the head of the Arabian Gulf between latitudes 28 and 30 north and between longitudes 46 and 48 east. Iraq lies towards the northern and western boundaries of Kuwait, Saudi Arabia lies to the south, while the Arabian Gulf marks the eastern boundary. The terrain is a at to slightly undulating desert plain. Much of the country is desert. Thus, the climate is typically arid with very hot summers and relatively cold and dry winters. The summer season in Kuwait falls between May and September while the winter season is between November and March. Summer temperatures may exceed 50  C, and in January, the coldest month, temperatures range between 2.8  C and 28.3  C. Kuwait has very little rainfall, most of it occurring as light winter showers brought by westerly depressions, especially in January. The annual rainfall varies from 10 mm to 370 mm. Dust and sandstorms are common throughout the year. They are more frequent in winter months and in midsummer. Relatively heavy trafc movement surrounds the area of study in the Khaldiya residential area and, therefore, it is mainly affected by the pollutants that are discharged from the trafc load in view of the proximity of major highways. The majority of the prevailing wind is from the west and northwest (NW). The monitoring site is situated downwind from the Shuwaikh industrial area and the Shuwaikh power plant in the event that the levels of pollution released from them are signicant.

3.2. Data collection

Kuwait Universitys mobile air pollution laboratory was used in the study. The location of the mobile laboratory in Khaldiya was selected as the sampling site on the basis of the availability of power and security and the topography of the area. Care was taken that no high buildings or trees were present within 500 meters of the site. The mobile laboratory was tted with chemical monitors and meteorological sensors. All the sensors were operated automatically. Measurements were recorded every ve minutes. Pollutants measured include CH4, NMHC, CO, CO2, NO, NO2, SO2 and O3. Meteorological parameters

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403 monitored simultaneously included wind speed and direction, air temperature, relative humidity and solar radiation. Methane and non-methane hydrocarbons were measured by gas chromatography using a ame ionization detector (Model MAS-1030A, Mine Safety Appliances Company) which had a detection limit of 0.05 ppm. Carbon monoxide and carbon dioxide concentrations were measured based on non-dispersive infrared absorption (Model 48 and Model 41/41H of Thermo Environmental Instruments, respectively). The detection limits for carbon monoxide and carbon dioxide were 0.1 ppmv and 5 ppb, respectively. The NOx concentrations were measured with a detection limit of 0.5 ppb (Thermo Environmental Instrument, Model 42). SO2 concentrations were measured by using model Model 43A with a detection limit of 1 ppb (Thermo Environmental Instruments, Pulsed uorescent). The ozone concentrations were measured by a non-dispersive UV photometer (Monitors Labs, Model ML 9812) with a detection limit of 1.0 ppbv. Suspended dust was measured gravimetrically (TEOM Series 1400a). This was a real-time device used for assessing particulate concentration for sizes smaller than 10 mm in diameter. It was a lterbased mass monitor which was composed of a TEOM sensor and a TEOM control unit. It had a detection limit of 5 mg/m3. Details of the mobile laboratorys meteorological sensors were given in previous studies (Abdul-Wahab et al., 1996; Abdul-Wahab, 2001; Bouhamra and Abdul-Wahab, 1999; Elkamel et al., 2001).
45.0 40.0 35.0 30.0 25.0 20.0 15.0 10.0 5.0 0.0 0

399

Mean ozone (ppb)

R2=0.965

Actual ozone Predicted ozone by PC

10

12

14

16

18

20

22

Time (hour)
Fig. 1. Predicted concentrations of ozone by principal component regression and their corresponding actual values according to the hours of the day. can be used as predictions of error terms for the PCR model. Detailed procedures for the hybrid approach can be found in Zhang (2003). The procedures for the application of ANNs to forecasting ozone can be found in AbdulWahab and Al-Alawi (2002).

4. Results and discussion Fig. 1 shows the results of applying the PCR method to model ozone concentration. While the adjusted coefcient of variations for this model are quite satisfactory, at 0.965, the gure clearly shows an underestimation of ozone concentration levels during part of the morning hours, and more so from the afternoon hours to midnight. When the data were tted using ANN, there was a signicant improvement, yielding an R2 value of about 0.986, as shown by Fig. 2. Nevertheless, the t resulted in an over-estimation of the ozone concentration levels during the evening hours. The application of ANN on the residuals from the PCR model resulted in an almost perfect t of the ozone concentrations, with R2 0.995 (Fig. 3). An examination of the residuals from the nal model showed that they were approximately normally distributed, with zero mean and no detectable serial correlation. Table 1 gives the actual hourly mean concentration levels and those computed by the three methods. The size of the root mean square error (RMSE) and the correlation coefcients between the actual and the predicted values were used to compare the models. The RMSE is dened as shown in Eq. (5). s P 2 Oi Ei RMSE 5 n
45.0 40.0 35.0 30.0 25.0 20.0 15.0 10.0 5.0 0.0 0

3.3. Methodology and the hybrid approach

The hybrid approach is based on the literatures ndings that the linear model has achieved success in outlining linear relationships in the data and has difculties when the data have multiple variables, multicollinearities and outliers. On the other hand, ANN provides good results for modeling non-linear data. Therefore, a hybrid approach can increase the chance to capture different patterns in the data and improve forecasting performance. Zhang (2003) outlined the successful methodology for combining ARIMA and articial neural networks. Using empirical data, he showed that for data containing linear and non-linear correlation structures, the combination method can be an effective way to improve forecasting technique. In this work, we have used the combining methodology proposed by Zhang. In the combining methodology, he expressed the relationship between linear and non-linear components as shown below in Eq. (1). yt Gt Nt 1

Where Gt is the linear component and Nt is the non-linear component, these two components are estimated using the following technique: First, the principal component regression (PCR) technique is used to estimate the linear component of the data (Gt) by using meteorological data and pollutants in order to forecast ozone concentration. Detailed procedures for using PCR to forecast ozone can be found elsewhere in Abdul-Wahab et al. (2005). Second, the residuals from the linear model containing the non-linear relationship is then found using Eq. (2). bt et yt G 2

Mean ozone (ppb)

Where et is the residual at the time t from the PCR model (linear model), yt is t is the forecasted value for the time t the observed value at the time t, and G resulting from the PCR model (Abdul-Wahab et al., 2005). Third, articial neural networks (ANNs) are then used to model the resulting residuals from the PCR model using Eq. (3). et f et1 ; et2 ; . ; etn 3t 3

R2=0.986

Where f is a non-linear function determined by the neural network and 3t is the t and random error. The forecast resulting from the ANN model is denoted by J the combined forecast will be expressed as shown in Eq. (4). bt b b yG Jt 4

Actual ozone Predicted ozone by ANN

10

12

14

16

18

20

22

Time (hour)
Fig. 2. Predicted concentrations of ozone by an articial neural network and their corresponding actual values according to the hours of the day.

Thus, the linear component is estimated by the PCR and the non-linear component is estimated by the ANN. The results from the neural network model

400
45.0 40.0 35.0 30.0 25.0 20.0 15.0 10.0 5.0 0.0

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403 Table 2 Root mean square errors (RMSE) of ozone concentrations (ppb) for the three models tted to the data for the whole study period and by days of the week Model Saturday Sunday Monday Tuesday Wednesday Thursday Friday Overall PC method 8.59 7.17 9.23 5.87 9.84 13.91 11.98 9.74 ANN method 3.42 3.05 3.54 4.16 3.62 3.68 3.10 3.51 Combined method 3.35 3.06 3.56 4.10 4.15 4.83 3.59 3.81

Mean ozone (ppb)

R2=0.995

Actual ozone Predcited Ozone by combined method

10

12

14

16

18

20

22

Time (hour)
Fig. 3. Predicted concentrations of ozone by the combined method and their corresponding actual values according to the hours of the day.

Where Oi is the actual observed ozone level every ve minutes, and Ei is the estimated concentration level by either of the models. The computed RMSEs are shown in Table 2, which includes the overall value for the duration of the study period, and for the days of the week. The range of the errors for the PCR model was 5.87 to 13.91 (ppb), while the errors for ANN ranged form 3.05 to 4.16 (ppb). The range for the combined model was from 3.06 to 4.83. A Wilkoxon signed ranks test showed that, while no statistically signicant difference was found between ANN and the combined models ( pvalue 0.146), the PCR model was signicantly different from either of the two ( p-values <0.01). Fig. 4 shows the graphs of the observed and tted values of hourly mean ozone concentration as produced by the
Table 1 Actual and predicted mean hourly ozone concentrations by the different forecasting methods Time (hour) Mean (ppb) Actual ozone 17.13 20.67 21.39 21.34 19.02 17.26 16.19 18.62 26.85 32.92 35.32 37.26 39.09 37.74 38.80 36.46 25.81 18.94 13.24 10.72 10.24 11.50 11.29 13.50 23.08 Predicted ozone by ANN 17.52 19.71 21.75 20.55 19.37 17.21 16.61 17.84 27.67 32.55 35.77 37.18 39.09 37.54 39.11 35.72 27.04 19.22 15.58 13.01 14.44 13.47 14.94 14.17 23.63 Predicted ozone by PC 16.10 19.89 21.86 21.47 19.65 18.26 14.75 17.29 24.24 29.28 33.91 37.33 39.19 36.03 34.58 31.05 21.53 15.09 7.80 7.08 7.24 8.08 8.62 11.89 21.02 Predicted Ozone by combined method 16.63 20.51 21.60 20.99 18.73 18.16 14.84 17.36 28.10 32.30 33.57 35.78 38.73 37.37 38.17 36.16 25.17 20.13 13.06 11.09 10.72 11.04 10.76 13.06 22.76

combined model for a typical working day, represented here by a Saturday, and for a weekend, represented by a Friday. Although there appears to be some discernable systematic underand over-estimations, the overall ts are adequate, with very high coefcients of variation. While the ozone concentrations are highest between 8:00 AM and 4:00 PM on weekdays, during the weekend the concentrations tend to increase after 1:00 PM to a peak between 6:00 and 7:00 PM. There is usually low ozone concentration during the morning hours on weekends and at night for all days of the week. This indicates low trafc levels at these times. Figs. 5e7 are the scatter plots of the logarithms of predicted ozone concentrations of each of the three methods against the corresponding logarithms of the observed values. The points tended to cluster along lines that are close to the 45 tangent line, indicating model adequacy. The slopes of the regression lines of the scatter plots, 0.823, 0.992 and 0.979, for the PCR, ANN and the combined models, respectively, are not signicantly different from unity, the slope of a perfect t. However, the R2 value, 0.818, for the PCR method

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Total

Saturday
Mean ozone (ppb)
50.0 40.0 30.0 20.0 10.0 0.0 0 2 4 6 8 10 12 14 16 18 20 22
Actual ozone Predcited Ozone by combined method

R =0.968

Time (hour) Friday

70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 0

Mean ozone (ppb)

R =0.975
Actual ozone Predcited Ozone by combined method

10

12

14

16

18

20

22

Time (hour)
Fig. 4. Predicted concentrations of ozone by the combined method and their corresponding actual values as produced for Saturday (e.g., working day) and Friday (e.g., weekend).

Zero hour represent 12:00 midnight.

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403

6.0 5.0 y = 0.823x + 0.4641 5.0 4.0 3.0 2.0 1.0 0.0 -1.0 -1.0 -2.0 0.0 1.0 2.0 3.0 4.0 5.0 R2 = 0.818

401

Log mean predicted ozone by PC

Log mean predicted by combined method

y = 0.9792x + 0.0527 4.0 R2 = 0.970

3.0

2.0

1.0

0.0 -1.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0

Log mean actual ozone

Fig. 5. Predicted values of log ozone by principal component regression vs the log actual ozone values.

Log mean actual ozone

Fig. 7. Predicted values of log ozone by the combined method vs the log actual ozone values.

is the lowest of the three. The R2 value for the ANN model is 0.845 while the one for the combined model is 0.970. These results yield correlation coefcients, with the logarithms of the actual ozone concentrations, namely, 0.904 for PCR, 0.919 for ANN, and 0.985 for the combined model. Thus, although the performance of ANN and the combined models are similar when judged by their RMSE results, the combined model is preferable to the ANN model. An added advantage of the combined model is that the principal component application identies the predictor variables that best explain the variation in the ozone concentration levels and the nature of the variation. In this case, the chief predictor variables, as selected by the principal component method, were the pollutant variables, [NO] and [SO2], and the meteorological variables of temperature and solar radiation, during day time, while only the polynomial function of the nitrogen compounds [NO] and [NO2] explained much of the variation at night time (Abdul-Wahab et al., 2005).

5. Conclusions This study has demonstrated that a more accurate prediction model for ozone can be obtained by using PCR as a variable selection technique to identify the most appropriate explanatory variable subset for a regression model of ozone concentration, followed by applying an ANN model on the resulting residuals. The ANN part of the combined model was able to t more accurately the non-linearity left in the residuals that the PCR analysis could not capture. Although the ANN model alone provided better t than the PCR model, the combined PCR and ANN signicantly improved the forecast accuracy for the ozone concentration levels in the 24-h period. Both the RMSE and MAPE were substantially reduced in the combined forecast, compared to either PCR or ANN forecasts. This approach in modeling offers great potential in other areas of modeling. Acknowledgements The authors would like to thank the referees for their constructive suggestions that have positively improved the clarity of the paper.

6.0 y = 0.9915x - 0.0002 R2 = 0.845

Log mean predicted ozone by ANN

4.0

2.0

0.0 -1.0 -2.0 0.0 1.0 2.0 3.0 4.0 5.0

References
Abdul-Wahab, S.A., 2001. IER photochemical smog evaluation and forecasting of short-term ozone pollution levels with articial neural networks. Trans ICHEME Process Safety and Environmental Protection 79, 117e128. Abdul-Wahab, S.A., Al-Alawi, S.M., 2002. Assessment and prediction of tropospheric ozone concentration levels using articial neural networks. Environmental Modelling & Software 17, 219e228. Abdul-Wahab, S.A., Bouhamra, W., Ettouney, H., Sowerby, B., Crittenden, B.D., 1996. A statistical model for predicting ozone levels in the Shuaiba Industrial Area in Kuwait. Environmental Science and Pollution Research 3, 195e204.

-4.0

-6.0

-8.0

Log mean actual ozone

Fig. 6. Predicted values of log ozone by an articial neural network vs the log actual ozone values.

402

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403 berger, K., Paksy, L., Ba nhidi, O., Rajko , R., 2004. Prediction Lengyel, A., He of ozone concentration in ambient air using multivariate methods. Chemosphere 57, 889e896. Maenhaut, W., Cornille, P., Pacyna, J., Vitols, V., 1989. Trace element composition and origin of the atmospheric aerosol in the Norwegian Arctic. Atmospheric Environment 23, 2551e2569. Makridakis, S., Anderson, A., Carbone, R., Fildes, R., Hibdon, M., Lewandowski, R., Newton, J., Parzen, E., Winkler, R., 1982. The accuracy of extrapolation (time series) methods: results of a forecasting competition. Journal of Forecasting 1, 111e153. Makridakis, S., Chateld, C., Hibon, M., Lawrence, M., Millers, T., Ord, K., Simmons, L.F., 1993. The M-2 competition: a real-life judgmentally based forecasting study. International Journal of Forecasting 9, 5e29. McAdams, H.T., Crawford, R.W., Hadder, G.R., 2000. A Vector Approach to Regression Analysis and its Application to Heavy-Duty Diesel Emissions. Society of Automotive Engineers, Inc. Contract with the Energy Division of Oak Ridge National Laboratory (ORNL), Contract No. DE-AC05e 00OR22725. Paterson, K.G., Sagady, J.L., Hooper, D.L., 1999. Analysis of air quality data using positive matrix factorization. Environmental Science Technology 33, 635e641. Peton, N., Dray, G., Pearson, D., Mesbah, M., Vuillot, B., 2000. Modelling and analysis of ozone episodes. Environmental Modelling and Software 15, 647e652. rez, J.C., Mayora-Ibarra, O.A., Torres-Jime nez, J., Ruiz-Sua rez, L.G., Ruiz-Sua 1995. Short-term ozone forecasting by articial neural networks. Advances Engineering Software 23, 143e149. , R., Stohl, A., Karatzas, K., Bohler, T., James, P., Pe rez, J.L., 2005. A San Jose modeling study of an extraordinary night time ozone episode over Madrid domain. Environmental Modelling & Software 20, 587e593. Sanchez, M.L., Casanova, J.L., Ramos, M.C., Sanchez, J.L., 1986. Studying the spatial and temporal distribution of SO2 in an urban area by principal component factor analysis. Atmospheric Research 20, 53e65. Schlink, U., Fritz, A.J., Herbarth, O., Richter, M., 2002. Longitudinal modelling of respiratory symptoms in children. International Journal of Biometeorology 47, 35e48. Schlink, U., Dorling, S., Pelikan, E., Nunnari, G., Cawley, G., Junninen, H., Greig, A., Foxall, R., Eben, K., Chatterton, T., Vondracek, J., Richter, M., Dostal, M., Bertucco, L., Kolehmainen, M., Doyle, M., 2003. Trace a rigorous inter-comparison of ground-level ozone predictions. Atmospheric Environment 37, 3237e3253. Schlink, U., Herbarth, O., Richter, M., Dorling, S., Nunnari, G., Cawley, G., Pelikan, E., 2006. Statistical models to assess the health effects and to forecast ground-level ozone. Environmental Modelling & Software 21 (4), 547e558. Seinfeld, J.H., 1989. Urban air pollution: state of the science. Science 243, 745e752. Seinfeld, J.H., Pandis, S.N., 1998. Atmospheric Chemistry and Physics. John Wiley and Sons, New York. Shi, J.P., Harrison, R.M., 1997. Regression Modelling of hourly NOx and NO2 concentrations in urban air in London. Atmospheric Environment 31, 4081e4094. , R., Kitwiroon, N., Fragkou, E., Pe rez, J.L., Middleton, D.R., Sokhi, R., San Jose 2006. Prediction of ozone levels in London using the MM5-CMAQ modelling system. Environmental Modelling & Software 21 (4), 566e576. Statheropoulos, M., Vassiliadis, N., Pappa, A., 1998. Principal component and canonical correlation analysis for examining air pollution and meteorological data. Atmospheric Environment 32, 1087e1095. Thompson, M.L., Reynolds, J., Cox, L.H., Guttorp, P., Sampson, P.D., 2001. A review of statistical methods for the meteorological adjustment of tropospheric ozone. Atmospheric Environment 35, 617e630. Tian, Y.I., Biswas, P., Pratsinis, S.E., Hsieh, W.M., 1989. Principal component analysis for particulate source resolution in cleanrooms. The Journal of Environmental Sciences 32 (6), 22e27. Utojo, U., Bakshi, B.R., 1995. Connection between neural networks and multivariate statistical methods: an overview. In: Baughman, D.R., Liu, Y.A. (Eds.), Neural Networks in Bioprocessing and Chemical Engineering. Academic Press, New York, pp. 435e459.

Abdul-Wahab, S.A., Bakheit, C.S., Al-Alawi, S.M., 2005. Principal component and multiple regression analysis in modelling of ground-level ozone and factors affecting its concentrations. Environmental Modelling & Software 20 (10), 1263e1271. Agirre-Basurko, E., Ibarra-Berastegi, G., Madariaga, I., 2006. Regression and multilayer perceptron-based models to forecast hourly O3 and NO2 levels in the Bilbao area. Environmental Modelling & Software 21 (4), 430e446. Arvanitis, A., Moussiopoulos, N., 2006. Estimating long term urban exposure to particulate matter and ozone in Europe. Environmental Modelling & Software 21 (4), 447e453. Bates, J.M., Granger, C.W.J., 1969. The combination of forecasts. Operations Research Quarterly 20, 448e451. Bloomeld, P.J., Royle, J.A., Steinberg, L.J., Yang, Q., 1996. Accounting for meteorological effects in measuring urban ozone levels and trends. Atmospheric Environment 30, 3067e3077. Bouhamra, W., Abdul-Wahab, S.A., 1999. Description of outdoor air quality in a typical residential area in Kuwait. Environmental Pollution 105, 221e229. Buhr, M.P., Trainer, M., Parrish, D.D., Sievers, R.E., Fehsenfeld, F.C., 1992. Assessment of pollutant emission inventories by principal component analysis of ambient air measurements. Geophysical Research Letters 19, 1009e1012. Buhr, M.P., Parrish, D.D., Elliot, J., Holloway, J., Carpenter, J., Goldan, P., Kuster, W., Trainer, M., Montzka, S., McKeen, S., Fehsenfeld, F., 1995. Evaluation of ozone precursor source types using principal component analysis of ambient air measurements in rural Alabama. Journal of Geophysical Research 100, 22853e22860. Chang, S.N., Hopke, P.K., Gordon, G.E., Rheingrover, S.W., 1988. Targettransformation factor-analysis of air-borne particulate samples selected by wind-trajectory analysis. Aerosol Science and Technology 8, 63e80. Clemen, R., 1989. Combining forecasts: a review and annotated bibliography with discussion. International Journal of Forecasting 5, 559e608. Comrie, A.C., 1997. Comparing neural networks and regression models for ozone forecasting. Journal of the Air and Waste Management Association 47, 653e663. Cybenko, G., 1989. Approximation by super precision of a sigmoidal function. Mathematics of Control Signals and Systems 2, 303e314. Davis, J.M., Eder, B.K., Bloomeld, P., 1998. Modeling ozone in the Chicago urban area. In: Nychka, D., Piegorsch, W.W., Cox, L.H. (Eds.), Lecture Notes in Statistics. Case Studies in Environmental Statistics, vol. 132. Springer, New York, pp. 5e26. Elkamel, A., Abdul-Wahab, S.A., Bouhamra, W., Alper, E., 2001. Measurement and prediction of ozone levels around a heavily industrialized area: a neural network approach. Advances in Environmental Research 5, 47e59. lez Cabrera, J., Fidalgo Mart nez, M.R., Mart n Mateos, E.J., Vicente Gonza Tavera, S., 2006. Study of the evolution of air pollution in Salamanca (Spain) along a ve-year period (1994e1998) using HJ-Biplot simultaneous representation analysis. Environmental Modelling &Software 21 (1), 61e68. Hair, J.F., Anderson, R.E., Tatham, R.L., 1987. Multivariate Data Analysis, second ed. Macmillan, New York. Heravi, S., Osborn, D.R., Birchenhall, C.R., 2004. Linear versus neural network forecasts for European industrial production series. International Journal of Forecasting 20, 435e446. Hoek, G., Schwartz, G.D., Groot, B., Eilers, P., 1997. Effects of ambient particulate matter and ozone on daily mortality in Rotterdam. Archives of Environmental Health 526, 455e463. Klaus, D., Poth, A., Voss, M., 2001. Ozone distributions in Mexico City using principal component analysis and its relation to meteorological parameters. Atmosfera 14, 171e188. Koop, G., Tole, L., 2006. An investigation of thresholds in air pollutionmortality effects. Environmental Modelling & Software 21 (12), 1662e1673. Kuntasal, G., Chang, T.Y., 1987. Trends and relationships of O3, NOx and HC in the south coast air basin of California. Journal of Air Pollution Control Association 37, 1136e1158. Lavecchia, C., Angelino, E., Bedogni, M., Bravetti, E., Gualdi, R., Lanzani, G., Musitelli, A., Valentini, M., 1996. Ozone patterns in the aerological basin of Milan (Italy). Environmental Software 11, 73e80.

S.M. Al-Alawi et al. / Environmental Modelling & Software 23 (2008) 396e403 Vaidya, O.C., Howell, G.D., Leger, D.A., 2000. Evaluation of the distribution of mercury in lakes in Nova Scotia and Newfoundland. Water, Air, and Soil Pollution 117, 353e369. Van Eijkeren, J.C., Freijer, J.I., Van Bree, L., 2002. A model for the effect on health of repeated exposure to ozone. Environmental Modelling and Software 17, 553e562. Volta, M., Finzi, G., 2006. GAMES, a comprehensive gas aerosol modelling evaluation system. Environmental Modelling & Software 21 (4), 587e594. Xu, J., Zhu, Y., 1994. Some characteristics of ozone concentrations and their relations with meteorological factors in Shanghai. Atmospheric Environment 20, 3387e3392.

403

Zannetti, P., 1994. Computer Modelling of Air Pollution, Science, Art or Fiction: Air Pollution II Volume 1, Computer Simulation. Computational Publications, Southampton, Boston. Zhang, G.P., 2003. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50, 159e175. Zhang, G.P., Patuwo, E.B., Hu, M.Y., 1998. Forecasting with articial neural networks: the state of the art. International Journal of Forecasting 14, 35e62. Zunckel, M., Koosailee, A., Yarwood, G., Maure, G., Venjonoka, K., van Tienhoven, A.M., Otter, L., 2006. Modelled surface ozone over southern Africa during the Cross Border Air Pollution Impact Assessment Project. Environmental Modelling & Software 21 (7), 911e924.