Conventional load forecasting techniques have been widely used over the past 30 years. These forecasts have tied load demand to the economic activity of the country and temperature variations while assuming inelasticity of the load to price sensitivities. Based on time-scale, they can be classified into three groups:
- Short-term load forecast (STLF): the time-period of STLF lasts for a few minutes or hours to one-day ahead or a week. STLF aims at economic dispatch and optimal generator unit commitment while addressing real-time control and security assessment.
- Medium-term load forecast (MTLF): the time-period of MTLF is a week to one year (possibly two years). MTLF aims at maintenance scheduling, coordination of load dispatch and price settlement so that demand and generation is balanced.
- Long-term load forecast (LTLF): the time-period of LTLF ranges from a few years up to 10 – 20 years ahead. LTLF aims at system expansion planning, i e. generation, transmission and distribution. In some cases, it also affects the investment in new generating units.
Different methods can be applied depending on the model identified. Short term load forecasting would require Similar Day Look up Approach, Regression based Approach, Time Series Analysis, Artificial Neural Networks, Expert Systems, Fuzzy Logic, Support Vector Machines, while Medium and Long-Term Load Forecasting will rely upon techniques such as Trend Analysis, End Use Analysis, Econometric Analysis, Neural Network Technique, Multiple Linear Regressions.
Enhanced load forecasting goes beyond the traditional linear regression method of forecasting load, which is based on economic activity and temperature forecast, considering the load inelastic to price sensitivities. It leverages the advancements in machine learning to predict the variability of the load at the customer level, this variability continuously increasing with the penetration of distributed energy resources, storage and electric vehicles. With the deployment of advanced metering infrastructures and variable tariffs offered on a retail level, new bot-tom-up methods are being tested that leverage the power of big data and predictive analytics to better understand customer decision-making mechanisms as well as its sensitivity to variable price signals for improved prediction of load demand.
Technology Types
In addition to the three main categories for load forecasting according to time horizon (see above: short, mid-, long- term), the literature pro-poses additional classifications to the mid and long-term load forecasting, more complex than the short-term, which concentrates most of the load forecasting techniques since the 60s.
Whereas the STLF rely on data modelling (fitting data to models, extrapolation) rather than on intimate information of how an electrical system works, the MTLF / LTLF require simultaneously the data analyst expertise and the expertise of how power system / markets behave. The manner in which all these impacting factors (such as temperature, weather, historical load, GDP forecast, demographic) are considered enables the classification of the MTLF / LTLF approaches:
- Either based on ‘time series’ (assuming that data have their internal structure and correlation), or on economic indicators affecting the load (the ‘econometric approach’), or on the ‘end-use approach’ concatenating, in a bottom-up mode, information gathered from the individual end-uses (presenting several advantages but very data intensive), or
- Another classification is based on the extent of these impacting factors. The ‘conditional modelling’ approach encompasses all historical load and weather data, socioeconomic indicators and energy policies, whereas the so-called ‘autonomous approach’ depends only on historical load and weather data. Researchers have proven that the latter provides better forecasts for a time horizon below one year.
Enhanced load forecasting techniques use new data mining techniques based on machine learning to classify different types of load behaviour provided by a large volume of input data. These methods can be used to estimate approximate functions that depend on many inputs when there is no accurate mathematical model to describe the phenomenon. Their validity is currently being tested to forecast loads using aggregated load data points (transmission level).
In this new methodology, load can become segmented on clusters of customers based on structural, demographic and financial factors, and the forecast outcome becomes probabilistic based on the likelihood of customer adoption of a technology and their reaction to variable retail prices.
Components & enablers
Mathematical techniques constitute the founding blocks for load forecasting. ‘Parametric methods’ should be distinguished from AI methods, or other ‘hybrid methods’.
- In parametric methods, the model of the load is built upon the relationship between load and load-impacting factors, the adequacy of the model being assessed based on the forecast errors as model residuals. ARMA, ARIM, ARMAX or ARIMAX are widely used time series methods . The Grey dynamic model is based upon three types of systems according to the degree of availability of the information (white system: all required information is available; black system: no information is available; grey system: partial information is available)
- AI methods are clearly part of the non-conventional, enhanced methods; they mostly include fuzzy logic, artificial neural network (ANN) and support vector regression (SVR).
- The hybrid category, also clearly non-conventional, includes heuristic optimisation algorithms such as genetic algorithms, expert systems or evolutionary computation algorithms.
Deploying a model for load forecasting (based on relevant data inputs) requires a series of generic steps, given below:
- Collection of historical load data
- Preparation of the load data, collection of historical weather data and of historical event data
- Analysis of the data
- Preparation of the model input and test the data
- Select a model, fit the data and run the model
- Select the best model and implement it
- Run and refine the model.
Focusing on enablers, they include data from smart meters and home energy management systems, retail electricity prices (variable), demand response subscribed customers, DERs owned by customers, data mining tools, etc.
Advantages & field of application
Relative load forecasting errors using conventional statistic methods seen at the level of transmission substations have been quite low (from [ 3 ], many large and medium utilities operate their systems with one day ahead load forecast error at 3 % or lower) as aggregation reduces the inherent variability in electricity consumption, resulting in increasingly smooth load shapes.
By leveraging the data of smart meters and customer sensors, enhanced load forecasting can play a major role in optimising grid investments and grid operations as machine learning techniques become more advanced and efficient in predicting customers’ behavior . Similarly, it is expected that market operations will become more efficient as aggregators will need fewer unbalancing reserves with reduced load forecast errors.
Technology Readiness Level
Although traditional linear regression method of forecasting load based on activity and temperature are mature (TRL 9), enhanced load fore-casting techniques remain at lower TRLs, such as TRL 5 for Machine Learning Technology validated in the relevant environment .
Research & Development
Current fields of research: The ETIP SNET R&I Implementation Plan 2021 – 2024 includes a specific topic on ‘Medium and long-term control (Forecasting (Load, RES), secondary & tertiary control: LFC, operational planning: scheduling / optimisation of active / reactive power, voltage control)’. The topic addresses the solutions for operational planning of the energy systems, with special reference, among others, to resources scheduling (through adequate generation and load forecasting).
Research on enhanced load forecasting includes:
- the design of machine learning algorithms with improved forecasting performance, lower memory requirements and scalable architecture
- the development of novel data-aware resource management systems that can provide powerful data processing in distributed parallel computing systems and clusters for real-time processing
- featuring engineering to select the most relevant information to feed the machines, for each specific case to increase accuracy while decreasing the risk of overfitting.
Improvement foreseen: Effective data sampling, improved categorisation of the information and extraction of load patterns. In addition, given the different regulatory regimes worldwide on data privacy, the technical full potential of data analytics in predicting residential load forecast will be likely more restrained in Europe as a result of the more stringent citizen’s data privacy protection framework.
Best practice performance
Depending on the choice of forecasting method/ strategy and the parameter configuration, the forecasting accuracy can be subject to significant variations.
Typical example of best practice performance can be mentioned from the literature:
- Study on ‘Neural Network Model for Short-Term Load Forecast Based on Long Short-Term Memory Network (LSTMN) and Convolutional Neural Network (CNN)’. A combined CNN / LSTM model was proposed to improve forecasting accuracy for the prediction of the next 24 h load forecast. The forecasting performance was based on evaluation indexes (Mean Absolute Error [MAE], Mean Absolute Percentage Error [MAPE] and Root Mean Square Error [RMSE]). The combined model can improve performance by at least 9% compared to the DeepEnergy, 12% compared to the CNN module and 14% compared to the LSTM module. The combined CNN–LSTM model can achieve the best performance in STLF.
Best practice application
Description
The EU funded GARPUR project worked on developing novel long-term load forecasting technique (~3years).
Design
The outcome is a new applicable methodology that uses multiplicative error model which incorporates the volatility experienced in long-term forecasts and outputs load forecast predictions based on aggregated historical demand data. The project focused on enhancing current conventional techniques of long-term load forecasting. No attempts were however made to compare these results to load forecasting based on machine learning techniques.
Results
The results are supported by 95% confidence intervals. The authors proved that using a multiplicative error model, they were a) able to forecast with higher accuracy the peak loads of a transmission operator compared to the conventional linear regression method by a margin of 7% b) improve the directional accuracy of the forecast considering unpredicted (historical) market movements (e.g. financial crisis of 2008).
Description
Researchers investigated a two-stage approach to forecast short-term electricity loads in the Polish transmission power system considering predictions from demand peak classification models. The dataset comprised of hourly load values of the Polish power system from 1 January 2008 and 31 December 2015 and weather data, including temperature, humidity, sunrise and sunset. The analysis was enriched by transforming 19 additional variables that impact load fluctuations from standard dummy encoding to binary codes to transform the data into fewer dimensions.
Design
In the first stage, the models are trained to classify peak load levels, which are equal to or above the 99th, 955h or 90th percentiles for the respective load distribution when considering weekly horizons. In the second stage, the scores from the classification models are fed to the forecasting models, which uses data mining techniques based on machine learning to identify patterns and underlying dependencies in the analysed data to predict the aggregated load 24 hours ahead.
Results
The results of the two steps approach (enhanced model) were compared to a basic forecasting model, which used the same data mining approaches but without including variables that came from the peak classification models for the different quantiles, i.e without learning probability distribution of variables affecting peak load variations and their interdependencies. The result show that the ANN ML technique proved to be the best method to peak load forecasting and was able to reduce the MAPE error by 6%, from 3.3% (basic model) to 3.1% (enhanced model). MAPE stands for mean absolute percentage error between the measured load and forecasted load.
Description
Researchers investigated different data mining techniques to forecast residential load. These techniques included basic techniques (ARIMA, basic regression) and machine learning techniques (ANN, support vector machine, random forest algorithm). The authors conducted two experiments. The first is on a single household in British Columbia Canada, consisting of two years of recorded energy consumption at one-minute intervals using 21 submeters covering 2012-2014. The second is on an aggregate of 46 households in Austin, Texas, with consumption measurements of one-hour interval from 24 circuits within each home.
Design
The machine learning techniques were in trained to detect household behavioural patterns by analysing appliances with similar switch ON probability distribution through the day or the week with the method of segmentation and sequence analysis.
Results
Results show that a) machine learning techniques had in general smaller errors and higher accuracy rates in forecasting load compared to basic techniques b) combining historical data usage and household behaviour data enhances the forecasting of consumer loads using machine learning techniques. With respect to a) the artificial neural network technique came out best in class in forecasting single household load with a MAPE of 23% and accuracy of 54% compared to basic techniques with an average MAPE of 41% and accuracy of 30%. With respect to b) enlarging the data set of the 46 households given to the machine showed that the ANN technique was able to reduce the MAPE on by 8% from 46% (basic data set) to 41% (richer data set). It is important to note that ANN was the only technique in which in an enlarged data set improved the forecast compared to the other machine learning techniques or basic techniques. The MAPE of other machine learning techniques or basic techniques with a basic data set ranged between 60% to 120% in the experiment with 46 households. The authors conclude that ANN could be the best method for solving short-term forecasting when dealing with large amount of high volatility data.
References
[1] Ken Seiden, Brian Eakin - Navigant. Solving the Utility Load Forecasting Conundrum, (2017). [Link]
[2] Khuntia, S. R., Rueda, J. L., & van der Meijden, M. A. Long-term load forecasting using a multiplicative error model. [Link]
[3] EISPC (2015). Load Forecasting Case Study. [Link]
[4] M.N.Q. Macedo, J.J.M.Galo, L.A.L. deAlmeida, A.C.de C.Lima (2014) Demand side management using artificial neural networks in a smart grid environment. [Link]
[5] Mishra, A.K., Saxena A.K.. Data Mining Technology and its Applications to Power Systems. [Link]
[6] Diamantoulakis P.D., Kapinas V.M., Karagiannidis G.K.,(2015). Big Data Analytics for Dynamic Energy Management in Smart Grids. [Link]
[7] Krzysztof Gajowniczek and Tomasz Ząbkowski, (2017) Two-Stage Electricity Demand Modeling Using Machine Learning Algorithms. [Link]
[8] Krzysztof Gajowniczek and Tomasz Ząbkowskii (2017) Electricity forecasting on the individual household level enhanced based on activity patterns. [Link]
[9] Khuntia, S. R., Rueda, J. L., & van der Meijden, M. A. M. M. (2016). Forecasting the load of electrical power systems in mid- and long-term horizons: A review. IET Generation, Transmission and Distribution, 10(16), 3971-3977. [Link]
[10] Vikas Gupta, Seema Pal, An Overview of Different Types of Load Forecasting Methods and the Factors Affecting the Load Forecasting, (2017). [Link]
[11] Gajowniczek K. Ząbkowski T. Data Mining Techniques for Detecting Household Characteristics Based on Smart Meter Data. Energies. 2015a; 8(7): 7407–7427. [Link]
[12] ETIP SNET R&I Implementation Plan 2021-2024 (2020). [Link]
[13] GARPUR, Deliverables [Link]
[14] Swasti R. Khuntia, Jose L. Rueda, Mart A. M. M. van der Meijden, Neural Network-based Load Forecasting and Error Implication for Short-term Horizon, 2016. [Link]
[15] Mahmoud A. HAMMAD, Borut JEREB, Bojan Rosi, Dejan DRAGAN, Methods and Models for Electric Load Forecasting: A Comprehensive Review, 2020. [Link]
[16] Chujie Tian, Jian Ma, Chunhong Zhang and Panpan Zhan, “A Deep Neural Network Model for Short-Term Load Forecast Based on Long Short-Term Memory Network and Convolutional Neural Network”, 2018. [Link]
[17] Statnett, Developing and deploying machine learning models for online load forecasts. [Link]
