We would like to announce some changes in the data centre data structure in the gas section. The columns of “gas flow interconnection point daily actual”, key: “GAS_FLW_INTRCON_D_A” will be changed in the next days. Continue reading Gas Analysis: Changes in Data Center
Starting on 13rd of March ECMWF Ensemble forecasts are disseminated 40 minutes earlier. Continue reading Ensemble forecasts disseminated 40 minutes earlier
MKonline has issued new consumption normals up to 2020.
A quick overview
- New consumption normals are published
- Nomals are now being dynamically simulated
- Be advised to update our consumption normal every day
How are the assumptions made
Our analysts have carefully gathered data and conducted literature surveys on the expected power consumption development for all price areas in Europe. Based on this MKonline has made and implemented assumptions on expected growth per year in our consumption forecasting system.
Notably, the assumptions are to be understood as expected underlying growth. The simulated normal for 2015 and 2016 will therefore not “hit” the assumed percentage growth perfectly. The reason is twofold:
- Calendar effects: The accumulated effects of weekends, holidays, and leap years vary from year to year. The easiest to understand may be the leap year effect. It will add, ceteris paribus, 1/365=0.27% to the annual growth in 2016. Going on, comparing 2017 to 2016 you will “lose” an 0.27% annual growth. Hence the simulated annual growth in percent will by definition deviate from the assumed underlying growth assumption.
- Learning from recent history: We collect and adjust the consumption model to actual numbers as we go. If, for instance, the consumption model (without adjustment) deviates from the recent received actuals, a Kalman filter adjusts the forecasts to fit the latest received actuals best. This procedure is also included for estimation of the normals. Further, the “recent” adjustment is kept constant for 60 days, and then linearly phased out over the next 30 days.
In layman words, the Kalmanfilter will help pull the forecasts, and the normal simulation, back on track during periods where consumption grows/shrinks faster than we assumed ex ante, or we experience exogenous shocks.
Consumption normals are updated each day
We recalculate and update the normals every day. That is of course not beaches we change our mind with regard to assumed underlying consumption growth. It is because we dynamically adapt the normals to reflect the latest received actual numbers. As a rule of thumb: the normal for the period “today” until 90 days ahead will experience minor adjustments each day. For this reason you should renew your consumption normal form MKonline each day.
We will keep you posted each time we will change the assumptions on underlying assumption growth.
Note: As I write this. 26 February 2015, the new consumption normal for Turkey is not yet simulated.
How is the normal simulated?
The consumption normal is simulated by running more than 30 weather years through our proprietary consumption models. One for each price area, on hourly level, from 2006 until 2020. Note that using the average of each weather parameter as input would not work. This is because the consumption model is nonlinear and contains input related to spatial definition of each model area.
One has to distinguish between preceding normal and expected normal ahead.
- Preceding normals: For the preceding period we take into consideration the received actual consumption. The deviation between actual consumption and simulated normal consumption indicates how “uncommon” the weather has been, in terms of MWh over a period.
- Future normal: Looking ahead we include our assumptions on expected underlying consumption growth, instead of actual received consumption numbers. In the years to come the normal consumption therefore indicates what we should expect the consumption to be, given no prior information on the weather.
Note: The normal for the current year is partly simulated on received numbers so far and partly on assumptions for the remaining part of the year.
A word of caution:
Normal consumption is also referred to as temperature adjusted consumption. Please be advised not to directly compare MKonline normals to that of third parties. MKonline adjusts for a wide range of weather parameters which enter the consumption model, not only temperature. We also include and simulate calendar effects and exploit the information embedded in received consumption data until “today”. At the end of the day, the model properties and weather period over which one simulates the normal will determine the result.
On Friday 23 January, we aim to update our historic data and implement a new consumption model in Germany.
Power load in Germany is an elusive variable . Not because one lacks sources but because of poor updates and internally inconsistent sources. This could be e.g. hourly numbers not summing up to aggregates over months or years, or summing up individual TSOs to national numbers. Another known issue is that they do not meet simple econometrical criteria one would expect from such data, e.g. sign of derivatives etc.
Please be informed that we have added the NTC forecast for individual price areas to our Data Center.
Each file contains commercial import and export capacity for a price area and all its adjoining power markets. The figure illustrates that Germany runs exchange with 9 price areas. Therefore, the file contains 9*2=18 variables in total. It follows the update schedule for Instant Spot.
How to find the file
Simply use the search field in the Data Center. Type in NTC and the price area you are looking for.
Click here to see an example of how net transfer capacity is presented for Germany. If you do not have access, please apply for a free trial here.
Why add a forecast for NTC
TSOs are perceived to issue forecasts on NTC. However, operating the Instant Spot model since January 2013, we have noticed a wide array of flaws, errors, and pit falls in the provided numbers. The quality varies over time. For some borders the issued numbers are literally useless.
Therefore, we have established a new routine. A panel of analysts validates the capacity numbers each workday. Furthermore, we plan to issue comments on the NTC numbers.
MKonline keeps you posted.
Following customers requests, we have started to publish all 51 ensemble runs in the Data Center.
At this round we added 4 variables: Temperature, Consumption, Photovoltaic, and Residual Load (Con-Wnd-SPV). Later we will add many more ensemble forecasts, such as Precipitation Energy, Price, Exchange, Hydrology and many more.
How to find data?
MKonline covers all of Europe (except Iceland and Ireland) in all price areas/areas 57. How can you find the data?
Simply go to the Data Center, place your cursor in the search field (top left corner) and type in the name of the price area/country and variable you are looking for. The system filters it out as you write.
Let us assume you search for Ensemble Switzerland. 2-3 letters in each word would filter out the files for you. We have also included some predefined filters, see on the left side of the Data Center.
Update and Structure of the files
The files follow the updated schedule for the ECMWF forecast and contain the two latest ensemble forecasts. The file will rotate in order that the first 51 columns always contain the most recent ensemble forecast, whether this is EC00 or EC12.
Each file contains only one variable for one price area, as indicated in the Ensemble Switzerland example.
In order to avoid any confusion regarding the data’s freshness, there is a row above the column names containing the issue date of the data.
As part of the continuous improvement of our analysis, the consultancy Perret Associates has now made relevant coal inventories data at main ports and delivery points worldwide available.
Below you can find a comprehensive list of the inventories available which go as far back as 2008 for instance for ARA. In some cases, we are also processing the raw data. This is the case for Indian power plants, for which we are also producing “like-for-like” inventories, which refer to stocks that we have been monitoring at the same power plants since 2010. Indeed as India is adding significant new coal-fired capacity each year, the analysis of total stocks is biased as it is based on an increasing number of power stations.
- Coal stocks at ARA terminals (EMO and OBA) since 2008
- Coal stocks at main Chinese ports since 2009
- Coal stocks at Newcastle Port since 2009
- Coal stocks at Indian power plants (total and like-for-like) since 2009
- Coal inventories at UK power plants since 1995
- Richards Bay Coal Stocks since 2006
- US coal stocks per sector (electric power sector, producers and distributors, end user sector) since January 2000
If you do not have access, please apply for a free trial here.
We have added some new files and augmented some existing ones in the past month:
- Allocated Implicit Exchange (IMP_ALC_H_A)
- Available Implicit Exchange Capacity (EXC_CAP_ATC_H_A)
Files for the Iberian area do now include FR-ES exchange.
- Wind Power Installed Capacity (WND_CAP_D_AF)
- Wind Power Production Hourly Actual (PRO_WND_H_A)
- Wind Power Production Hourly Forecast (PRO_WND_H_F)
- Wind Power Production Hourly Normal (PRO_WND_H_N)
Files for the SEE area do now include data from Croatia (HR) and a file for the Baltic area published in each category, containing Latvia (LV), Lithuania (LT) and Estonia (EE).
- Residual Load Hourly Actual (RES_LOAD_H_A)
- Residual Load Hourly Forecast (RES_LOAD_H_F)
- Residual Load Hourly Normal (RES_LOAD_H_N)
Files for the SEE area do now include data from Croatia (HR), Macedonia (MK), Bosnia and Herzegovina (BA) and Serbia (RS).
- Consumption Hourly Normal (CON_POW_H_N)
- Photovoltaic Hourly Normal (PRO_SPV_H_N)
- Residual Load Hourly Normal (RES_LOAD_H_N)
- Temp Hourly Normal (TT_CON_H_N)
- Wind Power Production Hourly Normal (PRO_WND_H_N)
Files for all areas cover the period 1 January 2014 until 31 December 2018.
It is now possible to download exchange data for Italy from the Data Center.