Forbes has a graphic showing the size of the errors in EIA gas production forecasts and actual values. It would be useful to have this sort of data (and some better visualisations) for all energy production forecasts (if nothing else it would bring clarity to which organisations produce useful forecasts and which don't) - Got A Favorite 2017 Energy Forecast? Technology Will Make It Obsolete.
We can take this logic to data, and, fortunately, the EIA has made this exercise easy to do. Since 1996, they have published retrospective analyses of their predictions. In this useful and under-appreciated effort, the EIA compiles 20 years of forecasts into a single report and compares those forecasts to what actually happened. The last of these analyses was published in May 2015, comparing forecasts as far back as 1994 to realized outcomes through 2013. For U.S. natural gas markets, I have extended this data (both forecasted and realized) through 2015 and characterized the forecasting error of each report year, prediction year combination as driven by unexpected shocks to supply or demand. I have plotted this data in the figure below, separating forecasts made before the start of the recent shale boom in 2005 and after.