Tourism competition data
The tourism forecasting competition described in Athanasopoulos et al (2011) was an important investigation into domain-specific time-series forecasting; a different approach from the broader-scope “M” series forecasting competitions which covered multiple areas. Tourism is important to me in my day job and the article referenced above has been influential in how we construct New Zealand’s official tourism forecasts. I have an interest in making the data from that competition as widely available as possible, and this week was able to release the
Tcomp R package on CRAN that has a conveniently shaped version of the data from that competition.
The best introduction is in the package vignette.
Only the univariate version of the data is available. This is unfortunate in a slightly counter-intuitive way - one of the more important findings from the competition was that univariate methods of forecasting performed better than more complex methods that relied on relationships between the variables of interest (eg tourism numbers and spend) and other economic explanatory variables (eg economic growth in market countries). So although we know from the competititon that those explanatory variables are of less value than might be hoped, we can’t explore that finding as much as we would like as the explanatory variables aren’t available for re-use. That’s not the fault of the organisers of the competition, just a reality of the availability of data and the conditions under which it was provided to them.
Contents of the Tcomp R package
In the new
Tcomp package, the data is stored in a single object,
tourism is a list of class
Mcomp that has 1,311 elements. The
Mcomp S3 class is inherited from Hyndman’s
Mcomp R package. Each element in
tourism has the information needed to fit and test a forecast to single univariate time series. For example, the training series is always a
ts object called
x. To plot the 34th training time series is as simple as:
We immediately see the strong seasonality that is characteristic of much tourism data.
tourism inherits classes from the
Mcomp package, methods provided in that package apply to the
tourism object and its elements. For example, the
plot method superimposes the test dataset on the graphic of the training dataset, so we can see what happened next after the first part of the competition training series #34:
Part of the development of the
Tcomp package was a moderately rigorous testing and validation process, which included attempting to reproduce the results in Athanasopoulos et al (2011). This was successful for the mean absolute percentage error of the naive forecasts, giving assurance that the data extract-transform-load was succesful (a naive forecast is that the value next period will be the same as last period for non seasonal data, or the the last period one cycle previously for seasonal data). One step to facilitate the testing was the
forecast_comp function which is now shipped with the
Tcomp package and designed to facilitate rule of thumb comparison of forecast results for a selection of standard forecast methods:
- auto-regression integrate moving average (ARIMA);
- exponential smoothing state space;
- theta method; and
- naive .
Here is that convenience function in action. It returns a data frame, the first four rows of which are the mean absolute percentage error (MAPE) for different forecast horizons (columns), and the final four rows are mean absolute scaled error (MASE). For both MAPE and MASE, smaller is better.
forecast_comp is a generic function that works with all
Mdata S3 class objects, it also works with objects from Hyndman’s
More comprehensive use
The main purpose of large collections of time series data like this is rigorous study of different approaches to forecasting. Here’s a simple taster of what I hope the
Tcomp package will be used for. In this section, I assess the relative accuracy of several forecasting methods: ARIMA, ETS, combination of ARIMA and ETS, naive, and Theta, using quarterly tourism data. Here’s the summary results (Theta wins, so long as the forecasting horizon of interest is four or more quarters):
Notice how much easier it is to predict four quarters ahead rather than six or eight.
“It’s tough to make predictions, particularly about the future.”
(attributed to Yogi Berra amongst others but it’s not sure who really said this first)
We see that, perhaps a little depressingly, it’s hard to beat the naive forecast that next quarter’s result will be the same as a year ago; or the simple theta method. The naive method is particularly effective for one cycle after the last data point (ie horizon equals four or eight, in this instance). Ho hum, c’est la vie. The average of the ETS and ARIMA methods performs better than either individually, consistent with previous findings about the value of ensemble forecasts. Many questions are begged by this set of results, which could be explored fruitfully with this and other real-life data.
Here’s the code for this mini-competition. It shows how to use the
tourism data and make the most of parallel processing to speed things up.