Getting it right with ABARE

27 Feb, 2002 10:00 PM

IT'S almost a yearly ritual. Every year ABARE hosts its OUTLOOK conference in February/March to detail where it expects of commodity prices to trend in the coming year.

And every year there seems to be a quote from a farmer saying he "will look at what ABARE is saying and do the opposite".

But is that tag fair, and does the taxpayer get value for money from the Government's official commodity forecaster?

A study of ABARE's forecast accuracy reveals some interesting facts.

When comparing ABARE's March forecasts for the year ahead, for beef, wool and wheat since 1991 against the actual prices received, the average mean error of their forecast was 17pc and the median error was 13pc.

A similar analysis of the Department of Treasury's one year forecast for wages, CPI and GDP change compared to the actual result in eight out of the past 10 years reveals an average mean error of 37pc in the Treasury's forecasts and a median error of 22pc. (see graph for breakdown)

To be fair Treasury is dealing with much smaller numbers, so a half a point difference between its GDP forecast and the actual figure can mean a 50-100pc error, which blows out its mean average.

Nevertheless, both organisations are expected to provide reliable forecasts, so the comparison is relevant.

In the late 1990's Treasury began releasing three and five year forecasts, similar to ABARE, but to date no accuracy data can be analysed.

A comparison of ABARE's three year forecasts to eventual prices achieved by those same commodities reveals the mean error was 19pc and the median error was 18pc.

So are those figures acceptable? In the above analysis it appears that ABARE has a more creditable forecasting record than the department responsible for Australia's purse strings.

Australian National University Research Fellow with the Centre for Bioinformation Science and statistical expert John Maindonald said the methodology used to derive those figures was more important than the figures themselves.

As long as there was no inherent bias in the forecasts, and ABARE was using proper methodologies, then its forecasts remained creditable, he said.

"Any kind of forecast is at best enlightened crystal ball gazing," he said.

"If the forecasts are taking proper account of the inputs we can't complain too much if they sometimes get it wrong."

Using the rubbish in - rubbish out model, ABARE chief commodity analyst Dr Terry Sheales said ABARE was more concerned with the quality and accuracy of its inputs rather than whether its forecasts eventuated 15 months or 35 months later as events changed throughout the forecast period.

The forecasts are based on certain supply and demand characteristics at the time they are calculated, rather than crystal ball gazing, he said, emphasising that the numbers ABARE produces were forecasts not predictions.

Predictions are based on definitive numbers. Forecasts look at all factors of supply and demand at the time which are factored in to determine the likely price outcome.

That's not to say that a computer spits out the answer each time without any human input, as analysts can modify certain effects and forecasts.

And it can get caught out. ABARE can't predict BSE outbreaks in Japan or droughts in Western Australia four months before the financial year if its forecasting begins.

It also uses a set of assumptions, such as the value of the Australian dollar and world economic growth, to determine its final figures. If it gets those wrong, then the whole forecast is skewed.

"We don't expect things to run out exactly the way we forecast them to be, because they are conditional on the information available at the time," Dr Sheales said.

But there are cases when ABARE has got forecasts wrong, when it could have been expected to table more accurate figures.

It was heavily criticised for failing to factor in the impact of the Asian crisis on Australian commodities in the late 1990's.

Although the effects of the crisis began to show through early in the year, ABARE stuck to its higher forecasts in its February predictions, and came unstuck as a result.

It did predict the effects of dairy deregulation right though, despite staunch industry criticism of its figures.

An analysis of whether ABARE picks its trends accurately, rather than its raw figures also provides some interesting facts.

On average it only gets the price trend right 50 per cent of the time, based on the figures gained from the earlier analysis. However, on a number of occasions (18pc) it was only slightly out, tipping a rise/fall of 10pc or less when the opposite rise/fall of 10pc or less occurred, for example tipping a 4c/kg rise in beef prices when prices fell 4c/kg.

It was very wrong (tipping rises/falls greater than 10pc, when the opposite rise/fall of more than 10pc occurred) 14pc of the time.

Also it was more likely to underestimate than overestimate prices when it picked the correct price trend.

The price and production forecasts released by ABARE regularly create newspaper headlines. "Wool boom to hit 800c/kg" or "Canola production to slump" dominate the bold print and introductions of news stories built around ABARE's forecasts.

But often the text accompanying the forecast and the confidence intervals of the graphs were as important as the raw figures, Dr Sheales argued, as it detailed the upside and downside risks to the expected trends, as well as factors which have been assessed to create it.

Armed with that extra information, people could then make a better decision on the credibility of those forecasts.

Dr Sheales also suggested ABARE's information should also be combined with other information providers such as stock agents or marketing bodies or buyer intelligence before making production or marketing decisions.

"The challenge for those who receive the information is to use it carefully and always use a range of sources," he said.

However few forecasters are prepared to give the long term forecasts produced by ABARE.

"A 15 month outlook is pretty brave," Dr Sheales admitted. " Not many people do that."

Commodity analysts and farm advisers appear to have little confidence in ABARE's predicted price trends.

Commodity analyst Malcolm Bartholomaeus, Callum Downs, said all price forecasts were notoriously inaccurate in agriculture.

"The best price forecast is the price somebody is prepared to pay," he said.

But Mr Bartholomaeus admitted ABARE provided a valuable role with data collection and analysis.

The University of Melbourne's Mackinnon Project senior consultant David Counsell suggested there were better price discovery providers than ABARE, particularly in the wool industry.

Those service providers had sprung-up as wool moved to a free trading environment, and hedge markets began to gain favour.

That meant woolgrowers and buyers alike could profit from accurate price forecasts using the hedge markets, so they were prepared to pay for accurate market intelligence, creating strong demand for service providers, he said.

At the opposite end of the spectrum, livestock producers could only adjust stocking rate or enterprise mix in response to forecasts so there was little demand for those service providers, he said.

Also markets were affected by subjective, emotional drivers which caused prices to trend higher or lower than where their the true value of the market lay, making it hard for organisations such as ABARE to make accurate predictions, he said.

Hasall and Associates senior consultant Ian Foreman said although he had stopped attending the OUTLOOK conference, his company always bought ABARE's forecasts for its farm performance data and to glean ABARE's view on supply and demand fundamentals.

"I use it as an indicator of where the market might by heading, and then analyse it according to the production variables on each farm," he said.

The information needs to be tailored to particular enterprises though. Just because ABARE predicts a lift in the wool market's average doesn't mean that a farmer struggling with his sheep flock will immediately make a profit.

Regular OUTLOOK conference attendee Wesfarmers Landmark's corporate affairs director David Coombes said although the company had cut back the number of staff it sent to the OUTLOOK conference, it regularly used ABARE's forecasts, even in its own five year plan.

"It's the analysis of the market conditions we find the most important," he said. Wesfarmers Landmark also sends a summary of the forecasts to all its branches.


light grey arrow
I'm one of the people who want marijuana to be legalized, some city have been approved it but
light grey arrow
#blueysmegacarshowandcruise2019 10 years on Daniels Ute will be apart of another massive cause.
light grey arrow
Australia's live animal trade is nothing but a blood stained industry that suits those who