Wilbur Smith Assoc forecasting record slammed in report for Reston VA group

Future Dulles toll road revenues are being used as the security for selling the debt needed to fund a $5 billion Dulles Metrorail branch line from West Falls Church through Reston County to Dulles Airport and out into Loudoun County. Over half the capital cost is proposed to be covered by toll revenue bonds of the Dulles Toll Road issued by the Metropolitan Washington Airports Authority (MWAA) that has a 50 year franchise on the tollroad and is building the rail line.

Half of the rail line is a done deal, financed and under construction but the second half remains to be approved and financed. The WSA/CDMSmith investment grade traffic and revenue study to be released any week now will be key to whether the project proceeds.

A cover letter addressed to Governor Bob McDonnell, FHWA administrator Victor Mendez and top officials of MWAA and the two local counties charges that "WSA has made many very large errors in its forecasts that have been costly to investors, bondholders, governments, and toll road users who have relied on them to approve construction of major toll road projects."

It continues: "we believe that it is imperative that a second, independent T&R forecast be completed by another forecasting group before any decision is made to move forward with the construction of Phase 2 of the Silver (Dulles Rail) Line."

The citizens association says that given Wilbur Smith's record of overestimating revenues and the unexplained discrepancies between the first two WSA studies the various parties to the Dulles project - FHWA, MWAA, the two counties Fairfax and Loudoun as well as Virginia DOT - should defer any further commitments until Wilbur Smith's forecasts have been checked against those of an independent forecaster.

VDOT is called to take the lead in having WSA's results checked out.

It calls on FHWA to develop a process of 'best practices' for traffic and revenue forecasts given that the problem of gross exaggeration of traffic and revenue (T&R) prospects is seen across the whole T&R forecasting business.

Maynard devastating about track record of forecasting

A key finding of the Maynard Report (named after principal author Terry Maynard) is that "optimism bias" is endemic in the toll forecasting business, and that Wilbur Smith which does about half the industry's forecasts is typical of the industry - driven by sponsors to produce the exaggerated forecasts they want. It cites work by Robert Bain, Bent Flyvbjerg, NHCRP, and ourselves but goes into several forecasts with original work.

Bain using work he started at Standard and Poors found that of 100 tollroad forecasts worldwide the average actual traffic was 77% of forecast or an overestimate of about 30% for year 1 and only minor improvement after that.

NCHRP looked at 26 toll road forecasts in the US over the first five years and found worse results.

Writes Maynard: "The atrocious overestimates of revenue by all the forecasters reflected in this data highlight the difficulty in forecasting demand in the 'ramp-up' period of a new toll road. They are inadequate for planning future revenues, financing, and toll rates."

And there is little indication forecasts improve much over time.

On the Dulles Toll Road forecasting WSA starts with an inflated number for current employment in the major county served (900k vs 700k) and then inflates that faster  than others with a higher growth rate (see graph nearby). Similarly on the Knik Arm bridge in Alaska WSA used exaggerated population and employment data, claiming to base them on numbers from a local University institute, which has denied its numbers are used.

Maynard picks up on what we reported (2011-04-11) as huge inconsistency in Dulles forecasting between WSA's modest projections of traffic and revenue for VDOT in 2005 and their highly bullish projections in 2009 for the new concessionaire MWAA. (see graphing of the inconsistency nearby)

Growth prospects declined in that period, and forecasts should have been lowered, not raised. WSA has never explained the basis for their large upward revisions.

"Our examination of vital population and employment input data used by WSA in its forecasts indicates it has almost always used the most optimistic data available to make its forecasts.

"This includes its 2005 and 2009 forecasts for the Dulles Toll Road. In its 2005 study, it utilized population and employment forecasts provided by GMU CRA which characterized official MWCOG forecasts as understating the area’s growth potential. Then, in 2009, it discarded the conclusions of the contractor it hired to do its socio-economic forecast, Linden Street Associates, Inc., of Alexandria, which had discounted the official MWCOG forecasts as overreaching.

"Instead, it used forecast data provided by Woods & Poole, Inc., another local demographic analysis shop, which was much more aggressive. As we have noted, in the one data point from those two studies we were able to check for Fairfax County—data from the US Census Bureau in 2010—the 2005 WSA forecast over-stated county employment by 25% and the 2009 study overstated it by 52%. In both cases, all the population forecasts we examined were within a reasonable five percentage points of the US Census 2010 count."

Detroit

In Detroit Michigan Maynard writes that an independent study by Halcrow found that WSA had used population and employment projections that were "far too high" and traffic was likely to be "about half" of WSA forecasts. He has an appendix commenting on the Detroit-Windsor DRIC/NITC bridge project.

Maynard says some of WSA's numbers are wrong, and that Halcrow's modeling is more convincing because it performs well in backward testing - not attempted by WSA.


NOTE: we're fully open to any response from WSA or any other forecasters. So far no response from WSA to the invitation to defend their work.