(Dr. David Robinson is an economist. He wrote and taught Laurentian’s Cost-Benefit Analysis course)
This week Tony Cecutti, the General Manager of Infrastructure Services for the City of Greater Sudbury asked council spend $80 million on a section of Maley Drive. To support his recommendation he presented to council a document called “Cost Benefit Analysis of Maley Drive Extension- Phase 1, prepared by AECOM, October 29, 2015; in support of application for federal funding under the Build Canada Fund.”
The study fails to make the case that the project is worth doing. The fact that Mr Cecutti presented it to council calls into question his judgement and his understanding of the economics of infrastructure projects.
What The Study Says It Is
There are three points to notice just in the title. First, the document is called a cost benefit analysis. Second, it was prepared to support an application for money. Third, the report deals with “Phase 1”. Before we get onto the problems inside the “study”, it is worth looking at what the document claims it is.
A Cost Benefit Analysis
A Cost Benefit Analysis lays out all the costs and benefits of a project and then assesses whether we can be sure the benefits are sufficiently larger than the costs. Everything depends on estimating the costs and the benefits accurately. In the “Statement of Qualifications and Limitations” for the report, however, the Consultant makes NO guarantees or warranties whatsoever, whether express or implied, with respect to the Report, the Information or any part thereof.” The reason given is that, “The information, data, recommendations and conclusions contained in the Report … may be based on information provided to Consultant which has not been independently verified.” What this appears to mean is that the costs and probably the impact on traffic flows have been estimated by the City. The AECOM report does not provide any independent data. Basically the consultant is saying “I did some calculations based on the numbers Tony gave me.” That would make it a very quick and very cheap, but not a very useful document.
And what if the numbers are not reliable? A Cost Benefit Analyst generally does what is called a “sensitivity analysis.” Here is an example. What if the project costs 20% more than the engineers estimated? Research tells us that could happen, so a good analyst does the calculation with higher costs and presents that result as well. That way a decision-makers can decide if the project is too risky. The analyst might also test the case of a 20% reduction in cost, a 50% reduction in benefits and even a higher interest rate. A robust project is worth doing even if things turn out badly. A cost benefit analysis without a sensitivity analysis is not generally acceptable. The study presented to council does not include a sensitivity analysis.
Or Maybe Not A Cost-Benefit Analysis?
A bit later the document says, “The purpose of this brief is to assess the economic feasibility of the Maley Drive Extension Phase I project,” and in the conclusion the consultant states, ”It is important to keep in mind that these positive results represent only the strict economic test of whether the project is feasible.”
There is a certain amount of confusion here. In fact, a financial feasibility study deals with whether the organization can pay for the project. The benefit of fixing your roof may be very big, but if you are broke, unemployed and have no credit, it simply isn’t feasible. The city didn’t need to pay a consultant to determine whether or not the project was feasible. Professional Cost Benefit analysts, however, know the difference between a cost benefit study and a financial feasibility study. Confusing the two tells us either that the consultant is not well trained or that the terms of reference were badly framed.
There are questions about financial feasibility that the city could ask. For example, council could ask if it is feasible to maintain social services, repair the existing roads, and complete the Maley drive extension while limiting tax increases to 1%. That is not the kind of question that the consultant was asked.
Maybe Not An Unbiased Cost Benefit Analysis?
The purpose of the document is part of the title: “prepared by AECOM, October 29, 2015; in support of application for federal funding under the Build Canada Fund.” – which means that it was intended to be a lobbying document, not a real analysis. Can you imagine Tony Cecutti asking for a study that would say his project was not worth doing? The title of the document tells us that this is not an independent analysis. The qualifying statements emphasize the same point.
Council is entitled to lobby the federal government and to make a strong case for funding. Council should not confuse a document prepared explicitly for the purpose of lobbying with a cost benefit analysis designed to help them make up their minds. That is like only listening to the defense lawyer in a murder trial.
Maybe not Complete Cost Benefit Analysis?
Finally, notice that the report is an assessment of Phase !. Phase I of the project is a combination of new road and reconstruction/rehabilitation of existing road which extends from the College Boreal entrance to 300 metres east of Lansing Avenue. There is some complicated and expensive work needed to connect the bypass to Falconbridge road that is not included in the project.
Or Maybe Not Well Done Cost Benefit Analysis?
The main benefits considered are travel time savings and vehicle operating cost savings. The Maley drive extension was estimated to save 457 Vehicle Hours Traveled (VHT) per peak hour. These savings are assumed to last until 2048. The major problem with this forecast is that a great deal of research has shown that the gains would be temporary.
Adding more roads decreases the time-cost of travel. When a price falls the amount demanded rises. The number and lengths of trips is so sensitive to the time cost that trips increase until the time cost reaches its old level according to the research. You get more trips with the same average time spent on each trip. In other words, VHT increases instead of decreases when you build a new road. If that is the case, the benefit estimates are simply wrong and the entire study is useless.
If you don’t know whether to trust the static forecasts of the traffic model or the long term results from traffic researchers, and if you don’t want to just flip a coin, try splitting the difference. Discount the benefits given in the study by 50%. If you do it cuts the estimated benefit-cost ratio in half, to 1.36, which is actually rather low for such a contentious project.
If you then take almost certain overruns into account your benefit cost ration falls to about 1 – which is to say the project will provide no net benefit. Perhaps that is why the consultant did not do the sensitivity analyses.
Lets assume that there will be savings and think about how they are distributed.
The savings in auto operating costs used in the study are estimated to be $1.15 million per year. For trucks, an operating cost per kilometer of $0.49 was used. The savings in truck operating costs are approximately $360,000 per year. These benefits (assuming you believe the number of hours saved) go to a small fraction of the population: to truck owners who operate in the Northeast of the City, and residents of the areas that will temporarily experience reduced congestion.
In Cost Benefit analysis all benefits and costs are included , no matter who gets them. I have argued that most of the benefits to truck owners will be captured by the mining companies when they renegotiate their trucking fees. In the end the rich owners of Vale will gain more than anyone in Sudbury. These benefits should still be counted.
There are reasons we look carefully at the distribution of benefits, however. The first is about justice. It makes sense that the people who benefit most should pay most. The second reason is political: It might be hard to convince people in the south end that they should pay to save time for people in the northeast.
The geography of the benefits presents a genuine political problem. The biggest winners will be people who live Garson and farther Northwest and who commute to the south or west to work. People living north of Lasalle in New Sudbury will gain less, but there are more of them.
There will be some gains to drivers using the Kingsway, some gains to drivers coming south on municipal road 80. The gains will be temporary, of course. There will be additional costs for drivers who currently use Notre Dame, and these will be permanent.
The project will do almost nothing for anyone south of the Kingsway and Elm except raise their taxes. In the long term the project will encourage development in the Northeast, and generally discourage increased density in the downtowns and in the south end. This is poor land use planning and will increase the overall cost of running the city.
The report claims that “greenhouse gas emissions savings amount to approximately $218,000 per year”. The report estimates the total benefit to be $6.6 million. This cannot be true; the total vehicle travel time will rise, not fall. Rather than decreasing greenhouse gasses, the Maley Drive Extension will increase them.
Even if you don’t believe that making more room for cars will increase driving and greenhouse gasses, there are good reasons to think that other mechanisms will reduce baseline fuel consumption. There will be carbon taxes. That will certainly reduce driving somewhat. Fuel efficiency is rising. That will also reduce the savings due to the Maley Drive Extension. There will be a massive increase in electrical vehicle use. And of course the world is likely to cut back on all use of fossil fuels. If emissions are going to fall anyway, the Maley Drive Extension would be cutting a piece out of a smaller pie. The benefit claimed will have to be reduced by at least 50%.
The estimate of greenhouse gas savings is simply nonsense. It may also be an attempt to “greenwash” a project that is not even remotely climate-friendly. This section is an insult to city councilors and an embarrassment to the economics profession.
What is the rate of return?
There is another flaw in the study. The author uses a discount rate of 3.5%. That is basically an estimate of the rate of return that the city can get on all other projects. It is a low rate but it could be justified on the grounds that it represents the cost of borrowing for the city.
The author then calculates that the rate of return on the entire Maley Drive project is 13.6%. The author is arguing that the city can borrow at 3.5% and invest in the Maley Drive Extension and earn 13.6% for a net return of 10.1%.
This is where we have to ask “How much can we earn on other projects?” If we can earn a similar return on other projects and we don’t do those projects because we money tied up in the Maley project, the the real cost is not 3.5% but 13.6%. At that rate of return the net present value of the project is much smaller and the economic return is zero.
Overall the study is badly done and Tony Cecutti, the General Manager of Infrastructure Services for the City has demonstrated that he does not know how to get a proper cost-benefit analysis done. He should be very embarrassed to be seen with it in public. And Council should be very worried about how projects are assessed at the city.