Time and Money in Cost-Inclusive Evaluation
Persaud & Yates, 2023
In his foreword to Nadini Persaud and Brian Yates’ 2023 book Cost-Inclusive Evaluation, Michael Scriven wrote:
As evaluation began to take shape as a semi-independent discipline in the mid-20th century, it became clear that the social science or educational research backgrounds of the early contributors were missing a key component. In the real world of evaluating real programs, the calculation of their costs, especially their opportunity costs, was being done without any professional skill. Because an accurate calculation of costs is often the governing—or at least one of the governing—elements in making practical decisions about whether to cancel or proceed with a project, this shortcoming led to an unacceptable defect of an evaluation in practical evaluation.
The idea of including costs in evaluations is intimidating for many evaluators who come from a social science, educational, or medical background. These evaluators may be used to working with many data types but worry that using monetary data crosses a line into economics or accounting. In this week’s newsletter, I want to persuade you that you may be more prepared to engage in cost-inclusive evaluation than you think you are. I propose to do this by taking three tricky concepts in cost-inclusive evaluation and showing you how to approach them. These concepts all involve the relation between time and money. I chose these three because I find they tend to be unintuitive at first and because you won’t get very far in cost-inclusive evaluation without them. However, if you bear with me for a few paragraphs, I think you’ll find that these concepts are within your reach.
Discount Rate
Imagine you park your car somewhere you really shouldn’t have (in front of the driveway of the first station?) and you get the world’s worst parking ticket – a $10,000 fine for stupidity. Now, I have a question for you: would you like to pay that ticket now or in five years? Any rational person, economists argue, will choose to pay the ticket in five years. Here’s the second question: why? Given recent economic events, you might have said “inflation” because you have a hunch that the dollar will be worth less in five years. But let’s imagine that inflation doesn’t occur, would you still prefer the ticket in five years? Certainly. If I knew I had to pay a $10,000 ticket in five years, I might start saving a small amount of every month in a special fund to cover the future cost. This is much less painful than trying to scrounge up money immediately.
Now, let’s try the opposite scenario. Say you win $10,000 as a prize in a contest. Would you prefer that money now or in five years? Again, ignore inflation. You want it now, you say? Why is that rational? You might say that the future is uncertain: perhaps the people who want to give you the prize will run out of money in five years, perhaps you won’t be in a position to spend the money, perhaps a great investment opportunity will come along in the next six months and you’ll miss your window if you don’t take the money now.
If you intuitively understand the two scenarios above, you can understand the discount rate. We just need to take one step further. For the traffic ticket, what extra percentage of the ticket would you agree to pay as a fee if you could defer the cost five years? Two percent? Five percent? For the prize, what percentage would need to be added as a bonus in order for you to accept a deal in which the prize pays out in five years instead of immediately? Five percent? Ten percent? These fees and bonuses are the rates that allow you to translate how much money will be worth to you in the future into how much it is worth to you right now. This future-to-present translation is the purpose of the discount rate, which is sometimes called “the time value of money.” It’s calculated exactly like an interest rate and it eats into the future value of both costs and benefits.
For example, if the discount rate is 2%, a $10,000 cost in year one is discounted by 2%, making its present value $9,804. A cost in year five is discounted more heavily due to compounding, using the formula:
resulting in $9,057. Over time, the present value of costs decreases as they are pushed further into the future. Our rational time preference to delay costs ate about $1k off our expected costs for the program’s fifth year. If you don’t calculate this, you are inflating the program’s costs and making it look more expensive in today’s terms than it really is. If this seems strange, remember that this is not rationally different from preferring to pay a traffic ticket later.1
How do we know the discount rate? Often the treasury rate of the country in which the program is located is used, but this is just one convention. The discount rate is a prediction, after all, and predictions court controversy. To determine what would be a good discount rate for the type of program you are evaluating, look for similar cost-inclusive evaluations and determine their methodology for setting the discount rate. If there are a range of plausible discount rates, choose the lowest and highest, median, and highest, and show how these discount rates affect results. To those of us who are used to looking for a single right answer, consider instead that the US OMB recommends using both 3% and 7% discount rates and comparing the results.
Net Present Value
By itself, understanding the discount rate is not terribly useful. The real leverage comes from being able to sum up all future costs and rewards in terms of today’s money. If we add up all the benefits of a project then subtract its costs, including the cost of the initial investment, all in terms of today’s money, then we get the net present value (NPV) of the project. If we formalize this, net present value is simply:
Where:
ΣPVBenefits is the sum total of the benefits of the project in present value;
ΣPVInvestment is the sum total of the benefits of the project in present value;
ΣPVCosts is the sum total of the benefits of the project in present value.
The reason we call out investments separately in this formula is that investments might be made up front, in which case we don’t need to discount them, or throughout the project, in which case we do need to discount them. Separating investments from operating costs (included in ΣPVCosts) helps with budgeting decisions as well, since operating costs can be cut, but up-front investments can’t.2
If you can calculate the discount rate for each year of costs and benefits, then do some arithmetic, you can calculate “the most widely used cost-analytical methodology in the industry” (p. 75). If NPV is positive, then the project will generate more value than it consumes. If it is negative, then the project will consume more value than it generates. NPV is not a ratio, it’s an amount of money that will be generated. A project with an NPV of $1,000,000 will make $1,000,000 in today’s money.
You might have heard of a similar concept – return on investment (ROI) – which many businesspeople like to use. In evaluation circles, you won’t hear much talk about ROI because it is a blunt instrument compared to NPV. There are a couple of reasons I say this. First, ROI uses undiscounted net costs and net benefits, so it doesn’t include any information about the time value of money. A project that takes 100 years to realize a benefit has the same ROI as one that takes 2 years to realize the same benefit as long as the costs are the same. Second, ROI is a ratio:
Where:
ΣUndiscountedNetProgramBenefits is the sum total of the undiscounted benefits of the project;
Σ Undiscounted Costs is the sum total of the undiscounted costs of the project.
And the final multiplication by 100 yields a percentage, e.g., ROI of 20% if the project has a 20% return. ROI makes for easy mental math if we have a variable amount of money to invest and we expect a certain linear monetary return, as in the stock market. However, it doesn’t do us any favors in thinking about more tangible assets such as we find in the non-financial sector, since these tangible assets come with fixed and variable costs, step costs, benefit ceilings, and lots of other nonlinearity.
Benefit-cost ratios, used in cost-benefit analysis, deal with the first of my problems with ROI but not the second. BCRs are calculated using discounted cash flows, so they include the time value of money. They are calculated like this:
Where all the terms are the same as in the formula for NPV given above. In fact, the only difference from the NPV formula is that the minus sign is replaced with a division sign to give us a ratio like ROI. Conventionally, the BCR is not rendered as a percentage, but rather as a classic ratio in which 1 is the break-even point, such that values greater than 1 indicate positive payoff for the project. Once again, however, I think that value ratios are a little misleading for evaluation because they subtly suggest linear relationships between inputs (costs) and outputs (benefits).
Discounted Payback Period
Metrics like NPV and BCR focus on money and put time in the background. Time is still considered because they use discounted cash flows, but neither of them actually yields a unit of time. If we flip things around a bit and focus on time, we can ask: how much time would it take for the total benefits to recoup the total costs of our project? If we do this naively without discounting, this is called the Payback Period. For example, if I invest $500 in training to learn a new skill and it takes me a week to earn $500 from plying my new trade, then the payback period is one week. For longer term investments, we will want to apply discounting and get the total discounted costs and benefits. Then we want to know how much time it takes for the benefits to arrive at the break-even point.
Here’s the example given by Persaud and Yates in their book (p.79), where we know the annual Net Cash Flows, that is, the value of benefits:
In Year 0 of the program, we invested $100k. In the following year, we brought in about $85k, which was good but not enough to recoup our initial investment. By Year 2, the value of benefits had fully recouped our initial investment. Note that these are all discounted cash flows that reflect the present value of the cash from the perspective of Year 0 when the funds were committed. After Year 2, our Cumulative Net Cash Flows are all positive. The formula for the Discounted Payback Period is:
In this case, the Year before full recovery is “1”, which we hold on to and add to the fraction of 14,284 / 72,560, yielding a Discounted Payback Period of 1.2 Years.
Phrasing the ratio of benefits to costs in terms of time is very helpful for understanding the time horizon at which we expect to see a net positive for a project. Projects with longer discounted payback periods are riskier to undertake. This is important not just for financial investments but also for public programs. While we might want to be a little more patient for public programs, we usually want to create more value than we spend, especially when there are alternatives available that are guaranteed to create net positives immediately (such as simply giving cash to people for whom it has a higher marginal utility).3
Evaluators need to be able to analyze the formal relationships between time and money
To close this newsletter, let me leave you with three scenarios in which being able to formalize the relationship between time and money would be critical to an evaluation:
You are making a recommendation about how to improve a program. Your recommendation is that the program spend more money to upgrade the quality of its services. The program head asks how much money they should reasonably budget to achieve the goals they have set.
At the end of an expensive “building” year, the program’s advisory committee members ask you whether, this year, the benefits of the program were worth the costs. If the operating budget were cut slightly, would the benefits exceed the costs next year?
You are taking part in a strategic planning session using evaluation results you have gathered from the previous several years. The advisory committee asks you to recommend a portfolio of programs to fund that will have a large long-term impacts and that will have an immediate and noticeable positive benefits for the community.
I think that these are all reasonable questions that an evaluator ought to be prepared to answer, even if we don’t always have all the required information at hand. Getting that information also seems like it should usually be a priority.
As with everything in economics, there are disagreements about whether discount rates are even the right idea conceptually. One objection is that we should use shadow prices of goods instead. If you’re interested in that possibility, here’s an example of that line of thinking from environmental economics.
Investments don’t include sunk costs, that is, money spent on things that came before the project that we don’t have the option to un-spend. For example, if a program uses expensive equipment that it already owned prior to the program, we don’t count this as part of the costs or investment, except for any funds required to maintain it in working order.
This footnote is to explain that parenthetical about “giving people cash to people for whom it has a higher marginal utility”. The marginal utility of a dollar refers to the additional utility or satisfaction gained from receiving one more dollar. his reflects the principle of diminishing marginal utility: after a certain amount of money, each additional dollar typically provides less utility than the previous one. While a dollar is still a dollar, the utility you get from it varies greatly. In general, people with less money tend to derive higher marginal utility from each additional dollar because it can meet more immediate and pressing needs. This is the point of Luke 21:1-4.