The value of a dollar is a complex thing. The value of a million dollars is a bigger complex thing. When used for philanthropic purposes, what good can $1 million do? There are many considerations to weigh. In order to even begin to answer the titular question of this post, it’s necessary to zoom out and discuss the basic concept of cost per impact and how we can possibly hope to quantify it.
A Simple View
According to Givewell, the cost to save a life by donating to one of its top charities ranges from $3,000-$4,500. I’m a big fan of Givewell’s work. They think as clearly and deeply about impact evaluation for direct service interventions as any other organization I have encountered. They state and consider assumptions; they rely on meta analyses of RCTs (What’s an RCT?); and they understand the prevalence of problems before and after solutions. All of this adds up to the refreshingly clean, buttoned up, and well-considered answer to the question, “What’s the most cost-effective way to save a life?” According to their calculations, a $1m donation made to one of their top charities saves an estimated 222-333 lives that would have been lost due to malaria, vitamin A deficiency (VAD), and other preventable causes. RTNF has donated $25M+ to their top charities in the past 7 years, saving 5,550-8,325 lives. Yay us.
Positive Externalities
It gets better though, because we miss a few important things when we just look at the cost per life saved for an intervention. For example, for every child saved who would have died from malaria, worms, or vitamin A deficiency, tens or hundreds more avoid serious and negative, but non-fatal cases of these ailments. For example, VAD causes cases of blindness, anemia, stunting, and increased risk of severe infections–so Vitamin A Supplementation is also great at reducing rates of those negative outcomes.
I wasn’t able to find good estimates of the additional benefit of avoiding non-fatal cases of VAD or malaria, but I think it would be a very conservative estimate to guess that for every life saved, at least 2-3 more lives may be profoundly improved (by avoiding blindness, severe infections, etc.). Further, if we look at basic outputs, such as how many bed nets or doses of vitamin A supplementation our funding provided, RTNF has in all likelihood helped provide interventions to close to 250,000 individuals for each $1M we’ve granted to these partners.
Criticisms and a More Complex View
As I said, I admire Givewell’s work immensely, and can’t name another organization that has done more for the field of effective altruism, nor has any other organization helped me clarify my thinking about impact and measurement. But while Givewell offers an excellent primer into thinking rigorously about how to estimate the cost of impact, there are some important limitations the field of effective altruism is still grappling with.
That is, while there is a lively and robust conversation about interventions with mature bodies of evidence underpinning them, they are not well-suited to estimate the impact of interventions that don’t have a stack of RCTs underpinning their work. It must be noted that a lack of evidence is not the same as a lack of impact.
Our work in Utah often supports interventions that have not performed their own randomized controlled trials or quasi-experimental studies. These interventions are offered by community organizations that have smaller budgets than our international partners, putting the dream of funding large studies far out of reach. In these cases, we encourage partners to be voracious students of the highest quality research in their field when making important programming decisions. Nevertheless, these programs offer valuable services to members of our own community. So while we don’t have the same level of confidence in our understanding of their impact as we would like, we think these programs are needed in our community. That being said, to add another lens through which we might respond to the main question of this post, we aggregated reported outputs for local partners and found that a total of $1 million led to all the following outputs:
Why would an impactful organization not have rigorous and mind-blowing evidence of profound impact? There are several possible reasons. Here are a few:
Some programs simply can’t be studied through randomization methods (but perhaps can be studied via less rigorous methods); however, just because such an intervention doesn’t translate well to an RCT, doesn’t mean that it is not generating a profound impact. In fact, it doesn’t even mean the program has less of an impact than the most effective, well-studied intervention. What it does mean is that impact is relatively harder to pin down and discuss with clarity. This is often the case with systems-change organizations. Whether a particular systems-focused intervention is more or less cost effective than direct-service interventions often remains–at this point–a theoretical question.
Some programs are too innovative (early) to have a mature body of evidence underpinning their work. That is, today’s innovative ideas may be the next decade’s evidence-based interventions, but studies just haven’t been done yet.
Garnering financial support for measurement and evaluation activities can be challenging, particularly for smaller, local organizations, where donors may not value rigorous evidence and/or avoid supporting administrative costs. Therefore, those most committed to pure effective altruism may be tempted to leave vulnerable members of their own communities behind in favor of supporting only the most cost effective global interventions.
Some humanitarian work delivers goods or outputs that are ends in themselves and may not need to be tied to long-term outcomes to be extremely valuable. For example, we support Shriner’s Wheelchair and Seating Program, which provides retro-fitted wheelchairs to children who need them. Providing a child with mobility is a profoundly life-changing and measurable end in itself. A program working toward the day when no child ever needs a wheelchair again is great, but in the meantime, kids need to get around.
Where We Land
In short, Givewell does excellent work in regards to calculating the cost to save a life for programs with ample rigorous evidence. They also do a great job at comparing the cost efficacy of established programs and making recommendations to the donor who doesn’t want to fuss with interventions in the saddest category of all–programs that have already been shown to be ineffective. Unfortunately, such programs litter the NGO field, and are often just as likely to receive big funding as actually effective programs…
The challenge is that by aiming for certainty, we also certainly miss impactful programs. Relying solely on RCTs could be compared to looking where the street light shines, but just outside the limits of the established literature are programs that are newer and messier, harder to understand with clarity, but possibly in the process of changing the world to a place of less suffering and more opportunity.
So, for today, we have some ideas about what $1 million means to RTNF, but it doesn’t feel like a complete picture yet. Admittedly, we don’t yet know how to bring all of the classes of newer and messier programs under the streetlight, but in our quest for certainty, we are trying not to throw out potential.
As always, we welcome resources and comments from our readers to help us refine our thinking about how to build a stronger understanding of impact for those areas where it remains elusive.