A/B Testing for Formative Evaluation

Easy version (IBE)

Note: in this paid post, I'm going to introduce a useful program evaluation method, explain when you would use it, work an example, and give you code you can copy to do the analysis yourself. I’m starting paid posts for a couple of reasons: 1) because certain posts are highly technical and take me a long time to write, and 2) I want to support Substack in its current form as an ad-free platform. I am still going to be writing free posts as well.

Bayesian A/B Testing in Formative Evaluation

Developing effective programs on the fly is part of formative evaluation. One important approach to refining these programs is through Bayesian A/B testing. This method allows program developers to experiment with alternative ideas, such as new treatment methods, before full-scale implementation. By integrating Bayesian A/B testing into the formative evaluation process, evaluators can help organizations make evidence-based decisions. Testing new ideas before full implementation also helps mitigate risks associated with unproven treatments or strategies, ensuring that only the most effective solutions are pursued.