Needs Assessment is a Type of Evaluation

Why and what type?

Needs assessment is a process employed to identify and prioritize the needs of a community or population. It’s also usually taught to practitioners as an activity distinct from evaluation. The typical explanation for the distinction goes something like this: evaluation is for when you are trying to make judgments about a well-defined thing (like a program), but needs assessment is for when you trying to investigate the needs of a more open system (like a neighborhood). This is not the sort of definition you would find in a textbook, but it is the common sense in practitioner circles.

First, a Contrarian Perspective

I side with Scriven in arguing that needs assessment is the process for determining what things are essential to the functioning of a system (e.g. an organization or community) and the extent to which these needs are being being met using available resources. Evaluation is about making judgments based on criteria to determine merit or worth of an evaluand. Criteria are key for evaluation because they are standards used to anchor value judgments: no criteria, no value judgments, no evaluation.

Now that we have this straight, we can see why needs assessments are actually not distinct from evaluation. In a needs assessment, the evaluand is the community or population, and the criteria are the identified needs. We compare the evaluand (e.g., the community) against these criteria to ascertain the extent to which needs are being met. Needs assessment obviously involves evaluative judgments because calling something a “need” is an assertion of its high value. Preferably, those evaluative judgments are made using non-arbitrary criteria.

The next hurdle to overcome is the idea that needs assessments differ from evaluations in that evaluations know what they are looking for and needs assessments must remain open to learning about unforeseen needs. However, this openness to new criteria (in this case, criteria related to needs) is also characteristic of goal-free evaluation. To paraphrase Youker (2024) the process of goal-free evaluation is 1) identify the relevant effects to examine while ignoring stated goals and objectives, 2) identify what occurred, 3) decide if if this can be attributed to the evaluand, and 4) determine whether the effects are positive, negative, or neutral.¹

GFE is distinct from classical program evaluation in that it does not have predetermined objects of study. The evaluator independently determines what the evaluand is doing and whether this is good or bad. In a needs assessment, the evaluator remains open to learning about new needs and proposes non-arbitrary criteria for judging whether they are met. Of course, many needs assessments are not even goal-free. Often, there is considerable agreement about the topics that the needs assessment will focus on. In these cases, needs assessments resemble classic evaluations even more closely.

What about the fact that needs assessments are generally of larger, open systems (communities, organizations), while evaluations are typically of smaller, closed systems (programs, products, personnel)? The answer is that there is nothing in principle that limits the size of the evaluand; we can evaluate communities, institutions, strategies, and nation states. If this sounds ambitious, then so is needs assessment!

Finally, we have to face up to what I like to call “intervention bias” in evaluation. This is the bias that causes us to think that evaluation is always an evaluation of an intervention. Program evaluation is the evaluation of an intervention, but evaluation is not reducible to program evaluation. Evaluation is a transdiscipline encompassing product evaluation, process evaluation, personnel evaluation, cost benefit analysis, and any other domain in which we seek to determine the value of something. One of the most popular and well-funded forms of evaluation in the world at this moment is AI evaluation, which is the application and development of benchmarks for these rapidly-changing technological systems. There is a lot more to evaluation than interventions. Thus, needs assessment can be a type of evaluation even if the evaluand is not usually a program.

Theoretical Consequences

The consequences of recognizing needs assessment as a type of evaluation are considerable for evaluation theory. Needs assessment is less taught in graduate programs and less written about in journals. Often in practice it is treated atheoretically, as though there were no major choices to be made and little rationale for making them. The result is sterile, aimless needs assessments with obvious logical problems, like treating gaps as needs, adopting a weight-and-sum approach to rank the needs, and failing to distinguish apparent needs from actual needs. I wish I had not seen all of these problems as a practitioner, but I can think of a real example for each of them.

I think a brighter future awaits: if needs assessments are a type of evaluation then we can bring more evaluation theory into needs assessment. Feminist evaluation can become feminist needs assessment, democratic evaluation can become democratic needs assessment, and so forth. Obviously, there is work that has already been accomplished in the theory of needs assessments (more on this in a moment, however), but clarifying the connection between evaluation and needs assessment will enrich the latter substantially.

Photo by Freysteinn G. Jonsson.

Two concepts that cross the bridge from evaluation to needs assessment will be the emphasis on applying plausible criteria and the importance of evaluator independence. Many needs assessments identify “needs” without stating the criteria they used to arrive at that designation. Identifying and articulating criteria is a key product of a needs assessment all on its own, and a product that may be more challenging to create than measuring apparent outcomes. Suppose we state that an organization has a need to limit staff turnover in order fulfill its mission. The process of setting the standard for what counts as high or low turnover is ultimately much more valuable for the organization moving into the future than the diagnosis of an apparent problem in the present. Work like this actually changes organizations.

Evaluator independence follows from the recognition that needs assessments are usually a type of goal free evaluation. If we really want evaluators to “discover” unknown needs, we need to allow them lots of autonomy. Attempting to point the evaluator in directions desired by leadership, limiting the exposure of the evaluator to certain people or places, or stating in advance that the findings will only be presented if they are perceived as “useful” for the current leadership are all ways of hindering the process of discovery, and all pose major problems for needs assessment as well. This is not to say that all needs assessments should be goal-free, but it is to say that we should never pretend to do a goal-free needs assessment if any of the foregoing tactics are in play. Readers of needs assessments should ask whether the evaluator was allowed independence in determining needs or whether they were steered – if we find no explicit statement of evaluator independence, the canary should be considered to be at the bottom of the cage.

Facing the Dominant View

Having laid out a picture of the position that needs assessment is a type of evaluation, we can now directly face the predominant view in the field today, which I think is best exemplified by Dr. Altschuld, one of the coauthors of the article I wrote about last week, and one of the authors of a popular book series on needs assessment. In answer to the question of why he wrote a whole book dedicated to needs assessment, Altschuld responds “At times, needs assessment is confused with evaluation of programs in regard to implementation and outcomes” (p.1).

If this is the motivation for writing a whole book on needs assessment, one would expect a slam-dunk argument for the difference between the two. Yet, the most we get is this paragraph:

“[Needs assessment and evaluation] are related processes. The assessment will be much more on the front (planning) end of a new program or project whereas the evaluation will be to see if the solution is working as planned and to determine whether outcomes were achieved by it. The needs uncovered and the outcomes being assessed have related elements and features” (p.82).

The vagueness of this paragraph startled me almost as much as its narrow view of evaluation. The desideratum for distinguishing a needs assessment from an evaluation here is timing, with needs assessments supposedly happening during planning and evaluation happening after a solution has been implemented. In reality, needs assessments do happen at all phases of a process or program, usually because the evaluand is bigger than a particular program. Likewise, program evaluation usually gets going before the program starts so that baseline data can be collected and formative advice can be provided.

The authors helpfully suggest that, if stakeholders are having trouble with understanding the difference between needs assessment and evaluation - as I suppose I would be if I were one - they should take the Watkins and Guerra (2002) questionnaire that helps distinguish between the two. The version provided in the text suggests that strong agreement with statements like these mean that the client requires a needs assessment:

The task is or has been to…

Make decisions regarding potential utility of a new intervention (course, process, product).

Make long-term recommendations for linking intervention to the organization’s strategic plan.

Ensure that all that the organization does/produces delivers added value for internal/external clients.

Meanwhile, strong agreement with statements like the following mean that the client requires an evaluation:

The task is or has been to…

Determine what interventions should be continued.

Make recommendations about an intervention that is currently being used.

Determine ways to improve efficiency of an intervention.

Once again, we see that timing and nature of the evaluand are the major ways of distinguishing needs assessments and evaluations. Apparently evaluating the potential utility of a new intervention would not be an evaluation. Long term recommendations or ensuring that an organization adds value are also apparently not part of evaluation either. These claims would probably be surprising to most evaluators! Likewise, needs assessment apparently does not help determine what interventions should be continued, make recommendation about those interventions, or make recommendations that could improve efficiency of interventions. On this definition it would be very hard to do a needs assessment of an organization that has several ongoing initiatives, some of which might be related to its key needs.²

A more recent book, Watkins and coauthors’ A Guide to Assessing Needs (2011) gets more explicit, in a section called “Isn’t an evaluation the same thing?”:

“Not really, though there are similarities. Both assessment and evaluation are important to improving results, but they serve different functions. The distinctive processes differ in the perspectives that they apply when collecting information and guiding decisions. Although many of the same tools are applied in both assessments and evaluations, understanding the difference about how the perspectives are translated into practice requires particular attention.

An assessment perspective, which you apply when conducting a needs assessment, collects information that identifies the gaps between the current results and the required or desired results (or needs), and then it appraises those needs for determining priorities and comparing alternative activities that may help improve performance. Hence, this approach to collecting and analyzing information takes place before any decisions are made about what to do, which vendors to use, or even what products are to be expected. Needs assessments are frequently completed in partnership with planning efforts (such as strategic planning or project planning) to define where a group or organization is headed and how it plans to get there.

Core to the distinction between evaluation and needs assessment in this argument is the function of each. It seems like the function of needs assessment is to collect information about gaps between actual and ideal results and then compare some possible ways to ameliorate the gaps. Again, this is consistent with the idea that needs assessment is a form of evaluation - evaluators carry out these functions all the time. Evaluation requires criteria, and criteria are what we use to see gaps between observed and desired performance. Evaluations make recommendations too, and to do this they have to compare some alternative potential courses of actions (at the very least, following or not following the recommendation).

By directly confronting the the mainstream view of needs assessments, I hope I’ve demonstrated that we are looking at a theoretical uniformity that has papered over a lot of interesting conceptual questions. Needs assessments are theory-driven activities that presuppose particular views of evaluation because they are a kind of evaluation. Bringing needs assessment under the category of evaluation clears away a lot of distinctions without a difference, but it also replaces this noise with some signal from evaluation theory.

1. Youker, B. W. (2024). Scriven's Goal-Free Evaluation. Journal of MultiDisciplinary Evaluation, 20(47), 28-31.

2. By the time it appears in the book, the poor questionnaire has already been changed substantially (including the addition of a couple of nonsense items) and its validity is not asserted.