Service Delivery
Evaluating the Process and Monitoring Outcomes
How are data collection and process and outcome evaluations related to service delivery?
Ultimately, the goal of any social service organization is to improve the results of the target population in some way by providing the right type of services and by providing them in an appropriate and adequate way. Data collection and evaluation are a means by which organizations can assess their progress on these two fronts. Essentially, data are the individual pieces of information that are collected on the activities of interest. Evaluation is the systematic way that data are assembled into a picture of (1) how well an organization is delivering its services and (2) the impact of those services on the target population.
The term "process evaluation" refers to a systematic method of assessing implementation. The goal of process evaluation is to use data to provide a description of how a program (or entire organization) is operating, compared with the manner in which the program was intended to operate. This comparison provides feedback to organizations on how well they are putting their plans into action and points out areas that may need improvement [1]. In short, a process evaluation is geared toward descriptions of programs and the activities that occur within them. Some questions that process evaluations might address include the following: How many clients were served per month? What are the characteristics of those clients? How much staff time is required per client? How long does it take for participants to complete the program [2].
An outcome (or impact) evaluation, on the other hand, provides a picture of the results or effectiveness of a program in achieving its intended goals . Depending upon the goals, questions addressed by an outcome evaluation might include these: Did school performance improve for participating children? Did participants become employed? Did rates of infectious disease drop in the county? Did the rate of teen pregnancy drop in the city? [2].
Outcome and process evaluations are closely related and interconnected because understanding how well a program has been implemented (process evaluation) is vital to interpreting the results of an outcome evaluation. For example, imagine conducting an outcome evaluation of a program intended to improve school readiness for disadvantaged children. We may find that children in the program do no better in early school performance than a comparison group of children who are not in the program. Thus, the outcome evaluation provided evidence that the program is not effective in meeting its goal. A process evaluation examining the manner in which the intended services were delivered might reveal, however, that the program was not fully implemented because targeted children did not receive all of the intended services. With this information, we would likely conclude that the program itself was not a failure, but the implementation was flawed. Thus, we would need to correct the shortcomings in service delivery before we can determine whether the program would be successful.
In short, data collection, and process and outcome evaluations are important to service delivery because they provide vital feedback on how services are being delivered and whether delivery of these services makes any difference in terms of achieving the program's specified goals. Organizations that are oriented around the continual use of data and evaluation to guide their operations employ what has been called Results Based Decision-Making.
Footnotes
[1] 2000 Annual Summary: Effective Prevention Principles and Programs, Rockville, MD: Center for Substance Abuse Prevention, 2000, p. 58. Available at: modelprograms.samhsa.gov (PDF file).
[2] Rossi, P. H., and H. E. Freeman, Evaluation: A Systematic Approach, Thousand Oaks, Calif.: Sage Publications, 1993. To order this textbook on evaluation, visit: www.sagepub.com.
Back to Top