Introduction

Outcomes are the results of a program’s services for its clients or community. They measure changes in clients’ circumstances, conditions and attitudes that result from the program’s work or changes in organizations and institutions directly affecting clients’ lives.

Although legal services providers collect and analyze a range of outcomes data, the outcomes data most widely used by legal services programs are non-financial and financial outcomes.

These data can be most effectively and efficiently collected and analyzed when they correspond to or are based on the LSC Case Service Report system “problem code” areas (e.g., Consumer, Housing, Family, Employment) and are collected only for extended services cases (LSC CSR case closing codes F-L), not for limited services cases (LSC CSR case closing codes A and B).

State funders (e.g., IOLTA programs) in Maryland, New York, Texas, and Virginia require their grantees to collect and report similar non-financial and financial outcomes data. Some programs collect these data even if they are not required to do so by state funders. Also, some programs combine different types of outcomes data to develop “higher-level” outcomes.

The following provides information about the Virginia IOLTA outcome measures used by legal services providers in the state; the outcome measures used by Cleveland Legal Aid Society; examples of “success rate” measures used by programs; and other outcomes data that are important but hard to collect.

The Virginia IOLTA Model for Using Outcome Measures

The Virginia legal aid programs collect and report the non-financial and financial benefits they provide clients through their extended services cases to the Legal Services Corporation of Virginia (LSCV), the state funder. The LSCV system is very similar to the outcomes measures systems used by the IOLTA funders in New York and Texas. (All three systems were developed with assistance from The Resource for Great Programs.)

1. Non-financial benefits. Programs collect and report outcomes for cases closed in 12 areas that correspond to the LSC CSR legal problem categories. See the full list.

Some legal problem categories have more outcome options than others. Here are some examples:

Consumer/Finance category—20 different possible outcomes, such as:

• “Obtained federal bankruptcy protection”
• “Stopped or reduced debt collection activity”

Family category—32 different possible outcomes, such as:

• “Obtained or maintained custody of children”
• “Obtained protection from domestic violence”
• “Obtained, preserved, or increased child support”

Information is also provided for the “number of persons directly affected” for each of these outcomes.

2. Financial benefits. Virginia’s legal aid programs collect and report two categories of financial outcomes data.
   • Direct dollar benefits for clients. These are for “affirmative dollar awards” to clients. Affirmative benefits are reported for the amounts of “lump sum awards/settlements” and “monthly benefits.” Specific financial benefits categories include “Social Security/SSI,” “Child Support,” and “Affirmative consumer judgments.” See the full list of the affirmative dollar award categories.
   • Dollar savings for clients. These outcomes are for “amount of dollar savings achieved for clients through judgments or payments avoided,” such as “Defensive Consumer Law Matters” (e.g., bankruptcy, garnishment). See the full list of the affirmative dollar award categories.

Cleveland Legal Aid Society’s Use of “Higher Level” Outcome Measures

Cleveland Legal Aid Society collects and analyzes both non-financial and financial outcomes data as part of the system it developed to meets its needs. However, Cleveland collects fewer categories of outcomes data than those used in the IOLTA models. In addition, Cleveland combines outcomes data for different case types to develop “higher-level” outcomes that correspond to the program’s strategic goals:

1. Improving safety and health
2. Promoting education and economic stability
3. Securing decent, affordable housing

1. Non-financial benefits. These describe the result of the case or matter, categorized by the LSC CSR problem code. For example, the outcomes for the Consumer/Finance area include “obtained monetary claim,” “reduced/avoided debt,” “increased assets,” and “obtained/restored utilities.” See the full list of the non-financial outcomes data that Cleveland collects.
2. Financial outcomes. These outcomes include data regarding changes in the amounts of a client’s monthly income, assets, or debts that can be attributed to CLAS’s work. The two basic questions Cleveland uses when calculating financial data are: “If Legal Aid had not been involved, what would the client’s [value/amount of income/asset/debt] be at the time the case was closed?” and “What was the client’s [value/amount of income/asset/debt] at the time the case was closed?” See the full list of the financial outcomes data that Cleveland collects.

“Success Rates” Data

Some programs collect data regarding the success rates of their case work. These are based on whether the program’s representation achieved the goal(s) established by the client and advocate when the case was opened. Some programs provide three options to indicate the extent to which the goal(s) was or were achieved. For example, BRLS uses: “Clients' Goals Achieved,” “Clients' Goals Partially Achieved,” and “Clients' Goals Not Achieved.” The case handlers select the option based on their judgments of the case results.

• Cleveland Legal Aid uses a system in which advocates indicate for each outcome whether that outcome was achieved or not (or whether it was not applicable for that case). In that way, Cleveland Legal Aid is able to calculate its success rates (for example, percentage of evictions prevented).  In addition, Cleveland Legal Aid gathers feedback from clients about these outcomes and compares those responses to staff responses to ensure accuracy of the data.

Important but Hard-to-Collect Outcomes Measures

Other outcomes are important for legal services programs. Examples include:

• Changes in clients’ assessments of their well-being or sense of empowerment
• Changes in the accessibility and responsiveness to clients of the courts or other public entities

However, these data are more difficult to collect and analyze than the outcomes measures discussed above. First, outcomes regarding clients’ “well-being” and the “responsiveness” of public institutions are less concrete and therefore more difficult to define than case outcomes such as “preventing an eviction,” “reducing debt,” or “obtaining SSI benefits.” Second, outcomes data from extended services cases can be readily identified and entered into the case management system when the case is closed. In contrast, data regarding clients’ well-being or the responsiveness of public institutions require more expensive and time-consuming data collection activities (e.g., surveys, interviews, administrative data from public agencies). Programs can incorporate the data they collect into reports that enable them to show and analyze the results of their client services on a variety of dimensions. Read more about this.