Introduction

The basic procedures and systems programs use to collect and analyze outcomes data about their client services can be relatively simple conceptually. Once a program identifies the specific types of data it wants to use (e.g., financial and non-financial benefits provided clients in a case), it must implement systems with the functionalities to:

1. Enable case handlers to easily enter these data into their program’s case management systems (CMS's) at the time they close a case

2. Aggregate these data and generate a range of reports that present data in formats that enable the program to effectively analyze and present data for a variety of purposes (e.g., assessing success in accomplishing mission and goals, developing strategies and allocating resources, and fundraising).

Overview of Possible Methods for Collecting Outcome Data on Limited Services

Limited service cases make up approximately three-quarters of the cases that Legal Services Corporation (LSC) grantees close in a given year. Such cases are closed with Case Service Report (CSR) Handbook closure codes A (“counsel and advice”) and B (“limited action”).

Despite their prevalence, little is currently known about the outcomes of these types of cases. In extended service cases, grantees usually know what the outcome of the case is because they are involved in the case and have contact with the client for an extended period of time. By contrast, in limited service cases, the interaction with clients is, by definition, limited, involving the provision of legal advice or taking some limited action on behalf of a client in a single interaction. Thus, grantees do not have the opportunity to learn about what happens with the client’s case.

LSC is committed to helping grantees think through ways they can address the challenge of collecting data on the outcomes of limited service cases. This document provides a high-level discussion of some potential methods for collecting outcomes data based on a scan of strategies being employed in the legal services community and in other industries.

Four methods are discussed:

• Follow-up phone calls

• Text message surveys

• Online surveys

• Tracking administrative records

Procedures and Data Management Systems for Collecting Outcomes Data

Several steps are required for programs to effectively and efficiently collect outcomes data.

1. Programs must first identify the types of outcomes data they want to use and therefore collect.

2. Case handlers must identify and record in the case file the results of individual extended services cases when they close the case. For example, the case handlers identify whether the client obtained a civil protection order, avoided eviction, obtained (or retained) SSI benefits, and so on. Since this is considered a case handling “best practice,” many programs require their case handlers to document this information in the case file, even if the program does not use a systematic outcomes data system. Learn more about the types of outcome measures used by many legal services programs.
   
   Programs typically collect these data only for extended services cases (LSC CSR case closing codes F-L), not for limited services cases (LSC CSR case closing codes A and B). It is relatively easy and inexpensive to capture outcomes data for extended services cases because case handlers know when they close a case, what the outcome(s) of a particular case is or are and can record this information on simple forms or enter it into specified fields in the CMS. In contrast, it is costly to identify the results of limited services cases—for example, follow-up services or similar means are required to obtain this information—and the ultimate outcomes of these cases may be determined by factors unrelated to the program’s representation (e.g., to what extent and in what ways did the client act on the advice provided by the program).

3. Programs must incorporate these functionalities into their CMS's:
   1. Enable case handlers or other staff to readily input into the CMSs (or similar data bases) the outcomes of extended services cases. See the CMS form that a CLAS case handler completes at case closing to provide financial and non-financial outcomes data for a Consumer case.
      
      BRLS case handlers do not directly enter case outcomes data into the CMS.  Instead, they complete a case closing memorandum which capturesthis information; the information is transferred into the CMS later. See the case closing memorandum form that identifies the outcomes data BRLS case handlers document at case closing.

1. 2. Capture that data in fields related to other CMS fields, so that the outcomes can be linked to type of case, level of service, and other key fields.
   3. Provide the efficient transfer of these data into the system(s) that will produce outcomes data reports.

“Off-the-Shelf” Versus Program-Specific Systems for Collecting Outcomes Data

Some programs use—or adapt—their vendor-provided case management system to collect and store outcomes data. The vendors of the different CMSs used by the large majority of LSC grantees (e.g., LegalServer, Kemps Case Works, Legal Files, Pika) have informed LSC that their systems have the necessary functionalities for collecting outcomes data or that these capacities can be incorporated into systems. Contact the vendor of your program’s CMS for more information.

Other programs develop and implement tailored CMS's to collect and store outcomes data. Learn more about the system CLAS developed and implemented to meet its particular needs.

Systems for Developing Analytical Reports with Outcomes Data

1. Once they have the CMS's—or other database systems—for collecting outcomes data, programs need to develop and implement system(s) with the capacity for aggregating these data and generating the types of outcomes data reports that will meet their particular needs.

2. Programs do not need sophisticated statistical and qualitative data analysis packages to generate useful and effective outcomes data reports. (Click here and here for information about some of those packages.) Instead, they can use database systems that are already used by many legal services programs.

For example, BRLS uses Excel to generate a variety of outcomes data reports, while CLAS uses Crystal Reports to compile data and generate a variety of reports with outcomes data. Learn more about reports that programs generate to analyze the results of their client services on a variety of dimensions.