In my immediate previous post, I highlighted some of the motivating inspirations for creating a particular platform in 2007: the Milwaukee County Community Justice Council. The desire was to examine our criminal justice system and to invite external partnerships to help us identify efforts from different systems that might help inform our desire to improve the Milwaukee “product” of justice.

Hidden in the request for help was a perhaps naive presumption that some system somewhere was “doing justice the right way”—such that our need was to discover it, adopt or adapt it, and make it our own. The reality in 2007 (and today) is that there are approximately 2,330 state-level criminal justice systems representing diverse populations and operating in myriad legal and cultural systems sometimes very different from Milwaukee. All are presumably trying in good faith to justly serve their particular communities. And while many of the dynamics of “the criminal justice system” are similar everywhere in the United States, you will find important nuances just by traveling outside your home county.

In all events, given the complexity and deeply structural challenges of the American legal system, how do you objectively identify a problem in your ecosystem, assess what might fix the problem, implement a reform, measure the impact of the effort, and then demonstrate a narrative of progress? Such a process comes with abundant loaded assumptions, each one challenging enough to derail any effort at reform (which helps explain why so few jurisdictions even try).

But perhaps the biggest issue confronting reform-minded practitioners can be distilled to this essence: the challenge of adequately and accurately capturing meaningful data.

Unifying all criminal justice systems in the past and no less in the present are grossly inadequate information management systems combined with sparse analytical capacity. One of the guiding principles adopted early in the Milwaukee reform process is captured by the phrase “You can’t effectively change what you don’t effectively measure,” and while the information collection process has been revolutionized in a short time, effective analysis remains a challenge to most systems.

The Milwaukee County justice ecosystem circa 2006 was predominately an analog, paper-based system. If you practiced criminal law in the 1960s and returned for a day as late as 2010, you would still recognize all the processes and procedures required to represent a client or prosecute a case. Data and information processing systems (including software at the later date) were a hodgepodge of commercial and proprietary products, with the police departments, sheriff’s office, prosecutors, courts, and corrections system all using different means to capture and store the information needed in their respective sphere, but rarely with any interoperability with other agencies. In a pre-Cloud, pre-AI world, a researcher needed to physically enter the space where the work unfolded to even attempt to capture data, and nonetheless he or she would be disappointed in the quality of the information.

Despite the obstacles, in 2005 the Milwaukee County District Attorney’s Office opened itself to outside, independent researchers. This occurred when the Vera Institute of Justice reached out and asked to be allowed to enter the complicated and risk-averse space of the elected prosecutor. It did so on a topic that was and is considered a third rail of police and prosecution controversy: race.