The Federal Circuit and a few other counterexamples notwithstanding, American courts are not substantively specialized.  By and large, the American judge is thus a generalist.  For better or worse, our judiciary seems to be holding out against the  pressures toward specialization that have so marked the contemporary legal and medical professions. 

Is this a good thing?  In the law review literature, there are plenty of calls for the creation of this or that new specialized court.  Certainly, specialization leads to quicker and more efficient decisionmaking.  But should we expect the specialized judge also to render decisions that are substantively better?

This is the question that lies at the heart of Chad Oldfather’s new article, “Judging, Expertise, and the Rule of Law.” 

Chad’s analysis rests on the burgeoning psychological literature on expertise.  His conclusion is that the claims about the ability of specialist judges to develop and successfully use a higher level of substantive expertise are probably much overstated.  Chad’s analysis has many nuances.  To highlight just one dimension that I found especially intriguing, he considers the likelihood that specialist judges are better able to make accurate predictions regarding the consequences of ruling one way or another.  He suggests that the answer depends on the certain personal qualities of the judge:

A final body of research that bears consideration is Philip Tetlock’s work on expert political judgment.  Tetlock’s project was, in effect, to attempt to isolate the components of “good political judgment,” and he did so primarily by getting experts to make predictions about future states of the world and then, over time, assessing whether those predictions came true. This is, as he readily acknowledges, a slippery task, in large part because disagreements turn on more than ascertainable factual claims. Instead, they involve “hard-to-refute counterfactual claims about what would have happened if we had taken different policy paths and on impossible-to-refute moral claims about the types of people we should aspire to be—all claims that partisans can use to fortify their positions against falsification.” While Tetlock acknowledges the impossibility of eradicating all the subjectivity from the inquiry, he maintains that it is possible to assess them by reference both to the correspondence between experts’ predictions and reality and the coherence of the processes by which they approach the task.

Tetlock concluded that, overall, experts’ judgment was not good. As he puts it, “Humanity barely bests the chimp.”  “[V]ariation in forecasting skill is roughly normally distributed, with means hovering not much above chance and slightly below case-specific extrapolation algorithms.”  Tetlock found that demographic and life-history factors bore very little relationship to forecasting success. “It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists, or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience in their chosen line of work.”  Nor did ideology or other factors relating to worldview or disposition correlate with forecasting success.

What did matter, Tetlock found, was the process by which experts approached the predictive task. Drawing on Isaiah Berlin’s famous fox-hedgehog distinction, he found a “dimension [that] did what none of the measures of professional background could do: distinguish observers of the contemporary scene with superior forecasting records, across regions, topics, and time.”

“Low scorers look like hedgehogs: thinkers who ‘know one big thing,’ aggressively extend the explanatory reach of that one big thing into new domains, display bristly impatience with those who ‘do not get it,’ and express considerable confidence that they are already pretty proficient forecasters, at least in the long term. High scorers look like foxes: thinkers who know many small things (tricks of their trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible ‘ad hocery’ that require stitching together diverse sources of information, and are rather diffident about their own forecasting prowess, and … rather dubious that the cloudlike subject of politics can be the object of clocklike science.”

Looking beyond the simple fox-hedgehog divide, Tetlock found that “hedgehog extremists making long-term predictions in their domains of expertise” were the worst performers.  The best were “foxes making short-term predictions within their domains of expertise.”  In all, he concludes, “the performance gap between foxes and hedgehogs … is statistically reliable, but the size of the gap is moderated by at least three other variables: extremism, expertise, and forecasting horizon.”

Tetlock reasons that these results are consistent with other research on cognition. Hedgehogs, he suggests, “bear a strong family resemblance to high scorers on personality scales designed to measure needs for closure and structure—the type of people who have been shown in experimental research to be more likely to trivialize evidence that undercuts their preconceptions and to embrace evidence that reinforces their preconceptions.”  Hedgehog experts perform especially poorly because their expertise better equips them to discount contrary evidence as well as to characterize evidence as bolstering their beliefs. Extremism magnifies the effect.  Meanwhile, foxes are more self-critical and more inclined to anticipate criticism from others, and consequently more likely to give due consideration to all information that bears on their position.  For foxes, expertise pays dividends, since it enhances their ability to assess all information.  (41-43)

In addition to questioning the advantages claimed for a specialized judiciary, Chad also discusses the case for generalists.  He identifies rule-of-law values as the most important justification for our generalist traditions:

The strongest arguments for a generalist judiciary seem to require the acceptance of a certain conception of law, and in turn of the judicial role. Stated simply, the conception of law that seems to underlie the generalist judiciary is one in which law strives to be something of a common language. There are at least two ideas at work here. One is that the expertise relevant to judging exists at a similarly broad level. On that view, the key is not expertise in or familiarity with the particulars of, say, tax law that matters, but rather an advanced ability to deploy the tools of legal analysis. The second idea extends beyond the act of judging to the impact of that act on the relationship between the law and those who are governed by it. It goes beyond the notion that we have a government of laws and not of men to the suggestion that we must have a government of laws that can be understood and adhered to by more-or-less ordinary people. The generalist judiciary can further these goals not only by preventing the sort of balkanization that is likely to occur if separate judiciaries have responsibility for their own areas of law, but also by serving as a more general barrier to needless technicality and complexity in the law. The presence of the generalist effectively requires specialist lawyers to translate their arguments into the common language of the generalist, which in turn facilitates the generality of law. This barrier provided by the generalist is hardly impermeable. It is not difficult to find examples of needless technicality and complexity in the law created by generalists. Nonetheless, one can easily imagine generalism serving as an antidote to some of the more severe pathologies that might afflict the law under a regime based more on specialization.  (48-49)

Chad’s very fox-like article is forthcoming in the Washington University Law Review.