On Testing Legal Skills

Since we’re all in the middle of finals right now, I thought I would share a quick problem, an incongruous message, that’s been bothering me.

Raise your hand if you’ve heard a law professor say something like, “No matter how well you think you know the rule, read it again every time you need to use it.” Almost all of you. Now keep it up if you’ve heard this from more than one professor. Still almost all of you. I agree that this is very good advice; rules change, subtleties escape us on first readings, and sometimes our memory is just wrong.

Now, raise your hand if you’ve heard a law professor explain a test like, “Although this is open book, you won’t have time to look up the rules. You need to know them very well yourself and just use the materials for the few you don’t know.” That’s almost the very definition of how an open-book exam is usually structured.

I’m not really sure the two are compatible.

This Post Has 7 Comments

  1. Sean Samis

    Good catch, Tom. I’ve noticed this, too. Perhaps another sign that “finals” and “real world” are not closely related.

  2. Andrew Golden

    Agreed. It’s even worse when professors give closed-book exams that require you to memorize statutes and rules. At least with open-book exams, you can memorize the gist of a rule and then double-check the nuances. With closed-book exams, you don’t even have that chance.

    Look, I get that there are certain things that you just HAVE to know in a subject. For example, any class dealing with the practical application of criminal law (so, beyond the 1L Crim Law class) is going to require that you can know Terry, Miranda, etc. without thinking. No way around that. But teachers should either teach our memorization of black-letter law or our ability to apply law to facts, not both at the same time. Or, alternatively, require us to memorize only the most basic rules, and then give us copies of the rest. Trust us to determine how much we need to keep in our heads.

  3. Bruce Boyden

    I can only speak for myself, but I’ve never said “Although this is open book, you won’t have time to look up the rules.” What I *have* said is “Although this is open book, you won’t have time to re-read any cases.” In fact, the whole reason I make my exams open book is so that people can look up rules, 3-part tests from cases, etc., to refresh their recollections of the exact wording. Heck, I even do that when I’m teaching. But I don’t allow time on my in-class exams for any more than that.

    I think that’s a lot like practice — you need to be able to spot basic issues, and recall relevant cases/statutes, on your own without having to research every single point. But if the exact wording matters, don’t rely on your memory.

  4. Irene Calboli

    I agree with Prof. Boyden. I also always give open book exams with at least enough time to review the rules and the basics of cases, yet students need to already know the materials very well since they will have no time to “prepare for the exam during the exam.” That said, I think the issue here is a little more complex. I totally agree (and think I also say that in class) with the suggestion that we cannot rely “just” on our memory for any rule, i.e., we need to confirm it. Yet, I also agree with the fact that exams should train our memory, i.e., we need to learn rules by memory and be able to repeat them without to much help (i.e., without books, or just with a quick look at them). We will, of course, forget the rule later on, but we will not forget (hopefully) where the rule is, i.e., the methodololgy, where we should look for it. Also, our memory (and our brain) needs to be trained, and this is exactly what often exams do: they train us, i.e., give us skills (not just knowledge) for our future profession. As Prof. Boyden exactly put it, lawyers need to be able to spot issue quickly and connect them to the relevant laws and cases. In my own legal education (and well before, since elementary school) I had to learn an enormous quantity of things by memory (articles, codes, entire pages of materials, etc.) and be able to repeat them aloud, very often in oral exams (I still remember as a nightmare the 831 articles of the Civil Procedure Code). Memory alone, of course, does not serve much, yet it is most often the basis for more developed reasoning, the sine qua non for building later a good practice. Good luck, however, with your finals!

  5. Chris King

    I don’t think the messages are incongruous or incompatible. To me, both statements convey the same message: know the material and know where to find it to check that you are applying the material correctly. I think the point professors are trying to make with these types of statements is that you won’t have time to look through the entirety of the Federal Rules of Evidence to determine what qualifies as other acts evidence, but that you will have time to refresh your memory on other acts evidence if you know where to find it in the Rules of Evidence. I think the situation would be simplified if instead of making an exam “open-book” the professor supplied any necessary statutes or rules with the exam.

  6. Andrew Golden

    “I think the situation would be simplified if instead of making an exam ‘open-book’ the professor supplied any necessary statutes or rules with the exam.”

    It’s funny you should mention that along with the Federal Rules of Evidence. When I was writing my earlier post on this thread, I deleted my last paragraph, which cited Professor Blinka’s Evidence exam as what I consider to be a good example of balancing requiring memorization and providing statute language. I personally think his was one of the fairest exams I’ve taken at the law school; he tells his Evidence classes what he will provide well in advance of the exam (as I recall, Rules 801-804), and then he refers to the rules he doesn’t provide in the questions either by their number or by their language. In that respect, the only way you’re not going to fare well on his exam is if you can’t recognize the issues being addressed.

    Let it just be noted (in case my original post wasn’t clear on this) that I completely agree with the approach Professors Boyden and Calboli are advocating. I mean, don’t get me wrong: I LIKE having a completely open-book, open-note, open-outline exam. I also have no problem with a professor telling me I can bring a copy of the applicable codes or statutes, but not my book or notes. My point is only that I find it unreasonable when some of my professors in the past have given me final exams that involve regurgitation of details that I am positive they themselves couldn’t recite without looking them up first (and I say this because, for some of these details, I’ve gone to office hours and asked them questions about it and I’ve SEEN them unable to recite it from memory). All I’d ever ask of my professors is that they don’t ask me to do anything on a final that they couldn’t do themselves. I don’t know that that’s too much to ask, is it?

  7. Kali Murray

    Andrew, although I agree with Bruce and Irene on all major points, and in fact, prefer open-book exams precisely because they mirror the process in practice (spot the relevant issue, then look up the test in statute books and/or cases), I have to speak to the one instance where memorization matters: the bar exam. Memorization matters a great deal for the multiple-choice section of the bar. While many of you will not take the bar exam (and at this point, I am contractually obligated to say: YAY, diploma privilege!!), memorization is a skill that is necessary to work one’s way through that particular task. It might be helpful to see that law school exams each have their own particular way of building skills. Some might be narrowly tailored to a particular test of the skill for the bar, while others might be keyed to a broader set of skills (like the first choices you make in practice). As you see, from Bruce’s, Irene’s, and my comments, we all think the design of a exam serves a broader pedagogical purpose, and a helpful question for both the professor and your fellow student is to ask in-class why we believe the design of a particular test is important. Such a question may ultimately reveal what we think is important about the law.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.