Published: March 2, 2012

Judicial Coup for Computer-Assisted Review

"This judicial opinion now recognizes that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases."

Magistrate Judge Andrew Peck, of the U.S. District Court for the Southern District of New York made e-discovery headlines late February with the first judicial endorsement of predictive coding. I'll leave it to the legal eagles to debate the courtroom implications, but it's a significant stamp of approval from the bench.You can read more from LTN, Forbes, Chris Dale, and Recommind, whose Axcelerate software is involved in the case.

In practice, many people have already adopted predictive coding. So the interesting question to my mind is: how are we going to agree on using it (or misusing it)?  Judge Peck stated further:

"The Court recognizes that computer-assisted review is not a magic, Staples-Easy-Button, solution appropriate for all cases. The technology exists and should be used where appropriate, but it is not a case of machine replacing humans: it is the process used and the interaction of man and machine that the courts needs to examine."

Arguments between the parties involved bear this out. First, how to agree on how many responsive documents are produced? Early on, the defendant volunteered to produce the 40,000 most-responsive documents determined by the computer once trained - which was dismissed by Judge Peck as potentially eliminating too many responsive results.

The Court rejected MSL's 40,000 documents proposal as a "pig in a poke." (1/4/12 Conf. Tr. at 51-52.) The Court explained that "where [the] line will be drawn [as to review and production] is going to depend on what the statistics show for the results," since "[p]roportionality requires consideration of results as well as costs. And if stopping at 40,000 is going to leave a tremendous number of likely highly responsive documents unproduced, [MSL's proposed cutoff] doesn't work."

This led to further arguments on how to create a seed set to determine responsiveness, etc., much of which are detailed in the court documents if you're curious. It should disabuse anyone of the notion that predictive coding will relieve us of lawyers.  Using it can require a lot of negotiation (and math) to agree on methods.

I've read a few industry arguments that this means predictive coding is too difficult or non-user-friendly - with which I disagree (depending on the tool).  It's just not always easy to agree on terms in our adversarial court system, particularly in an early adoption phase.  My hope is this opinion changes that for the better, or at least encourages parties to raise the bar and arrive more prepared.  The benefits are there, and the genie's not going back in the bottle.

The parties agreed to use a 95% confidence level (plus or minus two percent) to create a random sample of the entire email collection; that sample of 2,399 documents will be 5/ The Court also suggested that the best way to resolve issues about what information might be found in a certain source is for MSL to show plaintiffs a sample printout from that source. (2/8/12 Conf. Tr. at 55-56.) G:\AJP\DA SILVA MOORE - ESI Case 1:11-cv-01279-ALC-AJP Document 96 Filed 02/24/12 Page 9 of 49 10 reviewed to determine relevant (and not relevant) documents for a "seed set" to use to train the predictive coding software. (Dkt. No. 88: 2/8/12 Conf. Tr. at 59-61.) An area of disagreement was that MSL reviewed the 2,399 documents before the parties agreed to add two additional concept groups (i.e., issue tags). (2/8/12 Conf. Tr. at 62.) MSL suggested that since it had agreed to provide all 2,399 documents (and MSL's coding of them) to plaintiffs for their review, plaintiffs can code them for the new issue tags, and MSL will incorporate that coding into the system. (2/8/12 Conf. Tr. at 64.) Plaintiffs' vendor agreed to that approach. (2/8/12 Conf. Tr. at 64.)


Post a Comment
  • Leave this field empty
Please Enter Correct Verification Number

*All views and opinions expressed in ESG blog posts are intended to be those of the post's author and do not necessarily reflect the views of Enterprise Strategy Group, Inc., or its clients. ESG bloggers do not and will not engage in any form of paid-for blogging. Click to see our complete Disclosure Policy.



Katey Wood covers the e-discovery, archiving, content and records management, enterprise search, and text analytics markets. In this role, Katey produces qualitative and quantitative research including economic validations, technology validations, and user surveys regarding legal and IT systems maturity. She also advises users, software vendors, service providers, and financiers on strategic technology and business initiatives.