David Kessler of Fulbright and Jaworski weighed in on Judge Peck's recent predictive coding opinion this week, with the conclusion that a defensible process is key to the bench, not the particular technology used:
In short, the key takeaway here is not simply that Judge Peck approved the use of technology assisted review. While a precedent has now been set for the use of computer assisted review technology, the Opinion itself actually spends very little time discussing the specific technology selected by the parties. Judge Peck does, however, spend more than 25 pages discussing the process that the parties' attorneys applied to the use of the technology in this matter, going so far as to attach another 25 pages worth of protocol agreed to by the parties for the technology's use in this case. It appears that to this court, it is the process that is key, not the particular technology standing alone. Without a defensible, supportable, and explainable process, technology assisted review is no better and probably less defensible than other review approaches. As Judge Peck concludes: "As with keywords or any other technological solution to ediscovery, counsel must design an appropriate process, including use of available technology, with appropriate quality control testing, to review and produce relevant ESI while adhering to Rule 1 and Rule 26(b )(2)(C) proportionality." Opinion at 25-26.
I've bolded the "pull quote" for me. Remember that the initial approach on this case suggested by one of the parties was thrown out by Judge Peck as indefensible (judicial-coup-for-computer-assisted-review/index.html">as I discussed in my last post), since it would have missed too many potentially responsive items by only including the first 40,000. Judge Peck's opinion is being heralded as a "game-changer"--will the next inevitable step toward acceptance be adversaries' attempts at "gaming" discovery with their predictive coding process?
Which brings me to the importance of sampling. It's interesting: one predictive coding power-user I spoke to recently said he actually thought the random statistical sampling technology being built into many review tools was more significant and useful in his review projects than the predictive coding capabilities themselves. Another agreed that sampling was critical, but criticized the methods used by his particular tool, saying his quants had calculated mathematically that it required an overly broad sample. I've heard yet another argument that statistical sampling is insufficient--that samples should be weighted and constructed as "representative" of the rest of the document corpus, depending on the data contents.
I suspect these debates are just gearing up. It's not a matter of whether predictive coding is theoretically defensible anymore, but how you're using it and how effective and defensible it will be. Will predictive coding now go from the boogeyman of "automating attorneys" to becoming just a more quantitative stick to beat litigants over the head with? Maybe so--if they don't understand the tools and process involved.