Imagining the Evidence


Imagining the Evidence

  • 1 Tags
By Craig Ball

As a young lawyer in Houston, I had the good fortune to sip whiskey with veteran trial attorneys who never ran short of stories. One told of the country lawyer who journeyed to the big city to argue before the court of appeals. The case was going well until a judge asked, “Counsel, are you aware of the maxim, ‘volenti non fit injuria’?”

“Why, Your Honor,” he answered in a voice as smooth as melted butter, “In the piney woods of East Texas, we speak of little else.”

Lately, in the piney woods of e-discovery, the topic is technology-assisted review (TAR aka predictive coding), and we speak of little else. The talk centers on that sudsy soap opera, Da Silva Moore v. Publicis Groupe, and whether Magistrate Judge Andrew Peck of the Southern District of New York will be the first judge to anoint TAR as being “court approved” and a suitable replacement for manual processes now  employed to segregate ESI.

TAR is the use of computers to identify responsive or privileged documents by sophisticated comparison of a host of features shared by the documents. It’s characterized by methods whereby the computer trains itself to segregate responsive material through examination of the data under scrutiny or is trained using exemplar documents (“seed sets”) and/or by interrogating knowledgeable human reviewers as to the responsiveness or non-responsiveness of items sampled from the document population.

Let’s put this “court approved” notion in perspective. Dunking witches was court approved and doubtlessly engendered significant cost savings. Trial by fire was also court approved and supported by precise metrics (“M’Lord, guilt is established in that the accused walked nine feet over red-hot ploughshares and his incinerated soles festered within three days”). Whether a court smiles on a methodology may not be the best way to conclude it’s the better mousetrap. Keyword search and linear review enjoy de facto court approval; yet both are deeply flawed and brutally inefficient.

The imprimatur that matters most is “opponent approved.” Motion practice and false starts are expensive. The most cost-effective method is one the other side accepts without a fight, i.e., the least expensive method that affords opponents superior confidence that responsive and non-privileged material will be identified and produced.

Don’t confuse that with an obligation to kowtow to the opposition simply to avoid conflict. The scenario I’m describing is a true win-win:

  • Producing parties have an incentive to embrace TAR because, when it works, TAR attenuates the most expensive component of e-discovery: attorney search and review.
  • Requesting parties have an incentive to embrace TAR because, when it works, TAR attenuates the most obstructive component of e-discovery: attorney search and review.
  • Producing parties don’t just obstruct discovery by the rare and reprehensible act of intentionally suppressing probative evidence. It occurs more often with a pure heart and empty head as a consequence of lawyers using approaches to search and review that miss more responsive material than they find.

It’s something of a miracle that documentary discovery works at all. Discovery charges those who reject the theory and merits of a claim to identify supporting evidence. More, it assigns responsibility to find and turn over damaging information to those damaged, trusting they won’t rationalize that incriminating material must have had some benign, non-responsive character and so need not be produced. Discovery, in short, is anathema to human nature.

A well-trained machine doesn’t care who wins, and its “mind” doesn’t wander, worrying about whether it’s on track for partnership. From the standpoint of a requesting party, an alternative that is both objective and more effective in identifying relevant documents is a great leap forward in fostering the integrity and efficacy of e-discovery. Crucially, a requesting party is more likely to accept the genuine absence of supportive ESI if the requesting party had a meaningful hand in training the machine.

Until now, the requesting party’s role in “training” an opponent’s machines has been limited to proffering keywords or Boolean queries. The results have been uniformly awful.
But the emerging ability to train machines to “find more documents like this one” will revolutionize requests for production in e-discovery. Because we can train the tools to find similar ESI using any documents, we won’t be relegated to using seed sets derived from actual documents. We can train the tools with contrived examples–fabrications of documents like the genuine counterparts we hope to find.

I call this “imagining the evidence,” and it’s not nearly as crazy as it sounds.

If courts permit the submission of keywords to locate documents, why not entire documents to more precisely and efficiently locate other documents? Instead of demanding “any and all documents touching or concerning” some amorphous litany of topics, we will serve a sheaf of dreams—freely forged smoking guns—and direct, “show me more like these.”

Predictive coding is not as linguistically fussy as keyword search. If an opponent submits contrived examples of the sorts of documents they seek, it’s far more likely a similar document will surface than if keywords alone were used. As importantly, it’s less likely that a responsive document will be lost in a blizzard of false hits. This allows us to rely less on our opponents to artfully construct queries. Instead, we need only trust them to produce the non-privileged, responsive results the machine finds.

There’s more to documents than just the words they contain, so mocking up contrived exemplars entails more than fashioning a well-turned phrase. Effective exemplars will employ contrived letterheads and realistic structure, dates and distribution lists to insure that all useful contextual indicia are present. And, of course, care must be taken and processes employed to ensure that no contrived exemplars are mistaken for genuine evidence.

The use of contrived examples may ruffle some feathers. I can almost hear a chorus of, “How dare they draft such a vile thing!” But the methodology is sound, and how we will go about “imagining the evidence” is likely to be a topic of discussion in the negotiation of search protocols once use of technology assisted review is commonplace.

Another “not as nutty as it sounds” change in discovery practice wrought by TAR will be affording requesting parties a role in training TAR systems. The requesting party’s counsel would be presented with candidate documents from the collection that the machine has identified as potentially responsive. The requester will then decide whether the sample is or is not responsive, helping the machine hone its capacity to find what the requester seeks. After all, the party seeking the evidence is better situated to teach the machine how to discriminate.

For this to work, the samples must first be vetted by the responding party’s counsel for privilege and privacy concerns, and the requesting party must be willing to undertake the effort without fretting about revealing privileged mental impressions.

It’s going to take some getting used to, but the reward will be productions that cost less and that requesting parties trust more.

Volenti non fit injuria means “to a willing person, injury is not done.” When we fail to embrace demonstrably better ways of searching and reviewing ESI, we assume the risk that probative evidence won’t see the light of day and voluntarily pay too high a price for e-discovery.

About the Author

Craig Ball of Austin is a Board Certified trial lawyer, certified computer forensic examiner, and electronic evidence expert. He’s dedicated his globetrotting career to teaching the bench and bar about forensic technology and trial tactics. After decades trying lawsuits, Craig now limits his practice to service as a court-appointed special master and consultant in computer forensics and electronic discovery and to publishing and lecturing on computer forensics, emerging technologies, digital persuasion and electronic discovery. Craig writes the award-winning Ball in Your Court column on electronic discovery for Law Technology News and is the author of numerous articles on e-discovery and computer forensics, many available at Craig Ball has consulted or served as the Special Master or testifying expert in computer forensics and electronic discovery in some of the most challenging and well known cases in the U.S.

Craig Ball © 2012

Almanya sohbet anal yapan escort