30 September 2015
In the matter of Atrium Services Ltd: In the matter of Kimberley Scott Services Ltd sub nom (1) Robert Derek Smailes (2) Stephen Blandford Ryman v (1) John McNally (2) George McClean  EWHC 1755 (Ch) Ch D (Companies Ct), 22 June 2015, the claimant breached its disclosure obligations because relevant documents for disclosure were missed during the process of scanning hard copy documents, and then running keyword searches on them. The ruling is relevant to the English courts' approach to the use of modern technology to cope with vast quantities of potentially disclosable documents. The use of such technology, whilst acknowledged not to be "fail safe", has been accepted by courts in other jurisdictions (and by some English judges) as being a proportional way of limiting costs associated with disclosure.
Various issues arose during the disclosure process in this case. After the claimants initially provided an inadequate list of documents, Henderson J ordered that a fresh search be conducted. He also penalised the claimants in costs for embarking on a unilateral approach to disclosure, without sufficient collaboration with the defendants.
Following this order, new advisors for the claimants proposed to the respondents' advisors that an e‑disclosure provider (Unified) be used to scan (using Optical Character Recognition – OCR) and upload the hard copy documents, which involved "millions" of pages, to their platform for scanning and coding. It was agreed that Unified would undertake this task.
The disclosure exercise was eventually concluded and a new list (the List) produced.
The respondents argued that the List was defective because of shortcomings in the way in which Unified's coders had described documents in the database (eg "title" fields left blank; "author" fields blank; documents undated, generic descriptions such a "spreadsheet").
HH Judge Pelling found that "the omission of an appropriate concise description in breach of…CPR part 31 was serious and significant because the failure to comply would have added significantly to the time needed to carry out… a competent inspection exercise".
He noted that the position was mitigated because the documents had been made available on the shared platform in any event and declared that, although serious, he would have granted relief from sanctions on this point alone.
Failure to carry out a reasonable search?
The respondents alleged that the claimants had failed to carry out a reasonable search. Keyword searches were conducted over the scanned material to determine which documents should be reviewed for relevance and disclosure. Those to be disclosed were to be made available for inspection by uploading them to a further database to which all parties had access. The court was informed that, to render scanned documents keyword searchable, they first had to be subjected to a process whereby OCR was applied.
HH Judge Pelling was not impressed by the OCR process – stating that it produced "electronic versions of the documents that were seriously inaccurate".
Despite considering the evidence of an industry expert from Unified, HH Judge Pelling held that the fact that potentially relevant documents will have been missed as a result of the "OCR issue" was a serious and significant breach and amounted to a failure to carry out a reasonable search.
The expert explained the function of OCR and the tests that were carried out by the liquidators and their advisors to validate the selection of documents for further review, based on the OCR process. HH Judge Pelling was of the view that the evidence omitted sufficient detail for him to conclude that the shortcomings of the OCR had been rectified by other means (such as targeted sampling of documents that had not responded to the key word searches).
The judge suggested that a physical examination of a further 86,055 documents (the result of not being able to rely on the OCR) would be no more expensive or difficult than it was to examine the 94,247 documents that were physically reviewed in the first place.
Did Practice Direction 31B (electronic disclosure) apply?
No, because the documents were held by the party in paper form and only converted to electronic form for the purpose of carrying out the disclosure exercise, they did not constitute electronic documents.
Comment: OCR is not a new technology, it has been around for over 30 years. It can never be relied upon to give a 100% accurate rendition of a scanned document. What it is intended (and generally accepted) to do is to give a steer as to the words contained in a document. It is only as good as the quality of the scanned document and it cannot be conclusive, as, for example, it is not able to read manuscript or graphics at all and sometimes struggles with figures. The only way to be certain of what is contained in scanned documents is for a person to manually review each page – effectively a "leave no stone unturned" approach. This is rarely an appropriate way to conduct a disclosure exercise. OCR is generally regarded as a defensible way of interrogating large volumes of scanned material in a proportionate manner, accepting that its limitations mean that some documents containing key words may be missed. There is an important difference between missing a document and suppressing its disclosure.
There are references in the judgment to "documents being converted into an electronic format", "electronic versions of the documents that were seriously inaccurate" and "the garbled way in which the OCR process converted scanned documents into electronic format". OCR is however not a format of document, but, rather, a process that is run over the document. The source document does not change format at all as a result of OCR being run over it.
The court's finding that an established and mature process (ie OCR) had resulted in a significant failure (in that it caused potentially documents to be missed), highlights uncertainties concerning the courts' approach to innovative Technology Assisted Review (TAR) techniques and products such as predictive coding. These processes have court endorsement in both the U.S. and Ireland1 even though, like OCR, they are not fail safe.
The judge's comments concerning the adequacy of the document descriptions in the list is also interesting The amendments to CPR 31.5 introduced as a result of the Jackson Reforms provide the option of dispensing with Lists of Documents altogether. Given the trend whereby, almost invariably, parties require inspection of all material in any event, expending considerable time, effort and expense on manually perfecting the entries in the List may in certain circumstances be considered a disproportionate expense.
Footnote: 1. Irish Bank Resolution Corporation Ltd & ors v Sean Quinn & ors  IEHC 175.