Pyrrho Investments Ltd v MWB Property Ltd & Ors

Defendants are stored. To give an idea of the scale of the exercise, the total number of electronic files restored from the back-up tapes of the Second Claimant was originally more than 17.6 million. This has smce been reduced to some 3.1 million by a process of electronic de-duplication. But it is still a large and costly number to search.

When giving standard disclosure, a party is required to make a reasonable search for documents falling within rule 31.6(b) or (c).

The factors relevant in deciding the reasonableness of a search include the following

the number of documents involved;

the nature and complexity of the proceedings;

the ease and expense of retrieval of any particular document; and (d) the significance of any document which is likely to be located during the search.

Where a party has not searched for a category or class of document on the grounds that to do so would be unreasonable, he must state this in his disclosure statement and identify the category or class of document."

"20 The extent of the reasonable search required by lule 31.7 for the purposes of standard disclosure is affected by the existence of Electronic Documents. The extent of the search which must be made will depend on the circumstances of the case including, in particular, the factors referred to in rule 31.7(2). The parties should bear in mind that the overriding objective includes dealing with the case in ways which are proportionate.

21 The factors that may be relevant in deciding the reasonableness of a search for Electronic Documents include (but are not limited to) the following —

the number of documents involved;

the nature and complexity of the proceedings;

the ease and expense of retrieval of any particular document. This includes:

the accessibility of Electronic Documents including e-mail communications on computer systems, servers, back-up systems and other electronic devices or media that may contain such documents taking into account alterations or developments in hardware or software systems used by the disclosmg party and/or available to enable access to such documents;

"25 It may be reasonable to search for Electronic Documents by means of Keyword Searches or other automated methods of searching if a full review of each and every document would be unreasonable.

However, it will often be insufficient to use simple Keyword Searches or other automated methods of searching alone. The injudicious use of Keyword Searches and other automated search techniques

may result in failure to find important documents which ought to be disclosed, and/or

may find excessive quantities of irrelevant documents, which if disclosed would place an excessive burden in time and cost on the party to whom disclosure is given.

The parties should consider supplementing Keyword Searches and other automated searches with additional techniques such as individually reviewing certain documents or categories of documents (for example important documents generated by key personnel) and taking such other steps as may be required in order to justify the selection to the court."

PYITho

"The court must seek to give effect to the overriding objective when it(a) exercises any power given to it by the Rules; or (b) interprets any rule, subject to rules 76.2, 79.2, 80.2 and 82.2."

None of the last-mentioned rules is relevant to the present case. The overriding objective is set out in rule 1.1(1), and is well-known. It is "enabling the court to deal with cases justly and at proportionate cost". This is amplified in rule 1.1(2), and includes, so far as practicable, saving expense and dealing proportionately with the case.

"1. This judgment concerns a serious practical problem for the case management of disclosure which is now occurring on a regular basis. The reason is that, since certainly the beginning of this decade, increasing numbers of public bodies and private businesses, not to mention individuals, have gone over to creating, exchanging and storing their documentation and communicating with each other entirely by electronic means. The end result is that an enormous volume of information is now created, exchanged and stored only electronically. Email communication, word processed documents, spreadsheets and ever increasing numbers of other forms of electronically stored information ("ESI") now often form the entire corpus of the documentation held by companies and individuals who become involved in litigation. So the incidence of paper disclosure is becoming less and less prevalent though in some cases it may still be critical. and the incidence of the disclosure of electronically stored information, or ESI as it is known, is becoming more and more so.

What is more, the volume of the ESI, even in small organisations is immense, often, as in the case of email, because of the huge quantities of documents created (including wide-scale duplication) and the fact that the documents can exist in many different forms and locations so that they are not readily accessible except at significant cost. It is also commonplace for many individuals to have more than one email account — business, personal, webmail (for example, Yahoo, Gmail, Hotmail etc,) When ESI is available, metadata (literally data about data) associated with it can easily be unintentionally altered by the very act of collection, which in some circumstances can have a detrimental effect on the document's evidential integrity. What is more, ESI can be moved about nationally and internationally, indiscriminately and at lightening speed.

What is the problem with this in litigation? Disclosure is a tripattite exercise of search, disclosure, and inspection, and the problem, when it comes to ESI is often for a party to gauge the scope of a reasonable search for ESI under CPR Rule 31.7 and PD31(2). The problem is how the parties and (if disputed) the court determines what the scope of that search of ESI should be, how it is going to be made proportionate and how it is going to be carried out correctly first time , without the court having to order it to be done again, as has occurred, for example, in Digicel (St Lucia) Ltd and others v. Cable and Wireless Plc and Others 2008 EWHC 2522 (Ch) in which case Morgan J ordered the defendants re-do their ESI search exercise at an additional cost of something like €2 million.

By contrast, except in unusual cases, in the case of paper disclosure, parties usually know what paper they have. Often the problem is merely locating it physically and going through it to produce the documents required by the standard disclosure test. The problem with ESI is that, because of the matters mentioned above in paragraph 1, parties often do not know how much ESI they have, or where it is. They might have a idea as to which servers it is on or which personal computers it is on, or which back-up tapes it is on, but without a great deal more information, it is very difficult for them to know how much documentation will be revealed by searches of the media on which their ESI is stored and how much it is going to cost to search it and what the end result is going to be. A further issue might be that not all forms of ESI are searchable. Therefore, it has to be accepted that any search is not necessarily conclusive as to whether a particular document exists. Equally often the parties do not know where to begin their searches. In the case, for example, of email, the relevant servers are often not in their possession and sometimes not even in the jurisdiction. An ill considered search for ESI may produce far too few documents for review but more likely will produce such volumes that human review of every document is neither proportionate nor practical. Because of this a substantial industry has developed to handle the identification, collection, reduction and organisation for review of ESI. Often, this is carried out electronically, with technology aiding and supplementing human review."

That searching, because it is going to be done in a comparatively simple way, without using specialist software at this stage, is just going to give us the potential numbers of documents. Similarly, doing the same type of search in respect of the MEDS system for the 31 terms but only in respect of each of the key witnesses, will give us the potential number of documents in respect of that as well. It is at that stage, when that crude way of finding out what documents might be in existence is completed, that a service provider will have to be agreed between the parties, and will have to be instructed to look at what the next stage of the exercise should involve and how much it is going to cost, in order to produce a corpus of documents which is reviewable by both palties.

At the moment we are just staring into open space as to what the volume of the documents produced by a search is going to be. I suspect that in the long run this crude search will not throw up more than a few hundred thousand documents. If that is the case, then this is a prime candidate for the application of software that providers now have, which can de-duplicate that material and render it down to a more sensible size and search it by computer to produce a manageable corpus for human review — which is of course the most expensive part of the exercise. Indeed, when it comes to review, I am aware of software that will effectively score each document as to its likely relevance and which will enable a prioritisation of categories within the entire document set."

to (sometimes sceptical) litigation lawyers in the mid-1980s. I was one of them. But this was before the advent of personal computers, let alone of tablets and smartphones. There was no everyday or home computer culture then, and especially not amongst English lawyers. Now computers and computer technology are much more accepted as the norm, and, crucially, the technology is vastly better, for example in terms of storage size, portability of hardware and storage media, processor speed and programming, amongst other matters. A number of computer software companies now offer predictive coding software for use by lawyers.

PYITho

"The decision to allow computer-assisted review in this case was relatively easy — the parties agreed to its use (although disagreed about how best to implement such review). The Court recognises that computer-assisted review is not a magic, Staples-easy-Button, solution appropriate for all cases. The technology exists and should be used where appropriate, but it is not a case of machine replacing humans: it is the process used and the interaction of man and machine that the court needs to examine.

The objective of review in ediscovery is to identify as many relevant documents as possible, while reviewing as few non-relevant documents as possible. Recall is the fraction of relevant documents identified during a review; precision is the fraction of identified documents that are relevant. Thus, recall is a measure of completeness, while precision is a measure of accuracy or correctness. The goal is for the review method to result in higher recall and higher precision than another review method, at cost proportionate to the 'value' of the case.

The slightly more difficult case would be where the producing party wants to use computer-assisted review and the requesting party objects. The question to ask in that situation is what methodology would the requesting party suggest instead? Linear manual review is simply too expensive where, as here, there are over three million emails to review. Moreover, while some lawyers still consider manual review to be the 'gold standard' , that is a myth, as statistics clearly show that computerised searches are at least as accurate, if not more so, than manual review. [

Because of the volume of ESI, lawyers frequently have turned to keyword searches to cull email (or other ESI) down to a more manageable volume for further manual review. Keywords have a place in production of ESI — indeed, the parties here used keyword searches (with Boolean connectors) to find documents for the expanded seed set to train the predictive coding software. In

The evidence establishes, that in discovery of large data sets, technology assisted review using predictive coding is at least as accurate as, and, probably more accurate than, the manual or linear method in identifying relevant documents. Furthermore, the plaintiff's expert, Mr. Crowley exhibits a number of studies which have examined the effectiveness of a purely manual review of documents compared to using TAR and predictive coding. One such study, by Grossman and Cormack, highlighted that manual review results in less relevant documents being identified. The level of recall in this study was found to range between 20% and 83%. A further study, as part of the 2009 Text Retrieval Conference, found the average recall and precision to be 59.3% and 31.7% respectively using manual review, compared to 76.7% and 84.7% when using TAR. What is clear, and accepted by Mr. Crowley, is that no method of identification is guaranteed to return all relevant documents.

If one were to assume that TAR will only be equally as effective, but no more effective, than a manual review, the fact remains that using TAR will still allow for a more expeditious and economical discovery process.

As technology assisted review combines man and machine, the process must contain appropriate checks and balances which render each stage capable of independent verification. A balance must be struck between the right of the party making discovery to determine the manner in which discovery is provided and participation by the requesting party in ensuring that the methodology chosen is transparent and reliable. Ordinarily, as the rules in other jurisdictions provide, this is a matter of agreement between the parties at the outset. Agreement, as Clarke J. said in Thema, gives the parties "an added degree of comfort that a failure of the system to throw up a relevant document will be more likely to be viewed as unfortunate but unavoidable rather than a deliberate act".

Pursuant to the legal authorities which I have cited supra, and with particular reference to the albeit limited Irish jurisprudence on the topic, I am satisfied that, provided the process has sufficient transparency, Technology Assisted Review using predictive coding discharges a party's discovery obligations under Order 31, rule 12."

Experience in other jurisdictions, whilst so far limited, has been that predictive coding software can be useful in appropriate cases.

There is no evidence to show that the use of predictive coding software leads to less accurate disclosure being given than, say, manual review alone or keyword searches and manual review combined, and indeed there is some evidence (referred to in the US and Irish cases to which I referred above) to the contrary.

Moreover, there will be greater consistency in using the computer to apply the approach of a senior lawyer towards the initial sample (as refined) to the whole document set, than in using dozens, perhaps hundreds, of lower-grade fee-earners, each seeking independently to apply the relevant criteria in relation to individual documents.

There is nothing in the CPR or Practice Directions to prohibit the use of such software.

The number of electronic documents which must be considered for relevance and possible disclosure in the present case is huge, over 3 million.

The cost of manually searching these documents would be enormous, amounting to several million pounds at least. In my judgment, therefore, a full manual review of each document would be "unreasonable" within paragraph 25 of Practice Direction B to Part 31, at least where a suitable automated alternative exists at lower cost.

The costs of using predictive coding software would depend on various factors, including importantly whether the number of documents is reduced by keyword searches, but the estimates given in this case vary between €181,988 plus monthly hosting costs of El 5,717, to €469,049 plus monthly hosting costs of €20,820. This is obviously far less expensive than the full manual alternative, though of course there may be additional costs if manual reviews still need to be carried out when the software has done its best.

The 'value' of the claims made in this litigation is in the tens of millions of pounds. In my judgment the estimated costs of using the software are proportionate.

The trial in the present case is not until June 2017, so there would be plenty of time to consider other disclosure methods if for any reason the predictive software route turned out to be unsatisfactory.

The parties have agreed on the use of the software, and also how to use it, subject only to the approval of the Court.

There were no factors of any weight pointing in the opposite direction.