Burger v Office of the Independent Adjudicator for Higher Education

In September 2007 the claimant started the MRes/PhD Economics Track 1 programme at the London School of Economics and Political Science (“LSE”). For the first year the programme consisted of three core courses -- EC441, EC442 (Macroeconomics for MRes students) and EC443 (Econometrics for MRes students). In the second year the programme consisted of two elements -- a field course and a research paper. A MRes degree is awarded to students who pass the three core courses and the two elements. Then they can progress to Ph.D. registration in the third year. If a student fails an exam they may re-sit it on one occasion only.

In the examinations in June 2008 the claimant failed both EC442 and EC443. He re-sat them both in June 2009. While he passed EC443 he unfortunately failed EC442. Therefore, sadly for him, he could not complete the MRes and progress to Ph.D. registration.

On 28 July 2009 the claimant submitted a formal, wide ranging, complaint to the LSE about the examination process. That was rejected by Jenny Bone, the Director of the Planning and Corporate Policy Division on 30 October 2009. On 5 November 2009 the claimant launched an internal appeal; that was rejected by the Pro-Director for Teaching and Learning on 12 January 2010. On 2 February 2010 the claimant lodged a complaint with The Office of the Independent Adjudicator for Higher Education (OIA); that was rejected on 6 May 2011. On 7 July 2011 the claimant issued these judicial review proceedings against the adjudication by the OIA. On 22 August 2011 His Honour Judge Stewart QC refused permission on the papers. The claimant renewed his application and on 30 November 2011 Holman J granted permission on two limited grounds. 14 months later, and 3½ years after he failed the macroeconomics exam for the second time, I have to decide this limited application for judicial review.

The two ground are:

1.

Did the Deputy Adjudicator make an error when she said at Paragraph 22 of her decision issued on the 6^th May 2011 that: “I do not consider that paragraph 6.1 of [the Instructions for for Examiners for Taught Programmes 2008/2009] goes so far as to dictate that the assessment criteria or marking schemes to be disclosed to students in advance of examination”?

2.

Was the Interested Party (LSE) in breach of their own instructions and rules by not publishing their assessment criteria for examination EC442 in a form that was available to the Claimant as a student in advance of his taking that examination?

It is important to emphasise that the decision I must make is not in respect of the complaint made by the claimant to the LSE and its disposal. Rather, it concerns only the adjudication by the OIA, although it is fair to say that the second ground does relate to the scope of the LSE's obligations to publish certain information about examinations to its students.

It is noteworthy, however, that in his complaint to the LSE the claimant stated "the marking scheme of the final exam was not communicated to class teachers, so that they were unable to prepare their students appropriately (both EC 442 and EC 443)" and "I am convinced that the marking scheme applied in EC 442 is inappropriate. At the very least, at no point in time did the lecturers communicate their expectations with regards to the exams to the students, neither directly or indirectly via class teachers".

I have seen the marking scheme for the EC 442 exam in 2008. Basically, it sets out the answers to the questions. I have only seen question 3 and so it is difficult to make much sense of the rest of the answers but it is true to say that in most respects the suggested answers are detailed and formulaic. For question 3, however, the suggested answer was highly laconic. The question was:

“It is common to assert that a “reduced-form” way to capture an oil shock in a RBC model is a shock to TFP. Suppose, however that you wanted to provide an RBC model with a more structural (i.e. explicit) description of the oil shock. Describe how you would modify the standard RBC model to do so.”

The marking scheme merely said in relation to this:

“Not a unique “right” answers (sic). Examples are Finn (JCMB 2000), and Blanchard and Gali (working paper, 2007).”

That in this instance no model answer is given is hardly surprising. Whatever the field, for some exam questions there is no unique right answer. The claimant accepts that it is obviously impossible for him to argue that the marking scheme, whether detailed and specific in its other suggested answers, or vague and directionless as in the case of question 3, should be published to students before the examination.

In his complaint to the LSE the claimant did not raise any criticism of the failure by the LSE to publish examination “assessment criteria”, as opposed to the marking scheme, to its students. This did not surface until he complained to the OIA. In his complaint to the OIA the claimant raised 12 separate grievances. His fifth item was "the assessment criteria for the final exams were not disclosed to students".

I have been supplied with a copy of the rules applicable to the adjudication at the time. By rule 3.2 the scheme does not cover a complaint to the extent that it relates to a matter of academic judgement. By rule 3.5 a complaint will not be covered if in the opinion of the reviewer a matter complained about does not materially affect the complainant as a student. By rule 4.1 the complainant must have first exhausted internal complaints procedures before bringing a complaint to the OIA, although this requirement may be waived in exceptional circumstances.

In her ruling the Deputy Adjudicator wrote:

“21.

The School responded by clarifying that the marking scheme for EC442 had been approved by an internal examination board and external examiner and that Mr Burger's examination script was double-blind marked and assessed for a third time by an external examiner. The School explained that the Department of Economics had made a conscious decision not to disclose the marking schemes to students on the grounds that they should prepare themselves to do the best they possibly could, rather than 'targetting' parts of questions and the examination on the basis of accumulating sufficient marks to pass. The School acknowledged that this decision may not have sat kindly with some students but pointed out that the Department had been free to make the decision.

22.

In his complaint to the OIA, Mr Burger has argued that the failure to disclose the assessment criteria to both class teachers and students was in violation of paragraph 6.1 of the 'Instructions for Examiners for Undergraduate and Taught Graduate Programmes' (" Instructions"). I am not satisfied that this was the case. I have seen no evidence to support Mr Burger's inference that class teachers were not informed of the assessment criteria for EC442. Furthermore, I do not consider that paragraph 6.1 of the Instructions goes so far as to dictate that assessment criteria or marking schemes should be disclosed to students in advance of examinations. Indeed I have seen no other regulation requiring such disclosure to students. It is clear to me that the Department felt, in its academic judgment, that the information provided to students during the course, including mock questions and problem sets, was sufficient to prepare them for the examination.”

Para 6.1(b) of the Instructions for Examiners for Taught Programmes 2008/2009 provides that “the Chair of each Sub-Board of Examiners shall … ensure that the Department produces and publishes assessment criteria in line with the School requirements”.

It is clear to me that when making her ruling the Deputy Adjudicator assumed that a marking scheme and examination assessment criteria were synonymous.

In his application for judicial review the claimant complained forcefully at the failure of the LSE to publish its examination assessment criteria to its students in seeming violation of Para 6.1 (b). However at no stage prior to the oral hearing before Holman J did anybody focus on what the phrase assessment criteria actually meant. Characteristically Holman J went straight to the heart of the problem and a note of his remarks when granting permission records him saying:

“It is deeply regrettable that there are no samples of Assessment Criteria which Mr Burger says LSE did publish for other courses… If Assessment Criteria are just the expectations between different grades then they would not be very illuminating. If they are no more that indicators of the level of marks for the claimant to achieve grades then the publication of the Assessment Criteria would not be relevant. Mr Burger has said the Assessment Criteria is much more than this and involves verbal descriptors of the requirements of the course and examination which I would consider to be part of a traditional Syllabus.

If a properly understood Assessment Criteria involves elements of description of the Syllabus then it should be considered part of the Syllabus and the requirements for the examinations and should be published to students...

It is self evident that students must be able to ascertain the scope and subject matter of the course and ifthe Assessment Criteria is part of the means of doing this maybe they should be published to students”.

We now have two examples of published examination assessment criteria from the Department of Political Sociology and from the Department of Media and Communications. The former provides:

“Distinction (70 per cent or higher)

This class of pass is awarded when the essay demonstrates clarity of analysis, engages directly with the question, and shows an independent and critical interpretation of the issues raised by it. The essay shows exemplary skill in presenting a logical and coherent argument and an outstanding breadth and depth of reading. The essay is presented in a polished and professional manner, and all citations, footnotes and bibliography are rendered in the proper academic form.

Essays in the upper range of this class (80 per cent and higher) may make an original academic contribution to the subject under discussion. Answers in the upper range will be outstanding in terms of originality, sophistication and breadth of understanding of relevant themes and material.

Merit (60-69 per cent)

This class of pass is awarded when the essay attempts a systematic analysis of the issues raised, by the question and demonstrates independent thought. The essay shows appropriate skill in presenting a clearly reasoned argument, and draws on a good range of relevant literature. The essay is well-presented and citations, footnotes and bibliography are rendered in the proper academic form.

Pass (50-59 per cent)

This class of pass is awarded when the essay shows understanding of the issues raised by the question, and demonstrates an engagement with relevant literature. The discussion may rely more heavily on description than on independent analysis. There may be some inconsistencies, irrelevant points and unsubstantiated claims in the argument. Presentation and referencing is adequate but may contain inaccuracies.

Fail (40-49 per cent)

The essay shows limited understanding of the subject and lacks evidence of an independent response to the question. It may be based entirely on lecture material, poorly structured and contain significant errors of fact. The essay may be incomplete, including poor presentation and inadequate referencing, and fail to demonstrate an appropriate level of engagement with relevant literature.

Bad Fail (0-39 per cent)

The essay is incomplete or fails to address the question under study. It provides little evidence of reading or understanding. It may be poorly presented and lack referencing.”

The latter was in similar language and stated:

“70% and over: distinction

This is for outstanding work that achieves all that could reasonably be expected of an MSc student, and will feature many if not all of the following characteristics: original argument, creative selection of sources, highly critical appraisal and analysis, excellent integration of theory and evidence, excellent expression, citation and bibliography norms.

60 - 69%: merit

This is for work of good quality with a well-defined focus. Such work will feature many if not all of the following characteristics: thoughtful arguments, well researched selection of sources, good critical appraisal, well integrated theory and evidence, good, clear expression, accurate citation and bibliography.

50-59%: pass

This is for work that reaches the overall standard required of a Masters student and will feature many if not all of the following characteristics: standard argument and range of sources used, mainly fair synthesis of ideas, adequate presentation and flaws in citation and bibliography norms.

40-49%: fail

This is for work that does not reach the overall standard required of a Masters student. It will feature many if not all of the following characteristics: weak argument, narrow range of sources used, descriptive account, poor presentation, inaccurate citation and gaps in bibliography.

0-39%: bad fail

This is for work that shows a basic lack of knowledge and ability. Such work will feature many if not all of the following characteristics: very weak argument, little use of even standard sources, descriptive, with large gaps, very poor presentation with flawed expression and extensive flaws in citation and bibliography.”

During argument I described these verbal descriptions of what is required as banal and statements of the obvious. Ms McColgan, counsel for the OIA, and the claimant, who represents himself, agreed. Of course, given that the claimant failed the Macroeconomics exam for a second time I am only really concerned with the language used to describe a fail. Here the claimant points in the first example to “inadequate referencing” and in the second to “inaccurate citation”. He says that he came to the LSE having undertaken his undergraduate study in Germany where the supply of references in an essay answer is unknown and that none of his teachers ever told him that inadequate or inaccurate referencing may lead to him failing the exam. Ms McColgan points out that that had assessment criteria for the Macroeconomics exam in fact been published it may, or may not, have included a criterion of adequate or accurate citation or reference; we just do not know. In any event she suggests that the absence of published criteria of this nature simply cannot on any reasonable or plausible analysis lead anywhere near to a conclusion that the claimant's exam performance may have turned on this.

In truth, the examples of the assessment criteria which have now been obtained show them to be, as Holman J predicted they may well be, just the expectations between different grades and “thus not very illuminating”. They are merely verbal indicators of the standards needed for a student to achieve grades and are therefore not relevant. The concern of Holman J that assessment criteria “involves elements of description of the Syllabus” or that it contained information about “the scope and subject matter of the course” is shown to be unfounded.

I am satisfied that para 6.1(b) does indeed require the LSE to publish assessment criteria for examinations to students across the board. This is, I suppose, part of the drive for transparency in all things which is a feature of modern life. I am also satisfied that in paragraph 22 of her ruling the Deputy Adjudicator fell into error in holding that assessment criteria of the kind exemplified above should not be prepared and published to students. On the other hand, there cannot be even the faintest suggestion that marking schemes should be published to students before an exam.

Had the Deputy Adjudicator been aware that other Departments published assessment criteria of the kind exemplified above then she would no doubt have phrased paragraph 22 of her ruling differently and drawn a clear distinction between assessment criteria and marking schemes. She would, I am sure, have agreed with my conclusion that paragraph 6.1(b) did indeed require the publication of such criteria in this case but would also have concluded that that had such publication taken place it would have made absolutely no difference to the claimant's performance in the exam, and the complaint should therefore be dismissed under rule 3.5. Further, I consider it likely that she would have concluded that the claimant, not having raised the failure to publish assessment criteria within the internal appeals process, had not exhausted his remedies with the LSE under rule 4.1. I have no doubt that had this confusion not arisen the result would have been exactly the same -- the complaint would have been dismissed.

It is accepted that the OIA is amenable to judicial review, although the court will generally be very slow to interfere with its decisions (R (Siborurema) v OIA[2008] ELR 209). However, where an inferior tribunal has made an error of fact, relief by way of judicial review will only be granted if the error is material - see E v SSHD [2004] QB 1044 at para 66 per Carnwath LJ (as he then was).

In my judgment the error here was not material. It arose from regrettable confusion as to what assessment criteria actually comprised and its unhappy conflation with a marking scheme, publication of which most definitely cannot be made to students.

The powers of the OIA extend to making a recommendation to the educational body in question that the complainant should be allowed to re-sit the exam. Were I to quash the decision of the Deputy Adjudicator and to order the OIA to reconsider the complaint anew I have no doubt that it would again be dismissed for the reasons I have given. Therefore I consider it most improbable that the claimant will succeed in what he sincerely wishes for, which is to be able to take, and hopefully pass, the exam again so that he can go on to complete his Ph.D. at the LSE. I cannot see any prospect of that. However I am consoled by the claimant’s statement to me that if he cannot finish the Ph.D. at the LSE he will do so at the University of Zürich where he presently is working as a researcher. I hope he succeeds.

For all these reasons the claimant's application for judicial review is dismissed.