The following is a brief guide to the responsibilities and workload of an Examiner for the DFPH. It is intended to provide some background information on the role of the Examiner and the examination system in practice.
Job Description
Who are the Examiners?
Examiners are experienced Members or Fellows of the Faculty or other Colleges or Faculties who have specialist knowledge of at least one area of the curriculum, who are willing to give up some time to ensure that the education standards of both medically and non-medically qualified candidates are tested and maintained over time.
It is a responsible position, as the examinations are a key aspect of the Faculty’s role in setting standards and the whole process needs to be robust, confidential, thoughtful, unbiased and fair to all candidates, whatever their background.
Examiners are not paid, but they can claim reasonable travel expenses at Faculty rates for travel to meetings.
How are Examiners chosen?
Potential Examiners are usually nominated by members of the Faculty Board, Faculty advisers, and heads of academic departments. [Recently, it has been the practice to request nominations very widely and self-nominations are also acceptable].
All nominations and applications are screened and judged by the Chair of the DFPH Examiners against the criteria for appointment and then formally considered by the Education Committee.
Once approved by the Education Committee, potential Examiners are brought on to the panel as current Examiners retire. The term of service is usually for three years, with a maximum further three years extension.
All Examiners must be thoroughly conversant with the examination syllabus, with the contents of the supplied training pack, and must be able to demonstrate the knowledge required and their commitment to the principles underlying the examination to the Chair of Examiners before taking up their responsibilities.
Examiners are normally to "dummy mark" a series of papers in their subject area drawn from past sittings of the examination prior to undertaking full marking duties, in order to gain experience and to standardise their marks against those of previous examiners.
There are some 'specialist' examiners who are chosen for their connections with the background of candidates from particular sub-groups:
- there is always at least one statistician on the Board of Examiners,
- and there is always at least one examiner representing the Hong Kong College of Community Medicine, whose job it is to ensure that the total content of the examination and phrasing of questions does not put non-UK candidates at a disadvantage.
They are fully integrated into the main panel of Examiners however, and do not mark only Hong Kong scripts.
Keeping up-to-date
The Faculty expects all Examiners to keep up-to-date with developments in the specialty. For medically qualified Members and Fellows of the Faculty, examiners must meet the minimum continuing professional development (CPD) requirements. The CPD Department will be asked to confirm this for all Examiners on an annual basis.
Personal qualities needed
[E = essential D = desirable]
Knowledge:
* Specialist, up-to-date knowledge of at least one area of the curriculum (E)
Skills:
- Ability to respect and preserve confidentiality (E)
- Ability to stick to tight time schedules and turn around work quickly (E)
(deadlines for question setting, marking and commenting) - Good communication skills (E)
- Ability to work in teams (E)
- Ability to make thoughtful and unbiased decisions (E)
- Ability to argue their position coherently when taking a different view from other examiners and to arrive at shared judgements on contentious issues (E)
Experience:
- Past or present experience of weighing evidence, making important and difficult objective decisions and ensuring that judgement is not swayed by personal bias or sectional interests (E)
- Past or present experience of setting and marking examination papers (D)
- Past or present experience of training specialist registrars (D)
Workload
Examiners sometimes underestimate the workload entailed in marking. Most of the work is done within the three weeks following the examination, and examiners are advised to clear sufficient space in their diaries to ensure they can get the work done on time.
There will often be in excess of 70 scripts to mark for each examiner, and the length of time taken up by marking will vary with the type of question being marked: a critical appraisal question can take twice as long to mark as a short answer. Examiners, particularly those marking for the first time, are advised to leave plenty of time (around 15-20 hours is typical) for marking, and to spread it out: four hours at a stretch is enough for most people. Remember, deadlines are crucial, and the marks, agreed with your co-examiner, must arrive at the Faculty by the specified time and date.
Duties
Examiners will be scheduled to participate in one examination per year. However, they must be prepared to stand in at an additional sitting when needed, such as if a colleague should fall sick or retire early.
For each examination, every participating examiner is required to:
- Propose a minimum of two questions with key points for the paper they will mark
- Mark the scripts and consult with their co-examiner
- Help in finalising key points and comments on questions they have marked, for candidate feedback
- Meet specified deadlines for return of question proposals, marks and comments
One of each pair of co-examiners must attend the examiners' results meeting for each sitting in which they have participated. All examiners are expected, but not required, to attend the annual examiners' meeting for setting papers. Co-examiners should try to ensure that at least one member of each marking pair attends.
How the Examination is Run
Sittings
There are two DFPH examinations a year in March and October.
Setting the Examination Papers
There is a single question setting meeting each year, typically in September, to which all examiners are invited, in order to separate the setting and marking of papers. Examiners are strongly encouraged to attend this meeting. All examiners are required to propose a minimum of two questions with key points for the examination paper they will mark, which must be received by the DFPH Manager two weeks prior to the date of the question setting meeting.
If an examiner cannot attend the meeting, either for reasons of completely unavoidable other commitments, ill-health or geographical inaccessibility (for instance examiners from Hong Kong), they must ensure that the Chair of Diplomate Examiners receives proposed questions AND key points at least two weeks in advance of that meeting, and they must make every effort to ensure that their views are known and represented by an examiner in their subject area whom they know to be attending the meeting.
Stand-by Examiners
To ensure that there are examiners in reserve at any one sitting who can cover for the unexpected inability of colleagues to mark, there are two sets of examiners, one which marks the October paper, and one which marks the March paper. In circumstances where an examiner withdraws at the last minute, examiners must be prepared to participate at a sitting of the examination for which they are not scheduled to mark, as they function on these occasions as deputy or standby examiners. Examiners can be assured that their co-operation in this process will be much appreciated and that they will receive time in lieu for such emergency duties.
Marking Papers
For each examination for which they are scheduled to mark, or at which they function as a standby examiner, each examiner is required to:
- Mark the scripts and consult with their co-examiner, and return both initial and agreed marks to the DFPH Administrator by the agreed date.
- Help in finalising key points and comments for the general report on the examination.
Results meetings
There are two Examiners' meetings a year, to agree the pass list for the most recent sitting. These meetings are open to all those who have marked papers in the most recent sitting, but it is not required that all attend. It is necessary only that one of each pair of Examiners attends, attendance to be agreed between them. Assuming that only one will choose to attend on each occasion the group will typically consist of:
- Chairman/Deputy Chair of DFPH Examiners
- One Examiner from each of the five sections of Paper I
- One Examiner from each of the two sections of Paper II
- Administrative support
This group meets twice year a year in the Faculty offices, typically in February and July.
Examiner guidelines on Question Setting, Marking and Confidentiality
Key Points
Key points should be written by the examiner who sets the question and agreed with his or her co-examiner before submission. After the examination has taken place and candidates' scripts have been marked, additional points can be incorporated into the "Examiners' Comments" (comments on how candidates approached the examination question).
Key points are "key points" and not "model answers". Thus, they should be written in note form with bullet points and be succinct (usually not more than a page, when typewritten).
It is important to adhere to deadlines given by the Education Department as delayed submission of examination questions and key points by examiners creates difficulties in finalising the document for the Chair's approval and despatching to the printers.
Key points are divided into two categories:
- The first group (in bullet points) must contain the essential key points for a pass. They are headed: Most or all of the following would be required for a pass
- The second group (in bullet points) must contain additional or optional points (if any) which would improve an answer from a pass to a good or excellent answer. They are headed: The following are additional points which might improve the answer to "good" or "excellent"
Please note that the examination questions and key points produced by examiners are widely circulated to candidates, Faculty Advisers/Heads of Academic Departments, trainers and libraries.
Examiners are expected to compile key points, agreed with their co-examiner, for relevant questions which have been chosen for the examination paper. If additional key points are required for additional choices within a multi-part question, for example, these are written by the Specialist Group Examiners in consultation with the examiners setting the remainder of the key points for that section.
A copy of the Examiners' Key Points must be distributed to every unsuccessful candidate after notification of their result. It is therefore essential that these are agreed between examiners before being sent to the Education Department and that the deadline for submission is adhered to. If an examiner wishes to make revisions to their key points after marking, they must forward these with their marks.
Submission of marking sheets
Provisional marks must be submitted to the Faculty by the agreed deadline using the appropriate summary forms. Examiners are required to submit their individual marks before consultation with their co-examiner and their agreed marks after discussion with their co-examiner. Separate colour coded forms are provided for this purpose.
Examination Scripts
Due to issues of Data Protection examiners are requested to hold on to the examination scripts and any comments they have made until after the deadline for appeals, which is one calendar month after the despatch of results to candidates..
Examiners' Comments
All Examiners are expected to provide comments on the general performance of candidates. These should be kept concise and address issues such as how well questions were answered, any common deficiencies, points not referred to in the Key Points, and examination techniques.
Deadlines
A timetable of deadlines for the submission of marks, questions etc. is agreed on a yearly basis. Reminders are sent out approximately three to four weeks before each deadline. It is essential that the schedule is adhered to. If Examiners anticipate any problems they are asked to telephone the Education Department immediately.
Confidentiality
Confidentiality is of vital importance and examiners are asked to practice care in the exchange of marks and other information and to give clear guidance to their secretarial staff, especially if Specialist Registrars have access to department papers and the fax machine.
When returning marks, or submitting questions or key points, all correspondence must be marked "Strictly Personal and Confidential" for the attention of the DFPH Manager.
If marks or questions are emailed, they should be password protected and the password agreed before hand or emailed separately. The originals should be posted to the Faculty even if an email has been sent.
Examiners may not discuss candidates or their work with others apart from fellow DFPH Examiners.
Examiners may be asked to give advice to registrars on the preparation for the examination. They are warned to take care and are recommended to respond to registrars’ requested areas of revision rather than to initiate topics. During the period between the examination and up to the Examiners' meeting no direct feedback is allowed between Examiners and candidates.
Communication
Each question in the DFPH is marked by the same pair of examiners for all candidates. Examiners are expected to work closely with their co-examiner in order to agree key points and comments. After submitting individual marks to the Faculty, examiners are asked to discuss all candidates particularly where there has been a large discrepancy between the marks of two examiners. This can be done by telephone or by email - but with due regard to confidentiality.
Computers used for emailing should be password protected, and messages deleted from files as soon as possible after they have been read or printed out.
