ASSESSOR COMPETENCE

 

A note on assessor competence has been written, proposing a scheme for assessing the competence of lead assessors, and of the assessment team as a whole. It can be downloaded by clicking here, and using save target as... with a right click.

A statement on assessor competence for an HFI Process Risk Assessment is required because it is recognised by almost all forms of process assessment that assessor competence is an essential element of a valid assessment.

It may seem premature to discuss assessor competence when there is, as yet, no formal requirement for a statement of competence for HFI practitioners, or for many other key roles in MoD projects. This is recognised as being anomalous, but accepted, because without some form of statement on assessor competence, HFI Process Risk Assessment (HFI PRA) could get off to a bad start, or indeed be discredited on these grounds. It is also recognised that statements on competence are likely to become much more widespread in the future. This note may contribute to the current discussions on HF competence (e.g. within the Ergonomics Society), and may contribute to filling the current shortfall in the professionalisation of human aspects of safety.

A statement on competence is also required because an HFI PRA has consequences. An HFI PRA conducted as a Pre Contract Award Evaluation (PCAE) has major financial consequences for tendering companies (and for the client). Where an HFI PRA is being undertaken for a PCAE involving the expenditure of public funds, then there is a clear obligation (by extrapolation of requirements in SEI CMM and ISO 15504) for assessor competence to be formally considered. Apart from the case based on professionalism, there is the need here to make the assessment lawyer-proof. The losing companies may not just appeal the finding, they may sue, and it is necessary to demonstrate that the decision to spend public funds with a different company meets criteria for probity, accountability etc. Assessor competence considers training and competence for the assessment team as a whole, and for the Lead Assessor in particular.

HFI PRA may well be done in the context of a system where human error is significant is likely to form part of the relevant safety case. It would require SQEP (Suitably Qualified and Experienced Personnel) assessors as well as validated procedures.

Even an in-house Process Improvement (PI) programme has associated costs, and significant lost opportunity costs if conducted inappropriately.

It is particularly important to have a statement of competence in the field of HF and HFI because of the multiplicity of informal career paths and lack of a single recognised form of certification or accreditation of individuals or organisations. The consequence of this diversity of personnel is that the interim arrangements proposed here may remain in place indefinitely. Whilst the general requirements for competence can be identified, it is recognised that flexibility in the way that the requirements are met will be required in any competence scheme proposed.

Whilst the attached note refers to the generic requirements of a competence scheme (recognising the International approach being adopted in the HFI CMM study), the focus in the note is strongly on the needs of the UK MoD.

The experience criteria within the proposed scheme are shown in Figures 1 and 2 below.

 

Competence required

Level of collective competence available

In-depth (>10 yrs)

Some experience (> 5 yrs)

Limited or no experience (< 5 yrs)

HSL Model, or related model assessment

 

 

 

HFI (summarising Figure 2)

 

 

 

Process assessment experience

 

 

 

User experience (e.g. context of use)

 

 

 

Project, commercial framework

 

 

 

Type of system

 

 

 

Team facilitation

 

 

 

Figure 1: Collective competence assessment

HFI Domain

Interpreted domains for defining HFI experience.

Experience required and available

Expert
(10 yrs +)

Experienced (5 yrs +)

Familiarity (< 5 yrs or professional training)

No experience

Manpower, Personnel, Training (MPT)

Training Needs Analysis (TNA) experience (accreditation)

 

 

 

 

System interaction with MPT constraints

 

 

 

 

Organisational Development, Change Management (the HSL Human Resources process)

 

 

 

 

Human Factors Engineering

Physical aspects

 

 

 

 

Cognitive aspects, HCI

 

 

 

 

System Safety

Fault Tree Analysis etc, HRA etc.

 

 

 

 

Health Hazards

Occupational Health etc.

 

 

 

 

Figure 2: HFI experience requirements for Lead Assessor and assessment team.