Worked Examples

To illustrate the use of HFI PRA three exemplars have been devised.

The first case is a full implementation of the most formal use of HFI PRA, namely a Pre Contract Award Evaluation (PCAE) prior to down selection at Main Gate.

The second case is an operability improvement initiative conducted by an IPT as part of its working together breakthrough initiative.

The third case is a fairly formal Process Improvement initiative conducted within an organisation.

PCAE - Case 1

Prior to a decision to conduct a formal CE as part of tender assessment, there have been a number of relevant HFI activities. Early HF Analysis (EHFA) has identified risks, issues and opportunities. The phase outputs has included a URD and an SRD. The URD contains performance requirements for user tasks that have been assessed as part of the in-phase COEIA activity, and they have been taken as criteria for acceptance testing. The SRD contains specific performance criteria that constitute usability goals for system development.

In generating the ITT, the IPT has included programme requirements in the SMIR related to risk and opportunity areas. In reviewing the project risk register, it transpired that HFI risks appeared near the top of the impact assessment and that there were informal concerns about the ability of the candidate suppliers to address these issues.

It was decided that including the mention of an HFI CE in the ITT would

  1. give due emphasis to a structured approach to the topic and would encourage tenderers to assign resources to HFI and
  2. would provide a structured means of mitigating the risks, and managing their transfer to industry.

The alternative risk reduction strategies identified were making detailed demands for an acceptance test programme, or an expansion of the HFI requirements. The alternatives were rejected on the grounds that they could be easily circumvented by a supplier who was so minded, increased DPA and supplier costs, and limited the transfer of risk and design responsibility.

The risks included a specific risk associated with human reliability and a fairly generic risk associated with the difficulty of delivering a complex system to meet demanding usability goals in a short development time. .

The link between risks and processes from the reference model was made at an IPT meeting facilitated by an experienced assessor, who established maturity levels for specific processes that would be a) considered acceptable and b) considered definitely unacceptable. The range of previous experience, and the size of the organisational unit that was relevant was also agreed.

For the human reliability risk, the processes considered important were ‘Context of Use’ for the assessment of Performance Shaping Factors, and ‘Integration’ for the effective application of HRA information.

Possible causes of lack of HCD design influence were a lack of clear HFI management processes, a lack of integration between HFI and the design/development programme and a concern over resources

There was debate as to the specifics that might cause the lack of design influence that had been identified as a risk. The possible issues that might lead to the risk included lack of designer awareness and commitment, lack of structured HFI processes, lack of recognition of specific dependencies, and a lack of resources (due to a lack of commitment by management).

The processes selected for the generic risks were ‘Integration’ (HS.2.7), ‘Conception Lifecycle Process’ (HS.1.1). The next step was to assemble an evaluation team from the IPT, CE evaluation specialists and HFI specialists. The team went through a one-day intensive workshop on how to evaluate the processes in the specific context of the project risks. The site visit was planned to take two days, comprising a briefing, a set of parallel interviews with a range of members of the reference projects, and a debrief. A site visit was undertaken for each of the three suppliers.

The follow through actions comprised agreeing a PI programme with each supplier and feeding the weighted findings into the tender assessment process.

IPT Breakthrough working together initiative – Case 2.

In contrast to the previous study this use of CMM was

  1. a 360 degree assessment of all IPT stakeholders
  2. aimed at structured PI rather than an assessment.

The initiative started with a structured workshop.

The first part of the workshop aimed at identifying the key processes that cross IPT stakeholders, and which were considered critical to project success. The project risk register was used as reference material, but other risks were considered. The basis for the workshop was simple flip-chart material summarising the model.

The project was still at the very early stages and EHFA activity was still underway, and so it was decided to combine the initiation of PI with further identification of HFI risks, issues and opportunities. The life cycle processes were assigned to appropriate stake holders (from the DEC, DPA, industry, Customer 2 and the PPO) and issues were brainstormed.

The principal objectives for the second part of the workshop were

  1. for each stakeholder to start to develop their elements of an integrated HFIP, drawing on the reference model as a source of best practice and tailoring it to the needs of the project (in an auditable manner) and
  2. building a shared vision for HFI integration.

The exercise was focused by taking a number of specific phase outputs and discussing the means by which the stakeholders would contribute to specific elements, and how these could be kept coherent. The lifecycle processes (HS.1) were reviewed to establish the role of each of the stakeholders through life, and what preparatory and follow-up activities would be required to support each stage. The usability engineering (HS.3) processes were reviewed (with contractor leadership), aimed at identifying how involvement by other stakeholders would reduce risk and improve exploitation of opportunities.

The project risk register and EHFA outputs were updated, and milestones for reviewing PI progress were incorporated into the HFIP Part 1. The record of the event was incorporated into Part 2 of the HFIP.

Structured PI – Case 3.

This case study involves a contractor who wishes to gain provable track record on a current project. The aim of using this experience is support tendering for a future project where it was recognised that assurance of operability might be sufficient to offset the price disadvantage of a high quality bid against lowest-price based competition.

The commercial objectives here were to

  1. achieve customer buy-in
  2. gather cost metrics related to HFI and
  3. get up the learning curve and reduce costs and cost risk.

The contractor set up self-assessment workshops (with customer and supplier involvement) to establish the current level of maturity for the processes he considered key to future success. A separate workshop (with customer involvement) discussed the profile required for the prospective tender.

Process Improvement Groups were set up within the contractor organisation – one for each process targetted. These teams reported back to a joint contractor/customer steering group on a six-weekly cycle. Towards the time of the ITT, an external assessment was made of the processes concerned and included in the tender.