Types of Assessment

Intro to assessment, staged models vs risk-based

Intro to contents of next level

Intro to assessment

The model-based method of Capability Evaluation (CE) has  three styles of use.

Project monitoring,  and specifically monitoring of Process Improvement (PI).

Internal assessment of  its own processes by an enterprise (or part of an enterprise).

Contractor selection in major projects where quality in use has been identified as a risk area - Pre  Contract Award Evaluation (PCAE).

The scope of an assessment (in terms of processes assessed)  is tailored to the needs of the project. Similarly, the size of an assessment  (in terms of organisations assessed) can be tailored to circumstances. It is possible to conduct very low effort assessments at the early stage of a project,  on small projects and for informal process risk/PI purposes.

Process standards are principally concerned with two sorts of use, process improvement (PI), and capability evaluation (CE). However, they have proved to have other uses e.g. planning, competence definition.  PI is typically conducted by an organisation for its own benefit (but can be mandated by a customer) and comprises a cycle of self-assessment, structured improvement and re-assessment.  CE can vary in formality, but at its most formal is done by an accredited third party for the purposes of pre-contract award vendor assessment.  The systems community has developed COTS tools and methods for both PI and CE, one of the many benefits from adopting the process standard approach.

The diagram below shows how project risks and project  requirements provide the inputs to a Process Risk Assessment, which is followed by Process Improvement action and tracking.

Process assessment, capability and maturity

A  disciplined evaluation of an organisationís processes against a model is called process assessment. Process assessments generally focus on identifying improvement priorities (i.e. a formative evaluation).   Action taken to change an organisationís  processes so that they meet the organisationís business needs and achieve its business goals more effectively is called process improvement.   Process assessment and improvement are described in more detail below.   Process  assessment seeks firstly to establish whether processes are performed  successfully and secondly the degree to which processes are under control.   A scale of capability is used in this  assessment.  

A  capability scale is an ordinal scale of types of control. The table below describes the most common scale. This uses six levels to assess process capability.

Process  capability scale

Level

Description

0

No achievement of results  from a processes or processes

1

Performance in an ad hoc manner

2

Monitoring of time and product quality

3

Use of defined corporate procedures and infrastructure

4

Use of statistical control

5

Optimisation of each process to meet current and future business needs

Some words on the approach to 'capability' and 'maturity' are appropriate to the  reader familiar with the terms from CMM ®.   This paper is not the place for a debate on  the merits of 'staged' and 'continuous' models.   The HS model is a 'continuous' model in accordance with the ISO approach, whereby processes are selected for assessment and the organisation's capability  to perform a process is assessed against a capability scale.   It can of course be mapped to the stages of a staged model.

A  staged model such as CMM offers strong intuitive appeal by presenting a set of  organisational maturity levels, which are recognisable plateaux of aggregated  process capability.   The application of this concept to HCD featured in the research leading to the UMM, in the form of  a Human Centredness Scale (Earthy, 1998).   This scale portrays levels of human-centredness from X (Unrecognised) through to E (Institutionalised) via A (Recognised), B (Considered), C  (Implemented) and D (Integrated).  

Human-centred  methods developed for organizations with high levels of maturity cannot be  adopted by organizations at a lower level of maturity. The supporting processes  and information flows necessary to make the method work do not exist and the attitude necessary to put them in place is not widespread enough to make a  difference.   The scale has been used for  a variety of purposes, notably significant evaluations in the Chinese IT  sector.  

The  authors consider that the primary use of the HCS is educational and motivational, rather than formal assessment.   A limited process capability approach might be adapted for similar ends using sub-processes from the HS model selected from the strategic end of HS.2, such as HS.2.1, HS.2.2, HS.2.3.   More formally, as mentioned above, ISO PAS 18152 has an annex that identifies the  sub-processes that require or imply a level of process maturity above level  1.  

  

Lloydís Register of Shipping and Process Contracting Limited © 2003