On the use of criteria based on the SMART acronym to assess quality of performance indicators for safety management in process industries

Abstract

Management of safety, and barriers in particular, includes using information expressing performance, i.e. use of safety performance indicators. For this information to be useful, the indicators should demonstrate adequate quality. In other words, they should satisfy some predefined set of quality criteria. Without showing adequate quality, the indicators are generally unable to provide sufficient support for barrier management, which could result in poor decisions. In this article, the use of the SMART criteria is considered to assess the quality of safety performance indicators in process industries. SMART being an acronym for ‘specificity’, ‘measurability’ or ‘manageability’, ‘achievability’, ‘relevancy’ and ‘time-based’, covering five key aspects and criteria for assessing the quality of an indicator. A discussion on whether the indicators are able to demonstrate adequate quality by satisfying these criteria has been conducted. The finding is that all of the SMART criteria should be satisfied for a safety performance indicator to demonstrate acceptable quality and to be regarded as useful to support barrier management decision-making. However, it has also been observed that including the ‘M’ criterion in the assessment of quality is not needed. When all the other criteria are satisfied there is no way the conclusions could be misleading as a result of measurability or manageability aspects. Hence, for safety performance indicator quality, only four of the criteria are assessed and suggested for such situations to shorten the acronym to ‘STAR’. A key safety indicator used in downstream process facilities, i.e. ‘dangerous fluid overfilling events’, motivated from the 2005 Texas City refinery accident, is used to illustrate the situation. The indicator is also applied to another incident, the Buncefield oil storage depot's accident in 2005, to provide a broader context for using it. The findings in this article could also be applied beyond the context studied. This means that, despite focusing on safety indicators in the process industries, the findings are considered as relevant and applicable to other types of performance indicators and to other energy industries.