Semi-quantitative Approach in Risk Assessment: Inter and Intra-rater Reliability

Abstract

The semi-quantitative approach used in occupational risk assessments is quite popular among safety practitioners, namely because it allies the advantages of both the quantitative and the qualitative approaches and overcomes some of their limitations. However it still rises some concerns as the sort of methods involved haven’t been fully studied in order to assure high levels of confidence in the results they produce. This is a particularly relevant issue as the workers’ health and safety may be compromised if risk assessment results are not reliable. Therefore, a comparative study on the reliability of matrix based risk assessment methods was accomplished, particularly focused on their stability and reproducibility. The Krippendorff’s Alpha Coefficient (K) was chosen to assess inter-coder (reproducibility) and intra-coder (stability) reliability. In most cases, inter and intra coder reliability proved to be low (K <0.6) for both the risk level and the intermediate variables of each assessment method. Consequently, different priority interventions may be suggested, depending on the analyst involved and/or the moment of the risk assessment accomplishment. The obtained K values for the three groups of analysts seem contrary to the literature as the risk assessment results appear not to be dependent on their levels of experience and expertise.

Keywords: Reliability, Assessment, Inter-coder, Intra-coder, Krippendorff’s Alpha Coefficient

DOI: 10.54941/ahfe100170