4.3.4.4 Relative standard deviation

The RSD was calculated from the set of simulated ISM results for each iteration of the Monte Carlo simulation. Collectively, patterns in 95% UCL coverage and other performance metrics can be evaluated for different ranges of RSD. The following were noted:

  • Data sets with a high RSD are more likely to achieve specified coverage for 95% UCL of the (population) mean than data sets with low RSD. This effect is explained by the greater variability among replicates leading to higher 95% UCL values, resulting in better coverage.
  • A low RSD may intuitively appear to ensure specified coverage by the 95% UCL or low bias in a single estimate of the mean. However, the opposite is in fact the case when the underlying distribution is positively skewed (e.g., lognormal, gamma). For situations in which the UCL or one replicate mean is less than the true mean, the underestimate increases as RSD decreases. This phenomenon reflects the "proportionality effect," whereby the mean and variance are expected to be positively correlated for positively skewed distributions (Goovaerts 1997). Therefore, when the mean is relatively low, so too is the SD. Taken together, there is a greater likelihood that the UCL exhibits insufficient coverage.