4.3.1 Definitions of Performance Metrics

Performance metrics for a 95% UCL that can be evaluated when the true mean is known (or assumed):

  • UCL coverage
  • relative percent difference between UCL and true mean
  • bias
  • relative standard deviation of replicate means

Performance metrics provide a way to systematically evaluate and compare various sampling strategies, including sampling pattern, statistical sample size (both number of increments and replicates), and UCL computation techniques. Collectively, these results can help to establish an optimal decision process for using ISM given a particular set of site conditions and decision criteria. The following four metrics are defined below and evaluated through simulation studies:

  • coverage of the UCL
  • magnitude of UCL deviation from the mean (i.e., RPD between UCL and true mean)
  • bias of the mean of the samples
  • RSD of the estimates of the mean from each replicate ISM sample