A.1.1 Consensus Points
The presentation of simulation results is organized by points of consensus among the ISM workgroup. Each consensus point was guided by statistical theory and supported by results of simulation experiments. Table A-1 lists the consensus points, grouped by topics that are relevant to the overall sample design.
Table A-1. Consensus points guided by simulation experiments using probability distributions (PD) and maps (M)
Effects of the number of increments and replicates on the estimate of the mean | |
---|---|
1 | Increasing the number of increments and/or replicates reduces variability in the estimate of the mean. |
2 | Variability in the grand mean (i.e., the mean of the replicate incremental sampling estimates of the mean) is a function of the total number of increments collected (increments × replicates). |
3 | DUs with high heterogeneous contaminant concentrations have greater variability in the estimate of the mean and greater potential for errors in terms of both frequency and magnitude. Underestimates of the mean would be expected to occur more frequently than overestimates for heterogeneous sites with right-skewed contaminant concentration distributions. With equal numbers of samples (i.e., individual discrete samples vs. ISM replicates), the magnitude of error in estimating the mean would be expected to be lower using ISM. |
4 | The coverage of the 95% UCL depends on the total sample size (increments × replicates). For the typical number of increments of an ISM sampling design (e.g., 30–100), increasing the number of ISM replicates above 3 provides marginal return in terms of improving coverage; however, increasing the number of replicates decreases (i.e., improves) the RPD, meaning that it will produce estimates of the 95% UCL closer to the DU mean. |
5 | Simulations produced varying results in terms of improvement in coverage by increasing the number of increments. As with increasing replicates, increasing the number of increments decreases (i.e., improves) the RPD. |
6 | Coverage provided by the two UCL calculation methods depends on the degree of variability of the contaminant distribution within the DU. For DUs with medium or high heterogeneity, the Student’s-t method may not provide specified coverage. For DUs with high heterogeneity, the Chebyshev method may not provide specified coverage as well. |
7 | The Chebyshev method always provides a higher 95% UCL than the Student’s-t method for a given set of ISM data with r > 2. When both methods provide specified coverage, the Chebyshev consistently yields a higher RPD. |
Effects of sampling pattern | |
8 | If the site is relatively homogeneous, all three field sampling patterns yield unbiased mean estimates, but the magnitude of error in the mean may be higher with simple random sampling compared to systematic random sampling. All sampling patterns yield similar coverages. |
9 | While all three sampling options are statistically defensible, collecting increments within the DU using simple random sampling is most likely to generate an unbiased estimate of the mean and variance according to statistical theory. From a practical standpoint, true random sampling is probably the most difficult to implement in the field and may leave large parts of the DU “uncovered,” meaning without any increment sample locations. It should be noted that “random” does not mean wherever the sampling team feels like taking a sample, and a formal approach (e.g., based upon a random number generator) to determining the random sample locations must be used. |
10 | Systematic random sampling can avoid the appearance that areas are not adequately represented in the ISM samples. This approach is relatively straightforward to implement in the field. Theoretically, it is inferior to simple random sampling for obtaining unbiased samples and can be more prone to producing errors in estimating the true mean, especially if the contamination is distributed in a systematic way. Random sampling within a grid is, in a sense, a compromise approach, with elements of both simple random and systematic sampling. |
Subdividing the DU | |
11 | Sampling designs with this method yield unbiased estimates of the mean. |
12 | The principal advantage of subdividing the DU is that some information on heterogeneity in contaminant concentrations across the DU is obtained. If the DU unit fails the decision criterion (e.g., has a mean or 95% UCL concentration above a soil action limit), information will be available to indicate whether the problem exists across the DU or is confined to guide redesignation of the DU and resampling to further delineate areas of elevated concentrations. |
13 | Partitioned DU standard error estimates are larger than those from replicate data if the site is not homogeneous. Hence, 95% UCL estimates from a subdivided DU will be as high or higher than those obtained from replicate measurements collected across the DU. The higher 95% UCLs improve coverage (generally attain 95% UCL) and increase the RPD. These increases occur if unknown spatial contaminant patterns are correlated with the partitions. In most cases, the Student’s-t method provides adequate coverage. |
Relative standard deviation | |
14 | Data sets with a high RSD are more likely to achieve specified coverage for 95% UCL than data sets with low RSD. This tendency is explained by the greater variability among replicates leading to higher 95% UCL values, resulting in better coverage. |
15 | A low RSD does not ensure specified coverage by the 95% UCL or low bias in a single estimate of the mean. The opposite is in fact the case. For situations in which the UCL or one replicate mean is less than the true mean, the underestimate increases as RSD decreases. |
The simulation findings presented in this appendix do not represent the totality of simulation exercises conducted as part of this project. Some sets of simulations were subject to different interpretations, yielded inconsistent findings, or were repetitive. That some sets of the simulations were inconsistent or viewed differently within the ISM statistics workgroup is not surprising given that exploration of the statistical implications of ISM is a relatively new field. The reasons for differences were still being considered during development of this ITRC document. It is anticipated that additional research may be needed to further investigate the performance of alternative ISM sample designs. To avoid confusion and limit presentation to essential material, Table A-1includes only findings related to consensus points. Two of the simulation approaches below (Sections A.4 and A.5) have documented their work, and additional simulations can be found in technical documents (Hathaway and Pulshipher 2010; Singh, Singh, and Murphy 2009). The simulation approaches in Sections A.2 and A.3 are presented here for the first time.