Appendix A

Statistical Simulations

A.1 Introduction

This appendix presents additional details regarding the simulation studies used to evaluate the performance of alternative ISM sampling strategies applied to DUs with a range of heterogeneities. Monte Carlo methods were used to collect hypothetical incremental samples following various spatial sampling protocols. As noted in Section 4 of the main text, a range of DU scenarios was investigated to explore the effect of various factors on statistical performance metrics. The following factors were varied:

  • number of increments
  • range of variability
  • number of replicates
  • spatial patterns
  • sampling methods
  • methods of accounting for compositional and distributional heterogeneities
  • sampling patterns
  • choice of UCL calculation method

The following performance metrics were used to evaluate the influence of these factors on ISM results:

  • coverage of UCL–absolute and relative bias in the estimate of the population mean
  • absolute and/or relative percent difference between the UCL and true mean—standard deviation of relative bias in the population mean
  • relative standard deviation of replicate means

The main advantage of simulations is that population parameters are known. Therefore, alternative sampling approaches and calculation methods can be explored for a wide range of scenarios. With each simulation, the same sampling method and/or calculations are performed many times, as if a hypothetical field crew repeated the sampling effort over and over. Because each sampling event involves random sampling from the population, no two hypothetical events yield identical results. However, by repeating the exercise many times, we generate a distribution of results from which we can evaluate the various performance metrics noted above.

Not every performance metric is captured in every simulation, in part, because the simulations use different approaches to represent bulk material heterogeneity in a DU. Summary tables and discussions of each simulation clarify what metrics were evaluated and how this information can be used to guide in the selection of ISM sampling protocols. None of the simulations attempt to explicitly define all seven sources of error in estimates of the mean associated with bulk material sampling (refer to the main text and Appendix E for a discussion of Gy’s principles). The simulations focus on representing the compositional and distributional heterogeneities (CH and DH) that can be attributed to fundamental error (FE) and grouping and segregation error (GSE).

Simulations were conducted with defined distributions (statistical or spatial) to represent the variability in sample value results that may be expected given the combined effect of these errors. Simulations allow for the evaluation of different spatial sampling patterns that cannot be evaluated empirically because the true population parameters (e.g., population mean) are typically unknown. Naming conventions applied to each simulation experiment include a prefix "PD" for simulations with probability distributions and "M" for simulations with maps. The PD simulation approach involved randomly sampling from a two-parameter lognormal probability distribution with a specified mean and variance (PD). The ratio of the population parameters (i.e., standard deviation divided by the mean), also known as the coefficient of variation (CV), provides a measure of variability that facilitates comparisons of results across a wide range of conditions. The M approach involved the use of maps (2-D surfaces) to sample from alternative spatial distributions of soil contamination (M). Each set of maps has unique implementations that provide the ability to demonstrate a range of different DU conditions. The method to simulate the soil data for each set of maps follows:

  • M-1–Based on a real data set of more than 200 observations. The sample results were interpolated with inverse distance weighting techniques to yield a completely defined 2-D surface of concentrations (see Section A.3).
  • M-2–Maps are based on real DU data composed of bulk materials. The patterns and concentration values are established from extensive discrete data (100 increments per DU) gathered as a part of multiple ESTCP projects led by Jenkins and Hewitt. Hathaway and Pulsipher (2010) document the specific details for how the discrete data were used to establish the completely defined 2-D surface of increment values shown Section A.4.
  • M-3–"Bulk material" DUs, hypothetical homogeneous and heterogeneous DUs mimicking bulk material (e.g., soils) DUs are generated using the "MIS Module" of the USEPA software Scout 1.1 (USEPA n.d. "Scout 1.1"), as discussed in Section A.5.

Collectively, the simulation studies presented in this appendix provide a preliminary set of results intended to facilitate the development of ISM sampling designs and corresponding statistical analyses. More detail and underlying assumptions of the different simulation approaches are identified below.

Simulations presented in this appendix refer to different scales of heterogeneity as being "small" and "large." These are not intended to imply a precise dimension for a DU in terms of acres. Instead, the terms are relative to the size of the DU. "Small" scale refers to the immediate vicinity of the incremental sample, whereas "large" scale refers to the overall spatial extent of the DU.

Appendix E is a glossary of terms relevant to ISM. A glossary is also included at the end of this appendix to provide an expanded discussion of the definitions of key terms and concepts.