Industrial Statistics: Sampling and Weighting Procedures

1. Estimates and sampling errors for Establishment Surveys Estimates and sampling errors for Establishment Surveys International Workshop on Industrial Statistics International Workshop on Industrial Statistics Beijing, China, 8-10 July 2013 Beijing, China, 8-10 July 2013

2. Weighting procedures Weighting procedures Important to calculate appropriate weights based on probabilities of selection Important to calculate appropriate weights based on probabilities of selection Maintain information on geographic, stratification and identification codes of all sample establishments with corresponding probabilities of selection Maintain information on geographic, stratification and identification codes of all sample establishments with corresponding probabilities of selection This spreadsheet or database with the sample establishments can be used for calculating the weights This spreadsheet or database with the sample establishments can be used for calculating the weights

3. Calculation of weights list frame Calculation of weights list frame Probability of selection for sample establishments in stratum h (ISIC group, employment size) in list frame Probability of selection for sample establishments in stratum h (ISIC group, employment size) in list frame Weight for sample establishments in stratum h Weight for sample establishments in stratum h

4. Calculation of weights area frame Calculation of weights area frame Overall probability includes components from each sampling stage Overall probability includes components from each sampling stage Weight Weight

5. Adjusting weights for nonresponse Adjusting weights for nonresponse Unit nonresponse Unit nonresponse Important to determine reason for non-interview Important to determine reason for non-interview Sample establishment no longer exists (death) Sample establishment no longer exists (death) Respondent not available Respondent not available Refusal Refusal Weight adjustment should be done within each ISIC group and size stratum Weight adjustment should be done within each ISIC group and size stratum

6. Types of estimates in Establishment Surveys Types of estimates in Establishment Surveys Weighted totals Weighted totals Weighted total revenue, number of employees, etc. Weighted total revenue, number of employees, etc. W hi = weight of i-th establishment in stratum h W hi = weight of i-th establishment in stratum h Y hi = value of variable Y for i-th establishment in stratum h Y hi = value of variable Y for i-th establishment in stratum h

7. Adjusting weights (continued) Adjusting weights (continued) Very large establishments that do not respond may require special adjustment procedures Very large establishments that do not respond may require special adjustment procedures Special follow-up to complete response Special follow-up to complete response Imputation based on historical data Imputation based on historical data Adjustment procedure may vary by size of establishment Adjustment procedure may vary by size of establishment

8. Non-interview adjustment factor Non-interview adjustment factor Calculating weight adjustment factor Calculating weight adjustment factor W h = adjusted weight for the establishments in stratum h W h = adjusted weight for the establishments in stratum h n h = number of valid sample establishments selected in stratum h (excluding those which no longer exist) n h = number of valid sample establishments selected in stratum h (excluding those which no longer exist) n h = number of establishments with completed interviews in stratum h (including replacements) n h = number of establishments with completed interviews in stratum h (including replacements)

9. Types of estimates (continued) Types of estimates (continued) Means and proportions Means and proportions Expressed as ratio of two totals Expressed as ratio of two totals Variables X and Y can be continuous values or binomial (0,1) variables Variables X and Y can be continuous values or binomial (0,1) variables

10. Ratio estimates of totals Ratio estimates of totals Ratio estimation useful for estimating totals, using auxiliary information from frame or administrative sources and growth rate from survey data Ratio estimation useful for estimating totals, using auxiliary information from frame or administrative sources and growth rate from survey data Numerator and denominator of ratio should be highly correlated Numerator and denominator of ratio should be highly correlated

11. Ratio estimates of totals (continued) Ratio estimates of totals (continued) Improves accuracy of results when reliable information for denominator variable is available for entire frame Improves accuracy of results when reliable information for denominator variable is available for entire frame Example 1: estimating total value of production from annual survey when census data are available from previous year Example 1: estimating total value of production from annual survey when census data are available from previous year Example 2: estimating total rice production when total area of cultivated rice land is available, together with post- harvest data on rice production by hectare from a sample survey Example 2: estimating total rice production when total area of cultivated rice land is available, together with post- harvest data on rice production by hectare from a sample survey

12. Indexes Indexes Index of Industrial Production Index of Industrial Production Y = output Y = output I p = price index I p = price index I q = volume index I q = volume index t = current period t = current period t-1 = previous period t-1 = previous period

13. Types of error in estimates from sample surveys Types of error in estimates from sample surveys Sampling error of an estimate variability in value of estimate based on a sample of the population Sampling error of an estimate variability in value of estimate based on a sample of the population Measured by the standard error, or square root of the variance Measured by the standard error, or square root of the variance Nonsampling error all other types of error Nonsampling error all other types of error Examples: response error, other measurement errors, nonresponse, coding, data entry and processing errors Examples: response error, other measurement errors, nonresponse, coding, data entry and processing errors Can result in bias Can result in bias

14. Total error Total error Measured by the mean square error (MSE) Measured by the mean square error (MSE) MSE measures accuracy of survey estimate MSE measures accuracy of survey estimate MSE = Variance + Bias 2 MSE = Variance + Bias 2 Variance is measure of precision Variance is measure of precision

15. Accuracy of survey estimates Accuracy of survey estimates Accuracy depends on both sampling and nonsampling error Accuracy depends on both sampling and nonsampling error Publication of results from Economic Surveys should include annex with table of sampling errors and confidence intervals for most important estimates Publication of results from Economic Surveys should include annex with table of sampling errors and confidence intervals for most important estimates Calculation of variances should reflect any stratification and clustering in the sample design Calculation of variances should reflect any stratification and clustering in the sample design

16. Software for tabulating sampling errors from complex samples Software for tabulating sampling errors from complex samples SPSS Complex Samples module SPSS Complex Samples module Stata Stata Ultimate cluster or Taylor series variance estimator Ultimate cluster or Taylor series variance estimator Variance formula takes into account the stratification and clustering in the sample design Variance formula takes into account the stratification and clustering in the sample design See formulas See formulas

17. Preparing data files for calculation of standard errors Preparing data files for calculation of standard errors Data file should include sample design information Data file should include sample design information Stratum code Stratum code PSU code (for example, PSU identification) PSU code (for example, PSU identification) Weight Weight Sampling fraction for stratum Sampling fraction for stratum

18. Preparing data files (continued) Preparing data files (continued) Certainty establishments included in data file for calculation of variances Certainty establishments included in data file for calculation of variances Sampling rate = 1 Sampling rate = 1 Variance component from certainty stratum is 0 Variance component from certainty stratum is 0 Results in lower coefficients of variation Results in lower coefficients of variation