Model-Assisted Estimation

class: center, middle, hide_logo
background-image: url("img/daggett6.001.jpeg")
background-position: left
background-size: contain
background-color: #e9ecef

## Model-Assisted Estimation

#### Data Science in Official Statistics

#### 2024 International Conference on Establishment Statistics

#### Kelly McConville & Wesley Yung

####  Monday, June 17th, 2024

]

---

# Recap: Data Science Opportunities in Official Statistics

---

# Focus on Estimation Stage!

---

### Model-Assisted Survey Estimation

#### **Goal**: Estimate finite population quantities, such as means and totals.

#### Data Needed:

---

### Estimation Set-up

Enumerate the finite population of interest.

`$U = \{1, 2, \ldots, N\}$`

---

### Estimation Set-up

**Goal:** Estimate a population quantity, such as the total of a response variable, `$y$`.

`$t_y = \sum_{i \in U} y_i$`

---

### Estimation Set-up

Have auxiliary data for the entire population or summary auxiliary data.

Have `$\{\boldsymbol{x}_i\}_{i\in U}$` or `$\bar{\boldsymbol{x}}_U$`.

---

### Estimation Set-up

Construct a sampling design.  Assume a design-based approach to inference.

Have `$\{\pi_i\}_{i\in U}$` where `$\pi_i = P(i \in s)$`.

---

### Estimation Set-up

A sample is taken and you have access to the following data for estimation.

One option: `$\hat{t}_y = \sum_{i \in s} \frac{y_i}{\pi_i}$`

(Horvitz-Thompson Estimator)

---

### Estimation Set-up

Another option: Use the sample to .orange2[**build**] a model and then .orange2[**predict**] `$y$` over all the population units.

`$\hat{t}_y = \sum_{i \in s} y_i + \sum_{i \in U-s} \hat{m}(\boldsymbol{x_i})$`

---

### Estimation Set-up

Estimator: `$\hat{t}_y = \sum_{i \in s} y_i + \sum_{i \in U-s} \hat{m}(\boldsymbol{x_i})$`

Estimator can be rather .orange2[biased] when the model is misspecified.

---

### Estimation Set-up

**Another approach:** Use a bias-correction term.

Model-assisted estimator:

`$$\hat{t}_y = \sum_{i \in U} \hat{m}(\boldsymbol{x_i}) + \sum_{i \in s} \frac{y_i - \hat{m}(\boldsymbol{x_i})}{\pi_i}$$`

---

### Come back to the specifics of the estimator soon...

## But first, do we even have the necessary data?

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

> **Mission**: "Make and keep current a comprehensive inventory and analysis of the present and prospective conditions of and requirements for the renewable resources of the forest and rangelands of the US."

Focusing on Daggett County, Utah:

Quasi-systematic sample of ground plots

]

**Many** layers of remotely sensed data!

]

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

Maintain many different surveys:

* Job Openings and Labor Turnover Survey
* Occupational Employment and Wage Statistics

]

Quarterly Census of Employment and Wages:

* Size class

* Geographic information

* Industry type

* Whether or not its a multi-establishment firm

]

---

## Discussion Questions

### What survey data you have access to?

### What auxiliary data do you have access to?

### Are you currently combining these data sources for estimation?

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

Focusing on Daggett County, Utah:

Quasi-systematic sample of ground plots

]

**Many** layers of remotely sensed data!

]

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

* Want a simple model.
--

* Have lots of auxiliary data layers.

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

Use linear assisting model but estimate coefficients with the .orange2[elastic net]:

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

Use linear assisting model but estimate coefficients with the .orange2[elastic net]:

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

Use linear assisting model but estimate coefficients with the .orange2[elastic net]:

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

Use linear assisting model but estimate coefficients with the .orange2[elastic net]:

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

Use linear assisting model but estimate coefficients with the .orange2[elastic net]:

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

**Selected Predictors**:

* Normalized Difference Vegetation Index
* Slope
* Normalized Burn Ratio
* Elevation
* Slope : Forest/Non-Forest

]

**Non-Selected Predictors**:

* Forest Probability
* Eastness
* Forest Probability : Forest/Non-Forest
* Normalized Difference Vegetation Index : Forest/Non-Forest
* Normalized Burn Ratio : Forest/Non-Forest
* Elevation : Forest/Non-Forest
* Eastness : Forest/Non-Forest

]

---

###  Case Study 1: [US Forest Inventory and Analysis Program](https://www.fia.fs.fed.us/about/about_us/)

**Goal**: Estimate .orange2[MANY] finite population quantities!

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

**Goal**: Estimate the total number of certain job types.

Complex survey data from the BLS Occupational Employment Statistics Survey

]

Quarterly Census of Employment and Wages:

* Size class

* Geographic information

* Industry type

* Whether or not its a multi-establishment firm

]

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

**Goal**: Estimate the total number of certain job types.

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

**Goal**: Estimate the total number of certain job types.

* We have mostly categorical variables with many categories.

* The industry code is likely important and most job types probably only fall into a few of the industries.

* Additive model is too simple and the fully interactive is too complex with not enough data in many sub-groups.

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

Use regression trees instead!

* Recursively split sample into two groups based on an auxiliary variable.

* Industry has 24 categories.
    + Industry #71:  Arts, Entertainment, and Recreation
    + Industry #72:  Accommodation and Food Services

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

* .orange2[Recursively] split sample into two groups based on an auxiliary variable.

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

* .orange2[Recursively] split sample into two groups based on an auxiliary variable.

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

At each end node, the estimator is given by the .orange2[survey-weighted mean].

---

### The Regression Tree Estimator

`$$\hat{t}_y = \sum_{i \in U} \hat{m}(\boldsymbol{x_i}) + \sum_{i \in s} \frac{y_i - \hat{m}(\boldsymbol{x_i})}{\pi_i}$$`

Let `$Q = \{ B_{1}, B_{2}, \ldots, B_{q} \}$` be the of partitioning boxes from the tree.

For `$\boldsymbol{x} \in B_k$`:

`\begin{align*}
    \hat{m}({\boldsymbol x})=
   \widehat{\#}(B_{k})^{-1} \sum_{i \in s} \pi_{i}^{-1} y_i I ({\boldsymbol x}_i \in B_{k})
\end{align*}`

where

`\begin{align*}
   \widehat{\#}(B_{k}) = \sum_{i \in s} \pi_{i}^{-1} I ({\boldsymbol x}_i \in B_{k}).
\end{align*}`

---

### The Regression Tree Estimator

* Splits are selected based on a permutation test.

* Tree stops splitting based on a p-value threshold or minimum end node size.

* R package for regression trees built on complex survey data: [rpms](https://cran.r-project.org/web/packages/rpms/index.html)
    + Written by [Daniell Toth](https://scholar.google.com/citations?user=1lUPZikAAAAJ)
        + Software demo: Wed at 10am in the Sir Alex Ferguson Library

---

### The Regression Tree Estimator

#### Calibration

Can linearize the trees:

`$$\hat{m}({\bf x}) = \tilde{\mu}_{n1}  I({\bf x} \in B_{n1}) + \tilde{\mu}_{n2} I({\bf x} \in B_{n2}) + \ldots + \tilde{\mu}_{nq}  I({\bf x} \in B_{nq})$$`

* Regression tree is .orange2[calibrated] to population totals of the end nodes:

`$$\sum_{j \in U} I({ \bf x}_j \in B_{nk}) =   \sum_{j \in s} \tilde{w}_j I({ \bf x}_j \in B_{nk})$$`

----

#### Post-stratification

* Post-stratified estimator: Generalized regression estimator where the assisting model is the group mean model

* Regression tree estimator: End nodes of the tree are a set of .orange2[predictive] post-strata!

---

### Case Study 2: [US Bureau of Labor Statistics](https://www.bls.gov/bls/blsmissn.htm)

> **Mission**: "Measures labor market activity, working conditions, price changes, and productivity in the U.S. economy to support public and private decision making."

**Goal**: Estimate the total number of certain job types.

---

### General Thoughts on Selecting the Model

#### Consider your audience.

#### Figure out what data you have access to.

#### Do lots of exploratory data analysis.

#### Analyze the resulting survey weights.

---

## Questions?

---

## Up next:

### Break

### Implementation in `R`!