2 Survey design

This section specifies the statistical and sampling design of the JUSMER survey. It states the population targeted, the frame from which cases are drawn, the units and domains of inference, the selection mechanism and its field controls, the quota and sample-size structure, the longitudinal architecture, and the post-fieldwork weighting and error framework. Where operational parameters are still being negotiated with the panel provider, they are flagged explicitly so they can be completed once the scenario is awarded.

The design is sequential and conditional. A first cross-sectional study (Stage 1, Section 2.7) is fielded on its own, with a sample size fixed by the minimum power requirements of the conjoint module (documented in the experimental-design chapter). Only if its results are satisfactory is the study scaled to a three-wave panel (Stage 2, Section 2.8). The panel architecture, and the longitudinal weighting scheme in particular, therefore applies to Stage 2 and is flagged as conditional throughout; Stage 1 is a self-standing cross-section governed by the cross-sectional weight alone. The Stage 2 design is retained in full because it specifies the target should the go/no-go decision favor continuation.

2.1 Type of survey

The survey is fielded using the CAWI methodology (Computer-Assisted Web Interviewing): a self-administered questionnaire completed over the internet, with no interviewer present, distributed to eligible members of an online access panel through a personalized link (Biffignandi & Bethlehem, 2021; Callegaro et al., 2015). CAWI is well suited to this study for four reasons. It permits rapid, lower-cost collection of large samples across the national territory; its visual and programmatic environment supports the complex randomization, per-respondent profile generation, and tasks that the experimental module in Part A requires, something impractical in telephone or paper modes; the absence of an interviewer reduces interviewer effects and social-desirability pressure on the sensitive distributive and deservingness items; and automated routing, validation, and consistency checks improve data quality at the point of entry (Fowler, 2014; Groves, 2011). The instrument is implemented as an all-device (responsive) questionnaire, so that panelists may respond from desktop, tablet, or smartphone without loss of functionality.

These possibilities come with well-documented limitations that shape every downstream decision in this section. Because access to the internet is not universal and the panel is an opt-in, non-probability frame, the covered population does not coincide with the target population, and the classical apparatus of probability inference (known selection probabilities, design-unbiased estimators, quantifiable sampling error) does not apply without additional modeling assumptions (Biffignandi & Bethlehem, 2021; Callegaro et al., 2015; Scherpenzeel & Toepoel, 2012). Self-administration without an interviewer introduces measurement error and satisficing, and professionalized panel behavior (speeding, straightlining, duplicate participation) must be actively screened (Groves, 2011). These constraints are addressed where they arise: coverage and self-selection in Section 2.2 and Section 2.10, measurement and processing error in Section 2.10, and attrition in Section 2.8.

The questionnaire is designed for an approximate response time of 15 minutes per wave. Stage 1 is a self-standing cross-sectional study, sized for the conjoint module (Section 2.7) and analyzed on its own terms; it also serves to assess the instrument and the field operation before any longitudinal commitment. Stage 2, if activated, is longitudinal: the same panelists are re-measured across waves to observe change rather than a single cross-section. The staged structure and the wave timing are set out in Section 2.7 and Section 2.8.

2.2 Target Population and Survey Population

The target population is defined as persons aged 18 or older residing in the national territory during fieldwork. The conditions of a self-administered questionnaire do not substantively redefine this population (Groves, 2011).

The survey population (covered population) is the subset of the target population that has internet access, belongs to the provider’s panel, and is available during the field period. Because the panel is a voluntary opt-in, non-probability structure, the survey population does not fully coincide with the target population: persons without internet access and eligible persons who are not panel members have, in practice, no chance of selection. This coverage gap is a defining feature of the design rather than an incidental shortcoming, and it is carried forward explicitly into the interpretation of results and into the error assessment in Section 2.10.

2.3 Units of Analysis and Domains

The observation unit, information unit, and sampling unit coincide: the individual panelist aged 18 or older who completes the self-administered questionnaire. The unit of analysis is the population aged 18 or older resident in Chile, to which estimates are directed.

The study defines a single national-coverage domain. No territorial (regional or urban/rural) domains are defined, and the design is not powered to produce domain-level estimates below the national level. Sample size is therefore driven by the analytical and experimental objectives of each stage rather than by sub-domain precision targets (Section 2.6).

2.4 Sampling Frame

A sampling frame refers to the list, database, or operational mechanism that makes it possible to define, identify, and access the elements of the population covered by a survey. In practice, the frame defines the set of units from which the sample can be selected and, therefore, determines the empirical scope of the study and its potential sources of error (Groves, 2011).

In this case, the sampling frame is the online panel operated by the provider (Netquest). It is a non-probability, voluntary-membership frame that may exhibit self-selection bias and internet-access coverage bias. To document the frame’s properties and support the coverage assessment, the provider is required to report: the total panel size and the number of active panelists; recruitment mechanisms; eligibility criteria; the profile variables available for targeting and validation; updating and duplication procedures; and activity, invitation, and participation rates.

Because the frame is not a probability list of the target population, no frame-based inclusion probabilities can be derived from it; selection proceeds by quota-controlled invitation (Groves, 2011) (Section 2.5), and inference to the target population depends on the modeling and calibration described in Section 2.9.

2.5 Sampling Strategy and Sample Selection

The survey design employs a non-probabilistic quota-based selection strategy, implemented through targeted invitations (purposive sampling). In this type of design, the sample is not selected based on known probabilities of inclusion, but rather on sample composition specifications defined in advance, aimed at replicating certain population distributions across relevant control variables (Callegaro et al., 2015; Lohr, 2010). The provider draws invited panelists from profile information already held in the frame and directs fieldwork toward the quota margins defined in Section 2.6. Quota control operates in the Stage 1 cross-section and, if the panel is activated, in Wave 1 of Stage 2; the composition of Waves 2 and 3 then results from natural panel attrition (Section 2.6, Section 2.8).

The questionnaire is programmed and hosted by the research team in surveydown, and substantive responses are stored in the team’s database rather than the provider’s; quota management relies on a technical integration between panel and instrument. Multiple access is enabled in all phases: panelists who do not complete the questionnaire in a single sitting may re-enter and resume. In the final wave, the research team, as client, manages the corresponding stop-quota to prevent any surplus over the planned sample size.

A case is counted as effective when the questionnaire is completed and passes the quality filters.

2.6 Quotas and Sample Size

Quotas are simple (marginal) quotas: independent marginal control on each variable, with no crossed cells. They are defined by sex, age, and education, benchmarked to the 2024 Census (INE); universe: population aged 18 or older resident in Chile.

Table 2.1: Marginal quota targets, population aged 18+ resident in Chile

Variable	Category	N	Percent
Sex	Male	6780956	47.76
Sex	Female	7416995	52.24
Age	18–29	3074306	21.65
Age	30–44	4128793	29.08
Age	45–59	3396454	23.92
Age	60 or older	3598398	25.34
Education	Basic or less	4079158	28.73
Education	Secondary	5874173	41.37
Education	Higher technical	1402491	9.88
Education	University or more	2842129	20.02
Source: 2024 Census (INE), population aged 18 or older resident in Chile.
ISCED (CINE) correspondence: basic or less groups codes 01, 02, 03, 10, 14, and 98; secondary corresponds to codes 24 and 25; higher technical to code 35; university or more groups codes 46, 56, and 64.
The '60 or older' band has no defined upper bound; the provider must ensure the questionnaire captures respondents aged 75 and over where such panelists exist in the frame.

Quota control scope

Sociodemographic quotas are enforced in the Stage 1 cross-section and, if the panel is activated, in Wave 1 of Stage 2. Quota compliance is not guaranteed for the later panel waves: the composition of Waves 2 and 3 arises from the natural attrition of the panel, without quota enforcement. This is the design feature that motivates the longitudinal weighting scheme in Section 2.9, which applies only if Stage 2 proceeds.

Sample size is set by the analytical and experimental objectives of each stage, not as a fixed fraction of the population (Groves, 2011). The Stage 1 cross-section is sized by the minimum power requirements of the conjoint module (documented in the experimental-design chapter; see Section 2.7). The Stage 2 sample size, should the panel be activated, is the awarded three-wave structure detailed in Section 2.8.

Known feasibility constraint (basic education)

The provider (Netquest) has flagged a feasibility constraint on panelists with basic education or less: the active pool in this stratum is materially below the 28.73% marginal target in Table 2.1. The design therefore anticipates a documented deviation in this quota, to be addressed through controlled over-invitation of the stratum / a capped achieved share with the residual gap corrected at the weighting stage — see 2.9. The final treatment must be recorded here once agreed with the provider.

2.7 Stage 1: Cross-sectional Design

Stage 1 is a self-standing cross-sectional study and the first field of the project. Its sample size is set by the minimum power requirements of the conjoint module rather than by domain precision (documented in the experimental-design chapter), and it applies the sociodemographic quotas of Section 2.6. Because it is a single measurement, it carries no inter-wave attrition and no longitudinal weight; inference relies on the cross-sectional weight alone (Section 2.9).

Stage 1 may open with a soft launch: a first tranche of 500 effective cases, after which the field halts for a data-quality and quota-pacing review.

To confirm — Stage 1 validation

Soft launch: [yes, 500 effective cases with a review pause].
Focus of the review: conjoint module behavior, deservingness and meritocracy scales, redirection links, response times, and skip/filter logic.

2.8 Stage 2: Longitudinal Design

Stage 2, activated only if the Stage 1 results are satisfactory, follows a web-panel logic: the same panelists are re-measured at successive waves to identify within-person change, with the provider guaranteeing re-contact of Wave 1 respondents in later waves (Biffignandi & Bethlehem, 2021). The awarded Stage 2 design comprises three waves, with a Wave-1 target of 4,500 effective cases decreasing to 2,473 in Wave 2 and 1,500 in Wave 3 (Table 2.2).

Table 2.2: Awarded Stage 2 sample-size and wave structure (three-wave panel)

Wave	Effective cases	Retention from previous wave (%)	Cumulative retention from Wave 1 (%)	Quota control
Wave 1	4500	NA	100.0	Wave-1 quotas (Census 2024)
Wave 2	2473	55.0	55.0	Natural attrition
Wave 3	1500	60.7	33.3	Natural attrition
Stage 2 (conditional on the go/no-go decision). Within Stage 2, sociodemographic quotas are enforced in Wave 1 only; Waves 2 and 3 result from natural panel attrition. Attrition is above the standard for panel studies and is conditioned by the feasible re-contact windows (minimum four weeks between waves).

Two features distinguish this design from a standard panel and condition the weighting scheme:

First, sociodemographic quotas are enforced only in Wave 1; Waves 2 and 3 are the product of natural attrition (Section 2.6).
Second, inter-wave attrition is high, close to half the sample at the first transition, because the feasible re-contact windows (a minimum of four weeks between waves, subject to what the provider can retrieve at each interval) trade sample size against elapsed time.

No panel refreshment or controlled replacement is applied; the decreasing N reflects natural attrition alone. Retention is 55.0% from Wave 1 to Wave 2 and 60.7% from Wave 2 to Wave 3, i.e. 33.3% cumulative from Wave 1 to Wave 3. The balanced panel of respondents present in all three waves is therefore on the order of 1,500 cases, one third of Wave 1. This attrition is treated as non-ignorable and modeled at the weighting stage (Section 2.9), not assumed away.

Wave-1 validation. If Stage 2 proceeds, Wave 1 may itself open with a validation tranche, a technical pilot or a soft launch, to re-check the conjoint module at panel scale before committing the full Wave-1 N. This is distinct from, and additional to, the Stage 1 soft launch (Section 2.7); the two belong to different stages and are not a sequence within a single field.

To confirm — Stage 2 Wave-1 validation

Form: [technical pilot / soft launch — specify which].
If soft launch: a first tranche of 500 effective cases, a pause of 1–2 weeks for data-quality and quota-pacing review, and resumption on written go/no-go approval by the team.
Focus: the conjoint module.

Longitudinal re-consent. A re-consent model across waves is required: consent is framed for open-ended longitudinal re-contact (“at least one additional measurement”) and does not promise immediate anonymization, since immediate anonymization is incompatible with re-contact. The consent wording and its ethics approval are documented in the ethics chapter; this section only records that re-consent is a design requirement affecting retention.

2.9 Weighting and Calibration

Because the frame is a non-probability panel, there are no design-based inclusion probabilities and hence no classical design weights; weighting is calibration to known population margins (Biffignandi & Bethlehem, 2021; Lohr, 2010). The two stages differ in structure and therefore in the weights they require: Stage 1 needs only a cross-sectional calibration weight, while Stage 2 needs a per-wave cross-sectional weight and, for change analysis, a longitudinal weight defined on the balanced panel. The longitudinal machinery applies only if Stage 2 is activated.

2.9.1 Stage 1: cross-sectional weight

The Stage 1 cross-section is raked to the 2024 Census margins of sex, age, and education (Table 2.1). Raking (iterative proportional fitting) finds weights \(w_i\) that reproduce every population marginal simultaneously,

\[ \sum_{i} w_i \, \mathbb{1}\{x_i = k\} \;=\; N_k \tag{2.1}\]

for each category \(k\) of each quota variable, where \(N_k\) is the Census count in category \(k\). Because Stage 1 enforces the quotas at the point of selection, this weight corrects only residual departures from the target margins, and it is the sole weight the cross-section requires.

2.9.2 Stage 2: cross-sectional and longitudinal weights

Two features force two distinct weights in Stage 2: quotas operate in Wave 1 only, and attrition in Waves 2–3 is high and non-ignorable (Section 2.6).

Cross-sectional weight, one per wave. Each wave is raked to the Census margins by the same condition (Equation 2.1). In Wave 1 this reproduces the quota structure; in Waves 2 and 3 it additionally absorbs the marginal imbalance left by attrition. These weights describe the population at each wave.

Longitudinal weight, balanced panel. For within-person change, including the multilevel distributive model, the relevant sample is the balanced panel present in all three waves (\(\approx 1{,}500\) cases). Following recruitment-times-survey logic (Biffignandi & Bethlehem, 2021), the weight combines the Wave-1 base weight with an attrition correction. Let \(R_i = 1\) if respondent \(i\) completes the final wave, and estimate the retention probability by logistic regression on Wave-1 covariates \(z_i\) (sex, age, education, access device, total response time):

\[ \hat{p}_i \;=\; \Pr(R_i = 1 \mid z_i). \]

The longitudinal weight is the base weight times the inverse retention probability, re-anchored to the Census margins by a final raking pass:

\[ w_i^{L} \;=\; w_i^{(1)} \times \frac{1}{\hat{p}_i} \;\xrightarrow{\ \text{rake to } N_k\ }\; w_i^{L\ast}. \]

Anchoring to population margins is always available because the margins are known in every wave, which is what makes this feasible without a probability reference sample.

Caveats. Inverse-probability weighting corrects attrition only through the observed \(z_i\); selection on unobservables is not resolved. With attrition near half the sample, the weights are variable, so a trimming rule is applied and precision is reported through the effective sample size and design effect,

\[ n_{\text{eff}} \;=\; \frac{\left(\sum_i w_i\right)^2}{\sum_i w_i^{2}}, \qquad \text{deff} \;=\; \frac{n}{n_{\text{eff}}}, \]

so the loss relative to the nominal \(N\) is explicit (Section 2.10). The model, auxiliary variables, achieved-versus-target margins, and weight distribution are reported so the assumptions can be assessed (Lohr, 2010).

To confirm — weighting specification

Wave-1 covariate set \(z_i\), fixed in advance: sex, age, education, access device, total response time [add/remove any].
Raking convergence criterion and weight-trimming threshold: [values].
Calibration variables beyond sex × age × education: [e.g., macrozone, if a reliable benchmark is adopted].

2.10 Precision, Uncertainty, and Non-sampling Errors

The error framework applies to both stages, with one simplification: the Stage 1 cross-section has neither inter-wave attrition nor a longitudinal weight, so its nonresponse and precision assessment reduces to the coverage, self-selection, and measurement components below. The attrition and longitudinal-weight considerations apply to Stage 2 if activated.

Because selection is non-probabilistic, design-based sampling variance and classical margins of error do not strictly apply: there is no randomization distribution from which to derive them (Biffignandi & Bethlehem, 2021; Callegaro et al., 2015). Uncertainty for descriptive estimates is therefore reported on an approximate basis, with variance estimated on the calibrated sample and interpreted as conditional on the weighting model rather than as a probability-sample confidence statement. For the experimental module in Part A, inference rests on within-survey randomization of profile attributes, which is internally valid regardless of how the sample was selected; the causal quantities (e.g., average marginal component effects) are identified by the experimental design, while their generalization to the target population inherits the sample’s coverage and selection limitations (Hainmueller et al., 2014).

In Stage 2, because the weights, and the longitudinal weight in particular, carry non-trivial variance, precision is reported through the effective sample size and the design effect rather than the nominal case counts: the useful n in Wave 3 is appreciably below the 1,500 nominal cases of the balanced panel (Lohr, 2010).

Following the Total Survey Error framework, the design attends to the following non-sampling error sources (Biffignandi & Bethlehem, 2021; Groves, 2011):

Coverage error: non-internet users and eligible non-members of the panel are outside the frame (Section 2.2, Section 2.4). This is the dominant structural error of the design.
Selection / self-selection error: opt-in membership and differential willingness to be invited; quota control constrains the achieved margins where it is enforced (Stage 1 and Wave 1 of Stage 2) but does not equalize selection propensities within categories.
Nonresponse error: arising at two stages, panel recruitment and the specific survey, and, in the panel, attrition across waves, which under the Stage 2 panel removes roughly half the sample at the first transition and is the main threat to inference in Waves 2 and 3 (Section 2.8). Because this attrition is plausibly non-ignorable, it is modeled at the weighting stage (Section 2.9) rather than assumed away. Response rates in the classical sense are not fully computable for an opt-in panel; field-rate indicators (invitation, start, completion) are reported instead.
Measurement error: self-administration without an interviewer can produce satisficing, straightlining, and order effects; sensitive items may still attract social-desirability distortion despite the reduced interviewer pressure. Instrument-side mitigations (item wording, response formats, attention checks) are documented in the instrument chapter.

These sources are not independent, and trade-offs among them (e.g., between coverage and nonresponse) are managed at the design stage rather than assumed away (Biffignandi & Bethlehem, 2021). The residual, non-eliminable errors (chiefly coverage and self-selection) are reported transparently and treated as bounds on the external validity of the estimates rather than as quantities removed by weighting.

Biffignandi, S., & Bethlehem, J. G. (2021). Handbook of Web Surveys (Second edition). Hoboken, NJ: John Wiley & Sons.

Callegaro, M., Lozar Manfreda, K., & Vehovar, V. (2015). Web survey methodology. Los Angeles London New Delhi Singapore Washington DC Boston: Sage.

Fowler, F. J. (2014). Survey research methods (5. ed). Los Angeles, Calif.: SAGE.

Groves, R. M. (2011). Survey Methodology. (F. J. Fowler, M. Couper, J. M. Lepkowski, E. Singer, & R. Tourangeau, Eds.) (2nd ed (Online-Ausg.)). Somerset: Wiley.

Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2014). Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments. Political Analysis, 22(1), 1–30. https://doi.org/10.1093/pan/mpt024

Lohr, S. L. (2010). Sampling: Design and analysis (2. ed). Boston, MA: Cengage Learning.

Scherpenzeel, A., & Toepoel, V. (2012). Recruiting a Probability Sample for an Online Panel. Public Opinion Quarterly, 76(3), 470–490. https://doi.org/10.1093/poq/nfs037