Workflow / Data & workflow

What to do when some answers are missing

Choose and report how missing entries are handled, and understand when that choice changes the analysed cases or generated variables.

4 min read

Find out which rows the analysis used

Start by comparing the number of cases in your imported file with the number used in the result. A case is one respondent, organisation or other unit of observation. If some cases were left out, identify which required columns had missing values and check the rule used by the calculation.

With complete-case analysis, a missing answer on one required survey question can exclude that respondent from the whole model. That person may disappear from a relationship table even when they answered both questions you are looking at.

Adding a variable can therefore change the sample as well as the model. Before comparing two models, check whether both used the same cases. Otherwise, a difference in estimates may come partly from analysing different people.

Understand what filling a gap changes

Mean replacement fills an empty entry with an average calculated from available values. It supplies a number that the respondent did not provide. This changes the spread of values and can change the relationships between variables.

The right choice depends on why answers are missing and what you plan to estimate. A question skipped by mistake is different from one skipped because it did not apply. Importing the file successfully does not settle that issue.

When comparing software, match which observed values contribute to a replacement mean and when replacement occurs. In QuickPLS weighted PLS, mean replacement uses each indicator's observed arithmetic mean. Case weights remain unchanged and affect the subsequent model fitting; they do not turn this replacement into a weighted mean. Use the method guide to check the preparation used by the calculation you are reporting.

Check the order when creating interactions

An interaction asks whether a relationship changes with another variable. Its preparation may create a product such as X×W. Filling gaps in X and W before multiplying them can produce a different column from filling gaps in the product afterward.

Other preparation choices matter too. Centring subtracts an average; scaling changes the units. Save the choices used to create the new columns and follow the method's supported preparation steps.

When comparing QuickPLS with another program, check the prepared columns or the documented order of operations. This is especially useful when the original variables agree but an interaction result differs.

Keep the prediction test separate from training

A prediction check sets some cases aside, fits the model using the remaining training cases, then predicts the answers in the cases it set aside. The test is useful because the model has not used those answers to learn its relationships.

If you calculate replacement averages or scaling values from the whole dataset first, the test cases influence the preparation. This is called information leakage. It can make the predictions look better than they would for genuinely new cases.

Check that the chosen procedure repeats the necessary preparation inside each training sample. When comparing two models, use the same prediction task and compatible held-out cases. Save these choices with the result.

Tell the reader how you handled the gaps

State how missing values were recognised, which rule you used, and how many cases entered the analysis. If you filled gaps, explain how the replacement values were calculated. If you created interactions or other derived columns, record the preparation order.

For bootstrap or prediction work, say which preparation steps were repeated inside each resample or training group. Keep an unavailable result marked as unavailable. Entering zero instead would make a different statistical claim.