Interpretation / Results & interpretation

How to read confidence intervals and bootstrap results

Read interval endpoints with the estimate, identify the uncertainty procedure and check whether resampling failures affect interpretation.

7 min read

Read the estimate and interval together

An estimate is the value calculated from your sample, such as a path coefficient. The confidence interval shows uncertainty about that estimate under a stated procedure. A wider interval leaves a wider range of values compatible with the result. Read the endpoints in the same units as the estimate.

The confidence level describes the long-run coverage of the procedure under its assumptions: how often intervals from comparable repeated samples would contain the true value. It does not turn the displayed endpoints into certain limits for individual people or future observations. Report the confidence level and how the interval was calculated.

For a two-sided interval around a difference or coefficient, zero is the reference value for no difference or no effect. For an odds ratio, one is the no-change value. An odds ratio compares the odds of an event under the stated model. Use the reference value for your quantity rather than applying zero to every table.

The estimate can stay the same while its interval changes. Different sampling designs, assumptions or interval methods can produce different uncertainty calculations. When comparing reports, check those choices before deciding that one result contradicts the other.

A checked example from the regression exercise

In the independent Python reference calculation for the regression exercise, the coefficient for z is about 0.0018 and its HC3 95% t interval runs from about -0.1410 to 0.1446. HC3 adjusts the estimated uncertainty, and the interval uses a Student-t reference distribution. These are reference calculations, not output from a QuickPLS app run. The estimate is near zero, but the interval includes negative and positive relationships. Describe that uncertainty instead of concluding that z has no relationship with the outcome.

Know which bootstrap interval you selected

Bootstrapping repeatedly draws a sample from your existing cases and fits the model again. In a case bootstrap, a case can be selected more than once in a sample. The resulting collection of estimates helps assess sampling uncertainty. It is still based on the information in your original study.

A percentile interval takes its endpoints from the ordered bootstrap estimates. A bias-corrected interval adjusts for how those estimates are positioned around the original estimate. A bias-corrected and accelerated interval, usually called BCa, also adjusts for how the estimate responds to individual observations. A studentized interval scales estimation differences by an estimate of their uncertainty, called a standard error.

These methods can need different extra model fits or checks. Some are unavailable for particular outputs. Choose a supported interval method before examining which result is most favourable, and report the method you actually used. If it fails, investigate the cause; do not quietly replace it with whichever interval excludes the no-effect value.

Use the same random seed to help repeat a run in the same software, with the same data and settings. The seed is the starting value used to generate the sampling sequence. Two libraries may select different cases from the same seed number. To compare individual bootstrap fits across programs, their selected row indices must match. Otherwise use a comparison that allows for random sampling variation and explain that approach.

Screenshot to be addedPLS-SEM Bootstrapping settings with the intended dataset and completed selections

Check how many samples were usable

The requested number of bootstrap samples and the number of usable estimates can differ. A resample may contain too little variation for part of the model. A fit may fail to converge, meaning that its estimates did not settle, or return an inadmissible value, meaning that an estimate violates the method's requirements.

Read the usable and failed counts together with the failure reasons. Replacing failed fits with extra draws changes which results enter the reference distribution, the collection used to calculate uncertainty. Ignoring failures can also hide a model that is unstable when the sample changes.

A few failures and widespread failure need different levels of investigation. Find the variables, equations or requirements involved. Increasing the requested draw count does not fix the underlying cause. Keep the failure information with the result, especially if different statistics use different numbers of valid draws.

Screenshot to be addedRequested, usable and failed bootstrap samples

Decide whether the possible effects would matter

A p value describes how unusual a result at least as extreme as yours would be under the test's stated reference assumptions. The null is the specific claim being tested, often a zero coefficient or no difference. A p value is not the probability that your theory is true. A large p value also does not establish that two effects are equal.

Report the effect in its stated units and explain the range allowed by its interval. If that range still includes effects that would matter in your setting, a claim of no effect is too strong. An interval can be informative even when it does not settle the hypothesis: it shows how much uncertainty remains.

Consider how many comparisons you made and whether you chose them before or after inspecting the data. Testing many possibilities creates more opportunities for an apparently significant finding. Explain the group of tests you considered and any adjustment or exploratory purpose, rather than presenting the selected result as the only test performed.

Give readers enough detail to assess the interval

Keep the estimate, uncertainty and calculation settings together in the report. Test direction means whether the procedure examines a departure in either direction or only in a direction specified in advance. Resampling assignments record how the cases were selected or rearranged for the procedure.

  • Give the point estimate, its units and the reference value for no effect or no change.
  • State the confidence level, interval or test method, and whether the test was one-sided or two-sided.
  • Report requested, usable and failed resamples or assignments, including the stated reasons for failures.
  • Record the random seed and the settings used each time the model was fitted again.
  • Explain which tests were planned together and how you handled multiple comparisons. Identify analyses chosen after inspecting the results.