Independent calculations and simulation checks

Weighted PLS: check estimates and bootstrap intervals

Weighted partial least squares (PLS) gives some observations more influence through case weights. These tests check its estimates and confidence intervals after missing indicator values are replaced with the ordinary observed means. Bootstrap samples repeat that preparation and fitting, so the reference checks the same sequence.

Reviewed 4 November 2026
2.66e-12Largest difference in the first comparison

What the comparison shows

The evidence covers the earlier mean-replacement procedure, revised V2 estimates, bias-corrected and accelerated (BCa) intervals, studentized intervals and a simulation. The independent studentized check covers a stated subset of inner resamples. The simulation uses Gaussian single-indicator data with values missing completely at random (MCAR), so its findings apply to that design.

Keep data preparation inside each resample

The tested procedure first replaces missing indicator values with observed arithmetic means, meaning the ordinary average of the available answers. It then standardises the data using the case weights and fits the model. The weights affect those later steps, not the replacement means.

A bootstrap sample draws observations with replacement to study how estimates vary. Each sample must repeat the missing-value preparation and model fit. Comparing only the original fitted estimates would not check that repeated calculation.

Which software produced these results?

The first record is the earlier missing-data test. The following records use the revised V2 source. Their reference calculations use the packages listed in the table; the simulation has its own stated design. These are the versions actually tested, not new tests of every later release.

On a small screen, swipe the table sideways to see every column.

RecordQuickPLS build or result checkedReferenceComparison outcome
Historical weighted mean inferenceSaved 1.01.19 missing-data validation resultscSEM 0.6.1, statsmodels 0.14.4, NumPy 1.26.4, SciPy 1.15.2Passed the comparisons documented for this record
Weighted V2 mean-replacement1.01.20 correction sourcenumpy 1.26.4, scipy 1.15.2, statsmodels 0.14.4Passed the comparisons documented for this record
Weighted V2 BCa1.01.20 correction sourcenumpy 1.26.4, scipy 1.15.2, statsmodels 0.14.4Passed the comparisons documented for this record
Weighted V2 studentized1.01.20 correction sourcenumpy 1.26.4, scipy 1.15.2, statsmodels 0.14.4Passed the comparisons documented for this record
Weighted V2 calibration study1.01.20 correction sourceSee which mathematical and app-workflow checks were performedPassed the comparisons documented for this record

An earlier check using single-indicator constructs

Two test datasets represent a null effect and a signal: one has no effect in its data-generating design, while the other has an effect. Each construct has one indicator. In this setup, weighted least squares (WLS), a regression calculation that uses case weights, provides an independent check of the estimates.

  • These are the saved results of the earlier mean-replacement review.
  • The single-indicator result does not establish cSEM agreement for general models with several indicators per construct.
What was checkedValue / scope
Compared numbers and missing-value statuses35274
Outside the stated tolerance0
Different scenario names in the comparison record2
Largest absolute difference2.6558755195083e-12

Inspect the original estimate before the resampling summaries

These rows come from the original fitted model. The direct effect, path and total effect entries describe related outputs for the same x-to-y relationship; they are not three independent studies. The single-indicator loadings are also shown so you can see the measurement setup.

The table displays 15 significant digits. Absolute differences use the numbers before this display rounding. The download keeps the recorded precision and explains how the reference results were made comparable.

On a small screen, swipe the table sideways to see every column.

Quantity comparedQuickPLSReferenceAbsolute differenceComparison result
Direct effect: x → y
Result identifieroriginal/["direct_effect",["construct: x","construct: y"]]
0.09338015920664010.09338015920663991.2490009027033e-16Pass
Indirect effect: x → y
Result identifieroriginal/["indirect_effect",["construct: x","construct: y"]]
000Pass
Outer loading: x1 in x
Result identifieroriginal/["outer_loading",["construct: x","x1"]]
110Pass
Outer loading: y1 in y
Result identifieroriginal/["outer_loading",["construct: y","y1"]]
112.22044604925031e-16Pass
Path coefficient: x → y
Result identifieroriginal/["path",["construct: x","construct: y"]]
0.09338015920664010.09338015920663991.2490009027033e-16Pass
Total effect: x → y
Result identifieroriginal/["total_effect",["construct: x","construct: y"]]
0.09338015920664010.09338015920663991.2490009027033e-16Pass

V2: check replacement means and weighted estimates

V2 identifies the revised procedure in the test record. The Path, Factor and complete-data cases each use 128 rows. Path and Factor are weighting choices for the model calculation; the complete-data case checks behaviour when no indicator values need replacement.

Each case compares 269 values calculated by QuickPLS, 269 values in the standard result format and 4 replacement means. In the test record, Native labels the calculated values and Canonical labels the values prepared for reporting. Checking both can reveal a number that was calculated correctly but reported incorrectly.

  • Inspect the replacement means before the weighted estimates. That separates the missing-value calculation from the effect of case weights.
  • The same model contributes checks of its calculation, reported output and replacement means. These counts are not independent studies.
What was checkedValue / scope
PassedYes
Path weighting / Rows128
Path weighting / PassedYes
Path weighting / Calculation results / PassedYes
Path weighting / Calculation results / Numbers and missing-value checks269
Path weighting / Calculation results / Failures0
Path weighting / Exported results / PassedYes
Path weighting / Exported results / Numbers and missing-value checks269
Path weighting / Exported results / Failures0
Path weighting / Replacement means / Count4
Factor weighting / Rows128
Factor weighting / PassedYes
Factor weighting / Calculation results / PassedYes
Factor weighting / Calculation results / Numbers and missing-value checks269
Factor weighting / Calculation results / Failures0
Factor weighting / Exported results / PassedYes
Factor weighting / Exported results / Numbers and missing-value checks269
Factor weighting / Exported results / Failures0
Factor weighting / Replacement means / Count4
Complete data / Rows128
Complete data / PassedYes
Complete data / Calculation results / PassedYes
Complete data / Calculation results / Numbers and missing-value checks269
Complete data / Calculation results / Failures0
Complete data / Exported results / PassedYes
Complete data / Exported results / Numbers and missing-value checks269
Complete data / Exported results / Failures0
Complete data / Replacement means / Count4
Largest absolute difference2.66453525910038e-15

V2: bias-corrected and accelerated intervals

Bias-corrected and accelerated (BCa) intervals adjust bootstrap limits for bias and for how sensitive the estimate is to individual observations. The test uses 1000 outer bootstrap resamples, 48 leave-one-out samples and six summary results for a model with two reflective blocks and one path.

Leave-one-out samples, also called jackknife samples, omit an observation to assess that sensitivity. The reference checks the resampled estimates and the interval calculations described in the record.

  • The leave-one-out calculations check the sensitivity adjustment used by BCa. Keep them separate from the inner bootstrap fits used for studentized intervals.
  • The comparison count includes many values from the same model and resamples. It is not a count of independent studies.
What was checkedValue / scope
Absolute tolerance1e-10
Comparisons18585
Maximum absolute error4.4543257970985906e-12
Outer bootstrap samples1000
Leave-one-out samples48
Inner resamples checked independently0
Summary results6
Intervalbca
PassedYes

V2: intervals that use an estimated standard error

A studentized interval accounts for the estimated standard error, which measures sampling variability. In this nested bootstrap procedure, inner samples are drawn within outer samples to help estimate that variability.

The test uses 999 outer bootstrap resamples. The independent reference checks 297 inner resamples, rather than every inner fit calculated by QuickPLS. The agreement therefore applies to the checked inner fits and reported summaries.

  • Use the independently checked inner-resample count to judge how much of the nested calculation was verified.
  • The comparison total counts values from fits and summaries; it is not a count of separate studies or a claim that every inner fit was independently checked.
What was checkedValue / scope
Absolute tolerance1e-10
Comparisons17647
Maximum absolute error1.5543122344752192e-15
Outer bootstrap samples999
Leave-one-out samples0
Inner resamples checked independently297
Summary results6
Intervalnested_studentized
PassedYes

V2: repeat the procedure on simulated datasets

The simulation uses 200 datasets with Gaussian single-indicator data: each construct has one indicator, generated under a normal-distribution model. Values are missing completely at random (MCAR), meaning their absence does not depend on the observed or unobserved values. There are 100 datasets with a null effect and 100 with a signal.

Each dataset has N=300 observations, with 1000 outer resamples, 99 inner resamples and 300 leave-one-out samples. Results are assessed against 36 statistical criteria. This checks behaviour across repeated datasets under this particular design; it does not cover other missing-data mechanisms or general multi-indicator models.

  • The result applies to the stated normal-distribution, single-indicator and missing-completely-at-random design.
  • Keep the numerical comparison total separate from the number of simulated datasets and statistical criteria.
What was checkedValue / scope
PassedYes
Cases200
Numerical comparisons815600
Maximum absolute error7.551792524651546e-12
Statistical criteria36

Error limits for the earlier weighted test

The earlier test used different error limits for different outputs. A zero allowance requires exact agreement for the values to which it applies. Other rows allow small numerical differences. Use the rule attached to the particular output rather than applying a single limit to every result.

abs(native - aligned_reference) <= absolute_tolerance + relative_tolerance * abs(aligned_reference)

In this formula, native is the QuickPLS value and aligned_reference is the reference value expressed using the same definition. abs means absolute value, so the sign does not affect the size of the difference. Absolute tolerance is a fixed allowance. Relative tolerance adds an allowance that grows with the size of the reference value. A separate check verifies whether a result is available or missing where expected; it does not estimate another effect.

On a small screen, swipe the table sideways to see every column.

Absolute toleranceRelative toleranceNumeric or missing-value checksWhere the comparison rule was specified
1e-081e-0828672Specified separately for each compared value
003038Specified separately for each compared value
1e-071e-073564Specified separately for each compared value

What these checks tell you

Read each test alongside its own error limits and sampling design. The original-estimate check, BCa check, studentized check and simulation answer different questions about the weighted procedure.

All comparisons include multiple values from the same analysis. Use the dataset and resample counts to understand the scope, and the downloads to inspect precision beyond the displayed table.

  • The earlier weighted test uses the separate error limits listed above, including exact checks where required.
  • For V2 BCa intervals, the maximum allowed absolute error is 1e-10. This applies to the results named in that test.
  • For V2 studentized intervals, the maximum allowed absolute error is 1e-10. The reference covers the stated subset of inner resamples.

Inspect the comparison files

The JSON report separates the earlier weighted test, V2 estimates, interval checks and simulation findings. The CSV supplies comparison values where those values are available; a summary count alone is not a new table of estimates.

To repeat a test, obtain its original dataset or simulation design, model, settings and software versions. The downloads describe completed tests and do not include the original datasets or application source code.

Download the comparison results

See the values we checked, the differences we found and the software versions used. The download also identifies the original test files.

About these tests

These are QuickPLS development comparisons. Each record identifies the reference software or mathematical calculation, the tested build and the settings used.

How to understand these comparisons · Find checks for another method