Keep data preparation inside each resample
The tested procedure first replaces missing indicator values with observed arithmetic means, meaning the ordinary average of the available answers. It then standardises the data using the case weights and fits the model. The weights affect those later steps, not the replacement means.
A bootstrap sample draws observations with replacement to study how estimates vary. Each sample must repeat the missing-value preparation and model fit. Comparing only the original fitted estimates would not check that repeated calculation.
Which software produced these results?
The first record is the earlier missing-data test. The following records use the revised V2 source. Their reference calculations use the packages listed in the table; the simulation has its own stated design. These are the versions actually tested, not new tests of every later release.
On a small screen, swipe the table sideways to see every column.
| Record | QuickPLS build or result checked | Reference | Comparison outcome |
|---|---|---|---|
| Historical weighted mean inference | Saved 1.01.19 missing-data validation results | cSEM 0.6.1, statsmodels 0.14.4, NumPy 1.26.4, SciPy 1.15.2 | Passed the comparisons documented for this record |
| Weighted V2 mean-replacement | 1.01.20 correction source | numpy 1.26.4, scipy 1.15.2, statsmodels 0.14.4 | Passed the comparisons documented for this record |
| Weighted V2 BCa | 1.01.20 correction source | numpy 1.26.4, scipy 1.15.2, statsmodels 0.14.4 | Passed the comparisons documented for this record |
| Weighted V2 studentized | 1.01.20 correction source | numpy 1.26.4, scipy 1.15.2, statsmodels 0.14.4 | Passed the comparisons documented for this record |
| Weighted V2 calibration study | 1.01.20 correction source | See which mathematical and app-workflow checks were performed | Passed the comparisons documented for this record |
An earlier check using single-indicator constructs
Two test datasets represent a null effect and a signal: one has no effect in its data-generating design, while the other has an effect. Each construct has one indicator. In this setup, weighted least squares (WLS), a regression calculation that uses case weights, provides an independent check of the estimates.
- These are the saved results of the earlier mean-replacement review.
- The single-indicator result does not establish cSEM agreement for general models with several indicators per construct.
| What was checked | Value / scope |
|---|---|
| Compared numbers and missing-value statuses | 35274 |
| Outside the stated tolerance | 0 |
| Different scenario names in the comparison record | 2 |
| Largest absolute difference | 2.6558755195083e-12 |
Inspect the original estimate before the resampling summaries
These rows come from the original fitted model. The direct effect, path and total effect entries describe related outputs for the same x-to-y relationship; they are not three independent studies. The single-indicator loadings are also shown so you can see the measurement setup.
The table displays 15 significant digits. Absolute differences use the numbers before this display rounding. The download keeps the recorded precision and explains how the reference results were made comparable.
On a small screen, swipe the table sideways to see every column.
| Quantity compared | QuickPLS | Reference | Absolute difference | Comparison result |
|---|---|---|---|---|
Direct effect: x → yResult identifieroriginal/["direct_effect",["construct: x","construct: y"]] | 0.0933801592066401 | 0.0933801592066399 | 1.2490009027033e-16 | Pass |
Indirect effect: x → yResult identifieroriginal/["indirect_effect",["construct: x","construct: y"]] | 0 | 0 | 0 | Pass |
Outer loading: x1 in xResult identifieroriginal/["outer_loading",["construct: x","x1"]] | 1 | 1 | 0 | Pass |
Outer loading: y1 in yResult identifieroriginal/["outer_loading",["construct: y","y1"]] | 1 | 1 | 2.22044604925031e-16 | Pass |
Path coefficient: x → yResult identifieroriginal/["path",["construct: x","construct: y"]] | 0.0933801592066401 | 0.0933801592066399 | 1.2490009027033e-16 | Pass |
Total effect: x → yResult identifieroriginal/["total_effect",["construct: x","construct: y"]] | 0.0933801592066401 | 0.0933801592066399 | 1.2490009027033e-16 | Pass |
V2: check replacement means and weighted estimates
V2 identifies the revised procedure in the test record. The Path, Factor and complete-data cases each use 128 rows. Path and Factor are weighting choices for the model calculation; the complete-data case checks behaviour when no indicator values need replacement.
Each case compares 269 values calculated by QuickPLS, 269 values in the standard result format and 4 replacement means. In the test record, Native labels the calculated values and Canonical labels the values prepared for reporting. Checking both can reveal a number that was calculated correctly but reported incorrectly.
- Inspect the replacement means before the weighted estimates. That separates the missing-value calculation from the effect of case weights.
- The same model contributes checks of its calculation, reported output and replacement means. These counts are not independent studies.
| What was checked | Value / scope |
|---|---|
| Passed | Yes |
| Path weighting / Rows | 128 |
| Path weighting / Passed | Yes |
| Path weighting / Calculation results / Passed | Yes |
| Path weighting / Calculation results / Numbers and missing-value checks | 269 |
| Path weighting / Calculation results / Failures | 0 |
| Path weighting / Exported results / Passed | Yes |
| Path weighting / Exported results / Numbers and missing-value checks | 269 |
| Path weighting / Exported results / Failures | 0 |
| Path weighting / Replacement means / Count | 4 |
| Factor weighting / Rows | 128 |
| Factor weighting / Passed | Yes |
| Factor weighting / Calculation results / Passed | Yes |
| Factor weighting / Calculation results / Numbers and missing-value checks | 269 |
| Factor weighting / Calculation results / Failures | 0 |
| Factor weighting / Exported results / Passed | Yes |
| Factor weighting / Exported results / Numbers and missing-value checks | 269 |
| Factor weighting / Exported results / Failures | 0 |
| Factor weighting / Replacement means / Count | 4 |
| Complete data / Rows | 128 |
| Complete data / Passed | Yes |
| Complete data / Calculation results / Passed | Yes |
| Complete data / Calculation results / Numbers and missing-value checks | 269 |
| Complete data / Calculation results / Failures | 0 |
| Complete data / Exported results / Passed | Yes |
| Complete data / Exported results / Numbers and missing-value checks | 269 |
| Complete data / Exported results / Failures | 0 |
| Complete data / Replacement means / Count | 4 |
| Largest absolute difference | 2.66453525910038e-15 |
V2: bias-corrected and accelerated intervals
Bias-corrected and accelerated (BCa) intervals adjust bootstrap limits for bias and for how sensitive the estimate is to individual observations. The test uses 1000 outer bootstrap resamples, 48 leave-one-out samples and six summary results for a model with two reflective blocks and one path.
Leave-one-out samples, also called jackknife samples, omit an observation to assess that sensitivity. The reference checks the resampled estimates and the interval calculations described in the record.
- The leave-one-out calculations check the sensitivity adjustment used by BCa. Keep them separate from the inner bootstrap fits used for studentized intervals.
- The comparison count includes many values from the same model and resamples. It is not a count of independent studies.
| What was checked | Value / scope |
|---|---|
| Absolute tolerance | 1e-10 |
| Comparisons | 18585 |
| Maximum absolute error | 4.4543257970985906e-12 |
| Outer bootstrap samples | 1000 |
| Leave-one-out samples | 48 |
| Inner resamples checked independently | 0 |
| Summary results | 6 |
| Interval | bca |
| Passed | Yes |
V2: intervals that use an estimated standard error
A studentized interval accounts for the estimated standard error, which measures sampling variability. In this nested bootstrap procedure, inner samples are drawn within outer samples to help estimate that variability.
The test uses 999 outer bootstrap resamples. The independent reference checks 297 inner resamples, rather than every inner fit calculated by QuickPLS. The agreement therefore applies to the checked inner fits and reported summaries.
- Use the independently checked inner-resample count to judge how much of the nested calculation was verified.
- The comparison total counts values from fits and summaries; it is not a count of separate studies or a claim that every inner fit was independently checked.
| What was checked | Value / scope |
|---|---|
| Absolute tolerance | 1e-10 |
| Comparisons | 17647 |
| Maximum absolute error | 1.5543122344752192e-15 |
| Outer bootstrap samples | 999 |
| Leave-one-out samples | 0 |
| Inner resamples checked independently | 297 |
| Summary results | 6 |
| Interval | nested_studentized |
| Passed | Yes |
V2: repeat the procedure on simulated datasets
The simulation uses 200 datasets with Gaussian single-indicator data: each construct has one indicator, generated under a normal-distribution model. Values are missing completely at random (MCAR), meaning their absence does not depend on the observed or unobserved values. There are 100 datasets with a null effect and 100 with a signal.
Each dataset has N=300 observations, with 1000 outer resamples, 99 inner resamples and 300 leave-one-out samples. Results are assessed against 36 statistical criteria. This checks behaviour across repeated datasets under this particular design; it does not cover other missing-data mechanisms or general multi-indicator models.
- The result applies to the stated normal-distribution, single-indicator and missing-completely-at-random design.
- Keep the numerical comparison total separate from the number of simulated datasets and statistical criteria.
| What was checked | Value / scope |
|---|---|
| Passed | Yes |
| Cases | 200 |
| Numerical comparisons | 815600 |
| Maximum absolute error | 7.551792524651546e-12 |
| Statistical criteria | 36 |
Error limits for the earlier weighted test
The earlier test used different error limits for different outputs. A zero allowance requires exact agreement for the values to which it applies. Other rows allow small numerical differences. Use the rule attached to the particular output rather than applying a single limit to every result.
abs(native - aligned_reference) <= absolute_tolerance + relative_tolerance * abs(aligned_reference)
In this formula, native is the QuickPLS value and aligned_reference is the reference value expressed using the same definition. abs means absolute value, so the sign does not affect the size of the difference. Absolute tolerance is a fixed allowance. Relative tolerance adds an allowance that grows with the size of the reference value. A separate check verifies whether a result is available or missing where expected; it does not estimate another effect.
On a small screen, swipe the table sideways to see every column.
| Absolute tolerance | Relative tolerance | Numeric or missing-value checks | Where the comparison rule was specified |
|---|---|---|---|
| 1e-08 | 1e-08 | 28672 | Specified separately for each compared value |
| 0 | 0 | 3038 | Specified separately for each compared value |
| 1e-07 | 1e-07 | 3564 | Specified separately for each compared value |
What these checks tell you
Read each test alongside its own error limits and sampling design. The original-estimate check, BCa check, studentized check and simulation answer different questions about the weighted procedure.
All comparisons include multiple values from the same analysis. Use the dataset and resample counts to understand the scope, and the downloads to inspect precision beyond the displayed table.
- The earlier weighted test uses the separate error limits listed above, including exact checks where required.
- For V2 BCa intervals, the maximum allowed absolute error is 1e-10. This applies to the results named in that test.
- For V2 studentized intervals, the maximum allowed absolute error is 1e-10. The reference covers the stated subset of inner resamples.
Inspect the comparison files
The JSON report separates the earlier weighted test, V2 estimates, interval checks and simulation findings. The CSV supplies comparison values where those values are available; a summary count alone is not a new table of estimates.
To repeat a test, obtain its original dataset or simulation design, model, settings and software versions. The downloads describe completed tests and do not include the original datasets or application source code.
Download the comparison results
See the values we checked, the differences we found and the software versions used. The download also identifies the original test files.
About these tests
These are QuickPLS development comparisons. Each record identifies the reference software or mathematical calculation, the tested build and the settings used.
How to understand these comparisons · Find checks for another method