Regression: compare estimates and uncertainty with statsmodels

Match the regression and its uncertainty calculation

The response is the outcome being modelled; predictors are the variables used to explain it. The programs must use the same coding and predictor order. They must also agree on whether to include an intercept, the model’s baseline term.

Standard errors describe the uncertainty of estimated coefficients. HC3 is an adjustment that allows unequal error variance, meaning the spread of residual errors can differ across observations. An HC3 result should be compared with the same adjustment in the reference.

For logistic regression, check which outcome category is treated as the event. Reversing that choice changes the coefficient interpretation. The likelihood, the expression used to fit the observed responses, must also describe the same model.

Which software produced these results?

Both comparisons used the earlier program for testing QuickPLS calculations and the statsmodels versions listed below. They checked the calculation code directly, not the current Microsoft Store application.

On a small screen, swipe the table sideways to see every column.

Check linear regression, HC3 uncertainty and regression IPMA

The test covers ordinary least squares (OLS), which minimises the squared differences between observed and fitted outcomes. It uses HC3 standard errors and also checks importance-performance analysis (IPMA) based on regression. IPMA considers a predictor’s estimated importance alongside its performance level.

• The calculations used the historical 1.1.8 test executable, not the installed 1.01.19 release or the Store app.
• The comparison aligns the constants used in AIC/BIC, model-comparison statistics that account for fit and complexity.

Linear regression coefficients and HC3 uncertainty

The outcome is y, with x and m specified as predictors and z as a control. The table includes every coefficient in this test model, with its HC3 standard error, test statistic, two-sided p-value and 95% confidence limits.

The x coefficient is 1.5454545454544162: conditional on m and z, a one-unit difference in x corresponds to that difference in fitted y under this model. Its recorded interval runs from 1.1497707190770767 to 1.9411383718317556. The z interval spans zero. Numerical agreement is checked for both kinds of result, regardless of statistical significance.

The saved values are shown to 15 significant digits. Differences use the original numbers before display rounding. The 139-check record also includes model-fit, regression-IPMA and input or missing-value checks; it is not a count of 139 coefficients.

On a small screen, swipe the table sideways to see every column.

Regression importance and performance

This is regression IPMA from the OLS test model, separate from the construct-based historical IPMA example. Importance is the corresponding unstandardised regression coefficient. Performance uses the configured ranges: 0 to 15 for x, 0 to 8 for m and 0 to 2 for z.

For x, importance is 1.5454545454544162 and performance is 43.333333333333336. Interpretation depends on the input units and declared range. The historical observed-range PLS IPMA uses a different definition and cannot supply a substitute reference for this calculation.

On a small screen, swipe the table sideways to see every column.

Check test alternatives and a model without an intercept

The comparison covers three test cases with two-sided, greater and less alternatives. The greater case omits the intercept; all three name bin_y as the outcome and x, z and w as predictors. The calculation uses an unpenalised generalised linear model fitted by iteratively reweighted least squares (GLM/IRLS). Unpenalised means no extra shrinkage penalty is added to the fit.

IRLS updates a weighted least-squares calculation repeatedly to obtain the logistic estimates. These are algorithmic fitting weights; this description does not establish a test of user-supplied case weights.

• These calculations used the historical 1.1.8 test executable.

Logistic coefficients, odds ratios and 90% intervals

This table uses the test case named two_sided: outcome bin_y, predictors x, z and w, and an intercept. The saved confidence level is 90%. Each coefficient, standard error, test statistic, two-sided p-value, coefficient interval, odds ratio and odds-ratio interval is checked separately.

The coefficient of x is 1.0033956643544513 on the log-odds scale. Its odds ratio is 2.727527890546685, describing the multiplicative change in event odds for a one-unit difference in x, holding the other predictors fixed. This is not a probability increase. The 90% odds-ratio interval is 1.8975352557300182 to 3.9205639901789158.

These 36 displayed checks belong to the two-sided case, which has 637 numeric or missing-value comparisons in total. The full 1,899-check record also includes greater and less test alternatives; the greater case omits the intercept. Those alternatives do not mean that the event coding was reversed.

Values use 15 significant digits for display. The saved absolute differences are calculated before display rounding and are assessed with the recorded fixed-plus-relative tolerance.

On a small screen, swipe the table sideways to see every column.

Error limits for linear regression and regression IPMA

The rule below applies after aligning the output definitions, including the HC3 uncertainty calculation and AIC/BIC constants. It checks numerical agreement for this test, not whether a regression specification is suitable for a research question.

abs(native - aligned_reference) <= absolute_tolerance + relative_tolerance * abs(aligned_reference)

In this formula, native is the QuickPLS value and aligned_reference is the reference value expressed using the same definition. abs means absolute value, so the sign does not affect the size of the difference. Absolute tolerance is a fixed allowance. Relative tolerance adds an allowance that grows with the size of the reference value. A separate check verifies whether a result is available or missing where expected; it does not estimate another effect.

On a small screen, swipe the table sideways to see every column.

Error limits for the logistic examples

The logistic comparison uses both a fixed and a proportional allowance. A maximum absolute difference should therefore be read with the scale of the output and its assigned rule.

abs(native - aligned_reference) <= absolute_tolerance + relative_tolerance * abs(aligned_reference)

Use the same rule explained above: a fixed allowance plus one proportional to the reference value, with the limits in this table. Counts and missing-result checks are assessed separately from estimated values.

On a small screen, swipe the table sideways to see every column.

What these checks tell you

These checks support the tested linear and logistic calculations under matching settings. They compare coefficient uncertainty as well as fitted estimates. They do not establish that the same model will be appropriate or well-behaved for every dataset.

Separation occurs when predictors perfectly distinguish the response categories and ordinary logistic estimates may not remain finite. That situation, case weighting, bootstrapping and additional diagnostics are outside the stated cases. Several checked rows can describe the same coefficient, so the totals are not counts of independent studies.

• For linear regression, compare the same HC3 uncertainty method and the aligned model-comparison statistics.
• For logistic regression, preserve the event category and intercept choice before comparing numbers.

Inspect the comparison files

Use the CSV to inspect coefficients, standard errors, probabilities and interval limits. The JSON report identifies the test executable, reference package, model choices and comparison rules.

Repeating a test requires its original dataset, response coding, predictor order, intercept setting and software versions. The downloads report completed comparisons and do not include the original datasets or application source code.

• Understand the comparison rules ↗
• Find checks for another method ↗