Method guide / Configurations

Panel Set-fit Diagnostics

Compare set relationships across units and periods in panel data.

15 min read

What the method does

A pooled set relationship can conceal differences between units or over time. Panel set-fit diagnostics summarise the relationship within units, between units or across periods according to the selected procedure. Each summary must be interpreted with its available cases and membership definitions.

Within-group sufficiency consistency: Σᵢ∈G min(Xᵢ,Yᵢ) / Σᵢ∈G Xᵢ.

G identifies the cases in the unit or period being assessed. Groups with little membership mass or few observations can give unstable summaries. Pooled and within-group results answer different questions.

Data and analysis in QuickPLS

Use calibrated conditions and outcomes with unit and time identifiers. Check duplicate unit-period rows and missing periods before interpreting panel comparisons.

QuickPLS provides pooled and panel set-fit outputs. Compare the available cases and fit patterns across units and periods before generalising a pooled conclusion.

Follow a worked example

The following example shows the data, settings and decisions for one analysis. Use the linked practice files to follow it. For your own study, keep the procedure and choose the variables, model and settings that fit your research question.

Download panel-time.csv to begin. It contains 9 rows and 8 columns. The example below identifies the variables to use, their roles and any rows to exclude; not every column is an analysis variable.

Look inside a good overall result

A relationship across all observations can hide a problem in one unit or period. Panel and cluster diagnostics describe those differences in set fit. Choose whether you are examining necessity, where the outcome is contained in the proposed requirement, or sufficiency, where the selected condition or solution fits inside the outcome. Begin with a saved enhanced or advanced-calibration result.

Choose the actual target offered by the selector: a condition's presence or absence, or a whole-solution alternative from the saved result. A typed formula-like name does not select calculated memberships. The current controls do not provide a separate pathway picker; inspect individual pathways in the source solution investigation when that is the question you need to explore.

Choose the supported source and grouping fields

Use the saved enhanced fsQCA result for the original panel, or a supported advanced-calibration result. Choose a study containing all original source cases and explicit unit and period/group identifiers. A temporal result is a different source type even when it came from related data.

Select the target and preview the grouped evidence

  1. Open the panel workflow

    Use Data, More, Panel diagnostics. Select the intended saved source analysis.

  2. Load the source choices

    Choose Owning saved study and Specification revision, then Load panel choices. Set the Unit identifier column, Cluster or period identifier column and Set relation.

  3. Choose the targets

    Select 1 to 32 explicit targets. Use Condition targets for a condition's presence or absence. To select a whole solution, Load native solution alternatives and choose the actual saved alternative under Retained solution targets.

  4. Resolve and preview the selected evidence

    Choose Resolve exact native targets, check the returned selection, then Preview native panel diagnostics. Inspect the pooled and subgroup results with their original cases.

  5. Save, calculate and include the declared targets

    Choose Save panel revision and Calculate and retain panel diagnostics. For the report, select All explicitly selected panel targets and verify the saved target list.

Read subgroup fit with its observations and denominator

Pooled fit combines the observations. Within-unit fit follows each unit across its observations. Between-cluster fit groups observations by the selected cluster or period field. Compare their case lists, missing information and coverage of the unit-by-period grid. A measure with an empty denominator is undefined; it is not zero fit.

The teaching panel has three units and three periods, so you can check the complete grid by hand. That count alone does not verify every diagnostic statistic. Read the calculated measures and keep the identifiers and case lists needed to explain them.

Use the supported enhanced or advanced-calibration source. A temporal result does not replace it merely because it came from related data. Report the full selected target set and investigate whether a group difference reflects its observations, sparse support or a substantive feature of the setting.

Read consistency and coverage together. Consistency asks how closely the selected set follows the declared relation; coverage asks how much of the relevant set it covers. Keep undefined entries visible and use their membership totals to explain why they cannot be calculated.

The preview should identify nine observations, units u1 to u3 and periods 1 to 3. Find t06 in both the u3 and period-1 views. Keep period-1 coverage marked undefined and inspect its zero outcome-membership total; a six-case panel image cannot verify this example.

On a small screen, swipe the table sideways to see every column.

OutputHow to read itWhat to check
Selected conditions or whole-solution alternativesRead the exact calculated expressions selected for the diagnostics.Select the expression from the source result. Typing a similar formula or title does not select it.
Pooled, within-unit and between-period fitCompare the same selected expression across the chosen groups.Different membership totals or groups with few observations can change fit even when the expression stays the same.
Cases and undefined measuresInspect the original observations and keep undefined group measures labelled as undefined.An undefined fit is not zero and does not count as evidence against the relationship.
Included cases and selected targetsConfirm that the study contains all original cases and that the report includes every selected target.Panel diagnostics describe set fit. They do not provide fixed-effects regression estimates or establish causal effects.

Inspect the original panel study

Use panel-time.csv or a copy of Temporal and panel diagnostics. Compare the saved subgroup results with the original observations. Preserve the original files before changing the source or selected targets.

Find the observation hidden by the pooled fit

Use the original X and Y columns in panel-time.csv. For teaching, let X mean readiness and Y successful delivery. Both are binary memberships. The file also contains capacity and events for the separate temporal lesson; those derived variables are not needed for this X-to-Y panel check.

There are three units observed in three periods, giving nine observations. Pooled fit uses them together, within-unit fit examines each unit, and between-period fit groups rows by period. The different views help locate the observation behind an otherwise good overall result.

Load the original-panel result and its exact target

Choose the study based on original panel-time data, not Temporal teaching columns. The source specification is Pooled original-panel memberships. Its saved enhanced result uses X and Y with no deliberate case exclusion. The panel specification is Pooled, within-unit and between-period sufficiency diagnostics.

Choose Sufficiency for Presence of X, with Y as outcome. Use unit as the unit identifier and period as the cluster field. Select the original enhanced result or a supported advanced-calibration result. A temporal calculation in the same project does not become an eligible source merely because its observations originated in the same file.

Resolve the selected condition before calculating diagnostics

  1. Open the original-panel analysis

    Open fsqca_temporal_panel.qpls and select the original panel-time dataset. Choose Data > More > Panel diagnostics. Select the completed Pooled original-panel memberships Source analysis, its Owning saved study and Pooled, within-unit and between-period sufficiency diagnostics revision, then Load panel choices.

    What you should see: The selected analysis uses the intended original panel.

    How to read it: Confirm that it has not been replaced by a different temporal transformation.

  2. Set the two grouping fields

    Choose unit under Unit identifier column and period under Cluster or period identifier column. Confirm nine observations, three units and three periods.

    What you should see: Unit and time identifiers are assigned correctly.

    How to read it: Inspect duplicate and missing combinations.

  3. Choose the relation and condition target

    Select Sufficiency under Set relation and Presence of X under Condition targets. For a whole-solution analysis, load and select the actual saved alternative instead. Do not type a label as a substitute for selecting calculated evidence.

    What you should see: The relation and target specify the set-fit question.

    How to read it: Necessity and sufficiency use different denominators.

  4. Resolve the target

    Choose Resolve exact native targets. Check the source calculation and expression returned by the app before proceeding.

    What you should see: The target resolves to the intended condition or expression.

    How to read it: Correct an ambiguous assignment before previewing.

  5. Preview each view

    Choose Preview native panel diagnostics. Read the pooled row, all within-unit and between-period rows, and any undefined measures with their case lists.

    What you should see: Each preview identifies its available cases and grouping.

    How to read it: Compare pooled and within-group evidence without treating them as interchangeable.

  6. Save and run the declared selection

    Choose Save panel revision and Calculate and retain panel diagnostics. For reporting, select All explicitly selected panel targets so the report contains the complete target set you chose.

    What you should see: The saved calculation retains the selected relation and grouping.

    How to read it: Explain variation across units or periods alongside the pooled result.

Screenshot to be addedthe exact setup

Calculate the pooled fit from the original rows

Across nine observations, X has membership 1 seven times and Y has membership 1 six times. Their intersection is six. Sufficiency consistency is 6/7 = 0.857143 and coverage is 6/6 = 1. These hand counts match the supplied saved panel diagnostics.

The contrary observation is t06: unit u3, period 1, X = 1 and Y = 0. Every other X-positive observation also has Y = 1. Coverage of 1 says that X covers every Y-positive observation. It does not say that every X-positive observation succeeds; consistency answers that separate question.

On a small screen, swipe the table sideways to see every column.

ViewX membership sumY membership sumIntersectionConsistencyCoverage
Pooled7666/71
Within u122211
Within u222211
Within u33222/31
Within-unit sufficiency of X for Y in panel-time.csv. Units u1 and u2 have consistency 1; u3 has consistency 2/3 because t06 has X = 1 and Y = 0. Coverage is 1 for all three units.
Within-unit sufficiency of X for Y in panel-time.csv. Units u1 and u2 have consistency 1; u3 has consistency 2/3 because t06 has X = 1 and Y = 0. Coverage is 1 for all three units.

Explain why a subgroup measure is undefined

At period 1, only u3 has X = 1 and no unit has Y = 1. The intersection is zero. Consistency is 0/1 = 0. Coverage would require 0/0, so it is undefined: there is no outcome membership in that period against which to measure coverage.

At periods 2 and 3, all three units have X = 1 and Y = 1, giving consistency and coverage of 1. These period views locate the pooled contradiction. Leave undefined entries labelled as such, and consider the membership totals before averaging subgroup ratios.

On a small screen, swipe the table sideways to see every column.

PeriodX sumY sumConsistencyCoverage
1100Undefined: no outcome membership
23311
33311

Investigate the one contrary case

Units u1 and u2 have perfect sufficiency consistency in this small file, while u3 has 2/3. The period-1 result identifies the same contrary observation from a different grouping. These are two views of t06, not two independent contradictions.

Open t06 before proposing why readiness was present without delivery. A real study might investigate timing, measurement or context, but the synthetic file does not establish any such explanation. These descriptive checks do not estimate a causal effect, remove dependence between repeated rows or prove that a change in X caused a change in Y.

Check whether observations are absent or merely unusable

All nine unit-period combinations are present and usable for X and Y here. In an unbalanced panel, some combinations may have no observation. An existing row excluded because a required membership is missing is a different situation. Report both so the available information is clear.

When comparing targets or calibrations, show which original observations the source calculation includes. Changed exclusions can partly explain changed subgroup fit. The workflow accepts 1 to 32 selected targets. Choose them for the research question and report that full set rather than only the subgroup patterns that look convincing.

Check the selected source and subgroup evidence

An unexpected subgroup value can come from an empty outcome set, changed observations or a different selected result. Start with the exact target and its original case list, then use the checks below to explain the value or correct the selection.

On a small screen, swipe the table sideways to see every column.

SymptomCheckAction
Coverage is undefined in a periodThe sum of outcome membership in that subgroupReport the zero denominator rather than substituting a numerical value.
Pooled fit looks stronger than one unit’s fitMembership sums and the contrary casesInspect the subgroup and original observations alongside the pooled ratio.
The source result is unavailableWhether it is an eligible enhanced or advanced-calibration resultSelect the supported original-panel source rather than a related temporal result.
The selected expression cannot be resolvedThe source result and the expression selected from itUse Resolve exact native targets and confirm the returned expression.
A report omits a declared targetThe reporting target selectionChoose All explicitly selected panel targets and verify the saved report.
Some unit-period combinations disappearAbsent rows versus excluded existing observationsDescribe both kinds of incompleteness separately.

Keep the overall result and the revealing subgroups together

Describe the panel structure, exact source result, selected target and necessity or sufficiency relation. Report the pooled fit alongside the subgroup that explains the discrepancy. Preserve undefined measures and include every explicitly selected target in the saved report.

Reporting itemExample entry
Panel structure3 units by 3 periods; 9 of 9 combinations observed
TargetPresent X; sufficiency for Y
Pooled consistency; coverage0.857143; 1
Contrary observationt06, u3, period 1
Undefined measurePeriod-1 coverage because Y membership sums to 0
Numerical originHand counts and existing saved panel diagnostics

Example Methods paragraph

We assessed the sufficiency relation X to Y in nine synthetic observations from three units over three periods. Using the original enhanced source result, we selected the present X condition explicitly and examined pooled, within-unit and between-period fit. We retained undefined measures when a subgroup denominator was zero.

Connect the pooled value to t06

You could report: Pooled sufficiency consistency was 0.857 and coverage was 1. Within-unit consistency was 1 for u1 and u2 and 0.667 for u3. The contrary observation was t06 in period 1. Period-1 coverage was undefined because outcome membership was zero; periods 2 and 3 had consistency and coverage of 1. These findings describe the supplied teaching panel.

  • Recalculate 6/7 from the original observations.
  • Locate t06 in both the unit and period views.
  • Explain why zero consistency can accompany undefined coverage.
  • Check the observed and usable unit-period combinations separately.
  • Report the complete selected target set and its source calculation.

Keep the interpretation about set fit

These diagnostics describe a saved relationship across groups. They do not estimate a fixed-effects regression or a causal treatment effect, and do not make repeated observations independent. Choose an eligible source calculation and interpret the grouped set evidence on its own terms.

Calculation comparisons

Further reading

Related guides