QuickPLS What the method does
Robustness analysis varies justified calibration anchors, thresholds, cases or solution assumptions and compares the resulting evidence. Its purpose is to learn which conclusions depend on a particular choice. Repeatedly changing settings until a preferred pathway appears is not a robustness assessment.
Data and analysis in QuickPLS
Keep a saved baseline and define the alternatives before comparison. Record exactly what changes and hold the other choices fixed where possible.
QuickPLS supports named alternatives and robustness comparisons. Inspect the changed cases and pathways as well as summary stability measures.
Follow a worked example
The following example shows the data, settings and decisions for one analysis. Use the linked practice files to follow it. For your own study, keep the procedure and choose the variables, model and settings that fit your research question.
Download memberships.csv to begin. It contains 10 rows and 5 columns. The example below identifies the variables to use, their roles and any rows to exclude; not every column is an analysis variable.
Test a research choice that could affect the conclusion
A robustness check asks whether a finding depends on a reasonable choice in your analysis. For example, does the explanation remain when you require more cases per configuration? Name the starting analysis, or baseline, and explain the reason for each planned variation before comparing the results. Changing a frequency threshold tests something different from changing the meaning of set membership.
Each scenario needs a complete saved setup, even if its name mentions only the changed setting. Two scenarios with identical thresholds can still use different calibration anchors or exclusions. QuickPLS can reuse a calculation when all the relevant analytical settings match. It keeps the separate scenario names and their outcomes visible.
Start from a saved baseline and the correct plan type
Choose the original dataset and a completed baseline result. Use the robustness workspace that matches its method: the enhanced fsQCA plan or the calibrated-V2 plan. Calibrated-V2 is the app's separate workflow for advanced calibration. Save full settings for each variation so another reader can see exactly what changed.
Save the scenarios, run the queue and compare the evidence
Choose the baseline setup
In the appropriate robustness workspace, choose Saved setup to copy and Use saved setup as baseline. Check the source study and create the full list of planned scenarios.
Name and save the plan
Use Named baseline and Scenario to edit to check the choices. Choose Save explicit plan, then Start / resume queue. Scenarios are calculated one at a time.
Read every outcome
Inspect each scenario's status and history. Keep completed, empty, failed, cancelled and interrupted work distinct. Use Retry selected scenarios only when you decide to repeat an eligible unfinished attempt.
Choose the alternatives deliberately
For a completed scenario with a solution, select the Baseline solution and Scenario solution you want to compare. The app does not choose a preferred answer for you.
Compare the conclusion and the cases
Read the expressions, fit and membership differences together. Check which original cases are included and covered on each side. Keep the full scenario history with the comparison.
An unchanged formula is only part of the check
The same pathway formula can have different memberships or fit after calibration changes. Different printed formulas also need examination before you call their meaning different. Compare the actual alternatives, included cases, memberships and fit. A case excluded from one calculation should still be visible in the comparison's account of the original observations.
Keep empty, failed and interrupted items in the plan. Pause after current scenario prevents another calculation from starting once the current one finishes. Cancel remaining work preserves completed results; an active calculation may finish before cancellation is confirmed. A supported plan lists at most 100 scenarios and runs them one at a time. It does not rank the scenarios to select your preferred finding.
The comparison can check whether the selected expressions match after arranging their condition terms consistently. That is not a general proof that every differently written logical formula is equivalent. Use the original-case comparison to see which observations remain included and how their memberships change.
The plan should show the membership baseline and the two threshold scenarios. The higher-consistency result can be compared with the baseline’s complete expression; the empty frequency-2 result has no solution fit to report. Inspect each item’s status before reading its comparison.
On a small screen, swipe the table sideways to see every column.
| Output | How to read it | What to check |
|---|---|---|
| Baseline and scenario settings | Read the complete saved scenario, including the intended change from the baseline. | A scenario label alone does not specify its actual calibration, cases or cutoffs. |
| Scenario status | Separate completed, failed, cancelled and interrupted work before interpreting the comparison. | Report incomplete scenarios explicitly; they are not negative findings. |
| Formula and fit stability | Compare which conditions or their absences appear in each selected alternative, using consistent notation, then compare fit and memberships. | Different textual formulas may require further logical interpretation; no automatic best scenario is selected. |
| Case stability | Compare all original cases used by either analysis and check why a case is excluded on either side. | A single stability score should not hide changes in which cases were analysed. |
Check whether the delivery conclusion depends on sparse case support
This exercise compares the baseline with a higher consistency requirement and a higher frequency requirement. Consistency checks how closely a configuration fits within the outcome. Frequency counts the cases assigned to a configuration, regardless of their outcome. The two changes therefore examine different reasons why the conclusion might depend on the setup.
Open memberships.csv or fsqca_comparisons_robustness.qpls. A, B and C stand for capacity, reach and coordination; Y is delivery. These are invented teaching meanings. Keep m01 to m08. Explicitly exclude m09 because A = 0.5 and m10 because B is missing. The saved project includes the baseline, the sensitivity checks and the empty frequency-2 result discussed below.
Give each variation a reason before comparing results
Use a baseline of frequency 1, consistency 0.80 and PRI 0.50, with no contradiction band. The consistency-0.90 scenario asks whether the positive rows still qualify under a higher fit requirement. The frequency-2 scenario asks what happens when each configuration must have more than one supporting case. Keep calibration and included cases fixed for these two checks.
Write the reason for each scenario before looking for a preferred expression. If you later investigate calibration or exclusions, name those changes separately. Calibration can change membership values; exclusions can change the observations available. Both need interpretation beyond the effect of a truth-table threshold.
Save and run the delivery robustness plan
Copy the correct baseline
Open the completed membership baseline from fsqca_comparisons_robustness.qpls and choose Plan robustness study. Use its enhanced fsQCA workspace, not the advanced-calibration workspace. Under Saved setup to copy select that membership result and choose Use saved setup as baseline. Confirm A, B, C, Y and the eight included m-cases.
What you should see: The copied baseline contains A, B, C, Y and the eight intended cases.
How to read it: Correct the source before adding scenarios.
Add clearly named scenarios
Use Add explicit scenario to create two scenarios. For Higher consistency keep frequency 1 and set consistency to 0.90. For Higher frequency keep consistency 0.80 and set frequency to 2. Keep PRI 0.50, existing memberships, the same outcome and both exclusions. Under Scenario to edit inspect each full setup; keep the baseline in the plan.
What you should see: Higher consistency changes only consistency; Higher frequency changes only frequency.
How to read it: Check all settings, not just the scenario names.
Save the complete plan
Choose Save explicit plan. Check calibration, exclusions, outcome and PRI as well as the thresholds named in each scenario. A short scenario name does not capture all those choices.
What you should see: The saved plan retains the baseline and both alternatives.
How to read it: Calibration and exclusions remain as intended.
Follow the calculations
Choose Start / resume queue. QuickPLS runs one calculation at a time. Read each status and saved result. A completed calculation with no solution is different from an attempt that failed or was interrupted.
What you should see: Each queued calculation has its own status and result.
How to read it: A completed no-solution result differs from a failed attempt.
Compare the completed evidence
Select a completed scenario and load the alternatives. Choose a Baseline solution and Scenario solution when both have alternatives. Compare expressions, fit and case memberships. Keep the empty frequency-2 item in the report even though it has no solution alternative to select.
What you should see: The comparison identifies baseline and scenario alternatives where available.
How to read it: Keep the frequency-2 no-solution finding in the report.
Predict each result from the eight combinations
Each included configuration occurs once. Configurations 101, 110 and 111 have Y = 1; the other five have Y = 0. Raising consistency from 0.80 to 0.90 does not change those positive rows, because they already have consistency 1. Both setups return A*B + A*C, with solution consistency and coverage of 1.
With frequency 2, none of the eight configurations has enough supporting cases. The supplied saved calculation has no solution. Do not report 0 for its consistency or coverage: there is no evaluated solution whose fit could be given that value. You can explain the empty result from the original frequency counts and confirm it in the saved project.
Understand reused calculations and interrupted work
Two named scenarios can share one calculation when their complete scientific settings match. That includes selected columns, calibration, exclusions and other calculation choices, not just the consistency cutoff. A repeated baseline remains visible under its own name, but it is not an independent replication when it reuses the same calculation.
Pause after current scenario stops the next item from starting. Cancel remaining work retains completed work, and the active calculation may finish before cancellation is confirmed. Read the status and history of failed, cancelled or interrupted items, then explicitly retry where appropriate. Reopening the project does not turn unfinished work into a successful result. The ordinary plan allows at most 100 scenarios and calculates one at a time.
To revise a saved plan, use Copy into a new editable plan. Review and save the new plan before starting it. This keeps the earlier scenario list and its recorded outcomes available, instead of making old calculations appear to belong to newly edited choices.
Explain what the checks actually tested
The higher-consistency solution stays the same because each configuration contains one entirely successful or unsuccessful binary case. That is limited evidence about consistency across repeated cases. The frequency-2 result exposes the sparse support: no combination has a second case. Reporting both makes the dependence on single-case configurations clear.
A changed expression is not automatically a research failure. Ask whether the same cases are covered, whether a pathway loses its unique contribution, and whether changed calibration moves cases across the crossover. Unique coverage is the contribution beyond the other pathways in the selected solution. Explain the change that matters to your research claim instead of reducing unlike results to one unexplained robustness score.
Distinguish no solution from no completed calculation
Read the status first. An empty completed result can teach you something about the chosen evidence requirement. A failed or interrupted attempt has not supplied that finding. Use the table to decide what to inspect and whether a deliberate retry is appropriate.
On a small screen, swipe the table sideways to see every column.
| Problem | Check | Reasonable action |
|---|---|---|
| A scenario returns no solution | Whether any configuration meets its frequency rule | Report the empty result and its reason; do not replace missing fit with zero. |
| Two scenarios share one calculation | Whether their complete settings are identical | Keep their names but describe the shared calculation accurately. |
| The queue stops before every item finishes | Each item’s completion, failure or interruption status | Review why it stopped and explicitly retry unfinished items where the app allows it. |
| A specialist method is unavailable in the plan | Whether it is multi-value, temporal, panel, CNA, binomial or expanded | Use that method’s own supported workflow rather than relabelling it as ordinary robustness. |
| Expressions match but coverage changes | Included cases and calibrated memberships | Compare which cases were analysed and their memberships before claiming the same research conclusion. |
Show every planned change and its outcome
In Methods, explain why you chose each scenario and what stayed fixed. In Results, show the unchanged solution and the empty result with their reasons. Include unfinished attempts in the plan history rather than presenting them as negative findings or silently leaving them out.
On a small screen, swipe the table sideways to see every column.
| Scenario | Frequency | Consistency threshold | Observed outcome |
|---|---|---|---|
| Baseline | 1 | 0.80 | A*B + A*C; solution consistency 1 and coverage 1 |
| Higher consistency | 1 | 0.90 | Same complete expression and fit in this binary example |
| Higher frequency | 2 | 0.80 | No configuration meets frequency; no solution fit to report |
Example Methods paragraph
We specified two sensitivity checks around the synthetic delivery baseline. One increased consistency from 0.80 to 0.90; the other increased minimum frequency from 1 to 2. Conditions, outcome, existing memberships, PRI 0.50 and the eight included cases were held fixed. We retained the outcome of every planned scenario, including empty results.
State the conclusion the sensitivity checks support
You could report: Increasing the consistency requirement left the selected teaching solution unchanged. Requiring two cases per configuration left no observed configuration with enough support, because every configuration occurred once. The baseline explanation therefore depends on accepting single-case configurations in this example. This is a specific finding, not a claim that the study passed every possible robustness check.
Before closing the project, find every planned item and explain its status. For a non-empty result, inspect shared and uniquely covered cases. For the empty result, show the frequency counts. If work was interrupted, say which calculations are complete and which remain unfinished.
- Give a research reason for each changed choice.
- Show all planned items, including empty and unfinished outcomes.
- Describe shared calculations accurately instead of calling them independent replications.
- Use the separate workflows for specialist methods.
- Compare the same research claim across scenarios, including its case evidence.
Keep the plan within its supported methods
The ordinary plan supports up to 100 named scenarios, calculated one at a time. Report the whole planned set, including outcomes you did not prefer. Expanded fsQCA, Coincidence Analysis (CNA) and other specialist procedures have separate workflows and are not ordinary scenarios in this plan.
Open the supplied baseline and sensitivity checks
Use a copy of the Outcomes and robustness project or practise from memberships.csv. The saved calculations come from the 1.03.07 sample programme. They are existing teaching outputs, not fresh Store calculations or journal replications. Keep an original copy before changing the plan.