QuickPLS Ask about combinations, not just individual relationships
Dong, Chen and Shen combined survey-based structural analysis with fuzzy-set qualitative comparative analysis, or fsQCA, in a study of learning analytics and engagement. Their survey involved 400 undergraduates. The paper connects learning analytics with forms of learning presence and digital literacy.
In QuickPLS you will practise the fsQCA part: select the learning conditions, calibrate their raw scores and identify combinations associated with high engagement. The exercise uses original synthetic student data and supplies separate reference answers for comparison. The source describes survey timing inconsistently, so neither this adaptation nor its set relationships establish a time-ordered causal process.
Your study: different ways to support engagement
Imagine a learning team comparing 260 fictional students’ experiences. Some students may engage strongly when feedback is useful and digital skills are high. Others may rely more on teaching presence and peer connection. A configuration describes such a combination, rather than assigning one average coefficient to each condition.
Use four conditions: useful feedback, peer connection, teaching presence and digital literacy. The outcome is high engagement. Each column is a complete continuous teaching score from 1 to 7. These are observed exercise scores, not questionnaire scales copied from the paper and not factor scores from a fitted PLS model.
Write the set definitions before calibration. High teaching presence means a clearly organised and responsive teaching experience in this invented study. High digital literacy means confidence in navigating the learning activities. The file supplies values for practice; it cannot establish valid definitions for a real institution.
On a small screen, swipe the table sideways to see every column.
| Set | Column | Role |
|---|---|---|
| Useful feedback | feedback | Condition |
| Peer connection | peer | Condition |
| Teaching presence | presence | Condition |
| Digital literacy | literacy | Condition |
| High engagement | engage | Outcome |
| Case identity | case_id | Label for inspecting individual cases |
Step 1. Import the raw scores and open fsQCA
Keep a written record of the selected columns, calibration anchors and thresholds for each calculation. Results are saved separately in the project, so record which result is the baseline before changing a setting. The main calculation form is sufficient for this exercise.
Import and verify the raw-score file
In a writable project choose File → Import Data…, then Choose data file…. Select the lesson CSV. In Review & import, check raw data, Header row 1, comma separation and Decimal convention Dot (1.25). Keep the five score columns Continuous. Choose Import dataset and confirm Dataset imported and verified with 260 cases.
Open the calculation settings
With the imported dataset selected, choose Calculate…. In Find a method select Fuzzy-set Qualitative Comparative Analysis. Work in Method settings. A blank structural-model canvas is expected because fsQCA uses the dataset columns directly. Record Learning conditions for high engagement and Baseline anchors 2 4 6 in your analysis notes.
What you should see: Method settings shows Conditions and Outcome and case identity for the imported learning file.
Step 2. Turn the scores into explicit set memberships
Inspect the resulting memberships alongside raw scores. Membership near 1 means a case belongs strongly to the named set; it is not a 100% probability of the outcome. The crossover at 0.5 means the case is neither more in nor more out. Check any exact crossover rather than adding a tiny amount merely to push it onto a preferred side.
After completing the variable and anchor choices, choose Preview native calibration and read Original and calibrated values. Review borderline values and any missing-value or exact-crossover warning. Under Data: missing values and exact crossovers, record any deliberate treatment with its reason. The generated file is complete, but verify the case count used by the calculation before interpreting it.
Select the condition and outcome columns
Under Conditions click Add condition for each of feedback, peer, presence and literacy. Expand each condition and select its variable. Under Outcome and case identity choose engage as Outcome variable and case_id as Case identifiers. These are raw 1–7 scores, so each needs calibration rather than Existing membership [0, 1] / crisp 0 or 1.
Enter the direct calibration and anchor rationale
Choose Direct fuzzy-set calibration with full non-membership at 2, crossover at 4 and full membership at 6 for every teaching set. Fill in the anchor rationale for each set. These anchors express the exercise’s deliberately stated meaning: low scores are outside the set, the middle is ambiguous and high scores belong strongly. They are pedagogical choices, not established thresholds for real students and not the paper’s calibration values.
Step 3. Specify the truth-table rules before reading a solution
Under Truth table and solution assumptions set Frequency threshold to 3, Consistency threshold to 0.80 and PRI threshold to 0.50. The teaching baseline has no contradiction band. Frequency is the minimum number of cases assigned to a truth-table row; consistency and PRI assess their relationship with the outcome. Consistency measures how closely membership follows the proposed sufficiency relation. PRI helps identify a configuration that also appears to fit the absence of the outcome.
These are starting choices for this exercise. They are not a universal set of journal requirements, and they are not claimed to reproduce every setting in the source paper. Four conditions allow 16 possible present-or-absent configurations. Some may have few or no cases, so inspect the truth table rather than assuming the sample fills every row.
Calculate and inspect the baseline
Leave Request intermediate solutions with explicit directional expectations unselected for this baseline, then choose Start calculation. Open the completed result and use Search result outputs to find Truth table. Read its case counts, consistency and PRI before interpreting the solutions. Open Solution families and alternatives and read a complete Conservative alternative, then match its identifier in Overall solution fit — every alternative and Pathway fit and coverage — every alternative.
Check the answers for the 260-case file
Use direct logistic calibration for every selected condition and outcome: exclusion 2, crossover 4, inclusion 6 and inclusion membership 0.95. The baseline uses frequency 3, row consistency 0.80 and PRI 0.50. Keep the conservative family. 260 of 260 cases are included; excluded case identities, if any, are in the download.
For engage, 7 truth-table rows are positive, 9 negative and 0 remainders. There is 1 complete conservative alternative. Alternative 1 has consistency 0.800, coverage 0.764 and PRI 0.623.
About these answers: Calculated with R’s QCA package and independently checked for calibrated memberships, truth-table decisions, pathways and fit. These are comparison answers, not a saved QuickPLS run.
On a small screen, swipe the table sideways to see every column.
| Outcome | Selected alternative | Solution consistency | Solution coverage |
|---|---|---|---|
| engage | 1 of 1 | 0.800 | 0.764 |
What the separate checks cover
All 1,480 numerical comparisons were within 0.000000000001. A separate Python calculation checked the calibrated memberships, assignment of cases to truth-table rows, consistency and PRI for rows with observed cases, and fit of the pathways and complete solutions. It also checked that each conservative alternative covers exactly the positive rows in the Boolean truth table. This confirms the stated checks; it does not establish that every shortest possible expression was found, and it does not run QuickPLS.
Read the complete engage alternative
The table contains every pathway in alternative 1. Read * as AND, ~ as absence, and the rows together as OR. An omitted condition is not required either way by that term. Preserve this complete alternative; the pack retains every other returned alternative.
The engage outcome is a calibrated membership, not a pass/fail prediction. Raw coverages can overlap; adding them does not give solution coverage.
About these answers: Reference answer from R QCA 3.25, with independent checks of calibration, pathways and fit.
On a small screen, swipe the table sideways to see every column.
| Pathway | Consistency | Raw coverage | Unique coverage |
|---|---|---|---|
| feedback*peer | 0.843 | 0.604 | 0.128 |
| peer*presence | 0.847 | 0.568 | 0.092 |
| feedback*presence*literacy | 0.890 | 0.474 | 0.068 |
Follow one actual practice case: C0014
For case C0014, the membership in feedback*peer is 0.953. Take the smallest membership across the conditions in that term, complementing a membership first when it has ~. The outcome membership is 0.570. This is one case inside a pathway, not the pathway’s overall consistency.
About these answers: Values calculated from this CSV using the stated logistic calibration.
On a small screen, swipe the table sideways to see every column.
| Condition or outcome | Raw score | Calibrated membership |
|---|---|---|
| feedback | 6.0413 | 0.953 |
| peer | 6.0624 | 0.954 |
| presence | 3.6393 | 0.370 |
| literacy | 3.6191 | 0.363 |
| engage | 4.1921 | 0.570 |
Compare the planned threshold changes
The frequency / consistency column lists the rules applied to truth-table rows. The final consistency column measures the minimised solution. These are different quantities; a row threshold of 0.85 does not require solution consistency to exceed 0.85. A frequency increase can remove observed support and create more remainders.
These runs change frequency or row consistency while holding calibration fixed. They do not test other calibration anchors, outcome negation or another solution family. If the lesson asks you to explore those choices, save and report them as additional runs.
About these answers: External reference runs, with every returned conservative alternative retained.
On a small screen, swipe the table sideways to see every column.
| Outcome / change | Frequency / consistency | Positive rows | Alternatives | Solution consistency | Solution coverage |
|---|---|---|---|---|---|
| engage: baseline | 3 / 0.80 | 7 | 1 | 0.800 | 0.764 |
| engage: frequency | 5 / 0.80 | 7 | 1 | 0.800 | 0.764 |
| engage: consistency | 3 / 0.85 | 7 | 1 | 0.800 | 0.764 |
Step 4. Read a pathway as a conjunction
Suppose a solution term contains feedback AND literacy. Both conditions are required in that term. If peer connection is omitted, it is unrestricted within that term; omission does not mean low peer connection is required. A term that explicitly includes the absence of peer connection makes a different claim.
A solution can contain several pathways joined by OR. Read each complete alternative on its own. Do not take the most attractive pathway from one alternative and combine it with a pathway from another to create a solution the calculation did not report.
For each term read consistency, raw coverage and unique coverage. Raw coverage includes cases also covered by another term. Unique coverage measures the contribution that remains after accounting for the other terms in that solution. Solution coverage concerns their union, so adding raw coverages can double-count overlapping membership.
Hypothetical example
Illustrative reading example
Imagine two pathways with raw coverages 0.48 and 0.39, overlapping coverage 0.17 and solution coverage 0.70. Their unique coverages would be 0.31 and 0.22 in this simplified illustration. The union is 0.48 + 0.39 - 0.17 = 0.70, not 0.87. These are invented values for learning the distinction, not fitted results.
Step 5. Use cases to test the story you want to tell
Choose a reported pathway and inspect a case with high membership in both the pathway and engagement. Then inspect a case with high pathway membership but lower engagement. The first supports the descriptive pattern; the second helps you see where it fails or needs explanation.
For fuzzy memberships, AND is represented by the minimum and OR by the maximum. If feedback is 0.80 and literacy is 0.60, their conjunction has membership 0.60. If that student’s engagement membership is 0.30, the case does not closely follow the sufficiency claim. It should not disappear from the explanation merely because many other cases fit.
In Results open Case memberships and source rows and locate C0014 from the learning file. Read its raw scores and calibrated memberships together, then compare its pathway membership with engage. Use the supplied case calculation below the baseline answer to follow the same minimum-membership rule.
Do not confuse a pathway with a necessary condition
Sufficiency asks whether a combination is enough for the outcome within the analysed set relationship. Necessity asks whether the outcome requires a condition. A frequent ingredient in several pathways is not automatically necessary. Inspect the actual necessity result and its coverage before making that claim.
Low engagement needs a separate calculation. Reopen the fsQCA settings and change Analyse to Negation of the outcome (separate analysis), keeping the condition definitions clear. Record the new result separately; reversing every condition in a high-engagement solution would answer a different question.
An intermediate solution can simplify using directional expectations about unobserved combinations. Those expectations must come from the research argument. The first Conservative-solution exercise teaches you to see the observed evidence before adding assumptions. If you later use another family, report which assumptions changed and how the explanation depends on them.
Investigate empty or fragile configurations
On a small screen, swipe the table sideways to see every column.
| What happens | What to inspect | What to do |
|---|---|---|
| No positive row remains | Frequency, consistency, PRI and the calibrated outcome | Explain which rule excluded rows; do not lower thresholds simply to get a solution. |
| A pathway has high consistency but little coverage | How many outcome memberships it accounts for | Describe it as a limited route rather than a general explanation. |
| A case seems wrongly classified | Its original score, anchors and exact crossover treatment | Correct a documented setup error or save a reasoned calibration alternative. |
| Two pathways appear to cover more than the whole outcome | Their overlap and the solution’s union | Read unique and solution coverage instead of adding raw coverages. |
| A small anchor change removes a pathway | Borderline cases and changed truth-table rows | Report the sensitivity and avoid treating the pathway as settled. |
| Outcome absence gives a different explanation | The chosen outcome direction and configurations | Keep that result separate; asymmetry is not a calculation error. |
Step 6. Change one defensible decision at a time
Reopen the fsQCA settings with the same anchors and change Consistency threshold to 0.85. Calculate again and record the new result separately from the baseline. Compare which truth-table rows change and which pathways remain. Then restore the baseline consistency and calculate with Frequency threshold 5. These ask different questions: closer conformity to the outcome versus more cases behind a configuration.
For a calibration sensitivity exercise, retain full non-membership at 2 and full membership at 6 but examine crossover 4.2. State the interpretation: the exercise now requires a somewhat stronger raw score to count as more in than out. Follow the cases around the old boundary through the truth table. Do not report only the alternative with the largest coverage.
Save the baseline and every declared alternative. A useful conclusion can be that one pathway is stable while another depends on the treatment of a handful of borderline cases. That finding tells a reader more than a single attractive solution with no account of its choices.
Write a configuration result with its choices attached
Methods outline: We analysed [included count] synthetic student cases using four conditions and high engagement as the outcome. We calibrated each 1–7 teaching score with [method and anchors], applied [crossover and missing-value rules], and used frequency [value], consistency [value] and PRI [value]. We inspected the truth table before interpreting [solution family and alternative].
Results outline: The selected solution contained [pathways in words]. Its consistency was [value] and coverage [value]. The pathway [term] had raw coverage [value] and unique coverage [value]. Case inspection identified [supporting and contradictory examples]. Under the declared sensitivity choices, [describe what persisted and changed]. These statements describe the teaching cases; they are not evidence of an educational intervention effect.
| Report | Include |
|---|---|
| Sets | Definitions, original columns and calibration anchors. |
| Cases | Included count, exact crossover decisions and any exclusions. |
| Truth table | Frequency, consistency, PRI and contradictory rows. |
| Solutions | Family, complete alternatives and simplifying assumptions. |
| Interpretation | Consistency, raw and unique coverage, and original cases. |
| Sensitivity | Each saved alternative and the specific changes it produced. |
Write the result for this teaching file
We calibrated 260 synthetic cases using direct logistic calibration with anchors 2, 4 and 6 and inclusion membership 0.95. We used a frequency requirement of 3, row consistency of 0.80 and PRI of 0.50. We retained the conservative solution family.
Report the complete selected alternative and the changes shown in the sensitivity table. Explain case examples alongside aggregate coverage. These results describe the original teaching dataset, not the source paper’s observations. If several alternatives exist, explain the selection and preserve the others; do not combine favourite terms.
Finish with a case-based explanation
- Explain why the raw 1–7 columns needed calibration.
- Read one conjunction and distinguish an absent condition from an omitted one.
- Find a case that challenges a broader version of your preferred pathway.
- Explain why raw coverages cannot simply be added.
- Reopen the saved baseline and one sensitivity result, and identify the exact choice that differs.