A Guide to qcaERT

Robustness Diagnostics in QCA

Preface

Conducting a Qualitative Comparative Analysis (QCA) requires us to make decisions from the formulation of the research design through the interpretation of the solution. Some of these decisions establish the scope of the analysis, including which cases belong to the reference population and which conditions express the theoretical argument being made. Others determine how empirical information becomes set membership, how the truth table separates sufficient from insufficient configurations, and how logical remainders enter—or do not enter—the minimization.

These decisions act on different QCA objects throughout different analytical moments, and should never be bundled into one vague notion of “researcher discretion.” Calibration anchors determine the cases’ memberships in the condition and outcome sets. The frequency cutoff determines which truth-table configurations have enough empirical instances to be treated as observed for minimization, whereas the inclusion cutoff helps determine which of those configurations should be treated as sufficient for the outcome. The treatment of logical remainders then determines which counterfactual configurations may be used as simplifying assumptions. Case and condition selection reach further still, as they define the empirical scope and configurational space within which the truth table is constructed. Even when all of these choices are theoretically and empirically defensible, robustness appraisal seeks to respond what happens when another plausible choice is made (Skaaning 2011; Schneider and Wagemann 2012; Wagemann and Schneider 2015).

qcaERT version 0.1.2 addresses this question through several related, malleable, systematic, and highly auditable diagnostics. calib.test(), incl.test(), and ncut.test() move one analytical boundary at a time, whereas altset.test() samples combinations of calibration anchors and truth-table cutoffs. loo.test() and subsample.test() reconstruct the analysis after changing the composition of the cases. The remaining diagnostics organize structured comparisons: cluster.test() evaluates the selected sufficient expressions within substantively meaningful groups, while theory.test() compares solutions obtained from alternative condition sets.

Importantly, these functions are not a catch-all solution to QCA’s methodological choices—just convenient tools for appraising and presenting the robustness of choices made throughout the analytical process. As we will soon learn, the evidence stored by each reflects the comparison it actually performs.

This guide develops those diagnostics as parts of a replicable research workflow. Also, it presupposes familiarity with the basic mechanics of QCA, its terminology, and the epistemological context that guides the method. In other words, this is a “how can I strengthen my robustness appraisal?” book for readers who already know how to perform a QCA.

Why qcaERT?

qcaERT extends analyses conducted with the QCA package (Duşa 2019). Rather than leaving researchers to reconstruct, rerun, and document every alternative analysis by hand, its diagnostics organize recurring robustness tasks around a shared set of controls and returned objects. Depending on the question being asked, the stored evidence may include formula-preservation boundaries, sampled alternative specifications, deleted or retained cases, solution consistency, proportional reduction in inconsistency (PRI), coverage, parameters of fit calculated within clusters, pairwise comparisons among condition-set specifications, and explicit records of analyses that could not produce the requested comparison.

This emphasis differs from, but does not displace, the robustness tools provided by SetMethods (Oana and Schneider 2018, 2024). SetMethods offers the more explicitly set-theoretic protocol when the central task is to interpret relations among an initial solution, alternative test solutions, and their cases. qcaERT emphasizes operational control and transparency over how robustness tests are constructed, executed, compared, diagnosed, stored, and presented across a common workflow. They are siblings with different styles, and the useful choice between them depends on the question being asked and how much granular control you want from the robustness appraisal.

The use of qcaERT, therefore, rests on practicality and systematicity: it turns a claim about robustness into a visible record of what was varied, what was held fixed, what changed, and what could not be evaluated. Theoretical judgments and relevant decisions remain with the researcher, but now with a fuller basis for making and reporting them.

A Shared Family Grammar

The main qcaERT diagnostic functions are designed as a family, with shared grammar throughout. When the same analytical or operational control appears in several diagnostics, the functions generally use the same argument name and preserve the same meaning. Controls for the solution type, treatment of logical remainders, model selection, intermediate branches, progress display, and other recurring parts of the QCA workflow can therefore be learned once and recognized again. The function-specific arguments then identify the part of the analysis that actually changes, such as a calibration anchor, a truth-table cutoff, the composition of the cases, the grouping structure, or the condition-set specification.

The returned objects follow the same broad principle. Most diagnostics separate a clean results component from the more detailed diagnostics records and the settings that record the analytical design. Supporting components such as baseline, bounds, by_direction, by_case, by_run, or by_draw retain the evidence required by the particular comparison. Their print methods provide a concise entry point, while as.data.frame() exposes the main result and the supporting components remain available for closer inspection.

This consistent framework is meant to make the package easier to learn without concealing the differences among the diagnostics. Once we understand how one function selects a solution type, records a failed comparison, or separates a result from its supporting evidence, we should encounter familiar controls and inspection steps in the next function. The book follows the same principle: a recurring control is taught fully at its first meaningful appearance, while later chapters concentrate on changed defaults, new interactions, and function-specific consequences.

At the same time, the functions are not forced into an identical interface when their analytical objects differ. For example, cluster.test() begins from an existing truth table, while theory.test() constructs several truth tables and consequently returns several related result tables. If the common grammar supplies continuity, the function-specific arguments and returned components preserve the distinctions required by each diagnostic.

The chapters use one analysis of “robust civil society” to build the workflow from the ground up. The sequence (1) establishes and presents the reference QCA; (2) then moves calibration anchors, truth-table cutoffs, and bundles of settings; (3) alters case composition; (4) and finally evaluates pooled solution formulas across clusters and compares alternative condition sets.

For each function, the chapter begins with the analytical problem it addresses, develops the workflow from a narrow example into a fuller implementation, and interprets the resulting evidence within the limits of the design that produced it.

Readers new to robustness diagnostics are welcome to proceed in order. The next chapter establishes the meaning of robustness used throughout the book; the navigation guide that follows it then directs readers arriving with a particular problem to the relevant diagnostic.

This book occasionally allows itself a joke, an aside, or a mildly irreverent turn of phrase. Like any research approach or technique, QCA can be methodologically demanding, especially when it is narrated as though solemnity were another parameter of fit. The humor and ocasional colloquialism is not meant to trivialize the analytical decisions, the cases, or the reader; it gives dense explanations some room to breathe and keeps the conversation a bit lighter. Whenever precision and playfulness pull in different directions, however, precision wins.

Author

Breno A. H. Marisguia is a postdoctoral researcher in the Graduate Program in Political Science at the Federal University of Minas Gerais. His work focuses on research design, comparative methodology, quantitative and configurational methods in the social sciences. He is the developer of qcaERT, an R package for conducting and documenting robustness diagnostics in Qualitative Comparative Analysis; the Research Design Notebook, an interactive application and teaching tool for organizing, visualizing, and assessing the coherence of research designs; and coauthor of Análise Qualitativa Comparada para Ciências Sociais (2026). He has taught QCA, research design, and R programming in undergraduate, graduate, and intensive methods courses. His substantive research examines political institutions, presidential interruptions, legislative politics, and democracy in Latin America.

How to Cite This Book

Marisguia, B. A. H. (2026). A Guide to qcaERT: Robustness Diagnostics in QCA (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.21685193

Duşa, Adrian. 2019. QCA with r: A Comprehensive Resource. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-75668-4.

Oana, Ioana-Elena, and Carsten Q. Schneider. 2018. “SetMethods: An Add-on R Package for Advanced QCA.” The R Journal 10 (1): 507–33. https://journal.r-project.org/archive/2018/RJ-2018-031/index.html.

———. 2024. “A Robustness Test Protocol for Applied QCA: Theory and R Software Application.” Sociological Methods & Research 53 (1): 57–88. https://doi.org/10.1177/00491241211036158.

Schneider, Carsten Q., and Claudius Wagemann. 2012. Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139004244.

Skaaning, Svend-Erik. 2011. “Assessing the Robustness of Crisp-Set and Fuzzy-Set QCA Results.” Sociological Methods & Research 40 (2): 391–408. https://doi.org/10.1177/0049124111404818.

Wagemann, Claudius, and Carsten Q. Schneider. 2015. “Transparency Standards in Qualitative Comparative Analysis.” Qualitative & Multi-Method Research 13 (1): 38–42. https://doi.org/10.5281/zenodo.893091.