3  The Reference Analysis

All diagnostics in this guide start from one reference QCA, which supplies the calibrated memberships, truth table, and solutions that later checks will revisit. For its illustrative research question, the running example asks which configurations are sufficient for robust civil society among lower-income economies in 2024.

library(QCA)
library(qcaERT)
library(dplyr)

The baseline calculations use the QCA package for calibration, truth-table construction, necessity analysis, and minimization (Duşa 2019).

3.1 Cases, outcome, and conditions

The empirical information comes from the V-Dem Country-Year Dataset v15 (Coppedge et al. 2025; Pemstein et al. 2025). The universe is restricted before calibration to economies classified by the World Bank in FY2025 as low income or lower-middle income. Those classifications use 2023 Atlas-method gross national income per capita: at most USD 1,145 for low-income economies and from USD 1,146 to USD 4,515 for lower-middle-income economies (Metreau, Young, and Eapen 2024). The vector wb_fy25_low_lower_middle_income records the economies in those two categories.

wb_fy25_low_lower_middle_income <- c(
  "AFG", "BFA", "BDI", "CAF", "TCD", "COD", "ERI", "ETH", "GMB", "GNB", "PRK",
  "LBR", "MDG", "MWI", "MLI", "MOZ", "NER", "RWA", "SLE", "SOM", "SSD", "SDN",
  "SYR", "TGO", "UGA", "YEM", "SML", "AGO", "BGD", "BEN", "BTN", "BOL", "CPV",
  "KHM", "CMR", "COM", "COG", "CIV", "DJI", "EGY", "SWZ", "GHA", "GIN", "HTI",
  "HND", "IND", "JOR", "KEN", "KIR", "KGZ", "LAO", "LBN", "LSO", "MRT", "FSM",
  "MAR", "MMR", "NPL", "NIC", "NGA", "PAK", "PNG", "PHL", "WSM", "STP", "SEN",
  "SLB", "LKA", "TJK", "TZA", "TLS", "TUN", "UZB", "VUT", "VNM", "PSE", "ZMB",
  "ZWE", "PSG", "ZZB"
  )

The outcome is CIVSOC, membership in the set of countries with robust civil society. V-Dem constructs the underlying core civil society index from the conditions governing the entry and exit of civil-society organizations, state repression of those organizations, and the participatory environment (Coppedge et al. 2025; Bernhard et al. 2017).

The four conditions, named after their corresponding V-Dem indices, are:

  • FREEXP: freedom of expression and alternative sources of information;
  • PARTY: institutionalized political parties;
  • GENCL: women’s civil liberties; and
  • JUCON: judicial constraints on the executive.

The filter produces 77 eligible V-Dem observations in 2024. Eleven have no score for party institutionalization, while the outcome and other conditions are observed; complete-case analysis therefore leaves 66 countries.

None of the indicators used to construct these four conditions is a component of CIVSOC. Although the concepts are related—as one would hope in an analysis of civil society—the outcome is not mechanically built from any of these conditions.

region_pol retains V-Dem’s six-category politico-geographic classification, which combines geographical proximity with regional traditions in democratization research, while region_geo retains the finer nineteen-category geographic grouping, whose categories are defined from geographical location alone (Coppedge et al. 2025). The two variables are kept for the later subsampling and cluster diagnostics; neither enters the reference truth table.

vdem_selected <- readRDS("2025 V-Dem-CY-Full+Others-v15.rds") %>%
  filter(
    year == 2024,
    country_text_id %in% wb_fy25_low_lower_middle_income
  ) %>%
  transmute(
    case = country_name,
    region_pol = e_regionpol_6C,
    region_geo = e_regiongeo,
    CIVSOC = v2xcs_ccsi,
    FREEXP = v2x_freexp_altinf,
    PARTY = v2xps_party,
    GENCL = v2x_gencl,
    JUCON = v2x_jucon
  )

colSums(is.na(vdem_selected))
#>       case region_pol region_geo     CIVSOC     FREEXP      PARTY      GENCL      JUCON 
#>          0          0          0          0          0         11          0          0

vdem_raw <- vdem_selected %>%
  na.omit() %>%
  as_tibble() %>%
  tibble::column_to_rownames("case")

3.2 Calibration

Calibration turns the V-Dem indices into sets. In direct fuzzy calibration, the crossover marks the point at which cases shift from being more out than in to being more in than out, while the other two qualitative anchors define full nonmembership and full membership (Ragin 2008; Schneider and Wagemann 2012).

Whenever defensible theoretical and empirically informed breakpoints are available to clearly distinguish cases qualitatively in and out of a conceptual set, substantive knowledge should determine those anchors. Hence the name Qualitative Comparative Analysis.

Here, all five concepts are V-Dem indices ranging from low to high on a 0-1 scale, but their codebook definitions, obviously, do not provide external qualitative breakpoints for this restricted population. Moreover, this is a didactic example involving a subject on which I am not an expert. The reference analysis therefore uses QCA::findTh() to locate three empirical divisions within each distribution. Finding those thresholds makes the calibration easily reproducible and useful for teaching the diagnostics, although the resulting anchors should be understood as purely data-guided.

CIVSOC_th <- findTh(vdem_raw$CIVSOC, groups = 4)
FREEXP_th <- findTh(vdem_raw$FREEXP, groups = 4)
PARTY_th <- findTh(vdem_raw$PARTY, groups = 4)
GENCL_th <- findTh(vdem_raw$GENCL, groups = 4)
JUCON_th <- findTh(vdem_raw$JUCON, groups = 4)

vdem_calib <- vdem_raw
vdem_calib$CIVSOC <- calibrate(
  vdem_raw$CIVSOC,
  type = "fuzzy",
  thresholds = CIVSOC_th
  )

vdem_calib$FREEXP <- calibrate(
  vdem_raw$FREEXP,
  type = "fuzzy",
  thresholds = FREEXP_th
  )

vdem_calib$PARTY <- calibrate(
  vdem_raw$PARTY,
  type = "fuzzy",
  thresholds = PARTY_th
  )

vdem_calib$GENCL <- calibrate(
  vdem_raw$GENCL,
  type = "fuzzy",
  thresholds = GENCL_th
  )

vdem_calib$JUCON <- calibrate(
  vdem_raw$JUCON,
  type = "fuzzy",
  thresholds = JUCON_th
  )

list(
  CIVSOC = CIVSOC_th,
  FREEXP = FREEXP_th,
  PARTY = PARTY_th,
  GENCL = GENCL_th,
  JUCON = JUCON_th
  )
#> $CIVSOC
#> [1] 0.1670 0.5275 0.7590
#> 
#> $FREEXP
#> [1] 0.3055 0.4385 0.7450
#> 
#> $PARTY
#> [1] 0.2385 0.4890 0.6840
#> 
#> $GENCL
#> [1] 0.0900 0.3925 0.7155
#> 
#> $JUCON
#> [1] 0.1370 0.3605 0.6965

The calibrated outcome is almost evenly divided: 35 cases are more in than out of CIVSOC, while 31 are more out than in.

Inductive, data-driven calibration might weaken the epistemological presuppositions of QCA as a qualitatively led research approach. Indeed, there is some tension within the configurational comparative methods (CCMs) field over this and other topics—such as the “proper” treatment of logical remainders. Personally, I find this friction important to the development of QCA and other CCMs. Besides, an intellectual and decorous tug-of-war is always riveting!

Pragmatically, though, my advice is this: it doesn’t matter. We should be guided by our research questions and design, not the other way around. If the best way to answer our question and handle our data is through a more qualitative, theoretically led, in-depth treatment and analysis, great! Conversely, if the research points us toward a more empirically grounded route in which a data-driven calibration procedure is more appropriate—or if the search for a minimally sufficient, redundancy-free solution is justified regardless of the quality of the counterfactuals—go for it! Either approach is fine, as long as it is internally consistent with the research design and we are transparent about the entire process.

3.3 A Quick Necessary-Conditions Check

As good practice dictates, necessity is assessed before sufficiency (Schneider and Wagemann 2012). But, because qcaERT currently concentrates on robustness diagnostics for sufficient solutions, the chapter keeps this preliminary screen short, briefly reporting its findings.

vdem_outcome <- "CIVSOC"
vdem_conds <- c("FREEXP", "PARTY", "GENCL", "JUCON")
superSubset(
  vdem_calib,
  outcome = vdem_outcome,
  conditions = vdem_conds,
  incl.cut = 0.9,
  ron.cut = 0.6
)
#> 
#>            inclN   RoN   covN  
#> ------------------------------ 
#> 1  FREEXP  0.939  0.748  0.820 
#> ------------------------------

Only FREEXP passes both established inclusion and relevance thresholds. Its necessity inclusion of 0.939 indicates that membership in robust civil society is largely contained within freedom of expression; its RoN of 0.748 and necessity coverage of 0.820 also show that the condition is not simply a nearly universal superset with little empirical relevance.

3.4 Truth table and minimization

The reference truth table uses n.cut = 2, so a configuration needs at least two cases to count as observed; and the sufficiency cutoff is incl.cut = 0.8—not perfect, but not bad either. In practice, the three rows coded sufficient have inclusion scores from 0.896 to 0.927, comfortably above that baseline.

vdem_incl_cut <- 0.8
vdem_n_cut <- 2

vdem_tt <- truthTable(
  vdem_calib,
  outcome = vdem_outcome,
  conditions = vdem_conds,
  incl.cut = vdem_incl_cut,
  n.cut = vdem_n_cut,
  complete = TRUE,
  show.cases = TRUE,
  sort.by = c("incl", "n")
  )

vdem_tt
#> 
#>   OUT: output value
#>     n: number of cases in configuration
#>  incl: sufficiency inclusion score
#>   PRI: proportional reduction in inconsistency
#> 
#>      FREEXP PARTY GENCL JUCON   OUT    n   incl  PRI   cases                                                                                                                                                                                             
#> 16     1      1     1     1      1     23  0.927 0.905 Ghana,Honduras,Pakistan,Senegal,India,Kenya,Lebanon,Nigeria,Tanzania,Benin,Bhutan,Nepal,Zambia,Cape Verde,Timor-Leste,Jordan,Lesotho,Morocco,Sierra Leone,The Gambia,Sri Lanka,Somaliland,Zanzibar
#> 15     1      1     1     0      1     2   0.900 0.744 Bolivia,Comoros                                                                                                                                                                                   
#> 12     1      0     1     1      1     12  0.896 0.850 Mali,Philippines,Uganda,Mozambique,Ivory Coast,Liberia,Malawi,Papua New Guinea,Tunisia,Sao Tome and Principe,Solomon Islands,Vanuatu                                                              
#> 11     1      0     1     0      0     6   0.764 0.542 Angola,Cameroon,Chad,Kyrgyzstan,Madagascar,Togo                                                                                                                                                   
#>  9     1      0     0     0      0     5   0.759 0.401 Guinea,Mauritania,Central African Republic,Democratic Republic of the Congo,Somalia                                                                                                               
#>  4     0      0     1     1      0     2   0.728 0.231 Egypt,Burkina Faso                                                                                                                                                                                
#>  7     0      1     1     0      0     2   0.663 0.124 Vietnam,Uzbekistan                                                                                                                                                                                
#>  3     0      0     1     0      0     5   0.607 0.108 Ethiopia,Zimbabwe,Burundi,Republic of the Congo,Djibouti                                                                                                                                          
#>  5     0      1     0     0      0     3   0.545 0.074 North Korea,Rwanda,Tajikistan                                                                                                                                                                     
#>  1     0      0     0     0      0     5   0.472 0.050 Yemen,South Sudan,Cambodia,Nicaragua,Eswatini                                                                                                                                                     
#>  2     0      0     0     1      ?     1   0.764 0.226 Laos                                                                                                                                                                                              
#>  6     0      1     0     1      ?     0     -     -                                                                                                                                                                                                     
#>  8     0      1     1     1      ?     0     -     -                                                                                                                                                                                                     
#> 10     1      0     0     1      ?     0     -     -                                                                                                                                                                                                     
#> 13     1      1     0     0      ?     0     -     -                                                                                                                                                                                                     
#> 14     1      1     0     1      ?     0     -     -

The three solution types below come from the same truth table. While the conservative minimization treats logical remainders as false, the enhanced parsimonious and intermediate minimizations allow remainders via include = "?", except for contradictory simplifying assumptions identified by QCA::findRows(vdem_tt, type = 2). Moreover, in this didactic example, the intermediate solution uses the presence of every condition as directional expectation.

vdem_excluded_rows <- findRows(vdem_tt, type = 2)

vdem_dir_exp <- c(
  FREEXP = "1",
  PARTY = "1",
  GENCL = "1",
  JUCON = "1"
  )

vdem_con <- minimize(
  vdem_tt,
  include = "",
  details = TRUE
  )

vdem_par <- minimize(
  vdem_tt,
  include = "?",
  exclude = vdem_excluded_rows,
  details = TRUE
  )

vdem_int <- minimize(
  vdem_tt,
  include = "?",
  exclude = vdem_excluded_rows,
  dir.exp = vdem_dir_exp,
  details = TRUE
  )

The shared objects carried into later chapters are vdem_raw, vdem_calib, vdem_outcome, vdem_conds, vdem_calib_spec, vdem_calib_spec_with_outcome, vdem_incl_cut, vdem_n_cut, vdem_tt, vdem_con, vdem_int, vdem_par, and vdem_dir_exp. Before any of the choices in the reference analysis are perturbed, the next chapter places the three reference solutions in a common table and chart so that later claims about preservation and change have a visible baseline.