A Tutorial for Mahalanobis distance matching with MAGMA.R
Summary
This vignette complements the main MAGMA.R vignette by introducing Mahalanobis Distance Matching (MDM). Unlike Propensity Score Matching (PSM), MDM matches participants directly on multiple continuous covariates without requiring the estimation of propensity scores. Furthermore, MAGMA.R supports weighted Mahalanobis distance matching, allowing researchers to assign greater importance to selected covariates during the matching process. Importantly, Mahalanobis distance matching integrates into the MAGMA framework. Consequently, all features of MAGMA.R - including matching for up to four groups (and 2×2 designs), exact matching, and the comprehensive post-matching balance evaluation framework - are fully supported when using MDM.
When to use MDM instead of PSM?
PSM offers several advantages. Propensity scores can incorporate covariates measured on different scale levels, can account for nonlinear relationships and interactions between covariates and - depending on the method with which Propensity Scores are estimated - cases with missing values can be kept in the data set (Ridgeway et al., 2015). However, because a propensity score summarizes information from multiple covariates into a single value, individuals with identical propensity scores may still differ in their observed covariate values (Guo et al., 2020). This limitation is often less problematic, as comparisons take place at the sample level, where the joint distribution of covariates is independent of the treatment assignment after conducting PSM (Guo et al., 2020). However, when the matched sample size is small, differences between groups may remain. As MDM directly optimizes similarity across covariates, it can provide closer matches at the individual level and may therefore be advantageous in small samples.
Despite this advantage, MDM has several limitations. First, standard MDM cannot directly handle categorical covariates or missing data. Second, its performance may decrease when covariates deviate from normality (Gu & Rosenbaum, 1993). Third, MDM can become less effective when many covariates are included, partly because all covariates contribute equally to the distance calculation (Stuart, 2010). To address this issue, weighted MDM can be used to assign greater importance to selected covariates. Nevertheless, computational demands increase with the number of covariates and are typically higher for MDM than for PSM. In summary, MDM may be advantagous when PSM fails to achieve sufficient balance, the number of (metric) covariates is not too high, and the amount of missing data on the covariates is low.
Two-Group Example
This example illustrates MDM for two groups using the simulated data
set MAGMA_sim_data included in this package. The grouping
variable is gifted_support. The covariates used for MDM are
GPA_school, IQ_score, and
Motivation. To facilitate later estimations, we define a
vector containing these covariates:
Before matching, we examine descriptive statistics and initial unbalance.
# Estimate overall and group specific descriptive statistics and Cohen’s d
descs_gifted_pre <- MAGMA_desc(Data = MAGMA_sim_data,
group = "gifted_support",
covariates = covariates,
filename = "stats_gifted_pre.docx")
descs_gifted_pre %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"No Support N", "No Support Mean", "No Support SD",
"Support N", "Support Mean", "Support SD",
"d"))
#> Overall N Overall Mean Overall SD No Support N No Support Mean
#> gifted_support 800 0.38 0.49 495 0.00
#> GPA_school 800 3.29 0.93 495 3.13
#> IQ_score 800 100.51 15.00 495 96.32
#> Motivation 800 3.82 0.89 495 3.81
#> No Support SD Support N Support Mean Support SD d
#> gifted_support 0.00 305 1.00 0.00 -Inf
#> GPA_school 0.91 305 3.56 0.89 -0.48
#> IQ_score 14.20 305 107.30 13.75 -0.78
#> Motivation 0.90 305 3.83 0.88 -0.02
# Estimating the four balance criteria
unbalance_gifted <- initial_unbalance(Data = MAGMA_sim_data,
group = "gifted_support",
covariates = covariates,
round = 3)
#> Mean g was computed using random effects meta-analysis with metafor.
unbalance_gifted
#> Pillai's Trace d-ratio Mean g adj. d-ratio
#> Unbalance 0.129 0.333 0.427 0.331As evident from the descriptive statistics - particulary Cohen’s
d - and the initial imbalance assessment, the two groups differ
with respect to the covariates. Therefore, we conduct two MDM matchings.
First, a standard matching in which all cases from group one
can be matched to all cases from group two. Second, an exact
matching in which only cases sharing the same value on the variable
gender can be matched (for more details on exact matching,
see the main MAGMA.R vignette).
Standard Matching
The function MAGMA() enables standard MDM. The argument
Data specifies the data set, the argument
group specifies the grouping variable as a character, and
the argument covs defines the covariates used for MDM as a
character vector. Optional arguments can be used to further customize
the matching procedure. For weighted MDM, the argument
weights assigns weights to the covariates specified under
covs. This argument must be a numeric vector with the same
length as covs. The argument cores enables
parallel computation, and the argument verbose controls
console output. Although the matching algorithm differs from PSM, the
output structure of MAGMA() remains the same.
# Conducting matching for gifted support
MAGMA_sim_data_gifted <- MAGMA(Data = MAGMA_sim_data,
group = "gifted_support",
covs = covariates,
weights = c(1, 1.5, 0.5),
cores = 2,
verbose = TRUE)Balance Estimation
The procedure for post-matching evaluation is identical to the
procedure used for PSM. Balance can be estimated using the function
Balance_MAGMA(). Subsequently, balance results for specific
sample sizes can be extracted using Balance_extract, the
balance trend across different sample sizes can be examined using
Plot_MAGMA, and sample sizes at which each balance
criterion is optimized can be displayed using Table_MAGMA.
Finally, post-matching descriptive statistics can be computed.
# Estimating the four balance criteria iteratively over possible sample sizes
Balance_gifted <- Balance_MAGMA(Data = MAGMA_sim_data_gifted,
group = "gifted_support",
covariates = covariates,
step = "step") # Extracting balance criteria for 100 cases per group
Balance_100_criteria <- Balance_extract(Balance = Balance_gifted,
samplesize = 100,
effects = FALSE)
Balance_100_criteria
#> Pillai's Trace d-ratio mean g adj. d-ratio
#> 0.00 1.00 0.01 0.91
# Extracting pairwise effects for 100 cases per group
Balance_100_effects <- Balance_extract(Balance = Balance_gifted,
samplesize = 100,
effects = TRUE)
Balance_100_effects
#> GPA_school IQ_score Motivation
#> 0.02 0.01 -0.01
# Plotting balance trend over sample size
Plot_MAGMA(Balance = Balance_gifted,
criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
print = FALSE)
#> $Pillai
#>
#> $`d-ratio`
#>
#> $`mean g`
#>
#> $`adj-d-ratio`
# Sample Sizes with optimized balance criteria
Table_MAGMA(Balance = Balance_gifted,
round = 3)
#> Balance table successfully created!
#> # A tibble: 4 × 6
#> Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
#> <chr> <dbl> <dbl> <dbl> <dbl> <int>
#> 1 Best Pillai 0 1 0.002 0.947 132
#> 2 Best mean g 0 1 0.002 0.947 132
#> 3 Best adj. d-ratio 0 1 0.018 0.979 255
#> 4 Best d-ratio 0.01 1 0.099 0.803 295
# Computing descriptive statistics and pairwise effects for 100 cases per group
descs_gifted_post <- MAGMA_desc(Data = MAGMA_sim_data_gifted,
group = "gifted_support",
covariates = covariates,
step_num = 255,
step_var = "step")
# Displaying the table with defined colum names
descs_gifted_post %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"No Support N", "No Support Mean", "No Support SD",
"Support N", "Support Mean", "Support SD",
"d"))
#> Overall N Overall Mean Overall SD No Support N No Support Mean
#> gifted_support 510 0.50 0.50 255 0.00
#> GPA_school 510 3.48 0.85 255 3.47
#> IQ_score 510 104.04 12.39 255 103.82
#> Motivation 510 3.86 0.84 255 3.86
#> No Support SD Support N Support Mean Support SD d
#> gifted_support 0.00 255 1.00 0.00 -Inf
#> GPA_school 0.85 255 3.48 0.86 -0.01
#> IQ_score 12.32 255 104.27 12.48 -0.04
#> Motivation 0.83 255 3.86 0.86 0.00The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 255 cases per group), Pillai’s Trace decreased from .129 to < .001, d-ratio increased from .333 to 1, mean g decreased from 0.427 to 0.018, and adjusted d-ratio increased from .331 to .979. Thus, MDM was able to identify a solution in which the distributions of the covariates were comparable between the two groups.
Exact Matching
In exact matching, only cases sharing the same value on a variable -
here gender can be matched. The function
MAGMA_exact() enables standard MDM. This function has an
additional argument, where the name of the exact variable must be
defined as a character.
# Conducting matching for gifted support
MAGMA_sim_data_gifted_exact <- MAGMA_exact(Data = MAGMA_sim_data,
group = "gifted_support",
covs = covariates,
weights = c(1, 1.5, 0.5),
cores = 2,
exact = "gender")Balance Estimation
After the exaxt matching, the same procedure as for standard matching can be applied to evaluate the balance.
# Estimating the four balance criteria iteratively over possible sample sizes
Balance_gifted_exact <- Balance_MAGMA(Data = MAGMA_sim_data_gifted_exact,
group = "gifted_support",
covariates = covariates,
step = "step")
# Extracting balance criteria for 100 cases per group
Balance_100_criteria_exact <- Balance_extract(Balance = Balance_gifted_exact,
samplesize = 100,
effects = FALSE)
Balance_100_criteria_exact
#> Pillai's Trace d-ratio mean g adj. d-ratio
#> 0.00 1.00 0.01 0.91
# Extracting pairwise effects for 100 cases per group
Balance_100_effects_exact <- Balance_extract(Balance = Balance_gifted_exact,
samplesize = 100,
effects = TRUE)
Balance_100_effects_exact
#> GPA_school IQ_score Motivation
#> -0.02 -0.02 0.00
# Plotting balance trend over sample size
Plot_MAGMA(Balance = Balance_gifted_exact,
criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
print = FALSE)
#> $Pillai
#>
#> $`d-ratio`
#>
#> $`mean g`
#>
#> $`adj-d-ratio`
# Sample Sizes with optimized balance criteria
Table_MAGMA(Balance = Balance_gifted_exact,
round = 3)
#> Balance table successfully created!
#> # A tibble: 4 × 6
#> Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
#> <chr> <dbl> <dbl> <dbl> <dbl> <int>
#> 1 Best Pillai 0 1 0.001 0.758 24
#> 2 Best mean g 0 1 0.001 0.758 24
#> 3 Best adj. d-ratio 0 1 0.016 0.96 179
#> 4 Best d-ratio 0.01 1 0.112 0.792 292
# Computing descriptive statistics and pairwise effects for 100 cases per group
descs_gifted_post_exact <- MAGMA_desc(Data = MAGMA_sim_data_gifted_exact,
group = "gifted_support",
covariates = covariates,
step_num = 179,
step_var = "step")
# Displaying the table with defined colum names
descs_gifted_post_exact %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"No Support N", "No Support Mean", "No Support SD",
"Support N", "Support Mean", "Support SD",
"d"))
#> Overall N Overall Mean Overall SD No Support N No Support Mean
#> gifted_support 358 0.50 0.50 179 0.00
#> GPA_school 358 3.37 0.78 179 3.37
#> IQ_score 358 103.11 12.04 179 103.01
#> Motivation 358 3.79 0.77 179 3.78
#> No Support SD Support N Support Mean Support SD d
#> gifted_support 0.00 179 1.00 0.00 -Inf
#> GPA_school 0.78 179 3.38 0.78 -0.01
#> IQ_score 12.08 179 103.22 12.04 -0.02
#> Motivation 0.74 179 3.80 0.80 -0.03The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 179 cases per group), Pillai’s Trace decreased from .129 to < .001, d-ratio increased from .333 to 1, mean g decreased from 0.427 to 0.016, and adjusted d-ratio increased from .331 to .96. Thus, exact MDM was also able to identify a solution in which the distributions of the covariates were comparable between the two groups.
Three-Group Example
This example illustrates MDM for three groups using the same data set
(MAGMA_sim_data) and covariates (GPA_school,
IQ_score, and Motivation) as the two-group
example. As the grouping variable changes to
teacher_ability_rating (below average[BA], average [A],
above average[AA]), we start by estimating descriptive statistics and
initial unbalance.
# Computing descriptive statistics and all pairwise effects for three groups
descs_tar_pre <- MAGMA_desc(Data = MAGMA_sim_data,
group = "teacher_ability_rating",
covariates = covariates)
descs_tar_pre %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"BA N", "BA Support Mean", "BA Support SD",
"A N", "A Mean", "A SD",
"AA N", "AA Mean", "AA SD",
"d BA-A", "d BA-AA", "d A-AA"))
#> Overall N Overall Mean Overall SD BA N BA Support Mean
#> teacher_ability_rating 800 2.02 0.80 243 1.00
#> GPA_school 800 3.29 0.93 243 2.76
#> IQ_score 800 100.51 15.00 243 92.93
#> Motivation 800 3.82 0.89 243 3.77
#> BA Support SD A N A Mean A SD AA N AA Mean AA SD d BA-A
#> teacher_ability_rating 0.00 294 2.00 0.00 263 3.00 0.00 -Inf
#> GPA_school 0.82 294 3.33 0.85 263 3.73 0.86 -0.68
#> IQ_score 14.63 294 101.20 12.84 263 106.73 14.54 -0.60
#> Motivation 0.88 294 3.72 0.88 263 3.98 0.90 0.06
#> d BA-AA d A-AA
#> teacher_ability_rating -Inf -Inf
#> GPA_school -1.15 -0.47
#> IQ_score -0.95 -0.40
#> Motivation -0.24 -0.29
# Estimating and printing initial unbalance for teacher rated ability
unbalance_tar <- initial_unbalance(Data = MAGMA_sim_data,
group = "teacher_ability_rating",
covariates = covariates,
round = 3)
#> Mean g was computed using robust variance meta-analysis with robumeta.
unbalance_tar
#> Pillai's Trace d-ratio Mean g adj. d-ratio
#> Unbalance 0.215 0.111 0.537 0.158Similar as above, there are some substantial group difference, wherefore we conduct a standard and an exact matching.
Standard Matching
To conduct standard MDM, we only need to adapt the grouping variable compared to the two-group example.
# Conducting matching for gifted support
MAGMA_sim_data_tar <- MAGMA(Data = MAGMA_sim_data,
group = "teacher_ability_rating",
covs = covariates,
weights = c(1, 1.5, 0.5),
cores = 2)Balance Estimation
To evaluate the balance, we only need to change the specific object names and grouping variable compared to the two group example. Otherwise, the process of balance estimation and visualization remains the same.
# Estimating the four balance criteria iteratively over possible sample sizes
Balance_tar <- Balance_MAGMA(Data = MAGMA_sim_data_tar,
group = "teacher_ability_rating",
covariates = covariates,
step = "step") # Extracting balance criteria for 100 cases per group
Balance_100_criteria_tar <- Balance_extract(Balance = Balance_tar,
samplesize = 100,
effects = FALSE)
Balance_100_criteria_tar
#> Pillai's Trace d-ratio mean g adj. d-ratio
#> 0.00 1.00 0.04 0.87
# Extracting pairwise effects for 100 cases per group
Balance_100_effects_tar <- Balance_extract(Balance = Balance_tar,
samplesize = 100,
effects = TRUE)
Balance_100_effects_tar
#> GPA_school GPA_school GPA_school IQ_score IQ_score IQ_score Motivation
#> 0.00 0.01 0.03 -0.03 -0.03 -0.07 -0.03
#> Motivation Motivation
#> -0.03 -0.10
# Plotting balance trend over sample size
Plot_MAGMA(Balance = Balance_tar,
criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
print = FALSE)
#> $Pillai
#>
#> $`d-ratio`
#>
#> $`mean g`
#>
#> $`adj-d-ratio`
# Sample Sizes with optimized balance criteria
Table_MAGMA(Balance = Balance_tar,
round = 3)
#> Balance table successfully created!
#> # A tibble: 4 × 6
#> Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
#> <chr> <dbl> <dbl> <dbl> <dbl> <int>
#> 1 Best mean g 0 1 0.02 0.77 33
#> 2 Best Pillai 0 1 0.028 0.786 42
#> 3 Best adj. d-ratio 0.002 1 0.043 0.882 120
#> 4 Best d-ratio 0.008 1 0.086 0.822 148
# Computing descriptive statistics and pairwise effects for 100 cases per group
descs_tar_post <- MAGMA_desc(Data = MAGMA_sim_data_tar,
group = "teacher_ability_rating",
covariates = covariates,
step_num = 120,
step_var = "step")
# Displaying the table with defined colum names
descs_tar_post %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"BA N", "BA Support Mean", "BA Support SD",
"A N", "A Mean", "A SD",
"AA N", "AA Mean", "AA SD",
"d BA-A", "d BA-AA", "d A-AA"))
#> Overall N Overall Mean Overall SD BA N BA Support Mean
#> teacher_ability_rating 360 2.00 0.82 120 1.00
#> GPA_school 360 3.18 0.64 120 3.15
#> IQ_score 360 99.77 12.30 120 99.50
#> Motivation 360 3.81 0.80 120 3.76
#> BA Support SD A N A Mean A SD AA N AA Mean AA SD d BA-A
#> teacher_ability_rating 0.00 120 2.00 0.00 120 3.00 0.00 -Inf
#> GPA_school 0.66 120 3.18 0.65 120 3.20 0.62 -0.05
#> IQ_score 12.45 120 99.91 12.24 120 99.90 12.32 -0.03
#> Motivation 0.85 120 3.83 0.74 120 3.83 0.82 -0.09
#> d BA-AA d A-AA
#> teacher_ability_rating -Inf -Inf
#> GPA_school -0.08 -0.03
#> IQ_score -0.03 0.00
#> Motivation -0.08 0.00The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 120 cases per group), Pillai’s Trace decreased from .215 to .002, d-ratio increased from .111 to 1, mean g decreased from 0.537 to 0.043, and adjusted d-ratio increased from .158 to .882. Thus, MDM was able to identify a solution in which the distributions of the covariates were comparable between the three groups.
Exaxt Matching
Similar as for the two-group example, we need to use the function
MAGMA_exact() for exact MDM and specify the argument
exact.
# Conducting matching for gifted support
MAGMA_sim_data_tar_exact <- MAGMA_exact(Data = MAGMA_sim_data,
group = "teacher_ability_rating",
covs = covariates,
weights = c(1, 1.5, 0.5),
cores = 2,
exact = "gender")Balance Estimation
To evaluate the balance, we only need to change the specific object names and grouping variable compared to the standard matching. Otherwise, the process of balance estimation and visualization remains the same.
# Estimating the four balance criteria iteratively over possible sample sizes
Balance_tar_exact <- Balance_MAGMA(Data = MAGMA_sim_data_tar_exact,
group = "teacher_ability_rating",
covariates = covariates,
step = "step") # Extracting balance criteria for 100 cases per group
Balance_100_criteria_tar_exact <- Balance_extract(Balance = Balance_tar_exact,
samplesize = 100,
effects = FALSE)
Balance_100_criteria_tar_exact
#> Pillai's Trace d-ratio mean g adj. d-ratio
#> 0.00 1.00 0.05 0.86
# Extracting pairwise effects for 100 cases per group
Balance_100_effects_tar_exact <- Balance_extract(Balance = Balance_tar_exact,
samplesize = 100,
effects = TRUE)
Balance_100_effects_tar_exact
#> GPA_school GPA_school GPA_school IQ_score IQ_score IQ_score Motivation
#> -0.01 -0.01 0.04 -0.08 -0.04 -0.04 -0.07
#> Motivation Motivation
#> -0.03 -0.08
# Plotting balance trend over sample size
Plot_MAGMA(Balance = Balance_tar_exact,
criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
print = FALSE)
#> $Pillai
#>
#> $`d-ratio`
#>
#> $`mean g`
#>
#> $`adj-d-ratio`
# Sample Sizes with optimized balance criteria
Table_MAGMA(Balance = Balance_tar_exact,
round = 3)
#> Balance table successfully created!
#> # A tibble: 4 × 6
#> Criterion_optimized Pillai_Trace d_ratio mean_g adjusted_d_ratio n_per_group
#> <chr> <dbl> <dbl> <dbl> <dbl> <int>
#> 1 Best mean g 0.001 1 0.026 0.827 59
#> 2 Best Pillai 0 1 0.036 0.851 81
#> 3 Best adj. d-ratio 0.002 1 0.041 0.882 116
#> 4 Best d-ratio 0.009 1 0.094 0.801 145
# Computing descriptive statistics and pairwise effects for 100 cases per group
descs_tar_post_exact <- MAGMA_desc(Data = MAGMA_sim_data_tar_exact,
group = "teacher_ability_rating",
covariates = covariates,
step_num = 116,
step_var = "step")
# Displaying the table with defined colum names
descs_tar_post_exact %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"BA N", "BA Support Mean", "BA Support SD",
"A N", "A Mean", "A SD",
"AA N", "AA Mean", "AA SD",
"d BA-A", "d BA-AA", "d A-AA"))
#> Overall N Overall Mean Overall SD BA N BA Support Mean
#> teacher_ability_rating 348 2.00 0.82 116 1.00
#> GPA_school 348 3.18 0.62 116 3.15
#> IQ_score 348 99.54 12.13 116 99.31
#> Motivation 348 3.79 0.81 116 3.76
#> BA Support SD A N A Mean A SD AA N AA Mean AA SD d BA-A
#> teacher_ability_rating 0.00 116 2.00 0.00 116 3.00 0.00 -Inf
#> GPA_school 0.66 116 3.19 0.59 116 3.21 0.60 -0.06
#> IQ_score 12.37 116 99.56 11.96 116 99.74 12.16 -0.02
#> Motivation 0.86 116 3.80 0.74 116 3.80 0.82 -0.05
#> d BA-AA d A-AA
#> teacher_ability_rating -Inf -Inf
#> GPA_school -0.10 -0.03
#> IQ_score -0.04 -0.01
#> Motivation -0.05 0.00The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 116 cases per group), Pillai’s Trace decreased from .215 to .002, d-ratio increased from .111 to 1, mean g decreased from 0.537 to 0.041, and adjusted d-ratio increased from .158 to .882. Thus, exact MDM was also able to identify a solution in which the distributions of the covariates were comparable between the three groups.
2x2-design Example
This example illustrates MDM for a 2x2 designs using the same data
set (MAGMA_sim_data) and covariates
(GPA_school, IQ_score, and
Motivation) as the previous examples. The major change is
that now both gifted_support and enrichment
are used as grouping variable.
# Computing descriptive statistics and all pairwise effects for three groups
descs_2x2 <- MAGMA_desc(Data = MAGMA_sim_data,
group = c("gifted_support", "enrichment"),
covariates = covariates)
#> 2x2 groups are represented as 4 groups.
descs_2x2 %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"Sup & No En N", "Sup & No En Mean",
"Sup & No En SD",
"Sup & En N", "Sup & En Mean", "Sup & En SD",
"No Sup & No En N", "No Sup & No En Mean", "No Sup & No En SD",
"No Sup & En N", "No Sup & En Mean", "No Sup & En SD",
"d YesNo-YesYes", "d YesNo-NoNo", "d YesNo-NoYes",
"d YesYes-NoNo", "d YesYes-YNoYes",
"d NoNo-NoYes"))
#> Overall N Overall Mean Overall SD Sup & No En N Sup & No En Mean
#> group_long 800 2.63 1.07 175 1.00
#> GPA_school 800 3.29 0.93 175 3.44
#> IQ_score 800 100.51 15.00 175 105.09
#> Motivation 800 3.82 0.89 175 3.79
#> Sup & No En SD Sup & En N Sup & En Mean Sup & En SD No Sup & No En N
#> group_long 0.00 130 2.00 0.00 309
#> GPA_school 0.88 130 3.71 0.88 309
#> IQ_score 14.09 130 110.26 12.74 309
#> Motivation 0.90 130 3.89 0.86 309
#> No Sup & No En Mean No Sup & No En SD No Sup & En N No Sup & En Mean
#> group_long 3.00 0.00 186 4.00
#> GPA_school 3.01 0.93 186 3.31
#> IQ_score 94.27 13.30 186 99.74
#> Motivation 3.75 0.89 186 3.92
#> No Sup & En SD d YesNo-YesYes d YesNo-NoNo d YesNo-NoYes
#> group_long 0.00 -Inf -Inf -Inf
#> GPA_school 0.86 -0.31 0.47 0.15
#> IQ_score 15.01 -0.38 0.80 0.37
#> Motivation 0.90 -0.11 0.04 -0.14
#> d YesYes-NoNo d YesYes-YNoYes d NoNo-NoYes
#> group_long -Inf -Inf -Inf
#> GPA_school 0.76 0.46 -0.33
#> IQ_score 1.22 0.74 -0.39
#> Motivation 0.16 -0.03 -0.19
# Estimating and printing initial unbalance for teacher rated ability
unbalance_2x2 <- initial_unbalance(Data = MAGMA_sim_data,
group = c("gifted_support", "enrichment"),
covariates = covariates,
round = 3)
#> Mean g was computed using robust variance meta-analysis with robumeta.
unbalance_2x2
#> Pillai's Trace gifted_support Pillai's Trace enrichment
#> Unbalance 0.133 0.044
#> Pillai's Trace IA d-ratio Mean g adj. d-ratio
#> Unbalance 0.001 0.389 0.391 0.314Similar as above, there are some substantial group difference, wherefore we conduct a standard and an exact matching.
Standard Matching
To conduct standard MDM, we only need to adapt the grouping variable compared to the previous examples.
# Conducting matching for gifted support
MAGMA_sim_data_2x2 <- MAGMA(Data = MAGMA_sim_data,
group = c("gifted_support", "enrichment"),
covs = covariates,
weights = c(1, 1.5, 0.5),
cores = 2)Balance Estimation
To evaluate the balance, we only need to change the specific object names and grouping variable compared to the previous examples. Otherwise, the process of balance estimation and visualization remains the same.
# Estimating the four balance criteria iteratively over possible sample sizes
Balance_2x2 <- Balance_MAGMA(Data = MAGMA_sim_data_2x2,
group = c("gifted_support", "enrichment"),
covariates = covariates,
step = "step") # Extracting balance criteria for 100 cases per group
Balance_100_criteria_2x2 <- Balance_extract(Balance = Balance_2x2,
samplesize = 100,
effects = FALSE)
Balance_100_criteria_2x2
#> Pillai's Trace ME 1 Pillai's Trace ME 2 Pillai's Trace IA d-ratio
#> 0.00 0.00 0.00 1.00
#> mean g adj. d-ratio
#> 0.07 0.82
# Extracting pairwise effects for 100 cases per group
Balance_100_effects_2x2 <- Balance_extract(Balance = Balance_2x2,
samplesize = 100,
effects = TRUE)
Balance_100_effects_2x2
#> GPA_school GPA_school GPA_school GPA_school GPA_school GPA_school IQ_score
#> -0.09 -0.08 -0.08 0.01 0.03 0.05 -0.05
#> IQ_score IQ_score IQ_score IQ_score IQ_score Motivation Motivation
#> -0.04 -0.07 0.09 0.12 0.14 0.04 0.04
#> Motivation Motivation Motivation Motivation
#> 0.02 -0.05 -0.07 -0.13
# Plotting balance trend over sample size
Plot_MAGMA(Balance = Balance_2x2,
criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
print = FALSE)
#> $Pillai
#> $Pillai[[1]]
#>
#> $Pillai[[2]]
#>
#> $Pillai[[3]]
#>
#>
#> $`d-ratio`
#>
#> $`mean g`
#>
#> $`adj-d-ratio`
# Sample Sizes with optimized balance criteria
Table_MAGMA(Balance = Balance_2x2,
round = 3)
#> Balance table successfully created!
#> # A tibble: 6 × 8
#> Criterion_optimized Pillai_Trace_ME1 Pillai_Trace_ME2 Pillai_Trace_IA d_ratio
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 Best Pillai ME 2 0.009 0 0.002 0.889
#> 2 Best mean g 0.001 0.002 0 1
#> 3 Best Pillai ME 1 0 0.003 0 1
#> 4 Best Pillai IA 0.002 0.004 0 1
#> 5 Best adj. d-ratio 0.001 0.002 0 1
#> 6 Best d-ratio 0.001 0.005 0.001 1
#> mean_g adjusted_d_ratio n_per_group
#> <dbl> <dbl> <int>
#> 1 0.075 0.705 39
#> 2 0.046 0.836 83
#> 3 0.049 0.833 84
#> 4 0.069 0.824 106
#> 5 0.055 0.855 112
#> 6 0.076 0.829 139
# Computing descriptive statistics and pairwise effects for 100 cases per group
descs_2x2_post <- MAGMA_desc(Data = MAGMA_sim_data_2x2,
group = c("gifted_support", "enrichment"),
covariates = covariates,
step_num = 112,
step_var = "step")
#> 2x2 groups are represented as 4 groups.
# Displaying the table with defined colum names
descs_2x2_post %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"Sup & No En N", "Sup & No En Mean",
"Sup & No En SD",
"Sup & En N", "Sup & En Mean", "Sup & En SD",
"No Sup & No En N", "No Sup & No En Mean", "No Sup & No En SD",
"No Sup & En N", "No Sup & En Mean", "No Sup & En SD",
"d YesNo-YesYes", "d YesNo-NoNo", "d YesNo-NoYes",
"d YesYes-NoNo", "d YesYes-YNoYes",
"d NoNo-NoYes"))
#> Overall N Overall Mean Overall SD Sup & No En N Sup & No En Mean
#> group_long 336 2.50 1.12 84 1.00
#> GPA_school 336 3.53 0.74 84 3.58
#> IQ_score 336 104.87 10.41 84 105.58
#> Motivation 336 3.87 0.74 84 3.90
#> Sup & No En SD Sup & En N Sup & En Mean Sup & En SD No Sup & No En N
#> group_long 0.00 84 2.00 0.00 84
#> GPA_school 0.78 84 3.51 0.73 84
#> IQ_score 10.67 84 104.74 10.56 84
#> Motivation 0.77 84 3.85 0.74 84
#> No Sup & No En Mean No Sup & No En SD No Sup & En N No Sup & En Mean
#> group_long 3.00 0.00 84 4.00
#> GPA_school 3.53 0.71 84 3.50
#> IQ_score 104.87 10.06 84 104.30
#> Motivation 3.88 0.74 84 3.83
#> No Sup & En SD d YesNo-YesYes d YesNo-NoNo d YesNo-NoYes
#> group_long 0.00 -Inf -Inf -Inf
#> GPA_school 0.73 0.09 0.07 0.11
#> IQ_score 10.48 0.08 0.07 0.12
#> Motivation 0.73 0.07 0.03 0.09
#> d YesYes-NoNo d YesYes-YNoYes d NoNo-NoYes
#> group_long -Inf -Inf -Inf
#> GPA_school -0.03 0.01 0.04
#> IQ_score -0.01 0.04 0.06
#> Motivation -0.04 0.03 0.07The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 112 cases per group), Pillai’s Trace (gifted_support) decreased from .133 to .001, Pillai’s Trace (enrichment) decreased from .044 to .002, Pillai’s Trace (interaction) remained at < .001, d-ratio increased from .389 to 1, mean g decreased from 0.391 to 0.055, and adjusted d-ratio increased from .314 to .855. Thus, MDM was able to identify a solution in which the distributions of the covariates were comparable for the 2x2 design.
Exaxt Matching
Similar as for the previous examples, we need to use the function
MAGMA_exact() for exact MDM and specify the argument
exact.
# Conducting matching for gifted support
MAGMA_sim_data_2x2_exact <- MAGMA_exact(Data = MAGMA_sim_data,
group = c("gifted_support", "enrichment"),
covs = covariates,
weights = c(1, 1.5, 0.5),
cores = 2,
exact = "gender")Balance Estimation
To evaluate the balance, we only need to change the specific object names and grouping variable compared to the standard matching. Otherwise, the process of balance estimation and visualization remains the same.
# Estimating the four balance criteria iteratively over possible sample sizes
Balance_2x2_exact <- Balance_MAGMA(Data = MAGMA_sim_data_2x2_exact,
group = c("gifted_support", "enrichment"),
covariates = covariates,
step = "step") # Extracting balance criteria for 100 cases per group
Balance_100_criteria_2x2_exact <- Balance_extract(Balance = Balance_2x2_exact,
samplesize = 100,
effects = FALSE)
Balance_100_criteria_2x2_exact
#> Pillai's Trace ME 1 Pillai's Trace ME 2 Pillai's Trace IA d-ratio
#> 0.00 0.00 0.00 1.00
#> mean g adj. d-ratio
#> 0.06 0.83
# Extracting pairwise effects for 100 cases per group
Balance_100_effects_2x2_exact <- Balance_extract(Balance = Balance_2x2_exact,
samplesize = 100,
effects = TRUE)
Balance_100_effects_2x2_exact
#> GPA_school GPA_school GPA_school GPA_school GPA_school GPA_school IQ_score
#> -0.02 -0.05 -0.09 0.06 0.09 -0.03 0.03
#> IQ_score IQ_score IQ_score IQ_score IQ_score Motivation Motivation
#> 0.04 0.05 0.08 0.14 0.07 0.05 0.09
#> Motivation Motivation Motivation Motivation
#> 0.13 -0.03 -0.05 0.07
# Plotting balance trend over sample size
Plot_MAGMA(Balance = Balance_2x2_exact,
criterion = c("Pillai", "d_ratio", "mean_g", "Adj_d_ratio"),
print = FALSE)
#> $Pillai
#> $Pillai[[1]]
#>
#> $Pillai[[2]]
#>
#> $Pillai[[3]]
#>
#>
#> $`d-ratio`
#>
#> $`mean g`
#>
#> $`adj-d-ratio`
# Sample Sizes with optimized balance criteria
Table_MAGMA(Balance = Balance_2x2_exact,
round = 3)
#> Balance table successfully created!
#> # A tibble: 6 × 8
#> Criterion_optimized Pillai_Trace_ME1 Pillai_Trace_ME2 Pillai_Trace_IA d_ratio
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 Best mean g NA NA NA 1
#> 2 Best Pillai IA 0.005 0 0 1
#> 3 Best Pillai ME 2 0.003 0 0.004 0.944
#> 4 Best Pillai ME 1 0.001 0 0.002 1
#> 5 Best adj. d-ratio 0.001 0 0.002 1
#> 6 Best d-ratio 0.003 0.003 0.002 1
#> mean_g adjusted_d_ratio n_per_group
#> <dbl> <dbl> <int>
#> 1 0.044 0.692 20
#> 2 0.055 0.723 34
#> 3 0.052 0.811 77
#> 4 0.044 0.842 86
#> 5 0.046 0.845 90
#> 6 0.075 0.808 109
# Computing descriptive statistics and pairwise effects for 100 cases per group
descs_2x2_post_exact <- MAGMA_desc(Data = MAGMA_sim_data_2x2_exact,
group = c("gifted_support", "enrichment"),
covariates = covariates,
step_num = 116,
step_var = "step")
#> 2x2 groups are represented as 4 groups.
# Displaying the table with defined colum names
descs_2x2_post_exact %>%
purrr::set_names(c("Overall N", "Overall Mean", "Overall SD",
"Sup & No En N", "Sup & No En Mean",
"Sup & No En SD",
"Sup & En N", "Sup & En Mean", "Sup & En SD",
"No Sup & No En N", "No Sup & No En Mean", "No Sup & No En SD",
"No Sup & En N", "No Sup & En Mean", "No Sup & En SD",
"d YesNo-YesYes", "d YesNo-NoNo", "d YesNo-NoYes",
"d YesYes-NoNo", "d YesYes-YNoYes",
"d NoNo-NoYes"))
#> Overall N Overall Mean Overall SD Sup & No En N Sup & No En Mean
#> group_long 464 2.50 1.12 116 1.00
#> GPA_school 464 3.58 0.83 116 3.64
#> IQ_score 464 106.51 10.53 116 107.92
#> Motivation 464 3.85 0.81 116 3.91
#> Sup & No En SD Sup & En N Sup & En Mean Sup & En SD No Sup & No En N
#> group_long 0.00 116 2.00 0.00 116
#> GPA_school 0.85 116 3.58 0.86 116
#> IQ_score 11.30 116 106.57 10.53 116
#> Motivation 0.86 116 3.82 0.77 116
#> No Sup & No En Mean No Sup & No En SD No Sup & En N No Sup & En Mean
#> group_long 3.00 0.00 116 4.00
#> GPA_school 3.58 0.78 116 3.50
#> IQ_score 106.42 10.64 116 105.14
#> Motivation 3.82 0.83 116 3.84
#> No Sup & En SD d YesNo-YesYes d YesNo-NoNo d YesNo-NoYes
#> group_long 0.00 -Inf -Inf -Inf
#> GPA_school 0.85 0.07 0.07 0.16
#> IQ_score 9.54 0.12 0.14 0.27
#> Motivation 0.79 0.11 0.11 0.08
#> d YesYes-NoNo d YesYes-YNoYes d NoNo-NoYes
#> group_long -Inf -Inf -Inf
#> GPA_school 0.00 0.09 0.10
#> IQ_score 0.01 0.14 0.13
#> Motivation 0.00 -0.03 -0.02The post-matching balance and descriptive statistics show that the initial imbalance between the groups was substantially reduced. In the solution with the best adjusted d-ratio (n = 112 cases per group), Pillai’s Trace (gifted_support) decreased from .133 to .001, Pillai’s Trace (enrichment) decreased from .044 to < .001, Pillai’s Trace (interaction) remained low at .002, d-ratio increased from .389 to 1, mean g decreased from 0.391 to 0.046, and adjusted d-ratio increased from .314 to .845. Thus, exact MDM was also able to identify a solution in which the distributions of the covariates were comparable for the 2x2 design.
References
Gu, X. S., & Rosenbaum, P. R. (1993). Comparison of multivariate matching methods: Structures, distances, and algorithms. Journal of Computational and Graphical Statistics, 2(4), 405-420. https://doi.org/10.1080/10618600.1993.10474623
Guo, S., Fraser, M., & Chen, Q. (2020). Propensity Score Analysis: Recent Debate and Discussion. Journal of the Society for Social Work and Research, 11(3), 463–482. https://doi.org/10.1086/711393
Ridgeway, G., McCaffrey, D. F., Morral, A. R., Cefalu, M., Burgette, L. F., Pane, J. D., & Griffin, B. A. (2015). Toolkit for Weighting and Analysis of Nonequivalent Groups: A tutorial for the twang package. Rand.
Stuart, E. A. (2010). Matching Methods for Causal Inference: A Review and a Look Forward. Statistical Science, 25(1). https://doi.org/10.1214/09-STS313