Factorial ANOVA

Lecture 14

Dave Brocker

Farmingdale State College

AnOVA

What does ANOVA do?

ANOVA compares the means of different groups to determine if they differ significantly from one another.

  • ANOVA can examine independent variables with more than 2 groups.

  • ANOVA can also look at multiple independent variables.

2-way AnOVA

Prof Brocker believes the key to happiness involves 2 things: drinking coffee and Dark. He wants to see if drinking coffee makes participants significantly happier than not drinking coffee. He also wants to see if watching Dark show makes them significantly happier than not watching it.

  • What kind of design is he using?
  • 2 x 2

2-way AnOVA

We can write the design of a study analyzed using ANOVA as:

  • Number of groups “x” the Number of groups

  • Conditions in the first independent variable “x” Conditions in the second independent variable, etc.

  • 2 x 2

2-way AnOVA

Prof Brocker believes the key to happiness involves 2 things: coffee and Dark. He wants to see if drinking coffee makes participants significantly happier than not drinking coffee. He also wants to see if watching Dark show makes them significantly happier than not watching it. This is a 2 x 2 design.

  • IV1 = Number of groups in coffee: 2 (coffee or no coffee)

  • IV2 = Number of groups in Dark: 2 (Dark or No Dark)

  • DV

    • Happiness

2-way Anova

What are Prof Brocker’s Alternative Hypotheses?

  • H1: Participants who drink coffee will be significantly happier than those who don’t.

  • H2: Participants who watch Dark will be significantly happier than those who don’t.

2-way AnOVA

Prof Brocker believes the key to happiness involves 2 things: coffee and Dark. He wants to see if drinking coffee makes participants significantly happier than not drinking coffee. He also wants to see if watching Dark show makes them significantly happier than not watching it. This is a 2 x 2 design.

  • Participants who drink coffee will be significantly happier than those who don’t.

  • Participants who watch Dark will be significantly happier than those who don’t.

  • H3: Participants who drink coffee while watching the Dark will be the happiest.

ANOVA

The Three Types

  • Between-subjects: [One-Way or Factorial]

  • Within-subjects: (Repeated Measures ANOVA)

  • Mixed-designs: Combination of Between and Within

2-way Between subjects ANOVA

  • IVs must be nominal/categorical

  • DV must be continuous

2-way Between subjects ANOVA

  • IVs must be nominal/categorical

  • DV must be continuous

2-way ANOVA

Two types of variance:

  • Variance among all of the scores (error variance).

  • Variance between the groups of the first IV (between group variance 1).

  • Variance between the groups of the second IV (between group variance 2).

  • Variance between the combination of the groups (interaction variance).

2-way ANOVA

Code 
# Load libraries
library(dplyr)
library(ggplot2)
library(gt)
library(broom)

# Create a 2x2 grid structure for visualization
matrix_data <- expand.grid(
  Coffee = c("Yes", "No"),
  TV_Show = c("Yes", "No")
) %>%
  mutate(
    x = as.numeric(Coffee == "Yes"),
    y = as.numeric(TV_Show == "Yes"),
    fill_color = c("lightblue", "lightcoral", "lightgreen", "khaki") # Assign colors
  )

# Visualize the 2x2 matrix
fac_mat <- 
  ggplot(matrix_data, aes(xmin = x - 0.5, xmax = x + 0.5, ymin = y - 0.5, ymax = y + 0.5)) +
  geom_rect(aes(fill = fill_color), color = "black", alpha = 0.8) + # Draw rectangles
  scale_fill_identity() + # Use fill_color directly
  labs(
    title = "2x2 Factorial Matrix",
    x = "Coffee", y = "TV Show"
  ) +
  scale_x_continuous(breaks = c(0, 1), labels = c("No", "Yes"), limits = c(-0.5, 1.5)) +
  scale_y_continuous(breaks = c(0, 1), labels = c("No", "Yes"), limits = c(-0.5, 1.5)) +
  theme_minimal() +
  theme(
    axis.text = element_text(size = 12),
    axis.title = element_text(size = 14),
    plot.title = element_text(size = 16, face = "bold", hjust = 0.5)
  )

2-way ANOVA

Code 
fac_mat + 
  annotate(
    "text", x = 1, y = 1, label = "Coffee"
  ) +
    annotate(
    "text", x = 0, y = 1, label = "Coffee"
  ) +
    annotate(
    "text", x = 0, y = 0, label = "No Coffee"
  ) +
    annotate(
    "text", x = 1, y = 0, label = "No Coffee"
  )

2-way ANOVA

Code 
fac_mat + 
  annotate(
    "text", x = 1, y = 1, label = "Coffee\nDark"
  ) +
    annotate(
    "text", x = 0, y = 1, label = "Coffee\nNo Dark"
  ) +
    annotate(
    "text", x = 0, y = 0, label = "No Coffee\nDark"
  ) +
    annotate(
    "text", x = 1, y = 0, label = "No Coffee\nNo Dark"
  )

2-way ANOVA

To compare the means, we analyzing two types of variance:

  • Variance among all of the scores (within group variance).

  • Variance between the groups of the first IV (between group variance 1).

  • Variance between the groups of the second IV (between group variance 2).

  • Variance between the combination of the groups (interaction variance).

Calculating the Fs

Degrees of freedom

The number of observations (data points) in the data that are free to vary when estimating a statistic.

  • I own 7 pairs of sock. I want to wear a different pair of socks every day.

  • How many times do I get to pick a pair of socks before I get stuck?

  • The degrees of freedom is how many times things can vary before we are stuck with what’s left over.

Degrees of freedom

  • Between Groups (dfBG ) = number of groups (k) - 1

  • Error Groups (dfError) = number of participants (n) - number of groups (k)

  • dfTotal = n - 1

*k is the number of groups in the independent variable.

Degrees of freedom

  • Between Groups 1 (dfBG1 ) = k1 - 1

  • Between Groups 2 (dfBG2 ) = k2 - 1

  • Interaction (MSInteractioni) = (k1 - 1) x (k2 - 1)

  • Error (MSError) = n - (k1 x k2)

  • Total = n - 1

  • *k is the number of groups in each independent variable.

Calculating the MS in 2-way ANOVA

  • Between Group 1 Mean Squared:

    • \(MS_{BG_1} = \frac{SS_{BG_{1}}}{k_1-1}\)

  • Between Group 2 Mean Squared:

    • \(MS_{BG_{2}} = \frac{SS_{BG_{2}}}{k_2-1}\)

  • Interaction Mean Squared:

    • \(MS_{Interaction} = \frac{SS_{Interaction}}{(k_1-1) \times (k_2-1)}\)

  • Error Mean Squared:

    • \(MS_{Error} = \frac{SS{Error}}{n-(k_1 \times k_2)}\)

Calculating 2-way ANOVA anova

  • \(F = \frac{MS_{BG_{1}}}{MS{Error}}\)

  • \(F = \frac{MS_{BG_{2}}}{MS{Error}}\)

  • \(F = \frac{MS_{Interaction}}{MS{Error}}\)

Interpreting F

Code 
set.seed(123)
library(dplyr)
library(afex)
library(gt)
library(broom)
library(forcats)

twoway <- 
  tibble(
    id = 1:100,
    gender = sample(c("Male", "Female"), 100, replace = TRUE),
    education_level = sample(c("High School", "Bachelor's", "Master's", "PhD"), 100, replace = TRUE)
    ) %>%
  mutate(
    # assign numeric values for education level
    edu_num = case_when(
      education_level == "High School" ~ 1,
      education_level == "Bachelor's"  ~ 2,
      education_level == "Master's"    ~ 3,
      education_level == "PhD"         ~ 4
    ),
    # introduce a systematic difference based on gender and education_level interaction
    political_interest = 5 + 
                         edu_num * 1.2 +                     # main effect of education level
                         if_else(gender == "Female", edu_num * 0.8, edu_num * -0.5) +  # interaction
                         rnorm(100, mean = 0, sd = 1)        # some noise
  ) 

# Run factorial ANOVA
aov_ez(
  "id",
  "political_interest",
  twoway,
  between = c("gender","education_level"),
  return = "aov"
) |> 
  tidy() |> 
  mutate(
    p.value = ifelse(p.value < .001, "<.001",p.value),
    statistic = ifelse(is.na(statistic),"--",statistic |> round(2)),
    p.value = ifelse(is.na(p.value),"--",p.value)
  ) |> 
  rename(
    Source = term,
    SS = sumsq,
    MS = meansq,
    `F` = statistic,
    p = p.value
  ) |> 
  gt() |> 
  fmt_auto() |> 
  tab_footnote(footnote = md("*Dependent Variable: Political Interest*")) |> 
  tab_style(
    style = cell_text(weight = "bold"),
    locations = cells_column_labels()
  )
Source df SS MS F p
gender  1  216.57  216.57  244.15 <.001
education_level  3  198.312  66.104 74.52 <.001
gender:education_level  3   23.648   7.883 8.89 <.001
Residuals 92   81.608   0.887 -- --
Dependent Variable: Political Interest
Code 
twoway$education_level <- factor(twoway$education_level, labels = c("High School","Bachelors","Masters","PhD"),ordered = TRUE)

twoway |> 
  ggplot(aes(education_level, political_interest,color = gender, group = gender)) + 
  stat_summary(fun = "mean", geom = "point") +
  stat_summary(fun = "mean", geom = "line") + 
  stat_summary(fun.data = "mean_se", geom = "errorbar", width = .2) + 
  theme_minimal() + 
  labs(x = "", y = "Political Interest\n")

  • Interpreting the Fs in 2-way ANOVA

    • IV 1 = Gender

    • IV 2 = Education Level

Interpreting the Fs in 2-way ANOVA

  • F1 = Gender

  • F2 = Education Level

  • F3 = Interaction of Gender x Education Level

Interpreting the Fs in 2-way ANOVA

Code 
set.seed(42)

# Create a balanced design
df <- expand.grid(
  diet_type = c("Low Carb", "Low Fat"),
  exercise_regimen = c("Cardio", "Strength"),
  id = 1:25
) %>%
  mutate(
    # simulate main effects: both diet and exercise influence weight loss
    weight_loss = case_when(
      diet_type == "Low Carb" ~ 6,
      diet_type == "Low Fat" ~ 4
    ) + case_when(
      exercise_regimen == "Cardio" ~ 5,
      exercise_regimen == "Strength" ~ 3
    ) +
    rnorm(n(), mean = 0, sd = 1.5)  # add some noise
  )

# Run the two-way ANOVA
anova_result <- aov(weight_loss ~ diet_type * exercise_regimen, data = df)
anova_result |> 
  tidy() |> 
    mutate(
    p.value = ifelse(p.value < .001, "<.001",p.value |> round(2)),
    statistic = ifelse(is.na(statistic),"--",statistic |> round(2)),
    p.value = ifelse(is.na(p.value),"--",p.value)
  ) |> 
  rename(
    Source = term,
    SS = sumsq,
    MS = meansq,
    `F` = statistic,
    p = p.value
  ) |> 
  gt() |> 
  fmt_auto() |> 
  tab_footnote(footnote = md("*Dependent Variable: Political Interest*")) |> 
  tab_style(
    style = cell_text(weight = "bold"),
    locations = cells_column_labels()
  )
Source df SS MS F p
diet_type  1  101.266 101.266 41.9 <.001
exercise_regimen  1  114.057 114.057 47.19 <.001
diet_type:exercise_regimen  1    9.066   9.066 3.75 0.06
Residuals 96  232.023   2.417 -- --
Dependent Variable: Political Interest
Code 
df |> 
  ggplot(aes(diet_type, weight_loss,color = exercise_regimen, group = exercise_regimen)) + 
  stat_summary(fun = "mean", geom = "point") +
  stat_summary(fun = "mean", geom = "line") + 
  stat_summary(fun.data = "mean_se", geom = "errorbar", width = .2) + 
  theme_minimal() + 
  labs(x = "", y = "Weight Loss\n")

Interpreting the Fs in 2-way ANOVA

  • F1 =

  • F2 =

  • F3 =

Interpreting the Fs in 2-way ANOVA

  • F1 = Diet Type

  • F2 = Exercise Regimen

  • F3 = Diet Type x Exercise Regimen

2-way AnOVA

Prof Brocker believes the key to happiness involves 2 things: Coffee and Dark. He wants to see if drinking coffee makes participants significantly happier than not drinking coffee. He also wants to see if watching Dark show makes them significantly happier than not watching it. This is a 2 x 2 design.

He randomly assigns 100 participants to Coffee and Dark or No Coffee x Dark or No Coffee and No Dark.

Code 
# Make the Dataset
cd <- 
  tibble(
  id = 1:400,
  coffee = rep(c("No Coffee","Yes Coffee"), each = 200),
  dark = rep(c("No Dark","Yes Dark"), 200),
  happiness = c(rnorm(100,4,2) |> round(0),
                rnorm(100,7,1) |> round(0),
                rnorm(100,6,2) |> round(0),
                rnorm(100,8.5,1) |> round(0)
                )
  ) 

# Calculate ANOVA
aov(happiness ~ coffee * dark, data = cd) |> 
  tidy() |> 
  mutate(
    Source = term,
    SS = sumsq, 
    MS = meansq,
    `F` = ifelse(is.na(statistic),"--",statistic |> round(3)),
    p = ifelse(p.value < .05, "<.05",p.value |> round(3)),
    p = ifelse(is.na(p),"--",p)
  ) |> 
  select(Source,df,SS,MS,`F`,p) |> 
  gt() |> 
  fmt_auto() |> 
  cols_align(align = "center",
             columns = !Source) |> 
  tab_style(
    style = cell_text(weight = "bold"),
    locations = cells_column_labels()
  ) |> 
  tab_footnote(footnote = "Dependent Variable: Happiness")
Source df SS MS F p
coffee   1    322.202 322.202 74.543 <.05
dark   1      0.722   0.722 0.167 0.683
coffee:dark   1     11.903  11.903 2.754 0.098
Residuals 396  1,711.67    4.322 -- --
Dependent Variable: Happiness
Code 
cd |> 
  ggplot(aes(coffee, happiness,color = dark)) + 
  geom_jitter(alpha = .2) + 
  stat_summary(
    fun.data = "mean_se",
    geom = "errorbar",
    width = .2
  ) + 
  stat_summary(
    fun.data = "mean_se",
    geom = "line",
    group = 1
  ) + 
  theme_minimal() + 
  labs(
    x = "Coffee Drank\n",
    y = "Level of Happiness\n",
    color = "black"
  ) + 
  theme(
    axis.text = element_text(color = "black")
  )

Calculating the Fs in 2-way ANOVA

Code 
tribble(
~Source, ~SS, ~df, ~MS, ~F,
"Coffee",432, "","","",
"Dark",486,"","","",
"Coffee x Dark",540,"","","",
"Error",5184,"","",""
) |> 
  gt() |> 
    cols_align(align = "center",
             columns = SS:F)
Source SS df MS F
Coffee 432
Dark 486
Coffee x Dark 540
Error 5184

Calculating the Fs in 2-way ANOVA

Code 
tribble(
~Source, ~SS, ~df, ~MS, ~F,
"Coffee",432, 1,432,"8",
"Dark",486,1,486,"9",
"Coffee x Dark",540,1,540,"10",
"Error",5184,96,54,""
) |> 
  gt() |> 
    cols_align(align = "center",
             columns = SS:F)
Source SS df MS F
Coffee 432 1 432 8
Dark 486 1 486 9
Coffee x Dark 540 1 540 10
Error 5184 96 54

2-way AnOVA

Prof. Brocker recruit 500 children on Halloween. He assigns half to wear a costume with a mask, and the other half to wear a costume without a mask. He assigns participants to trick-or-treat alone or in groups. Then he measures how much candy each child collects while trick-or-treating.

Code 
set.seed(1031)  # Halloween-ish number for reproducibility

# Create a balanced factorial design
new_df <- expand.grid(
  mask_condition = c("Mask", "No Mask"),
  group_condition = c("Alone", "Group"),
  n = 1:125  # 125 per cell → 500 total
) %>%
  mutate(
    # Simulate DV with higher values for Mask + Alone
    candy_collected = case_when(
      mask_condition == "Mask" & group_condition == "Alone" ~ rnorm(n(), mean = 50, sd = 5),
      mask_condition == "Mask" & group_condition == "Group" ~ rnorm(n(), mean = 44, sd = 5),
      mask_condition == "No Mask" & group_condition == "Alone" ~ rnorm(n(), mean = 42, sd = 5),
      mask_condition == "No Mask" & group_condition == "Group" ~ rnorm(n(), mean = 40, sd = 5)
    )
  )

# Run a two-way ANOVA
model <- aov(candy_collected ~ mask_condition * group_condition, data = new_df)

model |> 
  tidy() |> 
    mutate(
    p.value = ifelse(p.value < .001, "<.001",p.value),
    statistic = ifelse(is.na(statistic),"--",statistic |> round(2)),
    p.value = ifelse(is.na(p.value),"--",p.value)
  ) |> 
  rename(
    Source = term,
    SS = sumsq,
    MS = meansq,
    `F` = statistic,
    p = p.value
  ) |> 
  gt() |> 
  fmt_auto() |> 
  tab_footnote(footnote = md("*Dependent Variable: Candy Collected*")) |> 
  tab_style(
    style = cell_text(weight = "bold"),
    locations = cells_column_labels()
  )
Source df SS MS F p
mask_condition   1   4,232.488 4,232.488 174.05 <.001
group_condition   1   1,156.661 1,156.661 47.56 <.001
mask_condition:group_condition   1     897.593   897.593 36.91 <.001
Residuals 496  12,061.683    24.318 -- --
Dependent Variable: Candy Collected
Code 
new_df |> 
  ggplot(aes(mask_condition, candy_collected,color = group_condition, group = group_condition)) + 
  geom_jitter(alpha = .2) + 
  stat_summary(fun = "mean", geom = "point") +
  stat_summary(fun = "mean", geom = "line") + 
  stat_summary(fun.data = "mean_se", geom = "errorbar", width = .2) + 
  theme_minimal() + 
  labs(x = "", y = "Candy Collected\n")

Calculating the Fs in 2-way ANOVA

Code 
tribble(
~Source, ~SS, ~df, ~MS, ~F,
"IV1",252, "","","",
"IV2",441,"","","",
"Interaction",882,"","","",
"Error",10416,"","",""
) |> 
  gt() |> 
  cols_align(align = "center",
             columns = SS:F)
Source SS df MS F
IV1 252
IV2 441
Interaction 882
Error 10416

Calculating the Fs in 2-way ANOVA

Code 
tribble(
~Source, ~SS, ~df, ~MS, ~F,
"IV1",252, "1","252","12",
"IV2",441,"1","441","21",
"Interaction",882,"1","882","42",
"Error",10416,"496","21",""
) |> 
  gt() |> 
  cols_align(align = "center",
             columns = SS:F)
Source SS df MS F
IV1 252 1 252 12
IV2 441 1 441 21
Interaction 882 1 882 42
Error 10416 496 21

Calculating 2-way ANOVA anova

  • \(F = \frac{MS_{BG_{1}}}{MS{Error}}\)

  • \(F = \frac{MS_{BG_{1}}}{MS{Error}}\)

  • \(F = \frac{MS_{Interaction}}{MS{Error}}\)

  • *All MS are divided by \(MS_{Error}\)

Reporting F

Reporting F

If asked to report findings in terms of the Null Hypothesis (H0), you should report findings as:

  • Reject H0, or

  • Fail to Reject H0

Reporting F

If asked to report findings in general or for publication, you need EACH F-value:

  • F(dfBG, dfW )

    • = F-value

      • Corresponding p-value

        • For significant results: Means and standard deviations of each group

Reporting F

The group that watched Dark (M=8.43, s=1.02) reported significantly more happiness compared to their peers in the control group who watched Jeopardy (M=6.12, s=0.98), F(1,496) = 7.12, p < 0.05.

Reporting F

Results IV1:

If significant: The group that watched Dark (M=8.43, s=1.02) reported significantly more happiness compared to their peers in the control group who watched Jeopardy (M=6.12, s=0.98), F (1, 496) = 7.12, p < 0.05.

If NOT significant: There was not a significant difference in happiness between the groups, F (1, 498) = 1.02, p = 0.07.

Reporting F

Results IV2:

If significant: The group that drank coffee (M=7.35, s=1.01) reported significantly more happiness compared to their peers in the control group who drank decaf (M=4.21, s=0.99), F (1, 496) = 9.12, p < 0.05.

If NOT significant: There was not a significant difference in happiness between the groups, F (1, 498) = 1.02, p = 0.07.

Reporting F

Results Interaction:

If significant: The group that drank coffee AND watched Dark (M=7.35, s=1.01) reported significantly more happiness compared to the other groups (M=4.21, s=0.99), F (1, 496) = 11.14, p < 0.05.

If NOT significant: There was not a significant difference in happiness between the groups, F (1, 498) = 1.02, p = 0.07.

Code 
library(dplyr)
library(ggplot2)

# Set seed for reproducibility
set.seed(123)

# Create levels for the independent variables
coffee <- c("Yes", "No")
tv_show <- c("Yes", "No")

# Create a data frame with all combinations of coffee and tv_show
factorial_design <- expand.grid(Coffee = coffee, TV_Show = tv_show)

# Simulate happiness scores for each group
# Adjust means to reflect interaction effects
n_per_group <- 30 # Number of participants per condition
data <- factorial_design[rep(1:nrow(factorial_design), each = n_per_group), ]
data$Happiness <- NA

# Define mean happiness scores for each condition
means <- c(
  "Yes_Yes" = 8,  # Coffee + TV Show
  "Yes_No" = 6,   # Coffee + No TV Show
  "No_Yes" = 5,   # No Coffee + TV Show
  "No_No" = 3     # No Coffee + No TV Show
)

# Assign happiness scores with some random noise
data$Happiness <- rnorm(
  n = nrow(data),
  mean = means[paste(data$Coffee, data$TV_Show, sep = "_")],
  sd = 1.5 # Standard deviation for noise
)

# Model
fac_mod <- 
  aov(Happiness ~ Coffee * TV_Show, data  = data)

data |> 
  ggplot(aes(Coffee, Happiness, color = TV_Show)) + 
  geom_jitter(alpha = .4) + 
   stat_summary(
    fun.data = "mean_se",
    geom = "line",
    color = "black",
    aes(group = TV_Show,
        lty = TV_Show)
  ) + 
    stat_summary(
    fun.data = "mean_se",
    geom = "pointrange",
    aes(color = TV_Show)
  ) + 
  theme_minimal() + 
  theme(
    panel.grid = element_blank()
  )

Code 
plot(TukeyHSD(fac_mod))