• random assignment to treatments
• poorer alternative: haphazard assignment
• stratification
  (i.e., randomize within groups)
• related: experimental blinding

• maximize desired variation
  • e.g. large doses (tradeoff with biological realism)
• minimize undesired variation
  • within-subjects designs
    • e.g. paired, randomized-block, crossover
  • reduce environmental variation
    • tradeoff with generality
    • e.g. environmental chambers, inbred/clones
• isolate desired effects: positive/negative controls
  (vehicle-only, cage treatments, etc.)
• measure/control for covariates

• power is the probability of successfully rejecting the null hypothesis
  • i.e., being able to see something clearly
• depends on
  • biological effect size
  • noise level
  • experimental design (sample size + …)
• uses null-effect counterfactual

an introductory video

• pilot studies? (danger)
• previous literature
• minimal interesting sample size (“SESOI”, “MCID”)
• estimates of uncertainty
  • particularly hard to think about
  • think in terms of coefficient of variation and translate
  • binomial tests easier, but watch out:
    assume no overdispersion

• seat-of-the-pants (“ask your supervisor”)
• web calculators (Lenth 2006)
• G*Power (Faul et al. 2009)

apropos("^power\\.")  ## base-R functions

## [1] "power.anova.test" "power.prop.test"  "power.t.test"

a1 <- available.packages(repos="https://cran.rstudio.com")
pow_pkgs <- grepv("power", rownames(a1), ignore.case=TRUE)
length(pow_pkgs)

## [1] 60

head(pow_pkgs, 10)

##  [1] "agpower"          "BayesianPower"    "BayesPower"       "CoRpower"        
##  [5] "crt2power"        "depower"          "easypower"        "ecopower"        
##  [9] "extraSuperpower"  "InteractionPoweR"

library("pwr")
apropos("^pwr")

##  [1] "pwr.2p.test"    "pwr.2p2n.test"  "pwr.anova.test" "pwr.chisq.test"
##  [5] "pwr.f2.test"    "pwr.norm.test"  "pwr.p.test"     "pwr.r.test"    
##  [9] "pwr.t.test"     "pwr.t2n.test"

also: library("sos"); findFn("{power analysis}")

dd <- read.csv("../data/ants.csv")
power.t.test(n=c(10,10),delta=2,sd=1)
power.t.test(power=0.8,delta=2,sd=1)

nvec <- 2:15
powfun <- function(n) {
   power.t.test(n, delta=2, sd=1)$power
}
powvec <- sapply(nvec,powfun)
plot(nvec, powvec, type="b",
     xlab="sample size (each group)",
     ylab="power",
     main = "delta = 2, sd = 1",
     ylim = c(0,1))

• e.g. Cohen’s \(d\)/\(g\)
• scaling not by standard deviation of a predictor, but by the noise
• removes units (and biological meaning)
• refers only to clarity, not biological impact
• if you need these for input to a program, start with biological effects and translate

• Cohen proposed standardized “small”, “medium”, and “large” scaled effect sizes
• convenient: but scaled/unitless/non-biological!

From Russ Lenth FAQs:

for a “medium” effect size, you’ll choose the same \(n\) regardless of the accuracy or reliability of your instrument, or the narrowness or diversity of your subjects

• running
• tautological: high \(p\)-value \(\leftrightarrow\) low power
• essentially useless
• instead:
  • show confidence intervals
  • (if necessary) pretend you’re doing prospective analysis
• push back: Thomas (1997), Gerard, Smith, and Weerakkody (1998), Hoenig and Heisey (2001)

• simplify the question
• use simpler designs (e.g. low/high vs continuous range)
• push treatments harder
• ask a different question

• simplify
• simulate
  • see chap 5 (Bolker 2008)
  • much more flexible - e.g. simulate effects of lack of balance - endpoints other than power (e.g. CV)

## experimental design
N <- 20; x_min <- 0; x_max <- 2
x <- runif(N, min=x_min, max=x_max)
## model world
a <- 2; b <- 1; sd_y <- 1
## setup
nsim <- 1000; pval <- numeric(N); set.seed(101)
for (i in 1:nsim) {
  y_det <- a + b * x  ## deterministic y
  y <- rnorm(N, mean = y_det, sd = sd_y)
  m <- lm(y ~ x)
  pval[i] <- coef(summary(m))["x", "Pr(>|t|)"] ## extract p-value
}
mean(pval<0.05)

## [1] 0.609

https://bbolker.github.io/powertargets/powertargets.html