A/B Test Design & Analysis

Design an A/B test for [WHAT YOU'RE TESTING] on [PLATFORM/PAGE].

Goal: [INCREASE CONVERSION / REDUCE CHURN / IMPROVE ENGAGEMENT]
Current baseline: [CURRENT METRIC VALUE]

Minimum detectable effect: [HOW MUCH IMPROVEMENT MATTERS]

Provide:
1. **Hypothesis**: "If we [change], then [metric] will [improve/decrease] because [reason]"
2. **Test Design**: Control vs variant(s) — exactly what changes
3. **Sample Size Calculation**: How many users needed per variant (with statistical power)
4. **Duration Estimate**: How long the test needs to run
5. **Segmentation**: Any user segments to analyze separately
6. **Success Metrics**: Primary and guardrail metrics
7. **Statistical Method**: Frequentist vs Bayesian, significance threshold
8. **Common Pitfalls**: Peeking, novelty effects, selection bias
9. **Analysis Template**: How to read and report the results
10. **Decision Framework**: Ship, iterate, or kill based on results

Design an A/B test for [WHAT YOU'RE TESTING] on [PLATFORM/PAGE].

Goal: [INCREASE CONVERSION / REDUCE CHURN / IMPROVE ENGAGEMENT]
Current baseline: [CURRENT METRIC VALUE]
Minimum detectable effect: [HOW MUCH IMPROVEMENT MATTERS]

Provide:
1. **Hypothesis**: "If we [change], then [metric] will [improve/decrease] because [reason]"
2. **Test Design**: Control vs variant(s) — exactly what changes
3. **Sample Size Calculation**: How many users needed per variant (with statistical power)
4. **Duration Estimate**: How long the test needs to run
5. **Segmentation**: Any user segments to analyze separately
6. **Success Metrics**: Primary and guardrail metrics
7. **Statistical Method**: Frequentist vs Bayesian, significance threshold
8. **Common Pitfalls**: Peeking, novelty effects, selection bias
9. **Analysis Template**: How to read and report the results
10. **Decision Framework**: Ship, iterate, or kill based on results