MAHMOUD ABDELNABY
Media Buying · CLI SCRIPT · 2

Ad Copy Split-Test Bayesian Significance Engine

Stop guessing on ad creative A/B tests. Compute exact Bayesian winning probabilities and expected loss without complex statistics.

Problem it addresses

Develop a zero-dependency Python script (bayesian_ad_tester.py) that computes Bayesian posterior probabilities for multi-variant creative tests (Beta-Binomial conjugate model for CTR and conversion rates). Features: 1) Calculates probability of variant A outperforming B, 2) Expected loss calculation (risk of being wrong), 3) Minimum sample size recommendation, 4) Generates a standalone visual HTML report.

Who it is for

PRO operating in high-growth e-commerce, digital publishing, and direct-response marketing across Saudi Arabia, UAE, and Egypt.

What is inside

Key files listed in the README repository architecture:

  • examples/ab_test_sample.csv
  • LICENSE
  • protocols/creative-testing-standard.md
  • README.md
  • reports/sample_decision_report.html
  • reports/significance_report.html
  • src/bayesian_ad_tester.py
  • src/bayesian_calculator.py
  • tests/test_bayesian.py

How to evaluate and use it

  1. Read the README first and confirm that the documented problem matches your use case.
  2. Inspect the listed files to find the data, tools, templates, and tests relevant to your task.
  3. Follow the repository’s Quick Start and check prerequisites before running commands.
  4. Compare outputs with documented examples and review limitations before using them in a decision or client workflow.

Quick Start from the README: python -m unittest discover -s tests · python src/bayesian_ad_tester.py --help

Scope and limitations

  • Bayesian vs Frequentist Framing: Reports posterior probability of winning rather than confusing p-values, preventing premature test stopping.
  • Informative Prior Calibration: Uses user-supplied priors; choose them from relevant account history rather than a claimed regional average.
  • Multi-Armed Bandit Integration: Calculates expected loss in conversions if the currently leading ad variant is chosen immediately.

This page summarizes the available documentation; it does not establish performance or business results. Verify data and prerequisites before use.

Related repositories