AnalyticsAttribution, incrementality and testsIntermediateAnalyses your data

A/B test result evaluation with statistical guardrails

Interprets an A/B test from raw counts with checks for sample-ratio mismatch, peeking and practical significance, without inventing a significance result.

For: Performance marketers, Marketing analysts, Growth marketers · Works with: Any capable chat model (ChatGPT, Claude, Gemini, others), Google Sheets / Excel

Public beta. This prompt was drafted with AI assistance and checked by automated rules, but it has not been reviewed or tested by a person yet. Treat it as a starting point and check the output. It is hidden from search engines while in beta. Use the “Was this prompt useful?” box to tell us what works.

Your prompt

Act as an experimentation analyst. Evaluate this A/B test using ONLY the data below. Test: new shorter checkout form (B) vs current form (A). Primary metric: purchase conversion rate per visitor. Planned split: 50/50. Planned duration and stopping rule: run for 2 full weeks, evaluate once at the end. Dates run: 15-28 Sep 2026. Data (variant | visitors/users | conversions): Variant | Visitors | Conversions A (control) | 10400 | 520 B (short form) | 9800 | 548 Tasks: 1. State assumptions (independence, one conversion per user, same time window for both variants). 2. Compute conversion rates and the absolute and relative difference, showing formulas, and verify. 3. Sample-ratio check: compare the observed split to the planned split and say whether a mismatch could indicate a problem; give the method (for example a chi-square test) and tell me to run it in a stats tool or calculator rather than asserting a p-value yourself. 4. Describe how to test significance (for example a two-proportion z-test) and which inputs to enter, and ask me to compute the p-value and confidence interval using a proper tool or the incrementality calculator. If I paste the tool output, interpret it; do not make up a p-value. 5. Discuss practical significance, peeking and early stopping, novelty effects, seasonality and multiple comparisons relevant to my setup. 6. Give a decision framework (ship, keep testing, stop) conditional on the tool results, and what you cannot conclude. 7. List missing data. Rules: do not invent statistics, p-values, confidence intervals or "statistical significance" without computed evidence. If a number is not provided, say so. Verify arithmetic. Correlation within a randomised test supports causal claims only if the design held.

Customize

The fields start with example values so you can see how the prompt works. Everything stays in your browser.

What was tested.

Decision metric.

Intended allocation.

Pre-planned duration and rule.

Dates.

Paste your counts. The default is small fictional sample data so the preview works; replace it with your own.

Example of what to expect

Illustrative only. It describes the kind of result this prompt aims for; real output varies by tool, model and run.

Conversion rates and differences with formulas, a sample-ratio check explained with instructions to run it, guidance on running the significance test in a proper tool, caveats (peeking, novelty), and a conditional decision framework. (Illustrative.)

Expected format: Assumptions, rates, SRM check, testing instructions, decision framework.

How to use it

  1. Run the significance test in a trusted tool and paste the result back.
  2. Predefine decision rules.
  3. Record the test in a log.

Limitations

  • Models can miscompute statistics; never rely on a model-produced p-value.
  • Tests stopped early or with mismatched splits can mislead.

Platform notes

Any capable chat model (ChatGPT, Claude, Gemini, others)

Official docs read · checked 2026-10-11

Works in any capable chat model. Paste a small anonymised sample and recompute the key numbers yourself: chat models can misread columns or miscalculate, and they cannot run your statistical tests reliably.

General notes on Any capable chat model (ChatGPT, Claude, Gemini, others)
  • These prompts are plain text and work in any chat assistant. For analytics prompts, paste a small, anonymised export; a chat model can misread columns or miscalculate, so recompute key numbers yourself.

Google Sheets / Excel

Not verified: third-party guidance only · checked 2026-10-11

Test any formulas on a small range first; syntax depends on locale and version.

General notes on Google Sheets / Excel
  • Formulas are written for common functions in both products. Locale settings (decimal and argument separators) can change what you must type.

Source, license and attribution

Origin
Original by MarketerTools
Publisher
MarketerTools
License
Original work by MarketerTools, free to copy and use. Informed by the linked documentation; no third-party text is reproduced.
Platform assumptions checked
2026-10-11

Documentation and references behind this prompt

These informed the structure and the platform notes. A reference is not a license, and no third-party prompt text is copied here.

Spotted an attribution error, or are you a source owner with a request? Use the chat button at the bottom right and quote prompt ID ana-035. We will correct or remove it.

Related prompts

  • AnalyticsAdvancedDesigns a test

    Incrementality test design: holdout or geo experiment

    Designs an incrementality test plan with hypotheses, units, duration decisions and pre-registered success criteria, without inventing sample sizes.

    Any capable chat model (ChatGPT, Claude, Gemini, others)

  • AnalyticsIntermediateAnalyses your data

    Conversion funnel drop-off analysis with hypotheses and tests

    Finds the biggest relative and absolute drop-offs in a funnel table and turns them into prioritised, testable hypotheses.

    Any capable chat model (ChatGPT, Claude, Gemini, others) · Google Analytics 4 · Google Sheets / Excel

  • VideoIntermediateDesigns a test

    Video A/B test plan: variants that change one variable

    Designs a creative test: one base video, single-variable variants, the constants to hold fixed, and an honest note on what the test can and cannot prove.

    Any capable chat model (ChatGPT, Claude, Gemini, others)