Mini Case Study Setup: A Landing Page Conversion Rate
Imagine you run a paid campaign that sends traffic to a new landing page. Your decision is operational: should you keep buying traffic, pause the campaign, or iterate on the page? The key unknown is the conversion rate p: the probability that a visitor converts (e.g., signs up). You have observed data from a short run: n visitors and x conversions. In this chapter, you will compute two decision-friendly summaries from the posterior distribution of p: the posterior mean (a single best estimate under squared-error loss) and a credible interval (a range of plausible values). The focus here is not on re-deriving Bayesian fundamentals, but on executing the workflow cleanly and interpreting the results for real decisions.
What You Will Produce: Two Numbers That Drive Decisions
For a conversion rate, stakeholders typically ask two questions: “What’s the best estimate?” and “How uncertain is it?” The posterior mean answers the first: it is the expected value of p under the posterior distribution. The credible interval answers the second: it gives a range [L, U] such that the posterior probability that p lies in that range is, say, 95%. In practice, you will report something like: “Estimated conversion rate is 4.8% with a 95% credible interval of [3.6%, 6.2%].” These summaries are easy to communicate, but they are also actionable: you can compare the interval to a business threshold (e.g., profitability break-even) or use the mean as an input to forecasting.
Case Data: A Short Campaign Run
Suppose you ran the campaign for one day and observed n = 500 visitors and x = 24 conversions. The naive estimate is 24/500 = 4.8%. But with only 24 conversions, uncertainty is non-trivial, and you want a principled way to quantify it and avoid overreacting to noise. You will compute the posterior mean and a 95% credible interval for p, then interpret them in business terms.
Choose a Prior That Matches Your Operational Context (Without Rehashing Theory)
In many product analytics settings, a simple, weakly informative prior is used to stabilize estimates when data are limited. A common choice is Beta(1, 1), which is uniform over [0, 1]. Another practical choice is Beta(0.5, 0.5), which is slightly more weight near 0 and 1 but still weak. If you have historical knowledge that typical conversion rates for similar pages are around 3% to 6%, you might encode that with a prior centered in that region. For this mini case study, use Beta(1, 1) to keep the arithmetic straightforward and to emphasize the mechanics of posterior mean and credible interval reporting.
Step-by-Step: Compute the Posterior Parameters
With a Beta prior and binomial data, the posterior is also Beta. Using Beta(α, β) as the prior and observing x conversions out of n visitors, the posterior parameters are: α_post = α + x and β_post = β + n − x. Here, α = 1, β = 1, x = 24, n = 500. So α_post = 1 + 24 = 25 and β_post = 1 + (500 − 24) = 477. Your posterior distribution is Beta(25, 477). This distribution describes your updated uncertainty about the conversion rate p after seeing the data.
Continue in our app.
You can listen to the audiobook with the screen off, receive a free certificate for this course, and also have access to 5,000 other free online courses.
Or continue reading below...
Download the app
Step-by-Step: Compute the Posterior Mean (Point Estimate)
The posterior mean of a Beta(α, β) distribution is α / (α + β). For Beta(25, 477), the posterior mean is 25 / (25 + 477) = 25 / 502 ≈ 0.0498, or 4.98%. Notice this is slightly higher than the naive 4.8% because the Beta(1,1) prior adds one pseudo-conversion and one pseudo-non-conversion, nudging the estimate toward 50% very slightly; with n = 500, that influence is tiny. In reporting, you might round to 5.0% depending on your audience.
Why the Posterior Mean Is Often the Default
In many operational settings, you need a single number to plug into downstream calculations: expected signups next week, expected revenue per click, or expected cost per acquisition. The posterior mean is the expected value of p under your uncertainty, which makes it a natural input for expected-value calculations. It also behaves sensibly when data are sparse: it avoids extreme values like 0% or 100% when x is 0 or x = n, which can otherwise break forecasting or ROI computations.
Step-by-Step: Compute a 95% Credible Interval
A 95% credible interval for p is typically obtained from posterior quantiles: the 2.5th percentile and the 97.5th percentile of the Beta(25, 477) distribution. In practice, you compute these using software, because Beta quantiles do not have a simple closed-form expression. The interval is: L = qbeta(0.025, 25, 477) and U = qbeta(0.975, 25, 477). If you compute this numerically, you will get an interval roughly around the posterior mean, with width determined by the amount of data and the observed conversion count. For this case, the 95% credible interval is approximately [0.032, 0.071] (3.2% to 7.1%). The exact numbers may differ slightly depending on rounding and the numerical routine, but the workflow is the same.
Practical Interpretation: What the Interval Means for the Campaign
The credible interval tells you which conversion rates are still plausible given your data and prior. Here, even though the posterior mean is about 5.0%, the interval suggests the true conversion rate could reasonably be as low as about 3.2% or as high as about 7.1%. If your break-even conversion rate is, say, 4.0%, then the interval straddles that threshold: some plausible values are below break-even and some are above. That is a signal that you may want more data before making an irreversible decision, or you may want to compute a decision-relevant probability such as P(p > 0.04). Even if you do not compute that probability explicitly in this chapter, the interval already provides a quick visual check against thresholds.
Step-by-Step in Code: Python (SciPy)
Below is a minimal Python snippet to compute the posterior mean and a 95% credible interval using SciPy. The key function is scipy.stats.beta for the Beta distribution. You will compute the posterior parameters, then use .mean() and .ppf() (percent point function, i.e., inverse CDF) for quantiles.
import numpy as np
from scipy.stats import beta
# Data
n = 500
x = 24
# Prior Beta(alpha, beta)
alpha = 1
beta_prior = 1
# Posterior parameters
alpha_post = alpha + x
beta_post = beta_prior + (n - x)
# Posterior distribution
post = beta(alpha_post, beta_post)
# Posterior mean
post_mean = post.mean()
# 95% credible interval (equal-tailed)
ci_lower, ci_upper = post.ppf([0.025, 0.975])
print(alpha_post, beta_post)
print(post_mean)
print(ci_lower, ci_upper)Step-by-Step in Code: R (Base qbeta)
In R, the Beta quantile function is qbeta. You can compute the posterior mean directly from α_post and β_post, and the credible interval via qbeta. This is often the fastest way to produce a report-ready summary.
# Data
n <- 500
x <- 24
# Prior
alpha <- 1
beta <- 1
# Posterior
alpha_post <- alpha + x
beta_post <- beta + (n - x)
# Posterior mean
post_mean <- alpha_post / (alpha_post + beta_post)
# 95% credible interval
ci <- qbeta(c(0.025, 0.975), alpha_post, beta_post)
alpha_post
beta_post
post_mean
ciEqual-Tailed vs Highest Density: Which Interval Should You Report?
The interval computed via posterior quantiles is an equal-tailed credible interval: 2.5% of posterior mass lies below L and 2.5% lies above U. Another option is a highest density interval (HDI), which is the narrowest interval containing 95% of the posterior mass. For a Beta distribution that is not too skewed, these are often similar; with small x or x near n, they can differ. In many business dashboards, equal-tailed intervals are standard because they are easy to compute and explain. If you are communicating to a technical audience or you expect skewed posteriors (e.g., very low conversion rates), an HDI can be a better summary of “most plausible values,” but it requires a bit more computation.
Reporting Template: Make the Output Decision-Ready
A practical report should include: the data window, the sample size, the posterior mean, and the credible interval. For example: “From 500 visitors and 24 conversions, estimated conversion rate is 5.0% (posterior mean) with a 95% credible interval of [3.2%, 7.1%], using a Beta(1,1) prior.” This format makes it easy for stakeholders to understand what was measured, how much data supports the estimate, and how uncertain the estimate remains.
Sensitivity Check: How Much Does the Prior Matter Here?
With n = 500, a weak prior like Beta(1,1) has minimal influence. But it is still good practice to sanity-check sensitivity by trying a couple of reasonable priors and seeing if the posterior mean and interval change meaningfully. For instance, compare Beta(1,1) to Beta(2,2) (slightly more concentrated around 50%) or to a prior centered near typical historical performance. If the posterior summaries barely move, you can be confident the data dominate. If they move a lot, that is a sign your data are too limited to support a stable estimate, and you should either collect more data or justify a stronger prior based on real historical evidence.
Operational Use: Forecasting Expected Conversions Next Week
Suppose you plan to buy enough traffic for m = 2,000 visitors next week. A simple forecast uses the posterior mean: expected conversions ≈ m × E[p | data] = 2000 × 0.0498 ≈ 100 conversions. The credible interval can be translated into a plausible range for the conversion rate, which then maps to a plausible range for conversions: 2000 × 0.032 ≈ 64 up to 2000 × 0.071 ≈ 142. This is not yet a full predictive interval for conversions (which would also include binomial sampling variability next week), but it is a quick, decision-friendly way to communicate uncertainty in the underlying rate and its impact on planning.
Operational Use: Comparing to a Profitability Threshold
Assume your economics imply a break-even conversion rate of 4%. With a posterior mean near 5%, the campaign looks promising, but the credible interval includes values below 4%. A practical next step is to decide whether the risk of being below break-even is acceptable. Even without computing an explicit probability, the interval tells you the decision is not “obvious” yet. If you can afford another day of data collection, you might continue running to narrow the interval. If the cost of running is high, you might pause and iterate on the landing page to improve the rate before spending more.
Common Pitfalls in This Mini Case Study
Reporting only the point estimate: A single number hides uncertainty and encourages overconfident decisions, especially when x is small.
Ignoring sample size: “5% conversion” means something very different with 24 conversions than with 2,400 conversions. Always report n and x.
Overinterpreting small differences: If two variants have posterior means of 5.0% and 5.4% but wide intervals, the difference may not be decision-relevant yet.
Using an interval without a decision threshold: An interval becomes actionable when compared to a business target (break-even, SLA, or KPI threshold).
Forgetting that the interval is about p: Mapping it to outcomes (conversions, revenue) requires multiplying by planned traffic and, for full prediction, accounting for future randomness.
Checklist: Your Repeatable Workflow for Conversion Rate Estimation
Collect data: x conversions out of n visitors for a clearly defined time window and funnel definition.
Select a reasonable prior for p that matches your context and document it.
Compute posterior parameters α_post and β_post.
Compute posterior mean α_post/(α_post+β_post) for a point estimate.
Compute a 95% credible interval using Beta quantiles (equal-tailed) or an HDI if appropriate.
Report: n, x, posterior mean, and interval; compare the interval to a decision threshold or planning target.
