Winner Criteria: Clicks vs Opens in Subject Line Tests
When you run a subject line A/B test, the variant that wins on opens isn't always the one that drives clicks. Choosing the right winner criteria — clicks vs opens — shapes what your AI agent optimizes for. Here's how to decide, and how marginal exposes both signals.
Why winner criteria clicks vs opens actually matters
Opens measure curiosity — they tell you whether your subject line earned attention in a crowded inbox. Clicks measure intent — whether the message and its call to action moved someone toward your goal. A subject line that over-promises can win on opens while losing on clicks, because the body fails to deliver on the hook.
Picking winner criteria up front prevents you from optimizing for the wrong number. If your campaign's job is to drive sign-ups or purchases, clicks are the closer proxy. If you're warming a list or building reach, opens may be the metric that matters.
- Opens: best for awareness, re-engagement, and top-of-funnel reach.
- Clicks: best for conversion-oriented sends with a clear CTA.
- Mismatch warning: a high-open, low-click winner often signals a subject/body gap.
- Open tracking is approximate; clicks are a harder, intent-based signal.
How marginal tracks both signals
marginal is a hosted email MCP server that runs subject line A/B tests and reports open and click tracking with within-test lift metrics. Because both numbers are available per variant, you don't have to commit to a single metric blindly — you can read clicks and opens side by side before declaring a winner.
The get_results tool returns per-variant opens, clicks, and lift, so your agent can apply whatever winner criteria you choose. The recommend_next tool then suggests the follow-up variant based on what actually performed.
- generate_variants — draft subject line options to test.
- launch_test — start a managed send across variants.
- get_results — pull open and click rates with within-test lift.
- recommend_next — propose the next variant from observed performance.
Choosing a winner with an AI agent
In AI agent email marketing workflows, the agent needs an explicit rule for what 'winning' means. Tell it to rank variants by click-through rate when the goal is conversion, and by open rate when the goal is reach. With marginal connected over MCP, the agent reads results directly and applies that rule without you exporting CSVs.
A common pattern: require a minimum open lift to rule out subject lines that aren't getting seen at all, then break ties on clicks. This keeps you from crowning a winner that nobody opened while still rewarding the variant that converts.
- Primary metric clicks, guardrail metric opens for conversion sends.
- Primary metric opens for reach or list-warming sends.
- Wait for enough sample before locking a winner — small tests mislead.
- Let recommend_next iterate from the metric you optimized.
Get started with marginal
Connect marginal to Cursor, Claude Desktop, Claude Code, Windsurf, Cline, Continue, Zed, or OpenAI Codex and run your first subject line A/B test today. Authenticate with a Bearer API key and point your client at the endpoint at https://marginal.sh/mcp.
The free tier includes 100 experiments per month — enough to test clicks-vs-opens winner criteria across several campaigns. Full tool reference lives in the docs at https://marginal.sh/docs/.
- Endpoint: https://marginal.sh/mcp
- Auth: Bearer API key (marg_live_...)
- Free tier: 100 experiments/month
- Docs: https://marginal.sh/docs/
Get started with marginal
marginal is a hosted email marketing MCP server at marginal.sh. Sign up free, create an API key, and connect https://marginal.sh/mcp from Cursor, Codex, or Claude Desktop.
- 100 experiments/month on the free tier
- Four MCP tools: generate_variants, launch_test, get_results, recommend_next
- Listed in the MCP Registry as sh.marginal/mcp