Seed test

Seed test, defined and delimited

A seed test is a controlled delivery check. You add a set of seed addresses, owned by a panel provider or by your own team, to a campaign or send the same message to them separately. Each seed mailbox is monitored, and the panel records whether the message reached the inbox, the spam folder, a tab such as Promotions, or did not arrive at all. Good panels also capture authentication results, link behaviour, image rendering and sometimes a screenshot of the message as it appeared.

What a seed test is not: it is not a spam score, it is not a reputation score, it is not a measurement of your whole list, and it is not a substitute for real engagement metrics. A spam score evaluates content rules. A reputation score summarises sending history at a provider. Inbox placement across your actual recipients is an outcome you infer from bounces, complaints, replies and panel samples, never from seeds alone. Confusing these categories leads teams to optimise a number that does not describe their customers.

The useful definition is narrower and more honest: a seed test is a repeatable experiment that tells you whether a specific message, sent from a specific infrastructure, at a specific time, can reach a set of monitored inboxes. Every one of those qualifiers matters, and dropping any of them turns a measurement into a marketing claim.

How a placement panel actually works

A panel provider maintains mailboxes at a set of domains. Some are consumer providers, some are corporate systems, some are regional providers that matter in a particular market. When you send, the provider either gives you seed addresses to include in the campaign or receives a copy through a dedicated seed address. It then checks each mailbox through provider APIs, IMAP or a proprietary agent, classifies the message and reports the aggregate.

The measurement method is where panels diverge. A panel that reads the inbox through IMAP sees the same thing a user sees, but it may be slow and it may miss tab placement. A panel with provider relationships may get richer signals but only at providers that cooperate. A panel that relies on its own agents can classify faster but may be affected by the agent’s behaviour. None of these is wrong; they answer slightly different questions, which is why the panel name and method belong in any report you cite.

There is also a seeding effect. Seed mailboxes are not real customers. They do not open consistently, they do not buy, they do not reply and they may share a domain with other panel customers. Providers know this. A panel result is therefore a sample of how the provider treats a message that looks like your campaign, not a prediction of how it will treat your campaign in front of an engaged subscriber.

Why two providers disagree about the same email

Providers run different filters with different weights. Gmail leans heavily on engagement and has a tabbed inbox, so a message can be delivered to Promotions rather than the primary tab, which some panels count as inbox and others do not. Microsoft’s consumer and corporate systems apply their own reputation signals and can defer mail with a temporary error that looks like a missing seed. Yahoo and smaller providers have their own histories. An Indian consumer domain may weight content, language and sender recognition differently again.

Infrastructure adds another layer. The same campaign sent through a shared IP with a noisy neighbour can be treated differently from the same content sent through a dedicated IP with a clean history. Authentication alignment matters at every provider, but the consequence of a failure varies: one provider may spam-folder an unaligned message, another may reject it outright, and a third may deliver it while quietly reducing future reputation.

The practical conclusion is that a single blended placement percentage is close to meaningless. A result of eighty percent inbox can hide a perfect score at four providers and a disaster at the fifth, which happens to be the one your enterprise customers use. Always ask for the breakdown by domain, and always compare the same domain across sends.

Why geography changes the result

Email is global, but filtering is local in ways that surprise teams. The provider mix differs: an Indian B2C list is dominated by a handful of consumer domains and corporate systems, while a US list has a different shape. The IP location and the sending infrastructure can affect how a regional provider treats the message. Content expectations differ too: a message in Hinglish may be entirely normal to one audience and look anomalous to a filter trained on a different corpus.

Send time is part of geography. A campaign scheduled at nine in the morning India time arrives in the middle of the night for a US panel, and the engagement pattern that follows is not comparable. A panel that observes for six hours will classify a message before the recipient would ever have opened it. If the panel provider is in another region, its observation window may not match your customer’s day at all.

Data residency and vendor choice also matter for Indian teams. If your email infrastructure sits outside India, the message crosses borders and may be subject to different scanning and logging. That is a compliance question as much as a deliverability one. The seed panel should include the providers your customers use in the market you sell to, and the report should say which market it represents.

Reading one result without over-reacting

A seed panel is a small sample. If a hundred seeds are split across six providers, each provider cell may hold fifteen to twenty mailboxes, and a single message landing in spam moves that cell by five to seven points. That is not a signal about your list; it is the resolution of the instrument. Reporting a provider cell as a precise percentage invites decisions that the sample cannot support.

The first check after a bad result is authentication, because an SPF or DKIM failure is a fact rather than a statistical fluctuation. The second is whether the message was actually sent to the panel at all, since a tagging error or a suppression rule can remove seeds silently. The third is whether the result is consistent with the previous three sends. Only after those three checks is it worth looking at content, links or subject line.

A useful habit is to write down the decision rule before the test. For example: if a provider cell drops below half inbox for two consecutive sends while authentication passes and volume is stable, we investigate content and list quality. A rule written in advance prevents a single alarming panel from triggering a rewrite of a campaign that was working.

Reading a trend without fooling yourself

One send is an anecdote. Three to five sends with the same panel, the same infrastructure and a comparable audience start to become a trend. Keep the variables stable: same panel composition, same send window, same volume band. If you change the subject line, the template, the IP and the time of day at once, you have run an experiment with four treatments and no control.

Seasonality is real. Festive campaigns in India, end-of-quarter pushes and sale events change both volume and recipient behaviour, and providers respond to the change in volume. A dip during a high-volume week may be about the week, not the copy. Compare with the same period last year where possible, or with a control segment that received the previous version of the message.

Track the panel alongside real metrics: bounce rate, complaint rate, unsubscribe rate, reply rate and conversion. A panel that says inbox while complaints rise is telling you something the placement number cannot. The panel is a leading indicator of infrastructure health; the real metrics are the outcome. When they disagree, trust the outcome and investigate the panel.

Authentication and rendering are separate questions

A good panel reports authentication separately from placement, and the separation is the point. SPF, DKIM and DMARC results tell you whether the receiving provider could verify the sender. Placement tells you what it did with a message that may have passed every check. A message can be fully authenticated and still filtered because of content, reputation or recipient behaviour. A message can also be unauthenticated and still reach the inbox at a lenient provider, which is the most dangerous result of all because it hides a problem that will bite elsewhere.

Rendering checks answer a third question: did the message look right? Images blocked, dark mode inversion, a broken link, a subject line truncated on mobile, a Hindi font that renders badly on one client. These are product defects, not deliverability signals, and they should be triaged by the team that owns the template. A panel screenshot is useful evidence in a bug report.

Keep the three reports separate in your dashboard. When they are blended into a single health score, an authentication failure looks like a content problem, and the team spends a week rewriting copy while the DNS record stays broken.

Designing a panel test worth trusting

Start with composition. A panel should include the providers that represent a meaningful share of your list, not just the ones the vendor finds easy to monitor. If sixty percent of your customers are on one consumer domain, that domain needs enough seeds to produce a stable cell. Add the corporate gateway your enterprise buyers use, and add a regional provider if you sell in a market where it matters. A panel of a hundred seeds spread evenly across ten domains is often less useful than sixty seeds across the four domains that matter.

Control the send. Use the same campaign build, the same IP pool and a comparable volume band. Tag seeds so they can be excluded from billing and from engagement metrics, and make sure your suppression logic does not remove them. Record the send timestamp and the observation window. If the panel observes for two hours and your recipients open over two days, say so in the report.

Finally, decide what the test is for. If it is a pre-launch check, a small panel and an authentication pass may be enough. If it is a deliverability programme, you need a stable panel, a baseline and a review cadence. If it is a vendor comparison, run both panels on the same send and compare by domain rather than by headline percentage.

• Weight the panel toward the domains your customers actually use.
• Keep panel composition and send window stable across tests.
• Tag seeds and exclude them from engagement and billing.
• Record the panel vendor, method, date and observation window.
• Decide the decision rule before the result arrives.

Worked example: a festive campaign, sampled

A Bengaluru retailer sends a festive campaign to 1,20,000 recipients through a shared IP. The panel has 150 seeds: seventy at a consumer domain, thirty at a corporate gateway, twenty at a second consumer domain and thirty across four smaller providers. This is a sample run, not a published benchmark. The panel reports eighty-six percent inbox overall, but the corporate gateway cell is at forty percent with twelve messages deferred and the rest in spam.

The team checks authentication first. SPF passes, DKIM passes, DMARC passes with alignment. The seed tags are present and none were suppressed. The previous three sends to the same gateway were at eighty to ninety percent. The difference appears after a template change that added a large image and a shortened link. The link shortener is on a domain with a poor history at that gateway.

The fix is unglamorous: replace the shortener with a branded link on the sending domain, reduce the image weight, and re-run the panel on the next send. The overall number barely moves, because the consumer cells were never the problem. Without the per-domain breakdown, the team would have concluded that the campaign was fine.

From a panel result to a deliverability programme

A seed test earns its keep when it feeds a programme rather than a launch checklist. The programme has four parts: authentication that is monitored continuously, list hygiene that removes hard bounces and honours suppression, engagement segmentation that stops mailing people who never open, and a panel that samples the domains that matter. None of these is expensive individually; the failure is usually that nobody owns the whole set.

Set thresholds that trigger investigation rather than panic. A single provider cell below a level for one send is a note. The same cell below it for two comparable sends is a ticket. Authentication failure is an incident at any volume. Complaint rate above your internal ceiling is an incident too, because it predicts the filter decision that the panel will show next week.

Write the result down where the next person can find it. A panel report that lives in a screenshot in a chat thread teaches nobody. A short log with the date, panel, send volume, per-domain placement, auth result and the action taken becomes the institutional memory that makes the next campaign calmer.