SPF

SPF, defined against the identity it checks

SPF is a published statement about sending infrastructure. A domain owner publishes a TXT record beginning v=spf1 that lists mechanisms: IP addresses, address blocks, other domains whose senders are authorised, and a final all mechanism with a qualifier. When a receiving server gets a message, it extracts the domain from the SMTP envelope sender, fetches that domain’s SPF record, evaluates the connecting IP against the mechanisms and produces a result: pass, fail, softfail, neutral, none, temperror or permerror.

The envelope sender is the address used in the MAIL FROM command, which usually appears to the recipient as the return path. It is not the From header rendered in the inbox. This is the most important distinction on the page. SPF answers whether the server that connected was allowed to send for the envelope domain. It says nothing, by itself, about whether the visible From domain authorised the message. That gap is what DMARC was designed to close.

What SPF is not: it is not encryption, it is not a spam filter, it is not a reputation system, and it is not a guarantee that a message is wanted. It is also not a property of the visible sender. A message can pass SPF for a bounce domain while displaying a forged From header, and a message can fail SPF after being legitimately forwarded while being entirely genuine.

What the receiver actually does

The receiving server has a connecting IP, a HELO or EHLO name, a MAIL FROM address and a message. SPF uses the MAIL FROM domain when it is present. When MAIL FROM is empty, which happens with some bounce messages, the check falls back to the HELO identity. The server then performs a DNS lookup for TXT records on that domain, finds the record that begins v=spf1, and walks the mechanisms from left to right until one matches.

The first match decides the result, qualified by the mechanism’s prefix. A plus sign means pass, a minus means fail, a tilde means softfail, a question mark means neutral. If no mechanism matches and the record ends with all, the qualifier on all applies. If there is no record, the result is none. A DNS timeout produces temperror, which is temporary. A malformed record, multiple records or an exceeded limit produces permerror, which is permanent for that check.

The result is written into the message headers, usually as an Authentication-Results field added by the receiver and sometimes as a Received-SPF field. That header is the ground truth when you debug. It names the identity that was checked, the result and often the client IP. A screenshot from a seed panel is useful, but the header is what tells you which domain was actually evaluated.

The ten-lookup limit and the include chain

SPF evaluation has a budget of ten DNS queries per check. Mechanisms that trigger a query include include, a, mx, ptr, exists and redirect. A literal ip4 or ip6 mechanism costs nothing. The budget is global across the evaluation: when an include pulls in another domain’s record, the mechanisms inside that record also consume queries. A record that looks short can therefore be expensive, because the domains it includes are themselves including other domains.

This is the single most common reason a working record stops working. A company adds a CRM, the CRM’s include pulls in three more domains, and the total crosses ten. The result is permerror, which receivers treat as no valid authorisation. Mail may still be delivered, but it fails authentication, and DMARC alignment will fail with it. The failure often appears months after the change, when a vendor updates its own record.

Headroom is the design goal. Keep the count at eight or below so that a vendor can add a nested include without breaking you. Audit the count quarterly and after every new sending tool. Flattening, which replaces includes with literal IPs, can reduce lookups, but it converts a live delegation into a snapshot that goes stale when the vendor changes ranges. Choose flattening deliberately, with a refresh process, rather than as a one-time fix.

The void lookup trap and the missing include

A void lookup is a DNS query that returns no usable answer, either because the name does not exist or because it exists without the requested record type. RFC 7208 caps void lookups at two per check. Exceed that and the result is permerror, even if the visible record looks reasonable. The cap exists because void lookups are expensive for resolvers and were being used to amplify load.

The trap is easy to fall into with mechanisms such as a, mx, ptr and exists, especially when a domain has been decommissioned but the mechanism remains. An include of a domain that publishes no SPF record is a different failure: the include cannot be evaluated, and the result is permerror. Both cases produce the same operational symptom, mail failing authentication, and both are invisible unless someone audits the record and its dependencies.

The defensive habit is to resolve every dependency yourself. For each include, fetch the target record and count its queries. For each a or mx mechanism, confirm the name has an address record. For each exists, confirm the macro expands to something that resolves. Do this before a campaign, not after a deliverability complaint, and record the date of the audit so the next person knows how stale the picture is.

Why SPF alone does not authenticate the From domain

SPF authenticates the envelope domain. The From header is a different field, and nothing in SPF requires the two to match. A sender can pass SPF for a domain it controls while displaying a completely different brand in the From header. That is not a bug in SPF; it is the scope of the protocol. The result is that a domain can have a perfect SPF record and still be impersonated in the field recipients actually see.

DMARC closes the gap with alignment. It requires that a passing SPF check use a domain aligned with the From domain, or that a passing DKIM signature use a d= domain aligned with it. Alignment can be strict, meaning an exact match, or relaxed, meaning the same organisational domain. This is why the standard advice is to publish SPF, DKIM and DMARC together: SPF authorises infrastructure, DKIM proves integrity and origin of signed content, and DMARC ties either one to the visible sender and tells receivers what to do on failure.

Forwarding is the other reason SPF is fragile on its own. A mailing list or a forwarder often re-sends a message with a new envelope sender, so the SPF check is performed against the forwarder’s domain rather than the original author’s. SPF can fail even though the message is genuine. DKIM survives many forms of forwarding because it signs content rather than the path. A programme that relies on SPF alone will misclassify legitimate forwarded mail.

Publishing for a multi-vendor estate

A typical Indian company does not have one sender. It has Google Workspace or Zoho for staff mail, an email service provider for campaigns, a CRM or support tool that sends notifications, a transactional provider for receipts and OTP-adjacent mail, and possibly an agency sending on a subdomain. Each sender has its own infrastructure, and each one needs to be represented in the record or delegated to its own subdomain.

There are two clean patterns. The first is one record on the main domain that includes every authorised sender, with a lookup budget that fits. The second is subdomain delegation: marketing mail sends as mail.example.in or news.example.in, each subdomain publishes its own record, and the main domain stays small. The second pattern is usually easier to operate because a marketing vendor change does not consume the transactional domain’s budget, and a compromise on one subdomain does not automatically authorise the other.

Every sending subdomain needs its own record. SPF does not automatically inherit the organisational domain’s policy for a subdomain in the way DMARC policy discovery does. A subdomain with no record returns none, which is not a pass. If you add a subdomain for a new campaign tool, add the record in the same change window as the DNS entry, not after the first send.

Soft fail, hard fail and the rollout order

The all mechanism is the default. A record ending in ~all returns softfail for anything unmatched, which most receivers treat as suspicious but not necessarily rejectable. A record ending in -all returns fail, which is a stronger statement and can cause rejection. Starting with -all on a domain with unknown senders is how an OTP flow breaks on a Sunday. Start with ~all, publish DMARC at p=none, collect reports, identify every legitimate source, then move to quarantine and reject.

The rollout has a sequence. Inventory the senders by reading DMARC aggregate reports and mail logs, not by asking around. Publish SPF for each sending domain and confirm each sender appears in the reports as passing. Publish DKIM keys and confirm signatures validate. Move DMARC from none to quarantine with a small percentage, watch the failure reports, then to reject. Only after that tighten SPF from ~all to -all if the inventory is complete.

Change control matters because DNS is shared. One team adding a vendor record can alter the lookup count for everyone. Keep the record in a repository, review changes, and require the lookup count and the sender list in the pull request. A DNS change that is not reviewed is a production change without a review, whatever the interface looks like.

Flattening, macros and the maintenance trade-off

Flattening replaces include mechanisms with the IP addresses those includes resolve to. It reduces DNS lookups and can rescue a record that has run out of budget. It also freezes a live delegation into a snapshot. When the vendor adds a range, changes a provider or retires an IP, the flattened record is wrong, and the failure appears as authentication loss for a sender you thought was authorised. Automated flattening services refresh the snapshot, which introduces a dependency on that service and a new availability question.

Macros are the opposite trade: they make a record dynamic without extra includes. A macro can insert the sender’s domain, IP or a validated value into a mechanism, and exists can test a computed name. Macros are powerful and easy to get wrong. A malformed macro can produce a permerror, and a macro that expands to an attacker-controlled string is a security problem. Use them only when the simpler mechanisms cannot express the policy, and test the expansion explicitly.

The honest position is that SPF is a small, static configuration that needs an owner. Whether you flatten or delegate, someone has to know the sender inventory and review it when a vendor changes. The record is not fire and forget; it is closer to a firewall rule that happens to live in DNS.

Worked example: four senders, one budget

A sample domain, example.in, sends through Google Workspace for staff mail, an ESP for campaigns, a CRM for notifications and a transactional provider for receipts. The record begins with v=spf1, includes the Google record, the ESP record, the CRM record and the transactional record, adds one literal IP for an in-house application, and ends with ~all during the rollout. The top-level record has four includes and one ip4, but the includes pull in nested records that also consume queries.

An audit resolves each dependency. Google’s include costs three queries in this example, the ESP costs two, the CRM costs two and the transactional provider costs one, plus one for the in-house a record. That is nine, leaving one query of headroom. A new vendor with a single include would fit; a vendor with a nested include chain would not. The team decides to move campaigns to a marketing subdomain with its own record, which brings the main domain down to six queries and gives each vendor room to change.

The marketing subdomain is published with its own SPF, its own DKIM selector and its own DMARC policy. The main domain keeps the transactional and staff senders. Reports are separated, so a campaign reputation problem does not contaminate the receipts stream. The change is a DNS project, not a product project, and it takes an afternoon once the inventory is written down.

Testing and the mistakes that keep repeating

Test from the outside. Query the TXT record with a DNS tool, count the lookups with an SPF checker that follows includes, and send a message through each sender to a mailbox you control. Read the Authentication-Results header and confirm the identity that was checked, the result and the client IP. If the header says spf=pass with smtp.mailfrom equal to your envelope domain, the check worked for that path. Repeat per sender and per subdomain.

The recurring mistakes are well known and still common. Publishing two SPF records on one domain produces permerror. Using include for a domain that has no SPF record produces permerror. Letting the lookup count drift over ten produces permerror. Using ptr is slow, unreliable and discouraged. Forgetting a subdomain produces none. Starting at -all before inventorying senders blocks legitimate mail. Flattening without a refresh process creates a slow, silent failure. And expecting SPF to protect the From header without DMARC produces a false sense of security.

Measure authentication per sender, not as one number. A dashboard that says ninety-eight percent SPF pass can hide a transactional stream at forty percent, and that stream is the one customers notice. Break the rate down by envelope domain, by sending IP and by receiving provider, and alert when a sender that normally passes drops. The alert is worth more than the average.

• One SPF record per domain, never two.
• Keep the lookup count at eight or below.
• Audit void lookups and dead mechanisms.
• Publish a record on every sending subdomain.
• Pair SPF with DKIM and a DMARC policy.
• Test from outside and read the Authentication-Results header.