Consent ledger

A system of record, not a column

A consent ledger is the append-only system of record for consent across an organisation. It stores one row per event, timestamped and attributed, covering every contact, every purpose and every channel, and it derives a current state from those rows rather than storing the answer in a field that somebody edits. Grant, withdrawal, re-grant after a fresh notice, a correction of a misspelled identifier, an expiry under a retention rule: all of them are rows. None of them is an overwrite.

The word ledger is a useful metaphor because accounting ledgers solved this problem centuries ago. You do not erase a wrong entry in a ledger; you post a correcting entry. The balance is derived. Anyone can replay the book and arrive at the same number. A consent ledger works the same way, except the balance is whether a person may be contacted for a purpose, and the replay is what lets you answer a regulator, an auditor or a customer without guesswork.

What the ledger is not: a marketing preference table with a few checkboxes, a column called opt_in with a timestamp beside it, or a copy of whatever each channel tool believes. Those are projections, and projections are allowed to exist for performance. They are not allowed to become the source. The moment two channel tools each hold their own truth, the organisation has no truth, only a disagreement it will discover during a campaign.

Why the boolean column always fails eventually

A boolean looks adequate for about six months. Then three things happen in sequence. A second channel is added and needs its own consent, so the single flag is stretched to cover email, WhatsApp and calls, and a withdrawal from one is applied to all three because there is nowhere to record the difference. A migration copies the flag into a new system and the timestamp, which lived in a different table, does not come along. And a customer disputes a message sent two years ago, at which point the flag says false and nobody can say when it became false or why.

The deeper problem is concurrency. Two processes touch the same contact on the same afternoon: an unsubscribe webhook and a re-import from an event list. With a mutable field, the last write wins, and the winner depends on queue ordering rather than on intent. With an append-only ledger, both events are recorded with their timestamps and sources, and the current state is a deterministic function of the sequence. You can then see that the import arrived four minutes after the withdrawal, which turns an invisible bug into an obvious one.

Audit is the third failure. A mutable field can answer what is true now and nothing else. A ledger can answer what was true on any date, which is the question that actually gets asked. When a customer says they unsubscribed in March and received mail in April, the answer has to be a reconstruction, not a shrug. The reconstruction is only possible if the March event was written down at the time and never overwritten by the April import.

Where the artefact ends and the ledger begins

The two terms are close enough that teams use them interchangeably, and that is the mistake to avoid. A consent artefact is the evidence of one event: the notice version and text, the purpose, the timestamp, the identifier, the channel, the exact action taken and a hash. A consent ledger is the system that holds all of those events for a contact in order and derives the current answer. The artefact is the document; the ledger is the filing system with a rule that nothing is ever removed.

Practically, the ledger references artefacts rather than containing them. Each row carries an artefact identifier, and the artefact payload lives in immutable storage where a bulk query cannot accidentally rewrite it. That split keeps the ledger fast and the evidence intact. It also gives you a defensible retention story: the ledger row can outlive the marketing data, while the artefact can be minimised after its retention period to the fields needed for proof.

The split has a failure mode worth naming. If the artefact store is unreachable, the ledger still answers current state, which is what sending needs, but cannot prove history. That is an acceptable degradation for a mail send and unacceptable for a regulatory response. Instrument both: alert when artefacts stop being written even though ledger rows continue, because that pattern means an integration is posting events without the evidence behind them.

The event taxonomy that stops arguments later

A ledger is only as good as its event vocabulary. The minimum set has eight types. Grant is a new permission with an artefact. Re-grant is a permission given again after a withdrawal or after a new notice version, and it must reference the new artefact rather than reusing the old one. Update is a change to the scope of an existing permission, such as adding a WhatsApp channel to an email consent. Withdrawal ends a permission and must record the channel it arrived through.

Expiry is the fourth, and it is the one most systems omit. Consent given for a campaign or a season has a natural end, and treating it as permanent is a quiet compliance failure. Objection is the fifth: a principal can object to a specific use without withdrawing everything, and the ledger needs a type for it. Correction covers a change to the identifier itself, such as a merged duplicate contact, which must be recorded because otherwise the history appears to belong to two different people.

The last two are administrative. Suppression is a technical block that is not itself a consent event, such as a hard bounce, and it should be a separate table or a separate type so nobody confuses a bouncing address with a person who withdrew. Audit is a read or an export, recorded because access to consent data is itself worth logging. With those eight types in place, most disputes become a matter of reading the sequence rather than arguing about definitions.

Naming conventions are part of the taxonomy. A purpose code that reads promo is ambiguous across email, WhatsApp and voice, and it will be reused for something else within a year. Use a short, stable code with a published definition, and version it when the definition changes rather than editing the code in place. The same applies to channel codes and event types: stable identifiers on one side, human-readable descriptions maintained separately on the other.

Designing the table and the projection

The core table needs a small number of columns and a large number of rows. An event identifier, a contact identifier, a purpose code, a channel code, an event type, an effective timestamp, a recorded timestamp, a source system, an artefact reference and an idempotency key. The idempotency key is what makes a webhook retry safe: the same withdrawal delivered five times writes one row and four no-ops. Without it, a retry-heavy integration inflates the ledger and makes the sequence unreadable.

Current state should be a projection, not a second source of truth. Materialise one row per contact, purpose and channel with the latest event type, the effective time and the artefact that produced it, updated by the same transaction that appends the event or by a consumer that can be rebuilt from scratch. The test of a good projection is that you can drop it and recompute it from the event table without loss. If you cannot, the projection has quietly become the source.

Queries at send time should be constant-time and cheap. A single lookup by contact, purpose and channel returning a state, a timestamp and a reason code is enough for the sending path. Do not return a boolean, because a boolean invites caching with no expiry. Return the effective time so the caller can decide whether the answer is recent enough for the volume being sent, and log the lookup so that a later dispute can show the gate was actually consulted.

Withdrawal propagation is the real service level

Capturing a withdrawal is easy. Making it true everywhere is the work. A withdrawal has to reach the email platform, the WhatsApp business account, the voice dialler, the CRM, the ad audience sync, any partner you pushed the lead to, and the local caches inside your own agent runtime. Each of those is a separate integration with its own failure mode, and the ledger is only credible if the fan-out is measured rather than assumed.

Define the target as a latency, not a state. For most Indian programmes a defensible target is fifteen minutes to the internal channels and two hours to external audiences, with a ninety-fifth percentile reported weekly. Measure from the effective timestamp of the event to the last successful propagation, and record the slowest channel by name. Averages hide the ad sync that takes a day, and the ad sync is often the one that matters most commercially.

Handle partial failure explicitly. A withdrawal that reaches email and WhatsApp but fails at the dialler is not a success with two out of three; it is a live risk that someone will call the person tomorrow. The ledger should carry a propagation status per downstream system, with retries and an escalation when a channel stays behind. Without that column, the only way to discover a broken integration is a customer complaint, which is the slowest possible feedback loop.

Ordering matters when a withdrawal and a re-grant arrive close together. If the fan-out is asynchronous, a re-grant can reach a channel before the withdrawal it was meant to follow, leaving the tool in the wrong state. Carry the ledger sequence number in every propagation message and have the receiver discard anything older than what it has already applied. That single field removes a class of race condition otherwise discovered by a customer receiving mail they thought they had stopped.

Retention, erasure and the proof you must keep

A ledger grows monotonically, which is a feature for audit and a problem for storage and privacy. Thousands of contacts a month produce hundreds of thousands of rows a year, and most of those rows reference artefacts that contain personal data. The answer is not to truncate the ledger. The answer is to separate the retention clock of the event from the retention clock of the payload.

Event rows can be kept for the period you might need to demonstrate compliance, minimised to identifiers and codes. Artefact payloads, which are richer, follow a shorter schedule and are then reduced to the fields that prove the act: notice version, purpose, timestamp, language, channel and hash. Message content, free-text notes and recording references are deleted. The chain of hashes stays, so a minimised artefact can still be shown to be the same artefact that was written on the day.

Erasure requests complicate this and should be handled as a documented exception rather than a conflict. When a principal asks for erasure, the marketing data goes, the contact becomes uncontactable, and the minimal proof of the consent and its withdrawal stays because the processing already happened and may need to be defended. Write that reasoning into the retention policy before the first request arrives, so the support agent is following a rule rather than improvising.

Backups are the part of the retention story that audits find first. A deletion job that clears the production table while nightly snapshots keep the same rows for a year has not deleted anything. Either exclude consent payloads from long-lived backups, or accept a documented restore-and-reapply procedure with a stated window. Write the choice down, because the alternative is an engineer improvising during an incident at two in the morning.

The India layer: consent managers, DLT and STOP

Three Indian mechanisms feed a ledger and each has a different shape. The DPDP framework introduces consent managers, registered intermediaries that let a principal give, review and withdraw consent across fiduciaries in one place. A withdrawal arriving from a consent manager will not look like your webhook. It needs an authenticated source, a signed payload or a verification step, and a mapping to your contact identifier that does not depend on an email address being current.

DLT is the second. Commercial SMS in India runs on registered entities, headers and templates, and the TRAI framework obliges senders to maintain consent records for the numbers they message. Operationally that means an SMS opt-out has to land in the ledger with the same weight as an email unsubscribe, and a DLT template that hard-codes a consent claim without a matching ledger entry is a gap an auditor will find quickly.

The third is the WhatsApp opt-out. A STOP keyword is a withdrawal for a channel and usually for a purpose, and the ledger should record it as such rather than as a global suppression. Treating it as global feels safe and creates a different problem: the customer who wanted to stop promotional WhatsApp messages but still wants order updates will stop receiving both, and will call support to ask why. Channel-scoped withdrawal is the correct model, and it is only possible if the ledger has a channel dimension.

Reconciliation: finding drift before a campaign does

Every channel tool keeps its own idea of who is subscribed, and those ideas drift. A suppression applied in one tool does not travel to another unless something moves it. A contact deleted in the CRM leaves an orphan in the WhatsApp console. A list import that ran with an older export re-subscribes people who withdrew in between. None of this is visible from the ledger alone, because the ledger is right and the tools are wrong.

A reconciliation job closes that gap. Once a week, pull the subscription state from each channel, join it against the ledger projection, and emit three counts: contacts subscribed in a tool but withdrawn in the ledger, contacts unsubscribed in a tool but active in the ledger, and contacts present in a tool but absent from the ledger. The first is the dangerous one and should trigger an immediate correction. The third is a design problem, because it means something is sending without consulting the record.

Report drift as a number with a trend rather than fixing it silently. Silent repair hides the integration that keeps breaking, and the same drift returns within a month. A named owner, a weekly count and an alert when the count rises after a release is enough to keep the system honest without building an elaborate observability stack.

The reconciliation query is easier to write than to schedule. Channel APIs rate-limit, some tools expose only a count and not a list, and the export that works today may be deprecated next quarter. Build the job to tolerate partial data and to report coverage alongside drift, so a channel that returned nothing is visible as missing rather than clean. A drift report that silently treats an unavailable API as zero is worse than no report, because it manufactures confidence.

Rolling it out without stopping the business

The migration path that works is to add the ledger beside the existing flags rather than replacing them on day one. Write every new consent event to both places, run the reconciliation job in report-only mode for a month, and compare. The differences tell you which legacy flags were wrong, which is usually a larger number than anyone expects. Once the ledger is at least as accurate as the flags, flip the sending paths to read the ledger and keep writing the flags as a temporary mirror.

Do not attempt to reconstruct history from campaign engagement. An open or a click is not consent, and a backfill that infers permission from behaviour creates a record you cannot defend. Start the ledger with today’s events and mark legacy contacts as unknown rather than granted. Then run a re-permission campaign to the ones who matter, which is slower, honest, and produces a smaller list that actually performs.

Sequence the channels by risk. Email first, because the volume is highest and the unsubscribe path is well understood. WhatsApp second, because template sends to people who never opted in are the most common complaint source in Indian programmes. Voice third, because a dialler that ignores the ledger is the fastest way to generate a grievance. Ads last, because the sync is slow and the fix is usually a scheduled job rather than a real-time gate.

Freeze the legacy writes at a known moment. Dual writing is safe only while one side is authoritative; once the ledger is authoritative, the old flags become a cache and must stop accepting writes from other systems. Teams that leave both writable end up with a third source of truth and a reconciliation job that can never reach zero. Announce the freeze, monitor for writes to the old path, and remove the permission rather than trusting a convention.