ADR 0007: Make ABI migrations first-class

Status: Proposed; amended 2026-09-30 for ABI 0.21 (see Amendment)
Date: 2026-09-09
Owners: Pina maintainers
Related: ADR 0001, ADR 0002, ADR 0003, ADR 0004

Context

Solana programs cannot rewrite every program-owned account during deployment. Account data is loaded only when a transaction names its address, and every transaction has account, privilege, compute, size-growth, and fee constraints. A schema upgrade must therefore remain able to read old bytes and migrate each account on demand.

Account bytes are only one part of the compatibility contract. An old client also sends an old instruction payload for a specific process and positional account list. Events written by earlier executables remain in immutable transaction logs.

The desired developer model is declarative:

[migrations]
version_type = "u8"

#![allow(unused)]
fn main() {
#[account(discriminator = AccountType::Profile, migrations)]
pub struct Profile {
	pub authority: Address,
	pub score: u64,
}
}

A program that wants every contract versioned should not have to repeat the annotation:

[migrations]
version_type = "u8"
auto = true # or a staged subset such as ["accounts", "events"]

Amended (ABI 0.21): neither setting lives in pina.toml any more. pina migrations create --version-type and --auto record them in migrations/manifest.json, their only copy, and pina.toml refuses the retired keys. See the Amendment.

The developer declares that a contract is migratable, but does not choose or maintain its version. The current Rust source describes the desired state. Pina owns version allocation, ABI snapshots, generated structural transitions, drift checks, and historical dispatch. A developer supplies code only when a schema diff cannot determine the intended value.

Decision

Pina will treat migrations as a versioned ABI system, backed by a checked-in ABI history and a content-addressed publication history. Pina migrates stored account data, instruction payload data, and the generated instruction process contract when it can prove that an old positional account list remains a valid request.

The first process proof is deliberately narrow. Existing account slots must remain an identical positional prefix and newly appended slots must all be optional. An old request then represents the appended suffix as absent. Reordering, insertion into the middle, removal, renaming, privilege or signer changes, PDA changes, known-address changes, and newly required accounts are genuinely breaking under the same discriminator. The developer creates a new instruction discriminator for those changes. Pina may add new formally specified proofs later, but it will not expose compatibility policy switches.

Version envelope

Every opted-in wire contract uses this envelope:

[existing discriminator][schema version][payload]

The discriminator remains at offset zero. The version is little-endian and is hidden from generated accessors and patches. Version zero is the first migratable representation.

[migrations].version_type is global for the program. Pina initially accepts u8, u16, and u32. It does not accept per-account or per-instruction overrides. The width may change while the program has no published migration-aware release. The first persistent publication freezes the width, byte order, and header offset for that program identity.

Amended (ABI 0.21): the width is recorded only as the manifest’s versionType. pina migrations create --version-type rewrites it while no receipt or pending record exists and fails once one does.

Each account, instruction contract, and event advances independently. An instruction version snapshots both its payload and its process account ABI. Either a payload change or a compatible appended-optional process change advances the instruction version after publication.

Amended (ABI 0.21): only accounts, events, and instructions that opt in with migrations use the envelope; an instruction covered by auto alone is recorded as a single snapshot without one. An appended-optional process change now extends the published version in place instead of advancing it.

Existing unversioned data is not silently treated as version zero. Its first payload bytes may be a valid version by accident. Adoption requires an explicit legacy bridge or a new discriminator.

Auto opt-in policy

A [migrations].auto list opts whole contract kinds in without per-item annotations. The vocabulary is exactly accounts, events, and instructions, and auto = true is sugar for all three. Unknown names, duplicates, and mixing the boolean with a kind list are configuration errors. Staging a subset is supported because the kinds carry different costs: an instruction envelope changes payload bytes and ripples into CPI call sites.

Amended (ABI 0.21): an auto policy no longer envelopes instructions, so covering them does not change a payload byte. This staging rationale applies to instructions opted in individually with migrations. The policy is set with pina migrations create --auto, which takes true, false, or a comma-separated kind list, rather than a pina.toml list.

The resolved policy is recorded in migrations/manifest.json, and the manifest remains the only policy source procedural macros consult. Macros must not read pina.toml: the manifest is the checked-in, hash-chained document that keeps builds deterministic and reproducible, and a proc macro does not re-expand when an unrelated toml file changes, so a toml read would leave stale expansions after a policy flip. pina migrations create therefore records the policy and snapshots every contract of the listed kinds, and a declaration without a snapshot still fails the build with the existing pina migrations create remedy.

Amended (ABI 0.21): the manifest is now the policy’s only home, not only the one source macros read: create --auto writes it directly, and pina.toml keeps no second copy to disagree with it. Neither document is hash-chained; the manifest never was, and the publication ledger dropped its receipt chain.

Per-item migrations = false overrides the global policy for one contract. Removing an envelope from a contract the manifest already records is an error rather than a silent opt-out, because stripping an envelope changes the wire format. The same rejection applies when a kind is dropped from auto. Recording an envelope removal as a deliberate migration is a future retirement flow; this ADR only fixes the fail-closed behavior.

Enabling auto on an already-launched program is a bulk wire-format change: create records one history entry per newly enveloped contract. Existing live bytes of those contracts have no envelope, so the developer must treat the addition like any other deliberate wire-format change. For a new program it is simply the version-zero baseline.

Because the macros read the manifest, a policy flip must re-expand contracts without a source edit. When a policy is recorded, pina migrations create scaffolds a build.rs that emits cargo:rerun-if-changed=migrations/manifest.json. Scaffolding is idempotent, never overwrites a hand-written build script, and reports the exact line to add when it cannot write safely; pina migrations check fails until the directive is present.

Current IDL and ABI history

The public IDL describes only the current program contract. For each opted-in account or instruction, it includes migrationVersion as an omitted constant and a constant discriminator at the byte immediately after the ordinary discriminator. Generated clients therefore write the current version without exposing it as an application argument, while an IDL captured from an older release continues to write its own frozen version. The IDL does not contain historical schemas or transition code; it is not the migration database.

Amended (ABI 0.21): each earlier version of an event is listed in the IDL as its own event node, <Event>V<n>, so clients can decode old log records. Historical account and instruction schemas and all transition code remain outside the IDL. A snapshot-only instruction has no migrationVersion field.

The checked-in Pina ABI history records the physical information needed to reconstruct every supported representation:

ABI format and generator versions;
contract kind and stable identity;
discriminator bytes and width;
migration version width;
fixed or compact storage mode;
ordered canonical wire types;
fixed offsets and sizes;
compact prefix widths, capacities, header offsets, tail order, and alignment;
referenced enum representations and explicit discriminants;
the process contract for every instruction version, including accounts, positions, optionality, signer and writable requirements, known addresses, and PDAs;
the adjacent proof that relates each pair of process versions;
canonical schema and transition hashes.

Stable identity derives from contract kind and discriminator, not a Rust type name. Renaming a Rust type does not create a new on-chain identity.

The ABI history comes from the same closed schema grammar used by Pina’s macros. Format 3 records the pinaPodV2 codec and a derived, payload-relative physical descriptor. Format 4 records the [migrations].auto policy alongside it. Historical generated types assert their compiled fixed size or compact header, maximum size, and tail alignment against that descriptor. The Codama IDL and unconstrained Rust type strings are not precise enough to be the long-term physical-layout authority. PinaPod does not own Solana migration policy.

Pina’s ABI document has its own formatVersion, separate from every on-chain contract version. All readers decode the document into a generic envelope, reject future formats, and run Pina-owned adjacent format migrations before deserializing the current typed model. The ABI library also provides adjacent downgrade paths. A downgrade fails closed when an older format cannot represent the current document without information loss. Normal manifest writes always use the current format. An internal ABI-format upgrade therefore does not consume an account or instruction migration number, and old checked-in manifests remain buildable as long as Pina retains their adjacent document migrators.

Drafts and publication

Local iteration has one replaceable draft head per changed contract. pina migrations create captures the current ABI. It replaces an unpublished draft without consuming another version. If the current head has appeared in a persistent release, it allocates the next version.

pina build never creates or changes migration history. It fails on ABI drift, an unresolved custom transition, a modified published schema or transition, version exhaustion, or a version-width mismatch.

Amended (ABI 0.21): with the width recorded once, in the manifest, there is no second copy for it to mismatch. Drift compares what a field stores rather than how its type is spelled, so a respelling such as PodU64 for u64 is not drift.

Developers do not mark versions as published. Before a non-local deployment starts, pina deploy atomically records the exact ABI candidate as pending. The pending versions are frozen because they may already be live. After success, Pina rechecks every planned input and converts the pending record into a local receipt. Each receipt records:

the cluster label and credential-free RPC URL;
the program identity;
the SHA-256 digest of the exact planned executable;
the ABI manifest digest and every current contract version, together with the pinned schema and transition-implementation hashes of every published version;
the preceding receipt digest.

The checked-in, hash-chained publication ledger is the current source of truth for version allocation. Both receipts and a pending deployment freeze versions. Loopback local deployments do not add publication state.

Amended (ABI 0.21): a receipt records only the credential-free RPC URL, the executable digest, the pinned schema and transition hashes of every version it made live, and the abandoned flag; the pending record adds the cluster label. The program identity is the manifest’s programId, the manifest digest and the preceding receipt digest are gone, and a contract’s highest published version is the position of its last pin. The checked-in ledger stays the source of truth for version allocation, append-only through version control rather than a hash chain.

The first implementation does not query the deployed program-data account, cluster genesis hash, deployment slot, or transaction signature. A process interruption can therefore leave the remote outcome ambiguous. The pending record preserves and freezes the candidate before the remote command starts. Rerunning the exact deployment resumes it; Pina rejects a different deployment until the pending attempt is reconciled. The local ledger is reviewable release evidence, not an independent on-chain attestation.

Generated data compatibility boundary

The generated dispatcher, not an ordinary current-only decoder, owns historical instruction payload compatibility. Its conceptual flow is:

Read the instruction discriminator and request version.
Load the exact process snapshot for that instruction version and verify its frozen compatibility path to the current process.
Decode the exact historical payload.
Validate the historical positional prefix, signer and writable privileges, known addresses, PDAs, and duplicate mutable aliases; represent a proven appended optional suffix as absent.
Inspect all typed migratable accounts required by the route.
Preflight every account migration without retaining account-data borrows.
Apply account migrations in generated deterministic order.
Reload and validate every account in its current representation.
Adapt the historical payload to the current command.
Apply current authorization and business invariants.
Run the current handler.

Application handlers see current types only. Historical instruction versions are attacker-selected input, so an adapter must not preserve obsolete weak authorization. Old bytes are decoded into a current command and then pass current checks.

The current-version hot path reads one version value and compares it with a generated constant. It does not scan history or trial-decode layouts.

Account migration runtime

Each generated adjacent account migration uses two phases:

#![allow(unused)]
fn main() {
pub trait MigratableAccount: private::Sealed {
	type Plan;

	const CURRENT_VERSION: u32;
	const VERSION_BYTES: usize;

	fn plan(data: &[u8]) -> Result<AccountMigrationPlan<Self::Plan>, ProgramError>;

	fn apply(plan: Self::Plan, destination: &mut [u8]);

	fn validate_current(data: &[u8]) -> ProgramResult;
}
}

Planning validates the exact source schema and returns owned, detached state plus that step’s target and working lengths. The plan cannot borrow account data. This lets the dispatcher drop every old borrow before funding, resize, mutation, or CPI. Applying a preflighted plan is infallible; the executor writes the adjacent destination version only after that representation validates.

The executor repeats this pair for each adjacent version in the bounded inline path. That sequencing is required for compact schemas: the allocation needed by v2 -> v3 can depend on the active tails produced by v1 -> v2, and copying a maximum-size account into stack scratch space is not viable on SBF. The first step may return a normal preflight error. Once any step mutates lamports, length, or bytes, a later planning, resize, or validation failure aborts the instruction and rolls back the complete transaction.

The executor has a one-way mutation boundary. Ownership, writability, historical decoding, step limits, size arithmetic, rent, funding authorization, and borrow availability fail with ordinary ProgramError values before that boundary. Once a funding CPI, resize, or byte rewrite succeeds, a later framework invariant failure aborts the instruction instead of returning a catchable error. This prevents application code from swallowing a migration error and committing partially rewritten bytes; Solana rolls every transaction effect back on the abort.

Generated structural changes include exact field copies, safe reordering, and defaults whose value is unambiguous. Type changes, narrowing, semantic splits or merges, authority changes, and other ambiguous changes generate an unresolved transition. A fixed-account transition is total and infallible after Pina validates its exact source shape. A manual instruction transition runs in scratch space and can reject a request. The build remains blocked until the developer implements the transition and provides semantic fixtures or invariants.

The default migration path retains every surplus lamport. It never uses the ordinary reallocation helper’s shrink-refund policy. Growth may charge only an explicitly declared, writable signer or a fixed program treasury policy, subject to a generated maximum. The target and payer must not alias.

Inline and dedicated execution

An ordinary instruction either completes all required migrations and its business handler atomically, or returns an error. It never returns success after migration without running the requested operation. Returning an error rolls migration writes back with the transaction.

Inline account migration is unavailable when any required condition is missing:

a stale account is readonly;
growth needs lamports and no authorized payer was supplied by the historical request;
growth exceeds the runtime’s per-instruction limit;
the bounded chain exceeds the supported compute or stack budget;
a custom transition needs an account absent from the historical process contract.

A stable migration instruction can advance a bounded amount of work and return success. Generated migration-aware clients may invoke it repeatedly, then retry the original operation. An old client cannot acquire this retry behavior after release. For that reason, Pina classifies compatibility at migration creation time instead of promising that every old request remains transparent.

Process compatibility

Pina versions the instruction account ABI with the instruction payload. The default adjacent proof accepts only two relationships:

the process account ABI is byte-for-byte unchanged; or
the destination retains the complete source list as an identical positional prefix and appends only optional slots.

The current Accounts parser treats missing trailing optional slots as None. Optional fields followed by another positional field still require the program-address filler, so absence cannot shift a later account into an earlier slot.

Every other relationship fails closed. Insertion into the middle, removal, reordering, slot renaming, signer or writable changes, PDA changes, known-address changes, and a new required account require a new instruction discriminator. Solana cannot synthesize a missing AccountView or escalate a privilege that was not present in the transaction. A change from the SPL Token program to Token-2022 therefore remains a new process unless both program slots already existed and the change is expressed entirely as current business logic.

Process compatibility proves only the generated account ABI. Arbitrary application semantics are not statically knowable. Every historical request is normalized and then runs current validation and authorization; developers must add semantic fixtures for manual migration logic.

CPI and mixed versions

Each program migrates only accounts it owns. Before a CPI, the caller migrates its own stale accounts and releases all typed guards. The callee’s generated boundary migrates its own stale accounts. After a CPI that received writable accounts, the caller reloads those accounts before further use because the callee may have changed their data and length.

For a current account interacting with a stale account, the dispatcher migrates the stale account before it constructs either current typed view. The handler therefore observes one coherent current model.

Account-local structural migrations cannot read or mutate unrelated accounts. A custom migration may declare read-only dependencies that are included in the historical request. Pina orders an acyclic dependency graph statically. Cycles and cross-account write invariants require an explicit coordinated migration instruction.

Events

Events are not migrated on-chain. New code emits only the current event representation. normalize_event_data and the generated with_current_event_data helper validate exact historical bytes and project them into caller-owned scratch space. The projection returns the source version with the latest-shape bytes, preserving whether a defaulted field was absent historically or was actually emitted with its default value. Golden event fixtures live in pina_test::HistoricalEvent; generated clients can apply the same manifest transitions off-chain.

Superseded (ABI 0.21): events are versioned, not projected. See the Amendment.

Compatibility verification

pina test --compatibility and pina_test consume checked-in historical fixtures, not fixtures regenerated from current code. The required suite covers:

every historical account version to the current version;
fixed, compact, fixed-to-compact, growth, shrink, and unchanged-size transitions;
current no-op, future-version rejection, truncation, malformed lengths, and maximum capacities;
rent deficit, surplus retention, unauthorized funding, aliasing, and arithmetic overflow;
historical instruction payload replay through every supported process version against the latest SBF artifact;
acceptance of appended optional account suffixes and rejection of required additions, insertion, reorder, removal, and privilege drift;
mixed-version sets, duplicate aliases, and missing privileges;
old caller programs performing CPI into the latest callee;
reload behavior after writable CPI resize;
rollback after failures injected at each migration phase;
event golden-byte decoding;
tampered histories, transitions, manifests, widths, and rollback releases;
compute, stack, and binary-size budgets for the oldest supported maximum-size state.

Native property tests cover parsers and state transitions. Miri covers borrow-guard lifetimes. Kani covers header arithmetic, bounds, and state machines. Mollusk and Surfpool exercise real SBF resize, rent, CPI, rollback, and old-client behavior.

Security consequences

Migration expands the program’s input surface to every supported historical layout. Generated code must reject unknown and future versions without fallback, validate the exact historical layout before reading fields, use checked length arithmetic, and validate the complete current representation after writing it.

Published transition code is immutable along with its source and destination schemas. Editing a live transition could let two accounts reach the same destination version under different semantics. A defect is repaired with a new version.

Account decoder history usually cannot be pruned safely because a Solana program cannot enumerate all of its accounts and prove that no old version remains. Instruction versions may be explicitly retired. Retiring account history requires an external completeness proof or an acknowledged risk of stranding old state.

An upgrade authority can bypass Pina and deploy arbitrary code. Publication guarantees therefore end at the program’s upgrade governance boundary.

Consequences

Developers manage desired schemas and semantic exceptions, not version integers.
Version bytes add permanent storage and request overhead to opted-in contracts.
Historical decoders and transitions increase SBF size and compute cost.
The runtime stays no_std and allocator-free.
PinaPod remains responsible for byte-layout validity; Pina owns versioning, rent, dispatch, publication, and compatibility policy.
Appended optional process accounts can share a discriminator; all process changes outside the proved relationship use a new discriminator.

Amendment (ABI 0.21, 2026-09-30)

ABI 0.21 narrows where the envelope applies and stops converting events. It also records the migration policy in the manifest alone and compares schemas by what they store. Accounts are unchanged: they keep the envelope, adjacent transitions, the reserved Migrate instruction, and MigrateAccount.

Instructions are snapshots unless they opt in

Under an auto policy, a plain #[instruction] is recorded with exactly one version and "envelope": false, and its wire format stays [discriminator][payload]. The snapshot still gates wire-breaking changes: the macro fails the build when the struct drifts from it, pina migrations create replaces it while nothing is published, and after publication create refuses a payload change (PublishedPayloadChanged) and asks for a new discriminator. Appending optional accounts extends a published version in place, for snapshot-only and enveloped instructions alike, because an older request that omits the new slots still parses.

#[instruction(discriminator = X, migrations)] opts one instruction into the envelope and the adjacent transitions this ADR describes. An envelope cannot be added to or removed from a published instruction (EnvelopeAddition, EnvelopeRemoval), because existing clients send exactly the bytes they were generated with. Programs published under ABI 0.20, where every recorded instruction was enveloped, keep their wire format by adding migrations to each such attribute; the build error names the contract.

The reason is cost without benefit. Most instructions never change their payload under one discriminator, yet the envelope added a byte to every request, a wire change at every CPI call site, and a conversion path to every dispatch. A new discriminator already expresses a new payload, and an instruction that must keep accepting old payloads can still opt in.

The dispatcher normalizes, not the handler

#[discriminator(entrypoint)] routes each instruction the manifest records with an envelope through its generated process_versioned, which converts a historical payload to the current layout and then calls ProcessAccountInfos::process_from_version(self, data, source_version: u32). The default implementation forwards to process(data), so handlers read current bytes without normalizing them. normalize_instruction_data returns a NormalizedInstruction with as_bytes(), source_version(), and was_migrated() for manual dispatch.

A transition that adds an instruction argument is always manual: a zero-filled argument is indistinguishable from a zero the client sent, so the developer chooses the value. Transitions that only move or drop bytes stay automatic.

Events are versioned, not migrated

The Events section above no longer describes the implementation. The program emits only the current version. Changing a published event appends a version with its own schema and no transition, so there is no migrations/transitions/event_* directory. normalize_event_data, with_current_event_data, MigratableEvent, and CurrentEventData are removed from the runtime.

Clients decode each version with its own schema. The IDL lists every earlier version as its own event node, <Event>V<n>, with its own codec, and the program-level log parsers (parse<Program>EventsFromLogs in TypeScript and Dart) dispatch each record by discriminator and version and fail closed on a version no generated event describes. Decoded records carry only the fields their version emitted, with no sourceVersion or wasMigrated. Client generation reads the event history from the IDL rather than the migration manifest. pina_test::HistoricalEvent fixtures are decoded with the generated event for their version.

Projection could only present a field an old event never emitted as zero, and a manual event transition left generated clients unable to decode that version at all. Decoding each version with the schema that emitted it is exact and needs no transition code.

Document shape

The ABI 0.21 manifest stores each fact once: the contract key is the identity, the wire codec is implied by abiVersion, instruction histories record "envelope": false when they have no envelope, and a process slot omits every field left at its default. The publication ledger keeps only what nothing else records. ADR 0009 records those changes.

The policy has one home

The auto policy and the version width are recorded only in migrations/manifest.json. pina migrations create --auto <POLICY> sets the policy (true, false, or a comma-separated list of accounts, events, and instructions), and --version-type <u8|u16|u32> sets the width; omitting a flag keeps what the manifest records. The width can change only while no receipt or pending record exists. pina.toml no longer accepts [migrations].auto or [migrations].version_type, and every command that reads pina.toml names the flag that replaces a retired key. The VersionTypeChanged and AutoPolicyChanged checks are gone, because there is no second copy to disagree with, and so is the IDL refusal for a pina.toml policy without a manifest: without a manifest there is no policy anywhere.

Keeping the policy in two places meant a check that they agreed, and the manifest still had to be regenerated before the pina.toml copy meant anything. One copy removes both, and it keeps the property this ADR relies on: macros read only checked-in bytes.

Because a hand edit could still change the manifest’s auto, the macros refuse to strip a recorded envelope that way. A declaration without a migrations token that the recorded policy does not cover, but that the manifest records with an envelope, fails the build, just as an explicit migrations = false does.

Equivalent spellings are one schema

Drift is decided by what a field stores. pina_abi::wire_type maps spellings that store the same bytes under the same reading to one form — the Pod* integer and boolean wrappers to their native names, Address to [u8; 32], PodString<N> to String<N>, and PodVec<T, N> to Vec<T, N>, recursively — and DataSchema::same_wire compares layouts, field names, and those forms. The macros’ snapshot check, pina migrations create and check, rename pairing, and the automatic transition planner all use it, so a respelling consumes no version and fails no build. Recorded spellings and pinned hashes are never rewritten. Types that only share a width, such as u64 and i64, still need a manual transition, because reading one as the other reinterprets a live value.

Rejected alternatives

Version fields written by developers

This duplicates state already known by the migration history and permits source constants to drift from deployed bytes.

Per-contract version widths

This saves at most a few bytes while making generic inspection, generated dispatch, configuration, and publication harder to reason about. A program-global width is a simpler permanent contract.

Loader-only migration

A loader cannot adapt historical instruction payloads. It may also retain a borrow when resize or CPI needs exclusive access.

Configurable account-list adapters

User-selected compatibility modes make the authorization boundary depend on configuration and make two Pina programs interpret the same diff differently. Pina instead ships a small set of versioned, tested proofs. The initial proof covers only an identical prefix plus an optional suffix.

Pina Book