Serialization

JSON by default, camelCased on the wire so a C# OrderId and a Java orderId are the same field. That default is what makes cross-language messages work without anybody configuring anything.

using var mq = await AceMqConnection.ConnectAsync(url);              // JSON
using var mq = await AceMqConnection.ConnectAsync(url, new XmlCodec());

Per consumer, when one queue carries a different format:

await mq.ConsumeAsync<Order>("legacy", ConsumerOptions.Defaults().As(new XmlCodec()), Handle);

What ships

Those five need nothing but the framework — the core's whole dependency list is two Microsoft packages.

JsonCodec accepts text/json as well as application/json, and a message with no content type at all. text/json is a legacy alias predating the registration of application/json, and it is still what some older producers and a few gateways stamp on a body that is plainly JSON. Go, Python and Ruby have always accepted it and Java now does too; refusing it here made a message four other libraries could read unreadable in this one. Only the accept set widens — JsonCodec.ContentType is still application/json, because writing an alias pushes the problem onto whoever reads next.

Formats that need an outside library get their own package, so an application that wants one does not acquire the rest:

That is every format the Java library has, plus payload encryption — which is in a package for the same reason and one of its own besides, covered in Encrypting the payload.

What each costs you. YamlDotNet has no dependencies of its own. Google.Protobuf has none either. Apache.Avro brings Newtonsoft.Json and System.CodeDom. Tomlyn brings System.Text.Json at a higher version than the core pins, and NuGet resolves to the higher one — so an application taking the TOML package moves from System.Text.Json 8.0.5 to 10.0.2. Nothing breaks, but it is the kind of thing worth knowing before it happens, and it is exactly why these are separate packages rather than part of the core.

AceMq.Amqp.Xml costs nothing: System.Xml is part of the framework and System.Text.Json arrives with the core, which already pins it. It is a separate package for the other reason these exist — so the core's list of formats does not grow by one every time somebody wants a different one.

AceMq.Amqp.Crypto brings BouncyCastle.Cryptography, which has no dependencies of its own. It is there because AES-GCM has to come from somewhere on netstandard2.0, and it is in its own package so a consumer publishing plaintext never sees it.

Protocol Buffers

dotnet add package AceMq.Amqp.Protobuf

using var mq = await AceMqConnection.ConnectAsync(url, new ProtobufCodec());

Or on one queue, while the rest of the service stays on JSON:

await mq.ConsumeAsync<OrderPlaced>(
    "orders", ConsumerOptions.Defaults().As(new ProtobufCodec()), Handle);

The content type written is application/x-protobuf, the same as Java's. Three are read, plus a suffix:

The asymmetry is deliberate. A wider read set costs a string comparison; a narrower one silently refuses a message it could have read perfectly well, and the refusal looks like a broken producer rather than a fussy consumer. ProtobufCodec.ReadableContentTypes is the list, if you want to assert on it.

It works with generated types, not with your own classes. Protobuf encoding is defined by a .proto and the code generated from it; there is no reflection-based fallback, because bytes produced that way would not be readable by anything else that speaks protobuf. A plain class is refused at the call rather than encoded into something only this library could read:

AceFatalException: <anonymous type> is not a generated protobuf message. This codec
encodes types generated from a .proto schema; there is no reflection-based fallback...

Generate the types with Grpc.Tools, which runs protoc at build time and contributes nothing at runtime:

<PackageReference Include="Grpc.Tools" Version="2.68.1" PrivateAssets="all" />
<Protobuf Include="order.proto" GrpcServices="None" />

A malformed body is fatal, not retryable. The same bytes fail the same way on every attempt, so the message is dead-lettered rather than retried forever.

Avro

dotnet add package AceMq.Amqp.Avro

Avro messages are not self-describing: a reader must already hold the schema the writer used, or the bytes cannot be read. That is the design difference from JSON, and it is why the codec is constructed with a schema rather than created empty.

There are two ways to say where the schema comes from, and the choice matters more than it looks.

A fixed schema

var codec = AvroCodec.Of(schemaJson);

Small, fast, nothing extra to run — and the writer's schema is whatever the reader happens to have compiled in. The moment a producer adds a field, every consumer still holding the old schema reads the new bytes wrongly, and Avro will not always notice. Sound only where the producer and every consumer are released together.

Content type avro/binary, and the body is nothing but Avro — no framing.

A registry

var codec = AvroCodec.Registered(new DbSchemaRegistry(Connect), schemaJson);

This writes the schema's identifier into the front of every message, so a reader looks up exactly what the writer used and lets Avro resolve it against its own. This is what makes adding a field safe, and it is the mode to use unless there is a reason not to.

[0x00][schema id: 4 bytes, big-endian][avro body]

The layout Confluent's clients use, and the same bytes the Java library writes — any of the three reads the others. Content type application/vnd.acemq.avro.

The registry has to be shared across processes. InMemorySchemaRegistry issues ids per process, so a message written by one and read by another refers to an id the second never issued — use DbSchemaRegistry or your own.

Both directions of evolution are tested: a V2 producer read by a V1 consumer, and a V1 producer read by a V2 consumer. The second only works because the added field has a default — without one Avro cannot invent a value and the read fails.

The reader schema

Resolution has two halves. One is the schema the writer used, which comes off the registry. The other is the schema this consumer was written against — the reader schema — and it is codec.ReaderSchema. Given both, Avro reconciles them: a field the writer added is skipped, and a field the writer has not started sending yet arrives as the reader schema's default. The handler sees the shape it was compiled against, whichever version produced the message.

By default the reader schema is the schema the codec was built with, which is what Python's reader_schema and Ruby's reader_schema: also default to. Two calls change it.

A different reader schema, for a service that publishes one version and consumes another. Only schema is registered; readerSchema is never written:

var codec = AvroCodec.Registered(registry, publishedSchemaJson, consumedSchemaJson);

That is Java's registered(registry, readerSchema) under the same name.

No reader schema at all, so every message is read with the shape its writer gave it:

var codec = AvroCodec.Registered(registry, schemaJson).WithoutReaderSchema();
// codec.ReaderSchema is null

The opt-out, and what Java and Go do when a registered codec is given no reader schema. It is for a consumer that wants to see exactly what a producer sent — a bridge, an inspector, a dead-letter drainer reading versions it was never compiled against. What it costs is what resolution buys: a field the writer added arrives rather than being skipped, and a field the writer has not started sending is absent rather than defaulted, so the target type has to match what the writer actually wrote, version by version.

WithoutReaderSchema() returns a new codec and leaves the one it was called on alone. On a fixed-schema codec it throws: nothing is framed, so there is no writer's schema to read with and the codec's own is the only one there is.

What it encodes

A type generated by avrogen (ISpecificRecord), a GenericRecord, or a plain class mapped onto the schema you supplied. There is no schema generation from a class: Avro's model is that the schema comes first.

Schema resolution

Handed the writer's schema and the reader's, Avro reconciles the two: a field the writer added that the reader does not declare is skipped, and a field the writer omitted is filled in from the reader's default. Handed only the writer's, there is nothing to reconcile, and the record arrives in the shape it was written.

So there is one rule, and it is the same rule in all five AceMQ libraries:

Resolution happens when the library has a reader schema to resolve onto.

What differs between the languages is where a reader schema comes from, and therefore how often there is one. Nothing about the bytes differs.

This is not an inconsistency waiting to be flattened. A library that resolves and a library that does not are both right about the same bytes — they are answering different questions, because only one of them was told what the reader expects.

The case that bites is a field the writer removed that the reader declares with a default. With resolution, the field arrives carrying that default. Without it, the field is simply absent: a missing key, whatever the language calls one. A consumer written against the reader schema then reads a value that was never on the wire, or fails to read a field it is sure it declared, and which of those happens is decided entirely by whether a reader schema was in play.

The other direction is the one people expect to be dangerous and is not. A field the writer added that the reader does not declare is skipped under resolution and present without it, and either way the fields the reader does declare come back correct — the unknown field does not shift the ones after it.

Both cases are pinned, with the bytes, in tests/AceMq.Amqp.Tests/fixtures/avro-resolution-fixtures.json, which every AceMQ library carries a copy of. It records the decoded value under each behaviour, as resolved and writerShape, and which library lands on which.

Asking for resolution in .NET

// Resolves: every message is reconciled with the schema this consumer holds.
// This is the ordinary Registered(...) call -- there is nothing to switch on.
var codec = AvroCodec.Registered(registry, schemaJson);

// Resolves onto a schema other than the one it writes. Only `schemaJson` is
// registered; `readerSchemaJson` is never written.
var consumer = AvroCodec.Registered(registry, schemaJson, readerSchemaJson);

// Does not resolve: the record arrives in the shape the producer wrote it.
var asWritten = AvroCodec.Registered(registry, schemaJson).WithoutReaderSchema();

Reach for the first unless there is a reason not to. Being able to redeploy a producer without its consumers is the whole point of putting a schema id on the front of the message, and resolution is the half of that which happens on the read side. See the reader schema above for what each of the three does to codec.ReaderSchema.

One edge worth knowing before you meet it: WithoutReaderSchema() throws on a fixed-schema codec, the one built with AvroCodec.Of(...). Nothing is framed there, so there is no writer's schema to read with and the codec's own is the only one there is — turning resolution "off" would quietly change nothing. The other edge is the decode target. Without a reader schema the shape changes from message to message, so a plain class mapped onto one version reads the versions that match it and nothing else; decode into GenericRecord and look at record.Schema.Fields if what you are reading is a queue of mixed versions.

YAML

dotnet add package AceMq.Amqp.Yaml

using var mq = await AceMqConnection.ConnectAsync(url, new YamlCodec());

For messages a person will read as much as a program: a configuration change broadcast to a fleet, a deployment instruction, a command replayed by hand from a dead-letter queue.

It costs more to parse than JSON and is a poor choice for high volume. It earns its place where somebody will actually look at the message.

Written in block style, which is the whole reason to pick it — flow style would produce something all but indistinguishable from JSON:

service: orders
version: 1.4.2
regions:
- eu-west-1
- us-east-1

Keys are camelCased, as with JSON, so a C# Service and a Java service are the same key. Unknown keys are ignored, so a producer adding a field does not break a consumer that has not been redeployed.

It never answers for a message with no content type

This is the part worth knowing. YAML is a superset of JSON, so this parser reads JSON bytes quite happily. If it volunteered for untyped messages it would give the right value from the wrong codec — and the mistake surfaces much later, as traffic recorded under a format nobody sent.

So CanDecode(null) is false, and it claims only application/yaml, application/x-yaml, text/yaml, text/x-yaml and anything ending +yaml — the four names YAML has gone by, which is the same set the Java, Python and Ruby codecs take. In a CompositeCodec, put JSON first and let YAML take only what is labelled.

Hostile input

A YAML tag naming a type is the deserialisation attack this format is known for. YamlDotNet does not honour one by default, and a test pins that rather than trusting it to stay true.

The billion-laughs shape — a small document whose aliases name an enormous graph — is bounded, because the parser shares aliases rather than materialising them. A 480-byte document nominally naming 10¹⁰ nodes reads in milliseconds. That is measured in a test, because if it stopped being true this codec would be a denial of service anybody could post.

TOML

dotnet add package AceMq.Amqp.Toml

using var mq = await AceMqConnection.ConnectAsync(url, new TomlCodec());

The same audience as YAML — a message a person reads and edits — with the ambiguity removed:

service = "orders"
enabled = true
regions = ["eu-west-1", "us-east-1"]

One way to write a string, no significant indentation, and no Norway problem. In YAML country: NO is the boolean false; in TOML an unquoted NO is not a value at all, so the mistake is a parse error rather than a country turning into false somewhere downstream. Where a human edits the message and a machine acts on it, that matters more than terseness.

Duplicate keys are refused rather than resolved — silently taking the first or the last would be a message that means something other than it looks like.

Keys are camelCased on the wire and read case-insensitively, so a message hand-edited by somebody who capitalised one still reads.

The shape has to suit it

TOML is a table format: a message body must be an object at the top level. A bare list or number is not a TOML document, and the codec says so rather than emitting something that is not TOML:

AceFatalException: List`1 cannot be written as TOML: ... TOML is a table format, so a
message body has to be an object at the top level. Use JsonCodec where the payload is
a list, a scalar, or a deep tree.

Deep nesting reads poorly too. Where the payload is a tree rather than a table, JSON is the honest answer.

Like YAML, it never volunteers for a message with no content type.

XML

dotnet add package AceMq.Amqp.Xml

using var mq = await AceMqConnection.ConnectAsync(url, new InteropXmlCodec());

Here because most estates have something that speaks XML and will not be rewritten, and a messaging library that cannot talk to it forces a translation layer nobody wants to own. New services should publish JSON; this exists so the ones that cannot are not a special case.

It writes application/xml, and reads application/xml, text/xml and any +xml suffix type — application/soap+xml, application/atom+xml, a vendor type. Like YAML and TOML it never volunteers for a message with no content type: XML is rarely what arrives unannounced, and a codec that guessed wrong there would turn a readable message into a rejected one. That case belongs to JsonCodec and BytesCodec.

Two XML codecs, and which one you want

The second row is the whole point. XmlSerializer binds element names case-sensitively, so Java's <orderId> does not bind to a C# OrderId.

Up to 0.3.0 it did not complain about that. It returned an object with every field at its default. A service that reached for the core codec to read a Java queue saw empty orders and no errors, which is the worst of both: the message was gone and nothing said so.

Since 0.5.0 XmlCodec throws instead:

AceFatalException: nothing in the body bound to Order: all 6 of its elements and
attributes were unknown to XmlSerializer, which matches names case-sensitively. This
is what a body written by the Java, Go, Python or Ruby AceMQ library looks like here
-- camelCase elements against PascalCase members. Read it with
AceMq.Amqp.Xml.InteropXmlCodec, from the AceMq.Amqp.Xml package, which reads what the
other four write.

It is an AceFatalException, so the message is dead-lettered rather than retried forever: the same bytes bind no better on the next attempt.

The refusal is narrow on purpose. It fires only when the document had content and none of it reached a member. A document with some elements bound and some unknown still decodes — that is what a producer adding a field looks like, and breaking forward compatibility would be a worse bug than the one being fixed. An empty document that legitimately decodes to an all-default object still decodes too, because nothing in it was unknown.

Why not just mark XmlCodec obsolete

That was the other candidate, and it was rejected. XmlCodec is correct for .NET talking to .NET and for a document that has to match an XSD; an obsolete warning on a correct use is noise, and noise is what teaches people to suppress warnings. The failure was silence, not existence, so silence is what was fixed — and the exception arrives at exactly the moment it is useful, naming the codec that would have worked, in front of somebody holding a stack trace.

The names still differ so that a consumer with both using AceMq.Amqp; and using AceMq.Amqp.Xml; gets a choice rather than a CS0104.

This is a breaking change: code that relied on getting an empty object back now gets an exception. That is the intent — there is no version of "relied on getting an empty object back" that was working.

No document type declaration, ever

A body carrying <!DOCTYPE is refused, and there is no way to relax it. The same decision Java, Python and Ruby took, for the same reason: a message body has no legitimate use for a DTD — it is one document produced by a serializer at the other end, and none of Jackson, encoding/xml, ElementTree or REXML writes one — so the configuration would only ever be wrong.

The usual justification given for this is external entities, and it is the wrong one here. There is no XmlResolver, so file:///etc/passwd is already inert. Internal entity expansion is not. A billion-laughs bomb needs no network access and no readable file; it expands inside the parser. Measured against this runtime rather than read off a table, with DtdProcessing.Parse:

Exponential in the depth, and the six-level case is still inside the SDK's own default 10,000,000-character entity cap — so that cap is not what saves a consumer. The measurement is a test, not a comment: if a future runtime made expansion inert, it fails rather than quietly becoming untrue.

The refusal is in two layers. The body is scanned for <!DOCTYPE before a parser sees it, so the failure names what is wrong instead of surfacing an XmlException about a security setting the caller never chose. Then the reader is created with DtdProcessing.Prohibit and XmlResolver = null, which catches what the scan cannot see — a UTF-16 body, where <!DOCTYPE is not a UTF-8 substring but the reader sniffs the encoding and would read the DTD perfectly well. Both are tested.

Both list shapes decode

Jackson wraps a list in an element of its own; Go's encoding/xml repeats the sibling:

<lines><lines>widget</lines><lines>gasket</lines></lines>   <!-- Jackson, over a POJO -->
<lines>widget</lines><lines>gasket</lines>                  <!-- encoding/xml, and Jackson over a Map -->

Both are real output from those libraries — the fixture in AceMq.Amqp.Xml.Tests was produced by running them — neither is going away, and both decode into the same List<string>. A one-item list is a third case, because <lines>widget</lines> on its own is indistinguishable from a scalar; the declared member is the only thing that can tell them apart, so the unwrapping is driven by the target type rather than guessed from the document. This codec writes the repeated-sibling form, which Jackson reads as well.

XML has no types

Every leaf arrives as text. <totalCents>4250</totalCents> is the four characters 4250, not a number, and <paid>true</paid> is the four characters true. Decoding reads numbers, booleans and enums back out of that text, so an int or a bool member works; decode into string and you get the text unchanged. There is no null either — an empty element reads as the empty string, and nothing here invents xsi:nil, because neither Jackson nor encoding/xml writes it.

The root element is named after the payload's type, the way Jackson uses the class's simple name. Coming back, the root's name is ignored: it names the message rather than being part of it, which is how Java's <Order> and Go's <order> read into the same type.

Attributes arrive as ordinary members beside the elements, and namespaces are cut back to the local name — a consumer wants the field, not the URI it was declared in, which is also what Jackson gives reading XML into a Map.

Changing format without stopping

var codec = CompositeCodec.Of(new JsonCodec(), new XmlCodec());

The first codec encodes; any of them can decode. So a service can start publishing JSON while still reading the XML already in its queues, and the old codec comes out once nothing is left that speaks it.

Choosing a format from configuration

var codec = CodecRegistry.ByName(settings.Format);   // "json", "xml", "string", "bytes"
CodecRegistry.Register("avro", () => new MyAvroCodec());

"text" names the same codec as "string": it is what Java and Python call it, so a format setting shared with services in those languages works here unchanged.

Registration is explicit. Nothing scans assemblies, so a format is available because something registered it — not because a package happened to be installed.

Encrypting the payload

TLS protects a message between your process and the broker. It does nothing about the message sitting in a queue, in the broker's storage, or in a backup of it.

dotnet add package AceMq.Amqp.Crypto

using AceMq.Amqp.Crypto;

var keyring = Keyring.Of(EncryptionKey.Generate("2026-01"));
var codec = EncryptedCodec.Wrapping(new JsonCodec(), keyring);

using var mq = await AceMqConnection.ConnectAsync(url, codec);

The body is encrypted. Headers are not. The envelope, routing key and application headers stay readable, because the broker routes on them and the library reads them. Anything secret belongs in the payload.

Rotating a key

var keyring = Keyring.Builder()
    .Add(EncryptionKey.Generate("2025-07"))      // still readable
    .Current(EncryptionKey.Generate("2026-01"))  // used for new messages
    .Build();

Keep the old key until the queues holding its messages are drained. Removing it too early produces:

this message was encrypted with key '2025-07', which is not on the keyring

EncryptedCodec.KeyIdOf(body) says which key a message needs without decrypting it, which is normally how that gets diagnosed.

What the construction is

AES-256-GCM with a 128-bit tag, and a fresh nonce per message.

0xAE  0x01  len  key identifier   12-byte nonce   ciphertext + 16-byte tag

Byte for byte what the Java, Python, Ruby and Go libraries write, and what they read. An encrypted message crosses languages; that is the point of the framing being this one and not one of its own.

The header — magic, version, length and identifier — is authenticated but not encrypted: GCM binds all of it as associated data, so a key identifier edited to point a consumer at a different key makes the message fail to open rather than quietly opening as something else. A fresh nonce per message means two identical payloads produce different ciphertexts; it matters more than that, because two messages under one key and one nonce do not merely lose some strength under GCM, they leak their difference outright.

A body that does not authenticate produces one error, whether the key was wrong or the body was altered:

this message did not decrypt with key '2026-01'. Either that is not the key it was
written with, or it was altered after it was written

The two are told apart by nothing — no separate message, no separate type, no earlier check that only one of them reaches. An error that said which it was would be a padding oracle with better manners.

Where AES-GCM comes from on netstandard2.0

It does not. System.Security.Cryptography.AesGcm arrived in .NET Core 3.0 and has never existed on netstandard2.0 or .NET Framework, which is why this library wrote AES-256-CBC with HMAC-SHA-256 up to 0.3.0 — sound cryptography, and a format no other AceMQ library could read, under a content type that said they could.

AceMq.Amqp.Crypto gets GCM from BouncyCastle, which has it on netstandard2.0. One implementation on every target rather than the built-in AesGcm on modern .NET and something else on .NET Framework: two cryptographic code paths behind one wire format is a divergence that only ever shows up at runtime, on one target, in somebody else's queue. The full reasoning, including what was rejected, is in src/AceMq.Amqp.Crypto/AceMq.Amqp.Crypto.csproj, next to the target it explains.

Bodies written before 0.5.0

Releases up to and including 0.3.0 wrote a .NET-only framing:

[version:1][keyIdLength:1][keyId][iv:16][ciphertext][hmac-sha-256:32]

Decode still reads it, and EncryptedCodec.KeyIdOf still names its key. The two are never confused and never guessed at: the family framing begins 0xAE, this one begins 0x01, and a body that is neither is refused naming both. The tag is still verified before anything is decrypted, exactly as it was.

Nothing writes it any more, and this reader will be removed in 1.0.0. Those bodies decrypt in .NET and nowhere else, so a queue holding them has to be drained — or republished by a .NET consumer running 0.5.0 or later — before anything in another language can read it.

if (EncryptedCodec.IsLegacyDotNetBody(body))
{
    // written by 0.3.0; only .NET can read this one
}

Keeping a large payload off the broker

A scanned medical report is tens of megabytes. Putting it on a queue is possible and is a mistake: it fills the broker's memory, it is copied to every bound queue, it makes a dead-letter queue impossible to inspect, and it turns a broker into a filesystem with worse tools.

What travels instead is a claim check — the payload goes to a store, and the message carries the key.

var store = new FilesystemClaimCheckStore("/mnt/payloads");
var codec = ClaimCheckCodec.Wrapping(new JsonCodec(), store);

using var mq = await AceMqConnection.ConnectAsync(url, codec);

Nothing else changes. Publishers publish documents and consumers receive documents; the codec is the only thing that knows the payload went somewhere else.

Only when it is worth it

Below the threshold — 64 KiB by default, the same number as Java — the payload travels inline, exactly as it would without this codec. That matters more than it sounds: offloading a two-hundred-byte message turns one broker round trip into a store round trip and a broker round trip, so an unconditional claim check makes the common case slower to fix the rare one.

ClaimCheckCodec.Wrapping(new JsonCodec(), store, threshold: 1024 * 1024);
ClaimCheckCodec.Wrapping(new JsonCodec(), store, threshold: 0);   // offload everything

The framing says which of the two it is, and a consumer handles both without being told. That is what lets the threshold be changed, or this codec be introduced, without a flag day: messages written before the change are still readable after it, and so are messages written by a publisher that does not use it at all.

What is on the wire

0xAC  0x01  0x00  payload      inline, and identical to what the delegate wrote
0xAC  0x01  0x01  key          a claim check, the key as UTF-8

Three bytes, and the same three bytes Java, Python and Ruby write. A payload of exactly the threshold size is offloaded — the comparison is strictly less than for inline — which is asserted a byte either side of the boundary in the test suite, because "roughly a threshold" is how two libraries end up disagreeing about a message.

The content type is the delegate's, unchanged. Unlike encryption, where the bytes really are something else: a claim-checked document is still a document, it is a document that is somewhere else, and a consumer that lacks the store gets a clear failure rather than a parser error.

For an operator looking at a dead-letter queue — which object does this need, and is it still in the store?

ClaimCheckCodec.KeyOf(body)          // the key, or null when it travelled inline
ClaimCheckCodec.IsClaimCheck(body)

The stores

IClaimCheckStore is three methods, so a store in front of S3 or Azure Blob Storage is a small class:

string Put(byte[] content);   // returns the key
byte[]? Get(string key);      // null when the store no longer holds it
void Delete(string key);

Two ship:

• InMemoryClaimCheckStore — for tests, and not for anything else. The payloads are held in the publisher's heap, which is where they were going to be anyway, so it removes them from the broker and nothing else. A consumer in another process gets "the claim check is not in the store".
• FilesystemClaimCheckStore — useful where the filesystem is shared and durable, an NFS mount or a persistent volume. Writes are staged and moved into place, so a consumer fast enough to read the key before the writer finished sees the whole payload or no payload. On a container's local disk it is the in-memory store with extra steps. Keys are checked rather than trusted before they become path segments: ../../etc/passwd is a key too.

Delete is never called by the codec. Deleting on read would break the second consumer of the same message and deleting on acknowledgement would break a replay, so when a payload may be removed is a retention decision, and retention decisions belong to whoever owns the data.

Retention is the part that goes wrong

The store and the queue have different lifetimes and nothing enforces a relationship between them. A message replayed a month later carries a key, and if the store expired that key the replay produces a message nobody can read — worse than a lost message, because it looks like a message and fails deep inside a consumer rather than visibly.

So the store's retention must exceed every retention that could bring a message back: queue TTLs, dead-letter queues, and however long somebody might sit on a message before replaying it by hand. When in doubt, longer.

Schemas

A registry maps a schema to a short id, so messages carry the id rather than kilobytes of schema:

var registry = new InMemorySchemaRegistry();
var id = registry.IdFor(new SchemaDefinition("json", "order.placed", schemaText));

The definition is opaque — JSON Schema, Avro, .proto, whatever. The registry provides identity, not validation: the same schema always gets the same id, a changed one gets a different id.

InMemorySchemaRegistry is per process. Ids are issued in registration order, so two processes disagree about what an id means and a restart renumbers everything. That makes it right for tests and wrong wherever a message outlives the process that wrote it.

DbSchemaRegistry is the one to use for that:

var registry = new DbSchemaRegistry(() => new SqlConnection(connectionString));

The fingerprint column is unique, which makes registering the same schema twice idempotent even when two processes do it at the same instant: the second insert is refused and the row already there is read back.