Wildmarks

WildNet

The server underneath Wildmarks.

A game server and network engine we wrote for one game. It began in C#, was moved to Rust from scratch, and continues in Rust. This page explains why it exists, how it works, what it has achieved so far, what the test suite checks, and where it is going.

Why we built it

An online world has one hard requirement that most software does not: hundreds of people act on the same state at the same time, and all of them have to agree on what happened. A sword that drops for two players is a bug that ruins an economy. A chest that empties after a crash is a bug that ruins trust. A client that can tell the server where it is standing is a cheat waiting to be written.

We wanted a server that makes those bugs impossible by construction instead of catching them case by case. That meant three decisions up front: the server is the only authority, every durable change is written to a log before the server answers, and the world is streamed to each player by what they can see rather than by what they ask for. Off-the-shelf servers each gave up one of those, or priced the fix as a service. So WildNet is ours, and it is shaped by Wildmarks rather than the other way round.

The engine contains no game rules. Accounts, combat, loot, chat and progression live in the game's own server module on top of it. WildNet only knows about tables, reducers, subscriptions and the wire.

How it works

Tables in memory

The world is a set of typed tables with primary and secondary indexes, held in memory. Reads never wait on disk.

Reducers

A client never writes a row. It asks for a named command, a reducer, which runs on a single writer thread, sees who sent it, and either commits all of its changes or none of them.

The commit log

Every committed transaction is appended to a log on disk before the client gets its answer. On restart the engine replays the log and arrives at the same world. A torn write at the tail is truncated, not guessed at.

Subscriptions

Clients subscribe to standing queries over a WebSocket. After each transaction they receive only the rows that changed inside their query, not a fresh copy of the table.

Interest windows

Space is divided into 32 m cells. A player's window is 7 by 7 cells around them and moves with them, so a client only receives the part of the zone it could see.

Two planes

Position updates are ephemeral: streamed at 10 Hz, never logged. Loot, inventory and progress are durable: logged, replayed, and kept across restarts. Mixing the two is the usual way a game server becomes slow or lossy.

Event streams

Combat hits, chat lines and other things that need to be seen once and not stored go through transient streams with per-identity rate limits and a bounded send queue per connection.

Identity and admission

Identity comes from a signed token bound to the connection, never from a field the client fills in. An admission gate decides what each connection may do before any reducer runs.

Exactly-once operations

Each durable operation carries an id. A client that reconnects and retries gets the original result from a ledger instead of running the reducer again. A claimed chest stays claimed.

Scheduled reducers

The engine can run a reducer at a time or on a cycle, so respawns and timers are server state rather than client timers.

Scaling and failure

Sharded hosts

One process can own several shards of the world, and several processes can share one world. Ownership of a shard is fenced: a host that lost ownership cannot keep writing.

Live handover

A player crossing a shard boundary is handed from one host to another with their durable state, over a signed exchange with explicit refusals and retry cooldowns. The client sees one continent.

Replicated log

The commit log can be replicated to a second host over a signed endpoint. The replica acknowledges, reports lag, and refuses writes from a host that no longer holds ownership.

Control plane

Shard ownership and host liveness are recorded in an external key-value store with compare-and-set, with a local witness that persists its own view so a host can answer quickly and still defer to the authority.

Snapshots and compaction

The log is periodically checkpointed into a snapshot so recovery does not replay from the beginning. Failed snapshots clean up after themselves, including when cancelled mid-write.

Schema migration

A log written by an older schema reopens under the new one. Old durable character logs from the game's early builds still load in tests.

Head to head

In August 2026 we rented a host with eight dedicated cores and ran WildNet against an established commercial engine of the same kind. Each engine was driven by its own client over its own wire protocol, through the same scenarios, with the same measurement: bytes counted at the server's port per scenario, CPU and memory sampled once a second. Each backend ran alone on the machine, with a fresh process and fresh data per repetition. The figures are medians of three repetitions.

MeasureWildNetThe other engine
Movement latency, p50, 100 to 500 bots0.3 ms1.7 to 2.5 ms
Movement latency, p99 at 500 bots1.9 ms19.8 ms
Paced throughput, 100 to 500 bots546 to 2,725 tx/s510 to 2,006 tx/s
Saturation throughput29,400 to 30,100 tx/s3,300 to 4,400 tx/s
Wire bytes per operation188 to 224 B263 to 358 B
Dense combat, 200 bots on 1,000 items, p50 / p955.3 / 8.2 ms18.7 / 20.8 ms
Dense combat, server CPU / peak memory79% / 207 MB136% / 415 MB
Retry after an abrupt process kill, wire per call1,843 B3,328 B
Saturation with every table durable11,431 tx/s4,023 tx/s
60-minute soak, server CPU average3.2%11.3%
60-minute soak, movement p95, first and last half1.1 → 1.1 ms2.5 → 2.4 ms
600,000 contested loot attempts1,000 owners, 0 duplicates1,000 owners, 0 duplicates
Ahead on every criterion of the short matrix, with equal correctness. The latency stayed flat from 100 to 500 bots; the other engine's p99 climbed tenfold over the same ladder.

The report also records two things against us, and we would rather state them than smooth them over. During the one-hour soak our memory grew by about 1 MB per minute, from the exactly-once operation ledger retaining an hour of loot commits; the growth is bounded by design, and a separate retention test the same day showed the plateau. And at the soak's gentle pace we moved more bytes on the wire than the other engine, because each result went out as its own socket write; a later measurement confirmed the cause and found no batch worth taking at that rate.

Game workload

Beyond the head-to-head, our own load generator drives the engine over the real wire with workloads shaped like Wildmarks. None of these is a capacity promise; the envelope was chosen as a reproducible test and will be replaced by numbers from the game itself.

75-cycle long run
108,000 movement events and 43,200 combat events delivered exactly once, in order, with every one of 3,600 durable results replayed unchanged after reconnect. Zero admission, backpressure or durability failures sampled during the run.
Contested loot
Every shared item had exactly one winner. Every loser received an explicit "already claimed". Both outcomes replayed identically after reconnect without running the reducer body again, and the final snapshot matched every winner.
Slow-peer isolation
With one subscriber fully stalled, a healthy subscriber in the same zone stayed exact. The stalled peer was cut off between 8,840 and 8,976 publications against the default 4,096-message guard, and nobody else noticed.
Fan-out latency
32 publishers in one zone at 10 combat events per second each, for 15 seconds, with an observer draining at a fixed rate. Delivery stayed exact at every point. At a 320 events/s drain the p95 publish-to-delivery latency met a 250 ms objective; at 280 events/s it first failed, with a 614-event backlog and every server guard still at zero.
Players per zone
500 to 800 concurrent players per zone on one server, measured with the original benchmark harness. Beyond that, the answer is another shard, not a bigger box.

What has been done

  • WildNet in C#. We wrote the first WildNet ourselves, in C#: the in-memory tables, the commit log, reducers, subscriptions, interest windows, sharding, signed handover and the replicated log, each with its own regression tests. The first Wildmarks builds ran on it.

  • The move to Rust. With Claude's help we ported the engine to Rust from scratch, slice by slice, with one rule: every slice ends with the original's regression tests green against the new engine, byte for byte on the wire. On 8 August 2026 the port matched the original on every milestone, all 24 hardening slices and the 15-slice replicated-log arc, with 666 tests passing. From that day the Rust engine is WildNet, and all work since is in Rust.

  • The head-to-head. On dedicated hardware, ahead on every criterion of the short matrix against an established engine, with equal correctness over 600,000 contested attempts and 2,400 abrupt-kill retries.

  • Five layers of game workload. Movement, zoned handover, durable loot and reconnect; combat fan-out with contested loot; a 75-cycle long run with CPU and memory observation; the slow-peer isolation curve; the paced delivery-latency objective. All exact.

  • Memory plateau proven. The ledger growth flagged in the soak was reproduced in a targeted retention test, shown to plateau, and then hardened.

  • Every line read. All 87 source files were reviewed line by line. The defects that turned up, in witness persistence, snapshot cancellation, replication receive bounds and handover retry timing, were fixed with regression tests that failed before and pass after. The full suite was green at the close, none ignored.

Tests

Every change runs the formatter, the linter on all targets, a build, and the whole suite. The counts below were taken from the source tree on 8 October 2026. Each group is one crate; the rows are its integration suites, and the group total also includes the unit tests inside the crate.

Test functions
782
Crates
10
Lines of tests
45,418
Lines of source
68,092
Integration suites
43
Ignored tests
0

Engine core

281 tests
  • tablesTyped tables, primary and secondary indexes, iteration that stays stable while rows change27
  • sharded_hostSeveral shards in one process, routing, and what happens when one is removed29
  • windowsInterest cells, the moving window, the per-window cell cap22
  • snapshotsCheckpoints, base offsets, failed and cancelled snapshots cleaning up19
  • commitlogRecord framing and torn-tail truncation15
  • databaseReplay determinism, kill and recover15
  • reducersSingle writer, atomic commit and rollback, sender identity15
  • subscriptionsStanding queries, initial snapshots, per-transaction deltas14
  • operationsThe exactly-once ledger: a retried operation returns its first result14
  • shard_ownershipOwnership fencing and compare-and-set authority14
  • remote_handoverMoving a player between hosts, refusals, retry cooldowns12
  • event_streamsTransient streams and their rate limits12
  • durabilityExact-boundary recovery and the group-commit contract11
  • ephemeralThe split between logged and unlogged state11
  • migrationReopening a log under a newer schema9
  • schedulerScheduled and cyclic reducers9
  • ledger_retentionRolling retention of the operation ledger and its memory plateau5

Server

279 tests
  • signed_wal_replicaThe signed replication endpoint, bounded refusals, disposal under load37
  • signed_handoverSigned cross-host handover end to end34
  • admission_gateConnection admission, limits, backpressure accounting26
  • production_authProduction authentication mode and its failure paths25
  • db_gateThe database protocol: handshake state machine, request ids, error frames20
  • mmo_gateWindows, streams and scheduled reducers over the wire18
  • auth_tokensSigned identity tokens and remembered-device sessions11
  • echo_gateWebSocket lifecycle and framing10
  • auth_fixturesShared authentication fixtures and cross-checks7
  • handover_exchangeThe handover request and reply exchange5
  • wal_adminOperator commands for the replicated log2
  • sdk_parityThe original C# client SDK talking to the Rust server unchanged1
  • durability_gateDurability observed from the client side1

Replication

66 tests
  • replicated_walAcknowledgement, lag reporting, fencing of a host that lost ownership40
  • replicated_wal_databaseA database running on top of the replicated log14
  • wal_replica_hostThe standalone replica process11
  • wal_operatorOperator tooling for the replica1

Protocol

63 tests
  • roundtripEncode and decode every message type and get the same bytes back26
  • rowsRow codec byte parity and bounds checks14
  • framesFrame bytes identical to the reference wire fixtures13

Control plane and load generation

93 tests
  • control_planeOwnership records, witness persistence and retry, compare-and-set against the store38
  • game_loadgenThe load generator itself: movement, handover, loot, reconnect, the fan-out curve and the latency objective30
  • bench_serverBenchmark server instrumentation21
  • bench_laneBenchmark run lanes and their output root4

The crash-host crate is a tool for killing a database mid-write during recovery tests and has no tests of its own.

What we are going for

The numbers above come from a provisional envelope and a one-hour soak. The goals below are the ones we will hold ourselves to, each with a measurement that can fail.

  1. Real traces instead of a guessed envelope. Zone population, event rates, payload sizes and acceptable lag taken from Wildmarks playtests, replacing every provisional default in the load generator.

  2. A thousand players in one zone, under a millisecond. Movement p50 below 1 ms and p99 below 5 ms at 1,000 or more concurrent players in a zone, held across several hosts rather than one process.

  3. Handover a player cannot feel. Crossing between hosts with no visible stall and no durable state lost, measured at the client, including when the target host refuses or the source dies mid-exchange.

  4. Replication on by default. A replica within a bounded lag at the measured game load, and a failover that loses no committed operation.

  5. Days, not an hour. Multi-day soaks with flat memory, zero failures and exact conservation, so the ledger plateau is observed rather than argued.

  6. Recovery in seconds. Restart from snapshot and log to a consistent world, timed on the real world size, with a bound we publish.

  7. Ship Wildmarks on it. The first playable slice of the game on this engine, then closed playtests, with the engine's numbers reported from real sessions.

Contact

Questions about the engine, or interest in using it for something that is not Wildmarks, are welcome.

[email protected]