Mock Services

Stand up protocol stubs entirely in Starlark — no Dockerfile, no sidecar, no python -m http.server dance. Faultbox starts an in-process listener for each declared interface, answers traffic on the wire protocol your SUT expects, and emits every handled request into the event log so assertions see it.

See RFC-017 for the design rationale and RFC-019 for how the protocol stdlib ships embedded in the binary.

When to use a mock vs. the real service

Mocks are stubs. They answer requests with canned data so the SUT boots and reaches the code paths you want to test. They are not production-faithful simulations.

Two flavours

Mock services split along protocol semantics:

1. Request/response protocols — HTTP, HTTP/2, TCP, UDP, gRPC — use the generic mock_service() builtin with routes={}.
2. Stateful / message-broker protocols — Kafka, Redis, MongoDB — use protocol-specific stdlib constructors loaded from @faultbox/mocks/<name>.star.

The split is purely at the Starlark surface: the same Go runtime infrastructure powers both. Stdlib constructors are thin Starlark wrappers that translate protocol-specific kwargs (topics=, state=, collections=) into the opaque config= map on mock_service().

The generic primitive

mock_service(name, *interfaces,
    routes:     dict = {},     # protocol-specific pattern → response
    default:    MockResponse = None,
    tls:        bool = False,
    config:     dict = {},     # opaque, used by stdlib wrappers
    depends_on: list = [],
)

Returns a ServiceDef — interchangeable with real services in fault(), events(), assertions, env-var references, etc.

Response constructors

json_response(status=200, body={...}, headers={})
text_response(status=200, body="...", headers={})
bytes_response(status=0, data="raw bytes")
status_only(code)
redirect(location, status=302)
grpc_response(body={...})
grpc_error(code="UNAVAILABLE", message="...")
dynamic(fn)

dynamic(fn) wraps a Starlark callable that receives a request dict (method, path, headers, query, body) and returns a response. Used for JWT signing, per-request flag lookups, anything where the canned answer depends on the request.

Handlers also receive raw_body (Starlark bytes), state (a frozen snapshot) and state_revision (an integer starting at 0 for each test). In typed gRPC mocks, body is decoded using protobuf JSON: proto field names (user_id), 64-bit integers as strings, enums as names, and bytes as base64 strings. Ordinary scalar defaults and empty lists/maps are included; absent optional fields and unselected oneofs are omitted. Well-known types follow their protobuf JSON representation. Without descriptors, body remains a raw byte-carrying string, as it does for other protocols.

Change mock state during a test

For services that read and cache configuration during boot, use per-test initial state instead of changing it in the body:

test("toggles_on", body=scenario,
     mock_state={geoconfig.name: {"toggles": {"enabled": True}},
                 wd2.name: {"status": "approved"}})

mock_state is keyed by mock service name, with concrete state dicts as values. Each value replaces that mock’s declared state= for this test. It is copied/frozen at declaration and applied at revision 0 before the mock listener starts, so dependent services see it during startup. Unspecified mocks retain their declared initial state. Every test and plan leaf resets independently; body-time set_state() changes never modify these defaults. Overrides emit mock.state_initialized events before service startup.

The lifecycle is: initialize mocks with test overrides → start dependents → setup= → body → teardown. Existing setup= remains after service startup. Unknown/non-mock targets fail before anything starts. A reused service that depends (directly or transitively) on an overridden mock is rejected: disable reuse for that consumer so cached boot configuration cannot survive across profiles. Unrelated reusable infrastructure can stay running.

mock.set_state(state) atomically replaces the whole state dict. Values may be dicts with string keys, lists, strings, integers, finite floats, bools or None. Faultbox copies and freezes them: later edits to the caller’s dict do not change the mock. Each dynamic invocation retains one snapshot even if the test switches state while that handler is running. A call already in flight may therefore still return the previous response.

load("@faultbox/mocks/grpc.star", "grpc")

def answer(req):
    if req["state"].get("down", False):
        return grpc.unavailable("upstream unavailable")
    key = str(req["body"]["id"])
    return grpc.response({"id": key, "name": req["state"]["users"][key]})

config = grpc.server(
    name = "config",
    interface = interface("main", "grpc", 9001),
    descriptors = "proto/config.pb",
    state = {"users": {"42": "old"}},
    services = {"/example.config.ConfigService/GetSetting": grpc.dynamic(answer)},
)

def test_aggregate_lag():
    # Start the business operation and check the old aggregate here.
    config.set_state({"users": {"42": "new"}})
    # Subsequent handler invocations now see the new aggregate.
    config.set_state({"down": True})
    # Exercise the upstream-error fallback here.

Use test steps/events to decide when to switch state, or a test-body sleep() for an explicit wall-clock lag. State changes are only allowed from test bodies, including their parallel branches, not during spec loading or from mock handlers. Handlers must return responses without mutating shared state. Each replacement emits mock.state_changed with revision and the JSON state; simultaneous replacements are ordered by their revision numbers. Every test and every choose() leaf starts again from state= at revision 0.

Mock implementation errors fail the test

mock.encode_error, mock.decode_error, mock.resolve_error and mock.dynamic_error prevent PASS, even if the SUT handles the resulting error successfully. CLI/JSON diagnostics name the mock, method and error (MOCK_ENCODE_ERROR, etc.). Static typed responses are validated at spec load, including wildcard routes and fallbacks for known descriptor methods. Explicit grpc.error(...) responses and HTTP error statuses remain valid test inputs and do not invalidate a run.

HTTP / HTTP/2 mocks

Routes match "METHOD PATH" with * (single segment) and ** (any segments) globs.

kp = jwt_keypair(kid = "test-1")

auth = mock_service("auth",
    interface("http", "http", 8090),
    routes = {
        "GET /.well-known/openid-configuration":      json_response(200, {
            "issuer":   "http://auth:8090",
            "jwks_uri": "http://auth:8090/.well-known/openid-configuration/jwks",
        }),
        "GET /.well-known/openid-configuration/jwks": json_response(200, jwt_jwks(kp)),
        "POST /token":                                dynamic(lambda req: json_response(200, {
            "access_token": jwt_sign(kp, {"sub": req["query"].get("user", "anon")}),
            "expires_in":   3600,
        })),
        "GET /health":                                status_only(204),
    },
)

jwt_keypair() / jwt_sign(keypair, claims) / jwt_jwks(keypair) are the low-level EdDSA primitives. For the common OIDC-issuer case, reach for the higher-level jwt.server() helper from @faultbox/mocks/jwt.star, which stands up the JWKS routes and gives you a .sign(claims) method.

HTTP/2 uses the same route table format. Protocol is "http2" instead of "http". Without TLS, served over h2c (cleartext HTTP/2). With TLS, served over h2 with ALPN.

TCP mock

Per-connection handler reads one newline-terminated line, matches it against patterns as byte prefixes, writes the configured response, closes the connection.

legacy = mock_service("legacy",
    interface("main", "tcp", 9000),
    routes = {
        "PING\n":    bytes_response(data = "PONG\n"),
        "VERSION\n": bytes_response(data = "2.0.0\n"),
    },
    default = bytes_response(data = "ERR unknown\n"),
)

For non-line-framed protocols (Cassandra CQL frames, custom binary), use the matching protocol’s stdlib mock if available, otherwise run the real service.

UDP mock

Default behavior is swallow-and-record: every datagram emits a mock.recv event but no response is written. Matches the StatsD / metrics-sink pattern where the SUT fires datagrams and you assert on receipt count.

statsd = mock_service("statsd", interface("main", "udp", 8125))

def test_metrics_emitted():
    api.public.post(path = "/order", body = {...})
    assert_eventually(events()
        .where(service = "statsd", op = "recv")
        .count() >= 3)

When routes={} is set, datagrams whose prefix matches receive the response written back to their source address.

gRPC mock

Routes match the full method path "/pkg.Service/Method" with /pkg.Svc/* (any method on a service) and /** (catch-all) globs.

flags = mock_service("flags",
    interface("main", "grpc", 50051),
    routes = {
        "/flags.v1.Flags/Get":  grpc_response(body = {"enabled": True, "variant": "B"}),
        "/flags.v1.Flags/List": grpc_response(body = {"flags": [{"name": "rollout", "enabled": True}]}),
        "/flags.v1.Flags/Fail": grpc_error(code = "UNAVAILABLE", message = "backend down"),
    },
)

Without a .proto file, responses are encoded as google.protobuf.Struct — the wire shape that reflection-based and loose-decoding clients accept. This works for clients that don’t compile proto stubs (typed Node clients, Python grpc with generic descriptors, hand-written Go that decodes into structpb.Struct).

It does not work for clients with compiled proto stubs — the standard Go pattern where pkg.NewServiceClient(conn).Method(ctx, &pkg.Request{...}) expects a specific generated message type back. For that case, use the typed gRPC mock below (v0.9.0+).

grpc_error() accepts canonical status codes by name ("OK", "CANCELLED", "UNKNOWN", "INVALID_ARGUMENT", "DEADLINE_EXCEEDED", "NOT_FOUND", "ALREADY_EXISTS", "PERMISSION_DENIED", "RESOURCE_EXHAUSTED", "FAILED_PRECONDITION", "ABORTED", "OUT_OF_RANGE", "UNIMPLEMENTED", "INTERNAL", "UNAVAILABLE", "DATA_LOSS", "UNAUTHENTICATED") or by numeric value.

Typed gRPC mocks (@faultbox/mocks/grpc.star) — v0.9.0+

For SUTs with compiled proto stubs (the standard Go gRPC pattern), load your protoc-generated FileDescriptorSet at spec time and let Faultbox encode typed responses against it on the wire. One Starlark file, no Go binary, no Dockerfile — RFC-023.

Step 1 — Build a FileDescriptorSet

Faultbox does not ship protoc and does not parse .proto files directly. You generate a .pb descriptor file once via your existing proto build pipeline:

protoc \
    --include_imports \
    --descriptor_set_out=./proto/all_upstreams.pb \
    proto/example/config/*.proto \
    proto/example/config_facade/*.proto \
    proto/example/user/*.proto

--include_imports is important when your protos depend on each other. The standard google.protobuf.* well-known types (Timestamp, Empty, Any, Struct, Duration, FieldMask, wrappers) are pre-registered by Faultbox automatically, so you don’t need to include them in the .pb.

Monorepo pattern: customers commonly maintain proto in a dedicated repo that publishes both .proto source and pre-built .pb artifacts. Point descriptors= at the published artifact — no protoc invocation in your faultbox spec or CI.

Step 2 — Declare the mock

load("@faultbox/mocks/grpc.star", "grpc")

config_service = grpc.server(
    name        = "config-service",
    interface   = interface("main", "grpc", 9001),
    descriptors = "./proto/all_upstreams.pb",
    services    = {
        "/example.config.ConfigService/GetSetting": {
            "response": {
                "id":       42,
                "name":     "primary",
                "scope":  "default",
                "currency": "USD",
            },
        },
        "/example.config.ConfigService/ListSettings": {
            "response": {
                "scopes": [
                    {"code": "default", "name": "Default"},
                    {"code": "other", "name": "Other"},
                ],
            },
        },
        "/example.config.ConfigService/AdminUpdate": {
            "error": {"code": "PERMISSION_DENIED", "message": "admin only"},
        },
    },
)

Each entry in services is one of:

Step 3 — Compose with fault rules

Fault recipes from @faultbox/recipes/grpc.star still apply to the typed mock’s interface — same wire-level behavior as the real upstream:

load("@faultbox/recipes/grpc.star", "grpc_faults")

config_unstable = fault_assumption("config_unstable",
    target = config_service.main,
    rules  = [grpc_faults.unavailable()],
)

deadline_exceeded = fault_scenario("deadline_exceeded",
    scenario = create_order,
    faults   = [config_unstable],
    expect   = lambda r: assert_eq(r.status, 504),
)

Multi-service mocks (order-service Phase 1 shape)

When one mock process backs multiple gRPC services on different ports — the order-service pattern — declare one grpc.server() per service sharing the same descriptor set:

descriptors = "./proto/all_upstreams.pb"

config_service = grpc.server(
    name        = "config-service",
    interface   = interface("main", "grpc", 9001),
    descriptors = descriptors,
    services    = { "/example.config.ConfigService/GetSetting": {...}, ... },
)
user_service = grpc.server(
    name        = "user-service",
    interface   = interface("main", "grpc", 9003),
    descriptors = descriptors,
    services    = { "/example.user.UserService/GetUser": {...}, ... },
)

api = service("api",
    interface("public", "http", 8080),
    binary = "./bin/order-service",
    env = {
        "GRPC_CONFIG_ADDRESS":   config_service.main.internal_addr,
        "GRPC_USER_SERVICE_ADDRESS": user_service.main.internal_addr,
    },
    depends_on = [config_service, user_service],
)

Each mock is its own service with its own interface, so fault rules target them independently.

Reflection support (grpcurl debugging)

When descriptors= is set, the mock automatically serves the gRPC reflection v1 service. grpcurl works against the mock out of the box — no client-side .proto file needed:

$ grpcurl -plaintext localhost:9001 list
grpc.reflection.v1.ServerReflection
example.config.ConfigService

$ grpcurl -plaintext localhost:9001 describe example.config.ConfigService
example.config.ConfigService is a service:
service ConfigService {
  rpc GetSetting(GetSettingRequest) returns (Setting);
  ...
}

$ grpcurl -plaintext -d '{"id":1}' localhost:9001 example.config.ConfigService/GetSetting
{
  "id": "42",
  "name": "default",
  "scope": "default",
  "currency": "USD"
}

Protocol compatibility

• gRPC (standard HTTP/2 + protobuf) — works natively.
• Connect / connect-go with WithGRPC() — verified; uses the same gRPC wire and hits the same handler.
• Connect gRPC-Web or pure Connect protocol — not yet supported (would need a separate Connect handler alongside the grpc-go server). File an issue if you need this.

What’s not in v1 (RFC-023 v2 candidates)

• Streaming RPCs. Unary only. Server/client/bidi streaming deferred until a customer explicitly needs it.
• Custom error details (google.rpc.Status with typed details). Plain status-code errors via {"error": {"code": "X"}} only.
• Load-time response-shape validation. Response-dict typos surface at request time (unknown field "cityid" naming the field and the target message). Load-time validation is a v2 ergonomic win.
• Raw .proto ingestion. v1 requires a FileDescriptorSet (.pb file from protoc). Parsing .proto directly is v2.

Go-binary fallback (for exotic cases)

If the typed Starlark mock can’t express what you need — streaming RPCs, dynamic server-side logic that grpc.dynamic() can’t capture, or protocols beyond unary gRPC — the Go-binary escape hatch from v0.8.6 still works:

upstreams = service("upstreams",
    interface("config_service",   "grpc", 9001),
    interface("user_service", "grpc", 9003),
    binary = "./bin/upstream-mocks",
    healthcheck = tcp("localhost:9001"),
    seccomp    = False,
)

The Go binary implements the standard Unimplemented*Server interfaces and imports your real *.pb.go. See the v0.8.6 release notes for a worked example. For new deployments, start with the typed Starlark mock above; drop to Go only when you hit something it can’t express.

Kafka stdlib mock

load("@faultbox/mocks/kafka.star", "kafka")

bus = kafka.broker(
    name      = "bus",
    interface = interface("main", "kafka", 9092),
    topics    = {"orders": [], "payments": []},
)

Backed by github.com/twmb/franz-go/pkg/kfake — a battle-tested in-process Kafka broker. Real kafka-go, franz-go, sarama clients connect, produce, consume, join consumer groups. Topics are seeded with empty partitions; pre-populating with messages requires producing them from the spec itself.

Redis stdlib mock

load("@faultbox/mocks/redis.star", "redis")

cache = redis.server(
    name      = "cache",
    interface = interface("main", "redis", 6379),
    state = {
        "config:max_retries": "3",
        "config:timeout_ms":  "5000",
        "flag:new_ui":        "true",
    },
)

Backed by miniredis — full RESP2 semantics for the 80% of Redis usage that’s string cache + counters + hashes + sorted sets. Real go-redis, redigo, raw RESP clients connect and GET/SET/DEL/INCR/EXISTS/TTL operate on seeded state.

Streams, pub/sub, and Lua scripting are inherited from miniredis but not all commands behave identically to a real Redis. Specs that depend on those features should run the real server.

MongoDB stdlib mock

load("@faultbox/mocks/mongodb.star", "mongo")

users_db = mongo.server(
    name      = "users-stub",
    interface = interface("main", "mongodb", 27017),
    collections = {
        "users": [
            {"_id": "1", "name": "alice", "role": "admin"},
            {"_id": "2", "name": "bob",   "role": "user"},
        ],
    },
)

Hand-written BSON OP_MSG + OP_QUERY responder. The official mongo-driver/v2 completes its handshake (hello / isMaster / buildInfo) and find/findOne returns seeded documents. Writes (insert/update/delete) acknowledge with ok: 1 but the data is discarded — this is a read-through stub. Unknown commands return ok: 1 (lenient, so unusual driver chatter doesn’t fail tests).

For round-trip CRUD with persistence, run the real server.

TLS

auth = mock_service("auth",
    interface("https", "http", 8443),
    tls    = True,
    routes = {...},
)

When tls=True, Faultbox:

1. Generates an ECDSA P-256 mock CA on first use (lazy, one per run).
2. Signs leaf certificates for mock and proxy with SANs ["localhost", "host.docker.internal", <service-name>, 127.0.0.1, ::1].
3. Writes the CA bundle to ${TMPDIR}/faultbox-ca-<timestamp>.pem.
4. Wraps the listener with TLS — HTTP, HTTP/2 (ALPN h2), gRPC.

The path is exposed as auth.ca_path at spec load, so it can be used in env values or as a volume source. For example, a host binary can set env={"SSL_CERT_FILE": auth.ca_path}; a container can set volumes={auth.ca_path: "/certs/mock-ca.pem:ro"} and env={"SSL_CERT_FILE": "/certs/mock-ca.pem"}.

SUTs trust the CA by reading that file. In binary mode, point your HTTP client’s RootCAs at it; in container mode, mount it into the SUT container at a path the client picks up.

TLS is supported on HTTP, HTTP/2, gRPC. Other protocols reject tls=True; if you need TLS Kafka/Redis/MongoDB, run the real service.

Kafka record events

The Kafka mock emits these events after sending successful broker replies:

Repeated fetches can emit the same offset more than once. Fetch is not proof of application processing; combine it with DB/API assertions. Commit offsets name the next record, so commit 3 covers offsets below 3. Fetch requests do not identify a consumer group; group attribution is available on commit events. Gzip/snappy/lz4/zstd record batches are decoded; transaction control records are excluded. A produce acknowledgement does not prove transaction commit. External clients using acks=0 emit mock.kafka.produce_unacknowledged, which makes no persistence claim. Malformed client traffic emits mock.kafka.protocol_error; observation-decoder failures emit mock.observation_error and invalidate the run.

Events

Request/response mocks emit events for handled requests; Kafka emits the record and commit events listed above. Example schema:

{
    "service":   "auth",
    "interface": "http",
    "kind":      "mock",
    "type":      "mock.GET /.well-known/openid-configuration/jwks",
    "fields":    { "status": "200", "body_size": "412", ... },
}

Use events().where(service=..., op=...) to assert call counts, call order, request contents — same DSL as for real services and syscalls.

Composing mocks with real services

Mocks coexist with real services in the topology:

load("@faultbox/mocks/redis.star", "redis")

auth  = mock_service("auth",  interface("http", "http", 8090), routes = {...})
cache = redis.server("cache", interface = interface("main", "redis", 6379),
                              state = {"feature:new": "true"})

api = service("api",
    interface("public", "http", 8080),
    image      = "myapp:latest",
    depends_on = [auth, cache],
    env = {
        "JWKS_URL":  "http://auth:8090/.well-known/openid-configuration/jwks",
        "REDIS_URL": "redis://cache:6379",
    },
)

The auth:8090 and cache:6379 spellings above are what you write; they are not what the container dials. A mock is an in-process listener on the host — it has no container and no entry in Docker’s DNS — so Faultbox rewrites those addresses in the SUT’s environment to reach the host instead (host.docker.internal:<port> for a container consumer, loopback for a host binary). Write the address as if the mock were a peer; the substitution makes it true.

v0.18.0. Before that release neither half of this worked: mock listeners bound host loopback, which nothing on the docker0 bridge can reach, and the injected address was a bare localhost:<port> that resolved to the SUT’s own container. mock_service() was unusable from a containerized SUT. Host-binary SUTs were unaffected.

The bind interface follows FAULTBOX_PROXY_BIND, which governs proxy and mock listeners alike.

Faulting a mock

Mocks accept the same fault() rules as real services:

auth = mock_service("auth", interface("main", "http", 8090), routes = {...})

jwks_unavailable = fault_assumption("jwks_unavailable",
    target = auth.main,
    rules  = [error(path = "/.well-known/**", status = 503)],
)

A proxy starts before dependent SUTs. Environment references to the mock are routed through it; mock.main.proxy_addr also works explicitly. Without a fault it forwards to the mock. An error/drop rule may short-circuit the call. TLS mocks keep TLS on both sides of the proxy using the shared CA.

A protocol-fault scope with no observed hit emits FAULT_NOT_FIRED and marks an otherwise passing test fault_bypassed. A scenario requiring faults to fire fails instead. Unresolved proxy_addr references fail before the SUT launches.

What mocks deliberately don’t do

• Stateful CRUD with persistence. MongoDB writes are dropped; Kafka topic message seeding is empty. Persistence is what real services give you.
• Production-shaped throughput. Mocks are correctness-first; benchmarks belong in real services.
• Schema validation. Routes match by pattern, not by request shape. Use dynamic() if you need to reject malformed requests.
• Replace WireMock / Prism / mountebank. Faultbox mocks exist to stand up dependencies of the system under test, not to be a general-purpose mock platform.