Skip to content

Trace, replay and failure briefs

Three commands read the development capture: they need DEBUG on when the request ran, and nothing else. Each takes last for the newest capture, and --json for agents. They run in the project's interpreter (its .venv, else the running compose service), like every runner. The same three are MCP tools on karni mcp (trace, replay, explain_failure).

The output below is real: EventHub (examples/eventhub) on SQLite with KARNI_CACHE=locmem, a Celery worker on kombu's filesystem broker, an attendee paying for a booking while the provider's webhook races ahead of the checkout.

karni trace <request-id>

Every response carries X-Request-ID. Pass it (or last) and Karni prints everything that carried that id, in order: the request, the Celery tasks it enqueued, their retries minutes later, email, webhooks, change-log entries (read from the database when it is reachable, and from the files alone otherwise).

$ karni trace 0caa83ad-4e00-4135-bb12-dbe23bad6ca1
Request 0caa83ad-4e00-4135-bb12-dbe23bad6ca1: POST /payments/webhook/test/ -> 200 in 75.473 ms, as anonymous
  view  karni_payments.webhooks.webhook
  started 2026-10-21 10:34:19 UTC

     +0.000s  request.start     POST /payments/webhook/test/
     +0.049s  task.enqueued     karni_payments.tasks.process_webhook_event [974694c5]
     +0.072s  webhook.received  provider=test event_id=evt_demo_1 event_type=checkout.session.completed duplicate=False refs=karni_payments.WebhookEvent:c5a247ba-…
     +0.076s  request.end       200 in 75.473 ms
     +0.081s  task.started      karni_payments.tasks.process_webhook_event [974694c5] (pid 45087)
     +0.109s  task.enqueued     karni_payments.tasks.process_webhook_event [974694c5] retry 1 eta 2026-10-21T10:34:21.083220+00:00
     +0.117s  task.retried      karni_payments.tasks.process_webhook_event [974694c5]: Retry in 1s: UnmatchedWebhookEventError('Webhook event test:evt_demo_1 (checkout.session.completed) matches no local Payment. It will be retried.')
     +1.115s  task.started      karni_payments.tasks.process_webhook_event [974694c5] retry 1 (pid 45087)
     +1.142s  webhook.matched   provider=test event_id=evt_demo_1 event_type=checkout.session.completed refs=karni_payments.Payment:990fbd5c-…,karni_payments.WebhookEvent:c5a247ba-…
     +1.154s  task.enqueued     events.listeners.send_booking_confirmation [378e08b3]
     +1.205s  task.succeeded    karni_payments.tasks.process_webhook_event [974694c5] in 90.246 ms
     +1.228s  task.started      events.listeners.send_booking_confirmation [378e08b3] (pid 45087)
     +1.232s  task.succeeded    events.listeners.send_booking_confirmation [378e08b3] in 4.099 ms

Linked requests (their events name the same rows):
  78c906a5-857f-4a1f-8836-826ea15012a9  POST /api/bookings/3c82f75e-…/pay/  payment.checkout  [karni_payments.Payment:990fbd5c-…]

The webhook's task ran in the worker (its pid), failed to find the payment, retried a second later under the same id, matched the payment the checkout request created, and queued the confirmation listener. Linked requests come from shared refs: the checkout named the same payment. Following it:

$ karni trace 78c906a5-857f-4a1f-8836-826ea15012a9
Request 78c906a5-857f-4a1f-8836-826ea15012a9: POST /api/bookings/3c82f75e-…/pay/ -> 200 in 56.722 ms, as attendee@example.com
  view  events.viewsets.BookingViewSet (events-booking, pay)
  started 2026-10-21 10:34:20 UTC

     +0.000s  request.start     POST /api/bookings/3c82f75e-…/pay/
     +0.025s  payment.checkout  provider=test amount=50.00 currency=usd refs=karni_payments.Payment:990fbd5c-…
     +0.035s  task.enqueued     events.listeners.send_booking_confirmation [9643a5fc]
     +0.056s  request.end       200 in 56.722 ms
     +0.765s  task.started      events.listeners.send_booking_confirmation [9643a5fc] (pid 45087)
     +0.826s  email.sent        to attendee@example.com "Your tickets for Open Source Summit"
     +0.829s  task.succeeded    events.listeners.send_booking_confirmation [9643a5fc] in 65.041 ms

The email went out with X-Karni-Request-ID: 78c906a5-…. Change-log entries of audited models appear in the same timeline (audit.change, audit.batch). With no capture, karni trace says so and how to enable it (E921).

karni replay <request-id> [--as <user>]

Re-sends the captured method, path, query, body and plain headers through Django's test client, in-process (no server), force-authenticated as the captured user or --as someone else (an email, a username or a primary key, or --anonymous for none). It runs against the current code, so no restart is needed after an edit. It prints the status, a short redacted body excerpt and what changed against the captured status.

$ karni replay 9788cc05-6397-46b8-9834-4b0c7467f75f
Replayed GET /api/events/2369b584-…/dashboard/ as attendee@example.com (rolled back)
  same as captured (403)
  body: {"detail":"Access denied: this row is readable but the action requires Owned('organizer') | HasRole('staff')."}
  new request 50903b6a-b0ff-45ec-b13a-7b9ff91fb358: karni trace 50903b6a-b0ff-45ec-b13a-7b9ff91fb358

$ karni replay 9788cc05-6397-46b8-9834-4b0c7467f75f --as organizer@example.com
Replayed GET /api/events/2369b584-…/dashboard/ as organizer@example.com (rolled back)
  was 403, now 200
  body: {"bookings":3,"tickets":5,"capacity":120,"revenue":"255.00"}
  new request e4fd02cd-e569-4678-b1a2-1ac805c58fc7: karni trace e4fd02cd-e569-4678-b1a2-1ac805c58fc7

Writes are rolled back by default. The replay runs inside transaction.atomic() on every database and rolls back: on_commit work (queued listeners, webhook tasks) is discarded, mail goes to Django's in-memory backend and is listed (email (not sent): to …) instead of sent, and a task the view enqueues directly runs eagerly inside the same transaction. Calls the view makes to outside services are not contained. --commit keeps the writes and uses the real mail backend and broker.

Redacted values cannot be replayed. The capture never stored the password. Replaying the literal "[redacted]" could set a password to it, so the replay stops and names what to supply:

$ karni replay 72f5ccf4-068e-4804-8eec-5084b1a10468
error: E923: These values were redacted when the request was captured, so they cannot be replayed: password. Supply them (--set password=<value>) or leave them out (--drop-redacted). See https://cbsshekhawat18-lab.github.io/karni/errors/#e923

$ karni replay 72f5ccf4-068e-4804-8eec-5084b1a10468 --set password=wrong-one
Replayed POST /api/auth/login/ as anonymous (rolled back)
  was 200, now 401
  body: {"detail":"No active account found with the given credentials"}
  new request 6ac58d6c-ed48-4d0b-b535-ddd268ae2380: karni trace 6ac58d6c-ed48-4d0b-b535-ddd268ae2380

--set takes a dotted body path (card.token=tok_1) or query:<name>. The value is parsed as JSON when it can be. Credential headers are never replayed (the client is force-authenticated), and uploaded files are not captured, so a multipart replay sends the fields only. karni replay refuses with DEBUG off (E920): it acts as any user on the configured database.

karni explain --failure <id>

An unhandled exception writes a failure snapshot. Its id is on the request record and in karni trace. karni explain --failure (a failure id, the failed request's id, or last) prints it as a brief. Read it before the traceback:

$ karni explain --failure last
Failure 9a434564036b: GET /api/events/ff9ab5c2-…/dashboard/ by organizer@example.com (id 16419f82-…; roles organizer)
  at 2026-10-21T10:35:50.933316+00:00  request 6247f4b2-bc14-47ce-83db-97557ddf9d00

Exception: TypeError: unsupported operand type(s) for /: 'NoneType' and 'NoneType'
Karni code: none

Where: events/viewsets.py:51 in EventViewSet.dashboard (project)
    51 | "per_ticket": f"{totals['revenue'] / totals['tickets']:.2f}",
  locals:
    self = <events.viewsets.EventViewSet object at 0x10e7d6990>
    request = <Request GET /api/events/ff9ab5c2-…/dashboard/>
    pk = ff9ab5c2-9f04-4aed-b552-332702627f73
    event = {"__model__": "events.Event", "pk": "ff9ab5c2-…", "id": "ff9ab5c2-…", "created_at": "2026-09-18T10:27:00+00…
    confirmed = <QuerySet events.Booking, not evaluated>
    totals = {"tickets": null, "revenue": null}
    capacity = 120
  call path (project and Karni frames, outermost first):
    karni/api/viewsets.py:846 in BaseViewSet.dispatch
    events/viewsets.py:51 in EventViewSet.dashboard

View: events.viewsets.EventViewSet (resource events-event, action dashboard)
  serializer karni.api.serializers.EventsEventSerializer
  kwargs {"pk": "ff9ab5c2-9f04-4aed-b552-332702627f73"}
Grants: none on this resource

Queries: 7 in 0.847 ms, 0 repeated statements

Next steps:
  1. Open events/viewsets.py:51 (EventViewSet.dashboard): it raised there.
  2. After the fix: `karni replay 6247f4b2-bc14-47ce-83db-97557ddf9d00 --as organizer@example.com` re-runs this request against the current code (was 500).
  3. `karni trace 6247f4b2-bc14-47ce-83db-97557ddf9d00` shows what else the request set in motion.

An event with no confirmed bookings makes both totals None. After the fix:

$ karni replay 6247f4b2-bc14-47ce-83db-97557ddf9d00
Replayed GET /api/events/ff9ab5c2-…/dashboard/ as organizer@example.com (rolled back)
  was 500, now 200
  body: {"bookings":0,"tickets":0,"capacity":120,"revenue":"0.00"}
  new request 73eb70e4-be4e-4327-a77c-a11ddfaa958d: karni trace 73eb70e4-be4e-4327-a77c-a11ddfaa958d

The brief adds the Karni code and karni explain <code> when the exception carries one, a hint for common database errors (a missing table: karni migrate), and the N+1 suspects when statements repeated. A Celery task that fails for good gets a snapshot too (its arguments by parameter name, redacted like everything else). karni explain E104 is unchanged. Pass a code or --failure, not both (C320).

Codes

Code Meaning
E920 karni replay needs DEBUG on
E921 nothing was captured for this request id (or it is not an id)
E922 no failure snapshot with this id
E923 a redacted value must be supplied with --set (or --drop-redacted)
E924 the user to replay as cannot be found
W920 RequestIdMiddleware runs after django-structlog's RequestMiddleware
C320 to C323 the CLI side: explain usage, a malformed --set, a project karni without these commands, an uncoded failure

Each is explained by karni explain <code> and in error codes.