Observability

An app wired to the socle ships its createLogger output (above a per-transport level), its unhandled errors and its React crashes to Axiom, each event enriched with a shared context. The generic code lives in @workspace/observability; an app only maps its env to a config, feeds the context and wires the pieces at boot. See Wiring per app for each app.

Principles

• One logging API. createLogger(name) from @workspace/shared/logger stays the only way to log. registerLogTransport is the extension point; call sites never change when a transport is added.
• One ingestion path, no SDK. Everything goes through createAxiomIngest (a batched fetch client with no dependency). No @axiomhq/* package in any app: the player's bundle budget on old devices rules it out, and a single path keeps behaviour identical across apps.
• No React, no env in the package. React 19 root handlers are plain functions typed structurally; configs are validated by parseAxiomIngestConfig and the env → config mapping lives in each app.
• Axiom is optional. No config → no transport, no ingest, nothing else changes. Tests and CI never set a token.
• No personal data by default. The context carries no identity unless an app adds userId explicitly.

The socle: @workspace/observability

One entry point per module, no barrel.

Data flow

createLogger("API").warn("…", ctx)
  ├─ console.warn(…)                          ← always, if above setLogLevel
  └─ every registered LogTransport
       └─ createIngestLogTransport            ← drops below minLevel
            └─ { kind: "log", level, name, message, ...context.get(), ...ctx }
                 └─ createAxiomIngest.ingest  ← batched POST to Axiom

window "error" / "unhandledrejection"  ─┐
React root handlers                      ├─ logger.error(…) → same path
process "uncaughtException" (Node)      ─┘

Error capture goes through a logger rather than straight to the ingest, so every other transport (console, in-app debug panels) sees the same events.

Canonical context keys

A log's own context wins over the bag: logger.warn("…", { route: "/x" }) overrides route for that event. Keys kind, level, name and message are reserved and cannot be overridden. Keep contexts flat.

Per-request context on a server (route, status, duration) travels in the log call's own fields, never in the shared bag: the bag is process-wide and requests are concurrent. The api keeps one request-scoped bag on top (src/lib/request-context.ts, AsyncLocalStorage), merged over the shared bag by its transport: that is where actorId lives for the span of an impersonated request.

Levels

• The global setLogLevel gates everything, console included; each transport then applies its own minLevel on top.
• Axiom transports ship warn and above (the default). Use info only for a targeted measurement worth paying ingest for.
• error: unexpected failures, uncaught errors, React crashes. warn: degraded but handled (expected mutation errors, SDK timeouts, React recoverable errors).

Guard-rails

• A transport never calls createLogger. createIngestLogTransport drops synchronous re-entry, but it cannot see an asynchronous loop.
• onError passed to createAxiomIngest never logs through createLogger while the ingest transport is registered. A failed flush would log, the log would be ingested, the next flush would fail again: one event per failed flush, forever. Drop the error, or write it to a channel that does not reach Axiom (an in-app debug buffer).
• Transports never throw out. createLogger wraps each transport call; the ingest transport also swallows its own failures.
• Dedup + cap keep a crash loop from flooding the dataset; one signature is reported once per session.
• Flush on the way out: pagehide with flush({ keepalive: true }) in browsers (covers bfcache and tab close better than beforeunload); flush() before exit on servers.
• Browser tokens are public. Use an ingest-only token scoped to one dataset per env; server tokens are distinct from browser ones. Rotate on abuse (a proxy in apps/api is future work).

Wiring per app

apps/app (Vite SPA)

• src/lib/env.ts — VITE_AXIOM_TOKEN, VITE_AXIOM_DATASET, VITE_AXIOM_EDGE (blank = unset) go through parseAxiomIngestConfig. Both token and dataset unset → env.axiom === null; one without the other, or an edge with a scheme, fails at boot.
• src/lib/observability-context.ts — getObservabilityContext(): the app's lazy context bag, seeded with app, release (BUILD_ID, the first 7 chars of BUILD_SHA), sessionId and host.
• src/lib/observability.ts — installAppObservability({ config, context, target, onIngestError }): creates the ingest and registers createIngestLogTransport only when config is set; always installs installGlobalErrorCapture (logger window) so global errors also reach the dev-tools log panel; returns { reactRootHandlers, flush, uninstall }.
• src/mount-application.tsx — calls installAppObservability right after getEnv(), before the first log; onIngestError writes to the dev-tools log buffer (pushLog), which never goes back to Axiom; flushes with keepalive on pagehide; passes reactRootHandlers to createRoot. It installs installDomTranslationGuard (logger dom-translation) on the same bag before the first render, and LanguageSyncProvider sets language next to <html lang>.
• src/hooks/wire-observability-identity-sync.ts — wireObservabilityIdentitySync(source, getUserId, context): router-agnostic (getCurrentRoute + subscribe), merges userId and route into the bag now and on every route change. useRouterWithAuth calls it once at router creation with an adapter over the router's onResolved event, and updates userId on sign-in / sign-out.

React root handlers: apps/app passes onUncaughtError and onRecoverableError only. Crashes caught below the root are logged where they are caught, so passing onCaughtError too would report each one twice:

• Route error components (RouteError, AuthenticatedRouteError, ExportError) only render the fallback; src/lib/router.ts sets defaultOnCatch: logRouteCatch (src/lib/route-catch-logging.ts, logger router), which logs render and loader errors as error with their componentStack, ApiErrors as warn with code and status, and skips errors forced by the boundary dev tools. Since __root has an error component, this covers almost every crash under the router.
• In-page boundaries (ErrorBoundary, section / item / cell boundaries) log what they catch themselves, with richer context, and skip forced errors too.

The former @axiomhq/js + @axiomhq/logging + @axiomhq/tanstack-start setup was dropped for the single ingestion path. What went with it: the router_navigation / router_performance events of observeTanStackRouter. route stays on every event through the context; navigation timings are out of scope with Web Vitals and tracing, and would come back as targeted createLogger calls if needed.

apps/learn (Vite SPA)

The same shape as apps/app, without the dev-tools buffer:

• src/lib/env.ts — the three VITE_AXIOM_* variables through parseAxiomIngestConfig, null when token and dataset are both unset.
• src/lib/observability-context.ts — the bag seeded with app: "learn", release (__BUILD_ID__, the first 7 chars of BUILD_SHA), sessionId and host; src/providers/learn-router.tsx merges route on every resolved navigation and userId on sign-in / sign-out.
• src/lib/observability.ts — installLearnObservability: the ingest and its transport when configured, installGlobalErrorCapture always; a failed flush is dropped, never logged.
• src/main.tsx — installs it before the first render, flushes on pagehide, passes onUncaughtError + onRecoverableError to createRoot; the router's defaultOnCatch (src/lib/route-catch-logging.ts) logs what route boundaries catch.

apps/player (Vite, SSR + legacy host)

• src/lib/telemetry/player-observability.ts — installPlayerObservability(), called once by main.tsx before the first render; null without Axiom or outside a browser, and then nothing is registered. It builds the single ingest shared by perf and logs, the context bag (app, sessionId, host, bundle, release = env.buildId, the first 7 chars of BUILD_SHA unless VITE_PLAYER_BUILD_ID overrides it, then the boot dealCode), registers createIngestLogTransport (warn+) and installGlobalErrorCapture, installs installDomTranslationGuard (logger player:dom-translation) on the bag, and owns the one pagehide listener: it runs the perf deal summary, then flushes with keepalive, so everything rides one batch.
• Loop guard: the ingest's onError writes to the app debug source, and the Axiom transport skips every player:debug:* logger — debug-panel diagnostics stay local, and a failed flush can never enqueue another event. The debug panel's own transport coexists unchanged.
• React: createRoot gets onUncaughtError + onRecoverableError; onCaughtError is left out because ErrorBoundary already logs what it catches (one event per crash, not two). The SSR markup is remounted, not hydrated, so there is no hydrateRoot to wire.
• createPerfTelemetryClient(observability) merges the device facts into the shared context and closes the deal on page hide; usePerfTelemetryContext merges dealCode, teachingMode and phase from their owners, so logs and perf events carry the same context.
• Config: src/lib/runtime-env.ts + server/env.ts + src/lib/telemetry/observability-config.ts. The same three VITE_AXIOM_* variables, RUNTIME on the SSR host (forwarded through the injected runtime-env element) and BUILD-TIME on the legacy host (VITE_AXIOM_* in apps/player/.env.legacy.local, baked into the bundle). The server validates them at startup and fails fast; the client's getAxiomIngestConfig() treats an absent or malformed pair as disabled, so a mis-filled env never breaks the player.
• Ads: the Ezoic client logs ready failed (reason: script-error | timeout, elapsedMs) and ready timing (elapsedMs) at warn through adsLogger, so both reach Axiom through the player transport.

apps/api (Node server + pg-boss worker)

• src/boot-observability.ts is the entry point's first import. When AXIOM_TOKEN + AXIOM_DATASET (server token, distinct from the browser ones; AXIOM_EDGE optional) are set, it registers createIngestLogTransport (warn+, context app: "api", role, release from API_RELEASE) and installProcessErrorCapture. Unset, nothing is registered and logs stay on stdout only.
• index.ts flushes on SIGTERM / SIGINT before closing.
• Per-request fields (method, path, status) travel on the log call's context, never in the shared bag. Ingest failures go to stderr, never back through createLogger.

apps/web and apps/admin (Next) — not wired

Neither app ships to Axiom, on purpose. apps/web is a static export (output: "export"): Next server instrumentation (instrumentation.ts, onRequestError) never runs in production, only under next dev and next build. apps/admin is internal, with no deploy and no users. If the public site ever needs its client errors, the player's wiring is the model: a root client component that creates the ingest, registers createIngestLogTransport, installs installGlobalErrorCapture on window and flushes on pagehide, plus a global-error.tsx that logs what it receives.

Adding another transport

Do not touch createLogger; register a transport at boot, before the first log.

import { registerLogTransport } from "@workspace/shared/logger";

registerLogTransport((level, name, message, context) => {
  // forward somewhere else; never call createLogger from here
});

registerLogTransport returns an unsubscribe, useful in tests; production code registers once and never unregisters.

Tests

• .env.test files never set an Axiom token, so Vitest and e2e runs never ship events. Don't add one.
• Unit-test wiring with a fake fetch (or a fake ingest) and a plain EventTarget as the capture target.

Player perf telemetry

The player ships animation perf events through the shared ingest. perf-telemetry.ts is a pure aggregator (createPerfTelemetry({ ingest, context, now })), create-perf-telemetry-client.ts wires it, perf-telemetry-provider.tsx subscribes the sampler and mode tracker, and usePerfTelemetryContext receives dealCode, teachingMode, phase, motionProfile and animationMode from their owners.

Events

Every event — perf and log alike, since both read the same context bag — carries the session context: app, sessionId (random per page load, no identity), bundle, release (first 7 chars of the bundle's BUILD_SHA), dealCode, teachingMode, phase, host, userAgent, viewportWidth, viewportHeight, dpr, hardwareConcurrency, deviceMemory, prefersReducedMotion, language, performanceMode (measured: full | reduced | minimal), motionProfile (applied: full | reduced | minimal), animationMode (the user's setting: auto | full | reduced | minimal) — and no personal data. Count animationMode != "auto" to see how many users force a profile.

Percentiles are nearest-rank on the sorted per-deal values. Expected volume: about one perf.summary per animation name per deal (play-card, bid, collect-trick, reveal, bid-drawer-open → ~5 events per deal), plus raw samples only on the slow tail and a handful of mode events per session.

Webhook pipeline logs

The webhook pipeline (ingest → worker → handlers → admin) is server-side (apps/api, packages/database) and uses plain createLogger calls, the same style as account.repository.ts above — no new API.

• No console.*. Every log goes through createLogger("<namespace>") from @workspace/shared/logger.

• No logger.child(...). createLogger doesn't expose a child API and this convention doesn't add one. Namespaces below are flat strings picked at each call site, same spirit as "stores via Provider, not getState" — a fixed convention beats a magic hierarchical API.

• Meta is a flat object, never nested. Canonical keys — include whichever are available, never rename them:

  { webhookId, provider, externalId, eventType, attempts?, durationMs? }

Namespaces (fixed, non-hierarchical)

Levels (fixed)

• info: persisted, claimed, done, skipped.
• warn: retryScheduled, duplicate (second ingest of the same event), missing webhook secret outside prod.
• error: exhausted, unexpected, a handler throwing an untyped error.
• debug: idempotence noop (incoming payload older than the persisted row).

Examples

// webhooks:ingest
const logger = createLogger("webhooks:ingest");
logger.info("persisted", { webhookId, provider, externalId, eventType });
logger.warn("duplicate", { webhookId, provider, externalId, eventType });

// webhooks:worker
const logger = createLogger("webhooks:worker");
logger.info("claimed", { webhookId, provider, eventType, attempts });
logger.warn("retryScheduled", { webhookId, provider, eventType, attempts });
logger.error("exhausted", { webhookId, provider, eventType, attempts });

// webhooks:handler:<eventType>
const logger = createLogger("webhooks:handler:user.created");
logger.info("done", { webhookId, provider, externalId, eventType, durationMs });
logger.debug("skipped: stale payload", { webhookId, provider, externalId, eventType });

// webhooks:admin
const logger = createLogger("webhooks:admin");
logger.info("done", { webhookId, provider, eventType });

// webhooks:boot
const logger = createLogger("webhooks:boot");
logger.warn("webhook secret missing", { provider });