Correlator
A request-tracing library that gives every customer request one ID and carries it through every service, language and background job, so a slow or failed request can be followed end to end.
- 23
- integration packages across JavaScript and Elixir
- 837
- shared test cases every language core must pass
- 8
- request-ID header formats read and written
The problem
A single checkout can pass through an API, a payments service, a background worker and a database, and each one writes its own logs with nothing linking them. Finding out why one request was slow or failed means guessing across systems. The standard tracing tools need agents and collectors on every server, and they handle a mixed Elixir and Node stack, or work that continues in background jobs, poorly.
What it does
Correlator gives every request one ID. It reads the ID from whichever header the request arrives with, keeps it available to all the code handling that request, stamps it on log lines, and passes it on to outgoing calls and queued jobs.
It can also write hop records: a start line and an end line for each step a request takes, such as an incoming request, a call to another service, a database query or a cache read. Each hop names its parent, so the full path and the time spent in each step can be rebuilt without comparing clocks between machines. Database hops record the table and the shape of the query with every value removed.
A service adopts it with one package and one line. Adapters for the frameworks and libraries it already uses are picked automatically, and hop records can be sent straight to Backwork, the studio's monitoring tool, so nothing extra has to be installed on the servers.
Key features
Speaks every common header
It reads and writes eight ID formats, from W3C traceparent to AWS, Google, B3 and Sentry, so a request keeps one ID as it crosses tools from different vendors.
Two languages, one behaviour
A JavaScript core for Node and Bun and a BEAM core for Elixir and Erlang pass the same shared test cases, so a mixed stack lines up instead of drifting.
Follows work into background jobs
The ID travels into queued jobs, scheduled tasks and spawned processes, where ordinary logging usually loses track of which request started the work.
Timed hops, database included
Start and end lines for every step, including queries, cache hits and misses and time spent waiting for a connection, show exactly where a slow request spent its time.
Private by default
Every incoming value is validated, sensitive-looking path segments are masked, and SQL is reduced to its shape, so traces don't leak customer data.
One package, one line
Adapters are detected rather than chosen by hand, and each part can be switched on or off in code or from the Backwork dashboard, so adoption takes minutes, not a project.
Where it stands
Not yet released: there's no public package or licence, and the specification is still a draft. It already runs inside the studio's own Inkress, FleekSite and Courier Portal services, and only the JavaScript and Elixir cores exist so far.
Under the hood: for the technically-minded
How it's built
The behaviour is defined in one written specification in which every rule has an ID. Each language implements a small core that does no network work of its own, and a shared file of test cases, generated by a reference implementation, is run against every core so the languages are checked against the same answers.
Anything tied to a third-party library is its own adapter package, so the cores stay dependency-free: Express, Fastify, Hono, NestJS, Remix, BullMQ, Postgres, Redis and Pino on the JavaScript side, and Plug, Phoenix, Ecto, Oban, Quantum, Redix, Tesla and HTTPoison on the Elixir side. A top-level package in each language bundles them, detects what a service uses, and owns the optional sender that posts records to Backwork.
Some scope lines are fixed for good: request bodies and function arguments are never recorded. Until it's published, services take a self-contained built copy into their own repository with one command, so nothing depends on an unclaimed package name.
The hard problems
Keeping two languages in agreement
Two separate cores could easily drift apart. A shared set of test vectors, each linked to the rule it checks, must pass in both languages before a change lands.
Not leaking data through traces
Query text is reduced to a value-free shape by a hand-written scanner in each language, which was fuzz-tested and fixed for a possible leak. Paths are masked only when a segment looks like an email, a long token or a JWT, after an earlier length-based rule proved wrong on real traffic.
Context lost at async boundaries
Timers, schedulers and queued jobs run with no request around them. An audit of the rollout found those gaps and closed them, and the library counts a missing context as a reported gap rather than failing the request.
Built with
- TypeScript
- Elixir / Erlang
- W3C traceparent
- Node and Bun
- Phoenix, Ecto, Oban
Building something in this space?
Work with us →