Technology note — this article reflects LangSmith SaaS, hybrid and self-hosted capabilities (the v0.13–v0.15 self-hosted line) available or announced as of July 2026. Feature maturity and edition requirements are labeled inline; deployment options and entitlements depend on plan and contract — verify against current documentation before committing an architecture.
Engineering AI for Production
In Part 1 of Engineering AI for Production, we looked at the retrieval layer and what embedding-space metrics can tell us about the systems underneath a RAG application.
Part 2 moved up the stack to the agent development loop: how prompts, evaluations, baselines and production feedback can make behavioural changes measurable and reversible.
Part 3 moves underneath that workflow.
Once tracing, evaluation and prompt management become part of the way you build and operate agents, you have another engineering decision to make:
Where should that platform actually run?
For some teams, data residency makes self-hosting the obvious choice. Others want the operational simplicity of SaaS. Hybrid models sit somewhere in between, but introduce their own questions around infrastructure ownership, data boundaries and migration.
The decision is not simply about where the software is deployed. It affects operational load, feature availability, trace-query performance, cost and how easily the architecture can change later.
In this article, Ernesto Martinez looks at the three deployment models for LangSmith, what each one really asks of an engineering team, and how to keep the observability layer portable enough that today’s infrastructure decision does not become tomorrow’s lock-in. The article then works through the decision factor by factor, including data residency, operational overhead, feature lag, scale and migration.
Engineering AI for Production · Part 3 of 5
Every team that gets serious about agents ends up operating an observability and evaluation platform. The previous post in this series described the workload: traces on every run, a versioned prompt registry, evaluation in CI, online scoring of production traffic. All of that has to run somewhere — and "where" is the first architectural decision that is genuinely hard to reverse.
LangSmith offers three answers: fully self-hosted in your own Kubernetes cluster, hybrid with the data plane in your VPC and the control plane in the vendor's cloud, and SaaS (US and EU regions). This post is the systems view of that decision — what each model actually demands, instrument-once patterns that keep the choice reversible, and the calculus we use to place workloads.
What "self-hosted" actually costs: thefootprint
Self-hosted LangSmith is a real distributed system, and honest capacity planning starts with its dependency footprint. The documented architecture (Helm chart on Kubernetes) comprises:
- ClickHouse — the trace store and the component that dominates sizing. Trace workloads are append-heavy with occasionally brutal analytical reads; disk and memory here scale with trace volume and retention, not with team size.
- PostgreSQL — transactional metadata: projects, datasets, prompts, experiments, users.
- Redis — or Valkey, supported as a drop-in replacement since v0.14 (April 2026), which matters to teams standardizing away from Redis licensing.
- Blobstorage (S3/GCS/MinIO) — payload offloading and bulk artifacts.
Two operational facts deserve more attention than they usually get. First, trace payloads have grown an order of magnitude as agents got more complex — the ecosystem's own reported figures put median payloads up ~6× and P99 into the tens of megabytes over two years. Whatever you sized for last year's chatbot is wrong for this year's multi-agent system; retention policy and payload sampling are first-class design inputs, not afterthoughts. Second, the platform is on a monthly release cadence: budget for regular Helm upgrades with real release note reading (the v0.15 line, for instance, changed how in-platform agents deploy and renamed service accounts — a mechanical upgrade, but only if someone reads the notes).
A back-of-the-envelope calculation worth doing before any procurement conversation: take your expected runs per day, multiply by average spans per run and your measured average span payload (not the vendor's, not last year's), and multiply out to monthly ingest; then apply your retention window to get steady-state trace storage before compression. Run the same numbers at your P99 payload — a handful of multi-megabyte agent traces can dominate storage that a million chatbot traces barely dent. Now do it again at 10× run volume, because agent traffic grows in steps, not slopes: one new use case shipping to production can multiply your span volume overnight. If the 10× number makes the storage line item uncomfortable, that is not an argument against self-hosting — it is the argument for deciding your sampling and retention policy now, while it is a design choice rather than an incident response.Full-fidelity tracing on 100% of traffic with long retention is a default you inherit, not a requirement you chose; most teams need 100% capture with tiered retention (days at full payload, months at metadata) far more than they need everything forever.
The enterprise-hardening story matured substantially through the v0.13–v0.15 line (January–May 2026), and it is the reason self-hosted is viable in strict environments at all: ABAC and audit logging enabled by default, mTLS support (including for ClickHouse), IAM-basedPostgreSQL authentication, workload-identity support for object storage, and per-project cost alerts. If your last evaluation of self-hosted LangSmith predates this line, it is out of date.
Instrument once, decide later
The deployment decision becomes far less frightening if the instrumentation layer does not encode it. Three patterns keep you portable:
Standards-based tracing. LangSmith supports OpenTelemetry-based ingestion alongside its native SDK, and cross-process trace linking is documented for distributed setups — an agent calling internal services can carry one trace across process boundaries. Building on OTel semantics rather than proprietary hooks means the instrumentation investment survives a change of backend, of deployment model, or of vendor.
Cost as telemetry, on every step. Since February 2026, custom cost metadata can be attached to any run — tool calls, retrieval, sandbox time — not just LLM invocations, and it aggregates into per-trace and dashboard views. We treat this as non-negotiable instrumentation: the question "what does one resolved ticket cost end-to-end?" should be answerable from the trace, in whichever deployment model the trace lives.
Bulk export as the escape hatch. Traces are also compliance artifacts and analytics inputs. A scheduled bulk export into your own data platform (object storage → warehouse) means the observability platform is never the only copy of the data — which is simultaneously your audit story, your data-science feed, and your migration insurance.
The decision, factor by factor
With portability secured, the choice reduces to five factors. The matrix below is the summary; the text is the reasoning.
Data residency is the eliminator. If regulation or contract says trace data cannot leave your perimeter, self-hosted is the answer and the rest of the matrix is commentary. The interesting middle case is "metadata may leave, payloads may not"— which is exactly the boundary the hybrid model draws: agent and trace data stay in your VPC's data plane while the vendor runs the control plane. Establish precisely which fields cross that boundary, in writing, before promising anyone the word "hybrid"; the EU SaaS region resolves asurprising number of residency conversations on its own.
Operational load is the honest number. A self-hosted deployment at meaningful trace volume is apart-time job that trends toward a full-time job: ClickHouse care and feeding, monthly upgrades, scaling events, backup drills. That cost is invisible in month one — the install is genuinely one Helm release — and dominates by month twelve. Teams whose platform engineering bench is already thin should weighthis factor double.
Feature lag is structural, not a defect. Cloud ships continuously; self-hosted releases monthly with some cloud-only capabilities arriving later or not at all (the vendor's own newest services — large-scale trace store, managed sandboxes and gateways — are cloud-first by nature). If your team lives on the newest evaluation and platform features, self-hosted means permanently trailing the blog posts by a quarter.
Query performance at scale is now a real differentiator. The vendor's purpose-built trace database (announced in production May 2026, with claimed order-of-magnitude improvements on core queries — vendor-reported; benchmark your own workload) is precisely the kind of component you cannot replicate in a self-hosted footprint. At small volume this is irrelevant; at hundreds of millions of spans it becomes the difference between debugging interactively and not.
Our resulting placement heuristic, bluntly: regulated data → self-hosted or hybrid, chosen by who runs the data plane best; everything else → SaaS in the matching region, and spend the reclaimed engineering time on the agents themselves.
The cutover, concretely
When a workload moves (in either direction —SaaS-to-self-hosted happens too, usually for residency), the migration is asymmetric and it pays to know which assets are portable before choosing what depends on them:
- Prompts — exportable via API, with commit history re-creatable; environment pointers (dev/staging/prod) are re-established per target. Applications that reference prompt: environment — as recommended in the previous post — migrate with a config change and zero code.
- Datasets and experiments — datasets export/import cleanly via API or bulk export; the golden datasets and their baselines are the crown jewels here. Migrate datasets, then re-run the baseline experiments in the target rather than trying to move result rows — evaluation results are only comparable when produced by the same environment.
- Dashboards, alerts, evaluator configs —treat as re-creation work and budget accordingly. This is the strongest practical argument for keeping evaluator definitions in code ( openevals -style) rather than only in the UI: code re-attaches anywhere.
- Historical traces — the deliberate leave-behind. Keep the old system read-only through your retention window, or rely on the bulk exports in your warehouse (which you set up in the instrument-once section — this is why). Migrating live trace history across trace stores is effort that buys almost nothing.
- The parallel run — tracing is fan-out-friendly: run old and new backends side by side for a week or two by environment, cut writes over when the new side's dashboards and online evaluators are proven, then hold the old side read-only until retention expires.
The pattern to notice: with prompts referenced by environment, evaluators kept in code, costs attached to traces, and bulk export always on, the platform becomes a replaceable component under a stable contract — which is exactly the property you want from infrastructure.
Takeaway: the decision matrix, as questions
Eight questions that resolve the deployment model in one meeting:
- Is there a regulatory or contractual line that trace data must not cross? (If yes: self-hosted or hybrid — and get the hybrid boundary's field-level detail in writing.)
- Does the EU SaaS region already satisfy what we are calling a "residency" requirement?
- Who— by name — will own ClickHouse sizing, monthly upgrades, and backups? Is that the best use of that person?
- What is our trace volume and P99 payload size today, and at 10× — and what retention do we actually need?
- Which platform features do we depend on that are cloud-first? How long can we trail them?
- Is our instrumentation portable (OTel-based, costs-as-telemetry, bulk export on) so this decision stays reversible?
- What does each model cost at our projected volume — including engineer time, not just invoices?
- If we had to move in 18 months, which assets would move (prompts, datasets, evaluator code) and which would we abandon (trace history, dashboards)?
Answer those eight and the matrix fills itself in. And whichever cell you land in, the workload from the previous post — the change loop, the baselines, the online evals — runs unchanged.
That separation between the loop and the platform is the real architecture.
This is the third post in a five-part series. The next two posts raises the stakes: agents that run for minutes or hours — how to design them (post 4) and the runtime engineering that keeps them alive (post 5).
References
- LangSmith self-hosted architecture and Kubernetes/Helm documentation —docs.langchain.com/langsmith (Valkey support, ABAC/audit defaults, mTLS,Postgres IAM auth: self-hosted changelog, v0.13.40–v0.15.0, April–May 2026)
- Custom cost tracking across the agent stack — LangSmith change log, February 5, 2026
- OpenTelemetry integration and cross-process trace linking —LangSmith documentation (linking guidance, June 2026)
- SmithDB trace database announcement —Interrupt 2026, LangChain blog, May 13–14, 2026 (performance figures vendor-reported)
- Trace payload growth figures — LangChainInterrupt 2026 keynote (vendor-reported)
- LangSmith deployment options and regions —langchain.com/pricing (verify entitlements per plan)
.png)
