Coding Agent Harness Selection and Integration: OpenCode, Pi, DeepSeek Harness, Codex, Claude Agent SDK, and OpenHands Compared
I have been evaluating OpenCode, Pi, DeepSeek Harness, Codex, Claude Agent SDK, and OpenHands recently, and took a look at all-in-one assistant projects like OpenClaw and Hermes Agent along the way. The motivation was concrete: I needed answers to two questions. Can it be deployed as a standalone process? And how does existing business software call it?
From the demos alone, all of these tools look alike: read files, run commands, edit code, call tools. The differences only surface once you wire one into a system — is there an HTTP API, can a session be resumed, how do you cancel a task, how are multiple users isolated, and when something goes wrong, can you reconstruct what a single run actually did.
So this article does not compare who writes smarter code. It compares two things: how to choose, and how to integrate once you have chosen.
First, versions and scope. The assessment is based on official documentation as of September 2026; the versions I checked were OpenCode v1.18.31, Pi v0.85.1, DeepSeek Harness v0.1.6-alpha.2, Codex v0.155.0, Claude Agent SDK Python v0.2.156 / TypeScript v0.3.276, OpenHands v1.20.0, OpenClaw v2026.9.4, and Hermes Agent v2026.9.14. This field moves fast — treat the APIs of whatever version you actually adopt as the source of truth.
A disclaimer up front: this is a documentation review, not a benchmark. There is no performance, concurrency, or cost data in this article; “effort” figures are order-of-magnitude judgments, not person-month commitments. For a few sections I read the docs without running the code, and I mark that inline.
Bottom line first
Given a typical set of goals, here is how I would choose:
| Goal | Start with |
|---|---|
| You already run Go, Java, or Python services and want a standalone Agent Runtime integrated quickly | OpenCode |
| You want to embed the agent core into your own Node.js product | Pi |
| You want to validate a plugin-based architecture, or you rely heavily on DeepSeek models | DeepSeek Harness |
| You have already committed to OpenAI or Claude | Codex / Claude Agent SDK |
| You need a full repository, containers, browsers, and a remote development environment | OpenHands |
| You want an off-the-shelf AI assistant for yourself or your team | OpenClaw / Hermes Agent |
This is not a capability ranking. OpenCode is strong on integration speed, Pi on ease of modification, DeepSeek Harness on its plugin architecture, and OpenHands on the heavier execution-environment problem. Different teams have different problems, so the answers differ.
One more thing I am certain of: being able to run a shell does not make something a business backend. A demo working is a long way from production, which needs sessions, cancellation, isolation, permissions, auditing, and task recovery.
Are these tools even the same kind of thing?
Not exactly.
I think of a harness as the execution environment around the model: it assembles context, lets the model pick tools, feeds tool results back to the model, and loops until the task ends.
1 | Agent = Model + Harness |
Under that definition, projects in this space fall into four camps:
| Category | Representative tools | What it really is |
|---|---|---|
| Finished harnesses / agent SDKs | OpenCode, Pi, DeepSeek Harness, Codex, Claude Agent SDK | Ship an agent loop you can run directly or embed in an application |
| Agent development frameworks | LangGraph, AutoGen, PydanticAI, Semantic Kernel | Parts for assembling your own execution engine |
| All-in-one assistant platforms | OpenClaw, Hermes Agent | Assistant products that ship their own agent core (or embed another vendor’s harness), plus chat channels and scheduled tasks |
| Protocols and infrastructure | MCP, ACP, A2A, Sandbox | Standards for connecting tools, agents, and execution environments |
LangGraph and OpenCode are often compared side by side, but they sit at different layers. With LangGraph you design task state, nodes, routing, and persistence yourself; with OpenCode the agent loop, sessions, and tools are already there.
OpenCode: an off-the-shelf agent backend
I am not very interested in OpenCode’s terminal UI. What is actually useful is how it wires the server, SDK, sessions, events, and extensions together.
Officially it provides opencode serve; your business service creates sessions and sends messages over HTTP, and receives events over SSE. Tools can be Bash, custom tools, plugins, or an MCP server; agents, subagents, and skills all come with ready-made configuration.
1 | Business Service |
A good fit for teams whose business service already exists and that do not want to rewrite it in Node.js for the sake of an agent; who want the runtime deployed independently; who want to get tasks, process, and results working end to end first; or who have not settled on a model yet.
Watch its boundaries. What it solves is “how one task executes”. Multi-tenant isolation, task queuing, cross-instance storage, credential management, and failure recovery are all out of scope. It is fine for an internal prototype; carrying real user volume and concurrent tasks means adding scheduling, permissions, storage, and execution isolation around it.
Pi: an agent core you can embed in a product
Pi feels different from OpenCode. OpenCode is like a process that serves others; Pi is like a set of small, clean building blocks. There are two practical integration patterns:
- Use the SDK in a TypeScript/JavaScript service to create and control agent sessions directly;
- Run a long-lived process in RPC mode, exchanging JSONL commands and events over stdin/stdout.
One session-related change is worth calling out: v0.85.0 added SessionManager.inMemory(), which restores sessions from entries your own external system stores — you can own persistence without much work.
An incident that illustrates Pi’s current state: 0.85.0 accidentally shipped internal experimental client/server code, and 0.85.1 immediately pulled back to source-only availability. There are exactly two officially supported interfaces — the local SDK and stdio RPC — and there is still no timeline for a native HTTP API. Plan your architecture as if no ready-made remote interface will ever exist; that is more reliable than waiting for one.
Extensions can register tools, commands, and event hooks; combined with skills, prompt templates, and packages, product-grade composition works nicely. If you want to own the external API, UI, session storage, and task state yourself and reuse only the agent loop, Pi is the best fit of the bunch.
The cost is equally direct: no enterprise control plane either — authentication, tenancy, database, worker scheduling, quotas, and operations interfaces are all yours to build. Go and Java services can integrate over JSONL RPC, but child-process exits, backpressure, restarts, and protocol versions all have to be handled by you.
In one sentence: the most freedom, and the most work.
DeepSeek Harness: worth watching for the plugin architecture, but leave room for change
First impression: everything-is-a-plugin. It is officially positioned as a plugin-based agent harness SDK; sessions, tools, MCP, sandbox, planning, and LLM access are all independent packages you swap and compose as needed. Putting replaceability first in the architecture is what distinguishes it from both OpenCode and Pi.
The repo is mostly TypeScript, MIT licensed, around 230k GitHub stars in late September. It ships both TypeScript and Python out-of-process SDKs, usable from the CLI, the web UI, plugins, and ACP; v0.1.6-alpha.1 added a headless mode that takes tasks on stdin, resumes sessions via --session-id, and emits run events line by line with --json — very close to Pi’s JSONL RPC shape. Python projects can genuinely integrate with it, but do not read it as “a harness for Python teams”. Models default toward DeepSeek, though Anthropic, OpenAI, Kimi, and GLM are also wired up. Sessions support persistence and replay, which is handy for troubleshooting and resuming.
It is just an agent runtime. No training, evaluation, or fine-tuning — do not expect a model iteration loop.
The risks are stated in plain sight: it is officially marked Developer Preview, with breaking changes announced. The September 17 v0.1.6-alpha.2 changed the client-side session API and slots to support multiple concurrent session instances, and switched plugin dependency resolution to runtime with runtime unloading — the release notes flat-out tell developers to audit their own loading and unloading logic. Before going to production, on top of the general checklist at the end of this article, verify three things specifically: plugin and SDK upgrade compatibility; whether the multi-model provider support is enough; and how event logs get persisted and redacted.
My advice: try it early, but do not let your business interfaces depend directly on its current SDK types. Wrap it in an adapter layer, and upgrades or switching back to another runtime become much easier.
Codex and Claude Agent SDK: once the platform is decided, do not fight it
If your company has already standardized on a model platform, model neutrality is not a requirement. Abstraction for abstraction’s sake has a real cost.
Codex offers three usage modes — non-interactive execution, SDK, and App Server — and manages threads, turns, events, approvals, structured output, MCP, and sandboxes; the App Server speaks JSON-RPC over WebSocket, so remote calls do not need another wrapper from you. The Claude Agent SDK exposes Claude Code’s agent loop, tools, sessions, hooks, and permissions to Python and TypeScript applications. If all you want is a remote call to a Claude agent, look at Anthropic’s Managed Agents, a hosted REST API — a different product from the Agent SDK.
Prepare yourself for the release cadence. Codex’s stable v0.155.0 shipped on September 17, and the 0.156.0 alpha came out the next day; the Claude Agent SDK Python package is still on 0.2.x and TypeScript on 0.3.x, with releases essentially daily. Fast iteration is good, but the APIs change with it — do not skimp on regression time when upgrading.
The advantage of vendor runtimes is the tight model-harness integration: new official capabilities are usable the moment they ship. The disadvantage is that account, model, protocol, and runtime become one bundled chain. Once you have picked the platform, use it; if you want a multi-model neutral platform, be careful.
OpenHands: look at it when you actually need a remote dev machine
OpenHands is heavier than everything above: SDK, agent server, remote sandbox, web UI — a complete execution environment for software engineering tasks. This is the area I looked into the least — I read only the SDK and agent server docs, without testing.
It earns its keep when a task has to clone repos, install dependencies, start services, run tests, open a browser, and run for a long time. If you just need to read a few business records and produce a structured result, this platform is too heavy.
One-sentence conclusion: be clear whether you want “one loop” or “a remote development machine”.
A word on all-in-one assistants
OpenClaw and Hermes Agent are a different species (around 390k and 250k stars respectively in late September): kernel, chat channels, scheduled tasks, and session management all pre-built, delivering an always-on assistant directly. They answer “how do I deploy an always-available assistant for myself or my team”, which is not the same question as this article’s “how does business software call an agent programmatically”. If you want a working agent in your company IM, deploying one of these with some config tweaks is the better deal; if you want to stuff a multi-tenant runtime into a business system, neither the session model nor the security assumptions support it. One piece of supporting evidence: OpenClaw’s core is built on Pi, and the app-server it uses for Codex is still marked experimental in the Codex repo — the Pi path works, and downstream projects have already stepped on the integration landmines for you.
What I look at during selection
First, separate three capabilities that are usually conflated; they also become the column definitions of the comparison tables below. Standalone operation: it can run a complete agent loop without being embedded in the business host process (this says nothing about being able to deploy the model offline). Native HTTP API: the vendor directly provides remotely callable HTTP, SSE, or WebSocket; SDKs, stdin/stdout, and JSONL RPC are programmatic interfaces, not remote services. Session management: whether tasks can continue, resume, query, and be reclaimed — independent of how the process starts and what protocol you call it with.
Once separated, here is how I verify:
Can it run standalone. Ask about deployment first: can it simply be handed to Docker, Kubernetes, or systemd, or must it be embedded in some host program? That decides how it enters your release process.
Does it have a native HTTP API. If all you can do is start a CLI once, wait for the process to exit, and read stdout, then cancellation, approvals, progress display, and failure recovery are all hard. And having an HTTP API is not grounds for relief either: check authentication, sessions, streaming events, cancellation, and concurrency semantics one by one.
Who owns the session. Do not just check for a session ID. Ask: can you create, query, continue, and delete sessions; is history in memory or persisted and replayable; can it recover after a process or container restart; how are multiple users, tasks, and instances isolated; and who handles timeouts, expiry, and archival. Also distinguish harness sessions from business tasks: the former continues context, the latter carries business state, retries, and auditing — I will get to the mapping later.
Do tools have contracts. Almost every coding agent can run a shell; what I care about is whether tools can be described as interfaces with a schema — parameter types, allowed values, stable return shapes, and which calls need approval. Prompting the model to “assemble the command in this format” works, but not for long.
Can a task be stopped. After a user cancels, flipping the page state to cancelled is not enough. Model requests, shell child processes, browser tasks, and remote tools all have to end together — otherwise the backend keeps burning money, or even keeps executing side-effecting operations.
Is isolation sufficient. A working directory is not a security boundary. Concurrent tasks need at least directory, process, network, credential, cache, and session isolation. If code execution or untrusted input is involved, look at containers and sandboxes.
Can results be consumed programmatically. Business systems need JSON, not a chunk of Markdown that “reads well”. Native JSON Schema or structured output is best; a dedicated tool for submitting the final result is second best; parsing free text is the worst option.
Putting them side by side
Only comparing the parts that matter for a business runtime, not the code-completion experience.
| Tool | Standalone | Native HTTP API | Session management | Other programmatic interfaces | Remote integration verdict |
|---|---|---|---|---|---|
| OpenCode | Yes, CLI / server process | Yes, HTTP / SSE | Built-in session API: create, query, and continue sessions | TypeScript SDK | Can be integrated directly as a standalone runtime |
| Pi | Yes, CLI / RPC process | No | Provides agent sessions; local persistence and lifecycle controllable via SDK; since v0.85 can restore sessions stored in external systems | TypeScript SDK, stdin/stdout JSONL RPC | Needs its own HTTP service wrapper |
| DeepSeek Harness | Yes, CLI / Web | Limited; the Web Host API targets localhost by default | Event-log-style sessions with persistence, replay, and recovery | TypeScript/Python SDK, ACP, headless (stdin / JSONL events) | Needs validation of API stability and remote exposure |
| Codex | Yes, codex / codex exec |
Yes, App Server supports WebSocket (HTTP + JSON-RPC) | Thread / turn; can hold, continue, and manage conversation lifecycle | SDK, App Server process protocol | Runs as a process; App Server callable remotely |
| Claude Agent SDK | Requires a host program | No | Supports session IDs and resumption; persistence strategy still has to be designed with the host | Python/TypeScript SDK | You must build the agent service yourself |
| OpenHands | Yes, Agent Server | Yes, Agent Server API | Conversations support persistence, pause, and resume | SDK | Natively a remote agent service |
Session management in this table only indicates whether there is a session identifier, history, and resumption. Where data is stored, whether multiple instances can share it, how it recovers from a crash, and who owns expiry and deletion all need separate verification.
Tool extension, model choice, path to a service, and effort, viewed together. Relative effort is estimated on the assumption “existing business service, internal tool first” — not a person-month commitment; going multi-tenant and production-grade moves every row up significantly:
| Tool | Tool extensions | Model choices | Path to a service | Relative effort | Main cost |
|---|---|---|---|---|---|
| OpenCode | Custom tools, plugins, MCP | Multi-provider, not tied to one model | Ships an HTTP/SSE server; shortest integration path | Small: authentication, result validation, task queue | You add the control plane — auth, isolation, scheduling |
| Pi | Extensions, tools, skills, packages | Multi-provider | You wrap an HTTP service and session store yourself | Medium to large: HTTP service, session storage, task scheduling, authentication, quotas, operations interfaces | The external service layer is essentially all yours |
| DeepSeek Harness | Plugins, tool and protocol extensions | DeepSeek by default; Anthropic, OpenAI, Kimi, GLM also wired up | Web Host API targets localhost; remote exposure must be validated yourself | Small to medium: version isolation layer, remote exposure plan, upgrade regression | Developer Preview; APIs may break |
| Codex | MCP, skills, sandbox, approvals | Extensible to Bedrock, Ollama, etc., but the protocol layer is the OpenAI Responses format | CLI / SDK / App Server (WebSocket) can all be made remote | Medium: service wrapping, approval flows, release regression | Strong platform coupling; migrating out requires protocol conversion |
| Claude Agent SDK | MCP, custom tools, hooks, permissions | Claude | A host program must wrap the SDK as a service | Medium: service wrapping, approval flows, release regression | Strong platform coupling; the service layer is yours to build |
| OpenHands | Tools, MCP, remote sandbox | Configurable | The agent server is itself a remote service | Medium: deployment, sandbox and resource management | Higher deployment and resource cost |
Likewise, having an HTTP API does not mean something can carry production business load. Authentication, concurrency, session persistence, cancellation, isolation, and failure recovery still have to be checked one by one.
Choosing by team situation
The bottom-line table above is looked up by goal; here is an exclusion path in reverse. The order also reflects my priorities:
1 | What you actually want is an always-online personal or team assistant, not a business backend? |
Two additions beyond the enumeration: DeepSeek Harness is worth pinning a version and running a batch of shadow tasks, using data to decide whether the plugin architecture is worth investing in; and on OpenCode, start with read-only tasks only, add authentication, result validation, and concurrency limits first, and only then allow writes.
Finally, effort. The relative effort in the combined table above is the most realistic variable in a selection exercise; routes can differ by an order of magnitude, which matters far more than “which model is smarter”. If your team has no one dedicated to building the control plane, the low-effort route is usually the better deal.
Why not write your own from scratch
Writing a minimal agent loop is not hard: assemble context, call the model, parse the tool call, execute the tool, send the result back. The trouble is everything after that — timeouts, interrupts, permission confirmations, oversized context, orphaned child processes, session recovery. None of them is complex individually; stacked together they consume a lot of time.
That is where an off-the-shelf harness pays off: it has already built the execution loop, so the business team’s time goes to more practical problems. Are the existing APIs and CLIs clear enough; can the model actually complete the task with the tools; which steps must have human confirmation; how do you accept the final result.
By the way, this is a good opportunity to test the quality of internal tooling. Many CLIs are built for humans: errors are all text, parameter naming is inconsistent, and output formats change. Let an agent call them repeatedly and the problems surface fast. Once parameters, error codes, and JSON output are cleaned up, the beneficiaries are not just agents — ordinary automation scripts get more reliable too.
Nor does business knowledge all need to go into one long prompt. Write procedures as skills, hand real actions to tools, put permissions in policies, and leave result judgment to a fixed evaluation set. If you switch models or runtimes later, most of this carries over.
What a harness covers for you
Put OpenCode, Pi, or DeepSeek Harness behind a business system and I treat them as the execution plane. Anything inside a single run should go to the harness where possible:
- Calling the model and maintaining the agent loop;
- Holding the current turn’s session context;
- Registering and calling tools;
- Loading skills, agents, and extensions;
- Handling files, the shell, and the working directory;
- Emitting message, tool, and error events;
- Basic permission confirmation and task cancellation;
- Constraining structured output where supported.
That is enough to get a business idea running. A support assistant looks up orders and past tickets before drafting a reply; a data assistant queries metrics, spots an anomaly, and drills into details; an engineering assistant edits code and runs tests. They all start here.
Do not hand the business control plane to the harness, too
A harness is only responsible for “how execution happens”. The following still belong in your own business system.
Business task state
A harness knows sessions and messages; the business knows tickets, review forms, report jobs, orders, and code changes. The two do not map one-to-one.
A single business task might time out and rerun twice, or start execution in OpenCode and later be handed to Pi for review. The better relationship:
1 | BusinessTask 1 ── N AgentRun 1 ── N HarnessSession |
I keep three IDs:
task_id: the business task;run_id: one execution attempt;session_id: the session inside a specific runtime.
That way retries, replay, and runtime swaps do not require touching the business data model.
Identity and permissions
A harness does not know which department a user belongs to, or whether they may see a given customer, dataset, or project. The tool permissions it offers are only the first layer; resource-level authorization belongs in the business system or a tool gateway.
Issue a short-lived identity per run so tools can check permissions by tenant, user, and task. Do not put long-lived admin tokens in the agent container.
Idempotency, retries, and approvals
A failed model call can be retried, and a timed-out query tool can be retried — but “submit a refund”, “publish content”, and “change configuration” cannot simply be replayed.
The business side needs to know whether an operation has already run, whether it may run again, and who approved it. The harness’s confirmation dialog can carry the interaction, but approval rules and final state live in the business system.
Whether the result is correct
A harness can make the model produce a report; it cannot know whether that report satisfies the company’s business definitions.
An analysis result may have to carry a statistical period, a data definition, and a sample size; a review result needs rule IDs and evidence; a code change must pass designated tests. Validate final results against a schema and business rules — do not just eyeball whether the prose reads well.
Scheduling, capacity, and high availability
Task queues, workers, rate limiting, priorities, failure compensation, cost quotas, and cross-datacenter deployment usually do not belong to a single harness either. It is cleaner to treat the harness as a worker; do not have business requests synchronously wait on a CLI process to exit.
Data compliance and auditing
A run’s context, tool outputs, and results usually carry business data. Confirm these things early rather than right before launch:
- Which model vendor the data goes to, whether it is retained or used for training, and whether contracts and model config make that explicit;
- Redacting event logs and tool output before they hit the database — credentials, personal information, and customer data must not go into logs verbatim;
- Whether high-risk actions can be traced to a person, a
run_id, and an approval record, how long audit records are kept, and whether they are tamper-proof; - Cross-border compliance for model calls and log storage when deploying across regions.
How to layer the integration
I prefer four layers:
1 | ┌───────────────────────────────────────────┐ |
The control plane stores business facts; the adapter masks runtime differences; the harness performs a run; the tool plane touches real systems.
How to wire up tools: CLI, native tools, and MCP
The three approaches are not mutually exclusive and usually coexist. If you have a mature CLI, reuse it first — it is ideal for a PoC; file processing, local scripts, and one-off migration jobs also suit a CLI.
Existing CLIs: the fastest integration
The simplest approach: bake the CLI into the runtime image and add it to PATH:
1 | biz-ticket get --id TICKET-001 --output json |
Then write a skill documenting when each command applies, what parameters it needs, and what it returns. Tools like OpenCode and Pi can execute them through the shell.
But do not temporarily copy the binary into each task directory and let the model freely compose commands. A directory is just a file location, not a permission boundary. Handle at least these:
- Pin the CLI version so the image is traceable;
- Expose only allowed commands, validate parameters, and never concatenate shell strings;
- Default to read-only, route writes through a separate entry point with short-lived credentials;
- Standardize JSON output and error codes, and set timeouts, output size limits, and resource caps;
- Log the caller, parameters, duration, and a result summary.
Schema wrappers: add contracts to high-frequency commands
The problem is that the model has to understand command-line syntax; argument escaping, long output, and error text all add uncertainty. Once you move toward staging, it is worth adding a schema wrapper around the CLI: wrap the command in a tool with a JSON schema describing the tool name, its purpose, and each parameter’s type and allowed values, so the model only passes structured arguments.
For example, define a schema for biz-ticket get first:
1 | { |
The model calls:
1 | { |
Once the wrapper validates that ticket_id matches the schema, it executes the fixed command biz-ticket get --id TICKET-001 --output json. The full shell string is never generated by the model; invalid arguments return a standardized error code and the command never runs.
Writing wrappers costs development time, so this layer only goes on the high-frequency commands the agent calls repeatedly. Anything used occasionally can just go through Bash.
Harness native tools: the tightest integration with the current runtime
Here, native tool means a tool registered through a harness’s own SDK, plugin, or extension mechanism — nothing to do with operating-system-native programs. OpenCode custom tools, Pi extensions, and custom tools in vendor agent SDKs all count.
These tools present name, purpose, and parameter schema directly to the model. When the model invokes one, the harness handles argument parsing, execution, event notification, permission confirmation, and error propagation. The implementation can live in the host process or call internal APIs instead — what matters is not whether it is in the same process, but that the tool uses the harness-specific interface.
For capabilities called frequently, with complex interactions and fine-grained permission needs, native tools are far more stable than having the model assemble shell commands. The cost is runtime coupling: an OpenCode custom tool cannot be used in Pi, and switching harnesses usually means rewriting the adapter.
MCP: better for shared capabilities
When capabilities like logging, orders, knowledge bases, and data queries need to serve multiple agent clients at once, MCP is the better fit. Business interfaces and credentials stay in a dedicated service that OpenCode, Pi, Codex, and Claude all access through the same tool protocol.
1 | OpenCode ─┐ |
MCP solves the connection problem; the enterprise still has to add identity, resource authorization, rate limiting, caching, redaction, and auditing around it.
Rollout order
How I would sequence it:
- Wire in existing CLIs first and get tasks working;
- Add schema wrappers to high-frequency CLIs;
- Turn a few strongly interactive capabilities into harness-native tools;
- Move cross-team, cross-runtime shared capabilities to MCP or a tool gateway;
- Route high-risk writes through deterministic workflow APIs with human approval.
You do not need to build a heavy platform on day one, and you will not stay in the “model executes arbitrary shell” state forever.
Add an adapter layer between the business and the runtime
Whichever you pick first, the business service should depend only on its own interface:
1 | type Harness interface { |
Then implement OpenCodeAdapter, PiAdapter, and DeepSeekAdapter.
The adapter does not need to flatten all products into one; it only unifies what the business actually depends on: run state, events, cancellation, and the final result. Capabilities like structured output, session forks, and specific sandboxes are exposed via capability flags.
Which business use cases to try first
Pick tasks that “need context understanding but whose mistakes can still be caught” first.
| Scenario | Good first candidates for agents | Keep manual for now |
|---|---|---|
| Customer support and tickets | Looking up orders, organizing history, triage, drafting suggested replies | Refunds, compensation, bans |
| Data analysis and reporting | Drilling into intermediate results, organizing definitions and conclusions | Direct production-database access or arbitrary SQL |
| Content moderation | Risk classification, rule matching, evidence assembly | Final rulings that affect user rights |
| Operations diagnosis | Querying logs, metrics, traces, and change records | Applying fixes to production without approval |
| Engineering automation | Reading code, editing files, running tests | Publishing without review |
| Enterprise process assistants | Policy lookups, document checks, form preparation | Treating a text reply as a completed process |
Tasks with fixed steps, fully enumerable rules, or a single mistake causing irreversible loss are better served by ordinary programs and workflows. Let the agent own just the nodes that need understanding and judgment — there is no need to take over the whole flow.
From PoC to production
Four steps — do not build all the infrastructure on day one.
Step one: prove the task has value
Pick one runtime, one model, and a handful of read-only tools; fix the inputs, tool set, and result format; run a batch of real tasks. Record at least:
- Task completion rate;
- Whether tool calls were correct;
- The share of unsupported conclusions;
- How often humans had to take over;
- p50 and p95 completion times;
- Token and infrastructure costs;
- Whether timeouts, cancellation, and recovery behaved.
A good single demo may just mean the model got lucky on the path. Running several dozen real tasks reliably is what shows that the harness, tools, and context design are basically right.
Do not chase multiple models and multiple agents at this stage. If a single agent with a few tools is not stable, adding roles only makes problems harder to localize.
Step two: tighten tools and results
On the tool side, upgrade following the rollout order above; swap high-frequency commands for schema wrappers and tighten argument and resource ranges. On the result side, validate with schemas, and separate facts, evidence, inferences, and recommendations.
Put every side-effecting tool behind approval.
Step three: add the business control plane
Add a task table, a queue, asynchronous workers, idempotency, layered retries, and cancellation. String together task_id, run_id, and session_id, and record model, tool, and approval events.
At this point the harness stops being a personal tool and becomes a business executor.
Step four: consider runtime swaps and a shared tool layer
Prepare a fixed replay set, run candidate runtimes on shadow tasks, and compare quality, cost, duration, and failure recovery.
When a new project shows up in the community, resist migrating immediately. The point of the adapter and the replay set is to make switching data-driven.
What I check before production
This checklist consolidates the acceptance points scattered above so you can walk through it directly:
- After a user cancels, do model requests and child processes actually stop;
- After a runtime crash, can a task recover or retry safely;
- Are directories, sessions, and credentials isolated across tenants;
- Can tool arguments and resource ranges be constrained;
- How are over-long or malicious tool outputs handled;
- Do final results pass schema and business-rule validation;
- Do write operations have idempotency keys and approval records;
- Can a run’s model, tools, approvals, and result be traced end to end;
- Are per-task time, token, tool-call, and resource limits enforced;
- Are event logs and tool outputs redacted before persistence;
- Has the model vendor’s data retention, usage, and cross-border policy been confirmed;
- Can a failed runtime upgrade be rolled back.
Until these are handled, the system can be used internally, but it should not carry important business actions.
Conclusion
The strongest takeaway from this comparison: standalone operation, native HTTP API, and session management must be examined as three separate things. Being able to run standalone does not mean you have a remote service — Codex’s App Server already speaks WebSocket, and you still add authentication, concurrency, and rate limiting yourself; having a session ID does not mean tenancy, persistence, and expiry handling are solved.
The selection itself is not hard — run through the decision tree and you have an answer. What really determines cost is two things: how much of the control plane you have to build yourself, and which side business data, permissions, and auditing live on. Whoever I pick, I will not let the business depend directly on its session and message formats — establish your own task_id, run_id, events, and results first, and call the runtime through an adapter. For a PoC, let the harness call existing CLIs; once it works, add schemas, permissions, auditing, isolation, and recovery. That is far more practical than building an “agent platform” from day one.
References
- OpenCode Server
- OpenCode SDK
- OpenCode Tools
- OpenCode Custom Tools
- OpenCode Permissions
- OpenCode MCP Servers
- Pi RPC Mode
- Pi SDK
- Pi Extensions
- DeepSeek Harness GitHub Repository
- DeepSeek Harness Release Notes
- DeepSeek Harness Session Persistence
- Codex as a Platform
- Codex Non-interactive Mode
- Codex App Server
- Claude Agent SDK Overview
- OpenHands Software Agent SDK
- OpenHands Agent Server
- OpenHands Conversation Persistence
- OpenClaw GitHub Repository
- OpenClaw Gateway Architecture
- OpenClaw Agent Runtimes
- Hermes Agent GitHub Repository
- Hermes Agent API Server
- Hermes Agent Messaging Gateway
This article is the English edition of a post originally published in Chinese.