Skip to content

Embedding BOS

You have an application, and you want an agent inside it. BOS gives you two ways to do that, and they are genuinely different shapes — not a beginner version and an advanced one. Choose before you write code. Switching later is a rewrite of your integration layer, because the two modes put the conversation in different places.

Which mode

Mode 1 — call the agent Mode 2 — mount the gateway
You write agent.run(chat_id, text) Nothing per turn
Who drives a conversation Your code BOS's actors and channels
Concurrency, queueing, turn takeover Yours to build Built in
Telegram / Lark / WebSocket clients No Yes
Install pip install bos-ai pip install bos-ai[gateway]
Needs a writable directory Only if you configure file-backed stores Always
Runnable example examples/embed_sdk.py examples/embed_gateway_fastapi.py

Pick mode 1 if your application already owns the conversation — a CLI, a batch job, a Slack bot you wrote, an HTTP endpoint that answers one question. You call the agent the way you call any other async function.

Pick mode 2 if you want what boscli gateway start runs — named actors, chat coordination, persistent channels, the WebSocket protocol — but inside your own process and behind your own auth, instead of supervising a second daemon.


Mode 1 — call the agent in your own process

pip install bos-ai

That is the library install. It ships no boscli command, no terminal UI and no gateway; see the install table for the extras. Configuration is a plain dict, so it can come from your database, your environment or a control plane — a config.toml on disk is the CLI's convenience, not a requirement.

import asyncio
from bos.sdk import BosApp

CONFIG = {
    "platform": {"extensions": []},
    "agents": {
        "assistant": {
            "system_prompt": "You are a concise assistant.",
            "model": "openai/gpt-4o",
        }
    },
}

async def main() -> None:
    async with BosApp(CONFIG, bos_dir="/var/lib/myapp/.bos") as app:
        agent = app.agent()
        first = await agent.run("chat-42", "hello")
        print(first.output)
        again = await agent.run("chat-42", "what did I just say?")
        print(again.output)

asyncio.run(main())

Four things that snippet is showing you:

agent() returns an AgentPort, not a reply. There is deliberately no app.ask(). run() takes ten parameters — streaming via event_sink, structured output via schema, interrupt, llm_args, turn_id and more — and any one-line façade over it sends you back down a layer the moment you want one of them. Dropping to the lower layer therefore hands you the same objects, not a different path: app.harness and app.workspace are exactly the ones BosApp is using.

Chat continuity is an id, not an API. Both calls above pass "chat-42", so the second turn sees the first in its history. There is no session object to create, hold or close. Use your own conversation key.

Agents are built once, at __aenter__, and cached. app.agent() is synchronous because of that. Every kind named in config["agents"] is built as the block opens; with no argument you get the default (the top-level default_agent key, or the only agent, or one named main). For a kind that only an @ep_agent factory registers — not named in your config — build it once with await app.build_agent("kind").

An agent that cannot be built costs only itself. If a kind your config names fails to build — a Claude Code agent refusing an ANTHROPIC_API_KEY in the environment, say — the block still opens with every other agent. The failure is logged, app.failed_agents maps the kind to its exception (for a picker to grey out), and app.agent(kind) raises chained to it. Fix the cause and await app.build_agent(kind) builds it without reopening the app. The default is the exception: if it cannot be built, async with BosApp(...) raises.

One BosApp per process. Bootstrap writes os.environ and rebuilds the agent registry, both process-global, so opening a second while the first is live raises rather than quietly corrupting it. Share the one app: agent() and run() are safe to call concurrently across different chat_ids.

And one mode per process: mount a gateway or hold a BosApp, never both. That exclusion is real but unguarded — nothing raises. Mounting a gateway runs the same bootstrap (on mount, and again on every POST /api/restart), which rebuilds the shared AgentRegistry out from under the BosApp, and vice versa; agents already built keep working, but every later build_agent(), actor start or restart resolves against the other side's workspace (docs/BACKLOG.md §4).

Turns on one chat_id are yours to serialize. In mode 1 nothing serializes them for you. Two turns running at once on the same chat_id each assemble context before the other has committed, so neither sees the other's message and their writes interleave — with no error and no warning. Queue per conversation, or reject the second turn, the way your application already handles two requests for one resource. (Mode 2 does this for you: the gateway's chat coordinator refuses a second turn on a busy chat with active_turn.)

Letting the user pick the model and thinking level

A turn can run on another model, or at another thinking level, than the agent's config names: pass them in llm_args. BOS keeps no selection between turns — remembering what the user picked is your app's job, as the chat_id is.

result = await agent.run(
    "chat-42", "refactor this",
    llm_args={"model": "anthropic/claude-opus-5-5", "reasoning_effort": "high"},
)

app.list_models(kind=None) says what to offer, for building those pickers:

for model, info in app.list_models().items():
    print(model, info["display_name"], info["efforts"], info["is_default"])

Keys are what llm_args["model"] takes; efforts are the llm_args["reasoning_effort"] values that model accepts (empty: it has no thinking level to pick); is_default marks what the agent runs when a turn names no model. BOS's own agent lists the models of every provider whose API key is set — ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY and so on, your config's envs and envfile included — in LiteLLM's ids. A Claude Code or Codex agent lists that runtime's native names instead.

The list is hard-coded in BOS and refreshed with its releases, and nothing is validated against it: a model it does not list yet still runs when you pass it, and a wrong one fails with the provider's own error.

Adding to a running turn, or stopping it

run(..., interrupt=poll) calls poll() while the turn runs — before each model call for a BOS agent, as each event arrives for Claude Code or Codex. What it returns is a message to deliver into the turn, never a request to stop. To stop, raise AbortTurn:

from bos.sdk import AbortTurn

def poll():
    if stop_pressed:
        raise AbortTurn()                                 # stops the turn
    if follow_up:
        return {"role": "user", "content": follow_up}     # delivered into the turn
    return None                                           # nothing new

result = await agent.run("chat-42", "summarize the repo", interrupt=poll)

A stopped turn returns (turn aborted before completion) … as its output, with finish_reason == "aborted". Any other truthy value — a dict without role and content, a bare string — raises TypeError from run() instead of reaching the turn (an external runtime raises RuntimeError, with the TypeError as its cause).

No extras required

The base install has no LLM client. Register your own provider and you need nothing else:

from bos.sdk import LLMResponse, ep_provider

@ep_provider(name="myco")
async def my_provider(messages: list[dict], model: str, **kwargs) -> LLMResponse:
    text = await my_client.complete(messages, model)   # your own client
    return LLMResponse(content=text)

Then use "model": "myco/whatever". Or install bos-ai[litellm] and use the built-in provider. Likewise, "platform": {"extensions": []} stops BOS auto-loading anything: import only the adapters you want, rather than bos.exts, which loads every built-in.

examples/embed_sdk.py is all of the above, runnable offline on a base install with no extras — two turns on one chat_id, a stub provider, and an event_sink showing the intra-turn events:

uv run python examples/embed_sdk.py

If you want less than BosApp

BosApp is a small object over one function. When you would rather own the harness yourself — a test, a script, an app with its own lifecycle object — open_harness is the same bootstrap with nothing on top:

from bos.sdk import open_harness

async with open_harness(workspace) as harness:
    agent = await harness.create_agent(kind="assistant")

Take workspace from app.workspace, or build one yourself with Workspace(workspace=".", bos_dir=..., config=...). The order inside open_harness is the contract: agent files are loaded before the platform registers them, and doing it the other way round drops every agent file with no error. That is why this exists as one function rather than two calls you repeat.

An agent backed by Claude Code or Codex

An agent can hand its turns to Claude Code or Codex instead of BOS's own loop — see External agents. Install bos-ai[claude-code] or bos-ai[codex], log in with the vendor's CLI, and build one like any other kind:

async with BosApp(CONFIG, bos_dir="/var/lib/myapp/.bos") as app:
    coder = await app.build_agent("codex", agent_cfg={
        "permission": "workspace-write",   # required
        "cwd": "services/api",
        "mcp_tools": ["CreateTicket"],     # your @ep_tools, exposed over MCP
        "timeout_seconds": 1200,
    })
    result = await coder.run("ticket-981", "Add a health endpoint and run the tests.")
    for message in await app.get_messages("ticket-981"):   # the runtime's own transcript
        print(message.llm_message["role"], message.llm_message["content"])

It is an AgentPort, not an Agent. app.agent(kind), app.build_agent(kind, ...) and harness.create_agent(kind, agent_cfg) all return the port: run(), ask(), request_stop() and name behave as they do for a BOS agent, and the same chat_id resumes the same native session on the next call — across restarts too, when the chat store persists. An agent your config declares with _parent = "codex" (or "claude-code") is prebuilt and reached with app.agent("<name>"), as is a bare [agents.codex] table. Without one, the bare kinds codex and claude-code are built with build_agent, whose agent_cfg carries the agent's keys; for a second variant, give it a new name with agent_cfg={"_parent": "codex", ...}.

Its config is checked when it is built. A missing permission, an unknown key, or a host the runtime refuses (a Claude Code subscription agent with ANTHROPIC_API_KEY in the environment, say) raises from build_agent. For an agent your config names, it raises when the block opens if that agent is the default, and from app.agent(name) otherwise (above). cwd resolves against the workspace root, which for BosApp(dict, bos_dir=...) is the process's current directory: pass a Workspace to fix it elsewhere.

Two records of the conversation. BOS stores two messages per turn — the user message and the final answer — and app.get_messages(chat_id, source="bos") returns that record, the one BOS guarantees. source="native" reads the runtime's own transcript, which holds more and promises nothing: the runtime prunes and compacts it on its own schedule. The default, source="auto", returns the native transcript for a chat an external agent served and BOS's record otherwise. Tool calls are in neither: they stream to an event_sink while the turn runs.


Mode 2 — mount the gateway

pip install bos-ai[gateway]

GatewayMount gives you the whole BOS runtime as an ASGI application you mount inside your own web app. Your host keeps its socket, its middleware and its auth; BOS never binds a port and never authenticates.

from contextlib import asynccontextmanager
from fastapi import FastAPI
from bos.config import Workspace
from bos.runner import GatewayMount

CONFIG = {...}   # the same dict shape as mode 1, plus [runtime] actors/channels

def make_app() -> FastAPI:
    mount = GatewayMount(
        lambda: Workspace(".", "/var/lib/myapp/.bos", CONFIG),
        public_base_url="https://myapp.example.com/bos",
    )

    @asynccontextmanager
    async def lifespan(_: FastAPI):
        await mount.start()
        try:
            yield
        finally:
            await mount.stop()

    app = FastAPI(lifespan=lifespan)
    app.mount("/bos", mount.build_app())
    return app

The first argument is a factory, not a workspace. A restart must re-read configuration, which means building a new Workspace — so GatewayMount is handed a callable it can call again.

build_app() is mounted once, before start(), and kept across restarts. The Gateway object behind it is replaced wholesale by a restart; the ASGI app in front is stable, so your host mounts it at import time and never re-mounts.

start() and stop() belong in your host's lifespan. They are what acquire and release the singleton lock and bring the actors and channels up and down.

public_base_url is yours to supply. You chose the socket and the mount path, so the gateway cannot discover its own URL. Passing it is what lets the status snapshot carry a usable base_url for clients and for boscli gateway restart.

bos_dir must be a real, writable path. The singleton lock, gateway.state and the channel cursors live in <bos_dir>/run/, so a mounted gateway cannot be entirely in memory even when the chat store is.

Mounted at /bos, you get GET /bos/api/status, GET /bos/api/actors, POST /bos/api/restart, the upload endpoints and WS /bos/ws. POST /api/restart re-reads configuration and rebuilds the runtime in place — new channels go live, a deleted agent stops being callable — without dropping your host process. mount.status() returns the same snapshot that route serves, so your own health endpoint can expose it directly.

examples/embed_gateway_fastapi.py is a complete host, runnable offline:

pip install bos-ai[gateway] fastapi uvicorn
uvicorn --factory examples.embed_gateway_fastapi:make_app --port 8123
curl -s http://127.0.0.1:8123/bos/api/status
curl -s -X POST http://127.0.0.1:8123/bos/api/restart

BOS performs no authentication

A mounted gateway is protected by whatever the host puts in front of it. A standalone gateway binds 127.0.0.1 by default; if you bind elsewhere, you front it yourself.


What both modes share

Almost everything. The two modes differ only in who drives a turn — below that line, it is one system:

  • Configuration — the same schema in both, dict or TOML. See the Configuration reference.
  • Agents — the same [agents.<kind>] tables and the same @ep_agent factories. Mode 2 additionally maps agents to named actors; mode 1 never touches the actor table.
  • Tools, plugins, providers, stores — the same @ep_tool, @ep_plugin, @ep_provider, @ep_chat_store registrations, resolved the same way. See Extending BOS.
  • Memory and skills — the same consolidation and the same skill directories. See Memory & skills.

So an extension you write for a mode-1 embed keeps working unchanged if you later mount the gateway. What you would rewrite is your own integration layer — the code that decides when a turn happens.


The contract

bos.sdk.__all__ is the promise. If a name is in it, it is supported; if it is not, it may move:

python -c "import bos.sdk; print(len(bos.sdk.__all__))"

It holds 41 names: BosApp, ModelInfo (what list_models returns) and open_harness; the agent surface (AgentPort, Agent, AgentHarness, AgentResult, Message, TurnContext, and AbortTurn, which stops a turn); the ports you can implement (LLM, ChatStore, Consolidator, ToolSet, TurnInterceptor, PromptProvider, TurnEventSink) together with every type those ports' own methods take or return, including the message-content types (MessageContent, TextPart, ImagePart, FilePart); the nine ep_* extension points; and Workspace, RootConfig, validate_config.

AgentPort is what the agent-returning calls are typed as — the four members a host actually uses (name, request_stop, ask, run). Agent is BOS's own implementation of it and satisfies it structurally; an agent backed by an external runtime implements the port without being an Agent (BEP 19 §3.3).

That last point is enforced, not aspirational: a test walks every Protocol in __all__ and fails if any type in one of its method signatures is unpromised. So you can implement any port in bos.sdk using only names from bos.sdk — writing a ChatStore never sends you hunting in bos.core for ChatCommit or TokenEstimate.

bos.sdk re-exports; it does not redefine. bos.sdk.Agent is bos.core.Agent, one class and one isinstance answer.

Everything else remains importable — including the _-prefixed helpers bos.core re-exports for extension authors — and is explicitly unstable. GatewayMount is mode 2's entry point and lives in bos.runner, outside the contract, because it depends on the [gateway] extra that bos.sdk must not require.

Where to go next

  • Configuration — every key the dict accepts.
  • Extending BOS — tools, plugins, channels, providers.
  • Runtime & gateway — what mode 2 is actually running.
  • CLI — boscli ask and the gateway commands, for driving the same project from a terminal.