Embedding BOS¶
You have an application, and you want an agent inside it. BOS gives you two ways to do that, and they are genuinely different shapes — not a beginner version and an advanced one. Choose before you write code. Switching later is a rewrite of your integration layer, because the two modes put the conversation in different places.
Which mode¶
| Mode 1 — call the agent | Mode 2 — mount the gateway | |
|---|---|---|
| You write | agent.run(chat_id, text) |
Nothing per turn |
| Who drives a conversation | Your code | BOS's actors and channels |
| Concurrency, queueing, turn takeover | Yours to build | Built in |
| Telegram / Lark / WebSocket clients | No | Yes |
| Install | pip install bos-ai |
pip install bos-ai[gateway] |
| Needs a writable directory | Only if you configure file-backed stores | Always |
| Runnable example | examples/embed_sdk.py |
examples/embed_gateway_fastapi.py |
Pick mode 1 if your application already owns the conversation — a CLI, a batch job, a Slack bot you wrote, an HTTP endpoint that answers one question. You call the agent the way you call any other async function.
Pick mode 2 if you want what boscli gateway start runs — named actors, chat
coordination, persistent channels, the WebSocket protocol — but inside your own
process and behind your own auth, instead of supervising a second daemon.
Mode 1 — call the agent in your own process¶
That is the library install. It ships no boscli command, no terminal UI and no
gateway; see the install table for the
extras. Configuration is a plain dict, so it can come from your database, your
environment or a control plane — a config.toml on disk is the CLI's
convenience, not a requirement.
import asyncio
from bos.sdk import BosApp
CONFIG = {
"platform": {"extensions": []},
"agents": {
"assistant": {
"system_prompt": "You are a concise assistant.",
"model": "openai/gpt-4o",
}
},
}
async def main() -> None:
async with BosApp(CONFIG, bos_dir="/var/lib/myapp/.bos") as app:
agent = app.agent()
first = await agent.run("chat-42", "hello")
print(first.output)
again = await agent.run("chat-42", "what did I just say?")
print(again.output)
asyncio.run(main())
Four things that snippet is showing you:
agent() returns an AgentPort, not a reply. There is deliberately no
app.ask(). run() takes ten parameters — streaming via event_sink,
structured output via schema, interrupt, llm_args, turn_id and more — and
any one-line façade over it sends you back down a layer the moment you want one
of them. Dropping to the lower layer therefore hands you the same objects, not
a different path: app.harness and app.workspace are exactly the ones BosApp
is using.
Chat continuity is an id, not an API. Both calls above pass "chat-42", so
the second turn sees the first in its history. There is no session object to
create, hold or close. Use your own conversation key.
Agents are built once, at __aenter__, and cached. app.agent() is
synchronous because of that. Every kind named in config["agents"] is built as
the block opens; with no argument you get the default (the top-level
default_agent key, or the only agent, or one named main). For a kind that
only an @ep_agent factory registers — not named in your config — build it once
with await app.build_agent("kind").
An agent that cannot be built costs only itself. If a kind your config names
fails to build — a Claude Code agent refusing an ANTHROPIC_API_KEY in the
environment, say — the block still opens with every other agent. The failure is
logged, app.failed_agents maps the kind to its exception (for a picker to grey
out), and app.agent(kind) raises chained to it. Fix the cause and
await app.build_agent(kind) builds it without reopening the app. The default is
the exception: if it cannot be built, async with BosApp(...) raises.
One BosApp per process. Bootstrap writes os.environ and rebuilds the
agent registry, both process-global, so opening a second while the first is live
raises rather than quietly corrupting it. Share the one app: agent() and
run() are safe to call concurrently across different chat_ids.
And one mode per process: mount a gateway or hold a BosApp, never both.
That exclusion is real but unguarded — nothing raises. Mounting a gateway runs
the same bootstrap (on mount, and again on every POST /api/restart), which
rebuilds the shared AgentRegistry out from under the BosApp, and vice versa;
agents already built keep working, but every later build_agent(), actor start
or restart resolves against the other side's workspace (docs/BACKLOG.md §4).
Turns on one chat_id are yours to serialize. In mode 1 nothing serializes
them for you. Two turns running at once on the same chat_id each assemble
context before the other has committed, so neither sees the other's message and
their writes interleave — with no error and no warning. Queue per conversation,
or reject the second turn, the way your application already handles two requests
for one resource. (Mode 2 does this for you: the gateway's chat coordinator
refuses a second turn on a busy chat with active_turn.)
Letting the user pick the model and thinking level¶
A turn can run on another model, or at another thinking level, than the agent's
config names: pass them in llm_args. BOS keeps no selection between turns —
remembering what the user picked is your app's job, as the chat_id is.
result = await agent.run(
"chat-42", "refactor this",
llm_args={"model": "anthropic/claude-opus-5-5", "reasoning_effort": "high"},
)
app.list_models(kind=None) says what to offer, for building those pickers:
for model, info in app.list_models().items():
print(model, info["display_name"], info["efforts"], info["is_default"])
Keys are what llm_args["model"] takes; efforts are the
llm_args["reasoning_effort"] values that model accepts (empty: it has no
thinking level to pick); is_default marks what the agent runs when a turn names
no model. BOS's own agent lists the models of every provider whose API key is set
— ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY and so on, your
config's envs and envfile included — in LiteLLM's ids. A Claude Code or Codex
agent lists that runtime's native names instead.
The list is hard-coded in BOS and refreshed with its releases, and nothing is validated against it: a model it does not list yet still runs when you pass it, and a wrong one fails with the provider's own error.
Adding to a running turn, or stopping it¶
run(..., interrupt=poll) calls poll() while the turn runs — before each model
call for a BOS agent, as each event arrives for Claude Code or Codex. What it
returns is a message to deliver into the turn, never a request to stop. To stop,
raise AbortTurn:
from bos.sdk import AbortTurn
def poll():
if stop_pressed:
raise AbortTurn() # stops the turn
if follow_up:
return {"role": "user", "content": follow_up} # delivered into the turn
return None # nothing new
result = await agent.run("chat-42", "summarize the repo", interrupt=poll)
A stopped turn returns (turn aborted before completion) … as its output, with
finish_reason == "aborted". Any other truthy value — a dict without role and
content, a bare string — raises TypeError from run() instead of reaching the
turn (an external runtime raises RuntimeError, with the TypeError as its cause).
No extras required¶
The base install has no LLM client. Register your own provider and you need nothing else:
from bos.sdk import LLMResponse, ep_provider
@ep_provider(name="myco")
async def my_provider(messages: list[dict], model: str, **kwargs) -> LLMResponse:
text = await my_client.complete(messages, model) # your own client
return LLMResponse(content=text)
Then use "model": "myco/whatever". Or install bos-ai[litellm] and use the
built-in provider. Likewise, "platform": {"extensions": []} stops BOS
auto-loading anything: import only the adapters you want, rather than bos.exts,
which loads every built-in.
examples/embed_sdk.py
is all of the above, runnable offline on a base install with no extras — two
turns on one chat_id, a stub provider, and an event_sink showing the
intra-turn events:
If you want less than BosApp¶
BosApp is a small object over one function. When you would rather own the
harness yourself — a test, a script, an app with its own lifecycle object —
open_harness is the same bootstrap with nothing on top:
from bos.sdk import open_harness
async with open_harness(workspace) as harness:
agent = await harness.create_agent(kind="assistant")
Take workspace from app.workspace, or build one yourself with
Workspace(workspace=".", bos_dir=..., config=...). The order inside
open_harness is the contract: agent files are loaded before the platform
registers them, and doing it the other way round drops every agent file with no
error. That is why this exists as one function rather than two calls you repeat.
An agent backed by Claude Code or Codex¶
An agent can hand its turns to Claude Code or Codex instead of BOS's own loop — see
External agents. Install bos-ai[claude-code] or
bos-ai[codex], log in with the vendor's CLI, and build one like any other kind:
async with BosApp(CONFIG, bos_dir="/var/lib/myapp/.bos") as app:
coder = await app.build_agent("codex", agent_cfg={
"permission": "workspace-write", # required
"cwd": "services/api",
"mcp_tools": ["CreateTicket"], # your @ep_tools, exposed over MCP
"timeout_seconds": 1200,
})
result = await coder.run("ticket-981", "Add a health endpoint and run the tests.")
for message in await app.get_messages("ticket-981"): # the runtime's own transcript
print(message.llm_message["role"], message.llm_message["content"])
It is an AgentPort, not an Agent. app.agent(kind), app.build_agent(kind, ...)
and harness.create_agent(kind, agent_cfg) all return the port: run(), ask(),
request_stop() and name behave as they do for a BOS agent, and the same chat_id
resumes the same native session on the next call — across restarts too, when the chat
store persists. An agent your config declares with _parent = "codex" (or
"claude-code") is prebuilt and reached with app.agent("<name>"), as is a bare
[agents.codex] table. Without one, the bare kinds codex and claude-code are built
with build_agent, whose agent_cfg carries the agent's keys; for a second variant,
give it a new name with agent_cfg={"_parent": "codex", ...}.
Its config is checked when it is built. A missing permission, an unknown key, or a
host the runtime refuses (a Claude Code subscription agent with ANTHROPIC_API_KEY in the
environment, say) raises from build_agent. For an agent your config names, it raises
when the block opens if that agent is the default, and from app.agent(name) otherwise
(above). cwd resolves against the workspace root, which for BosApp(dict, bos_dir=...)
is the process's current directory: pass a Workspace to fix it elsewhere.
Two records of the conversation. BOS stores two messages per turn — the user message
and the final answer — and app.get_messages(chat_id, source="bos") returns that record,
the one BOS guarantees. source="native" reads the runtime's own transcript, which holds
more and promises nothing: the runtime prunes and compacts it on its own schedule. The
default, source="auto", returns the native transcript for a chat an external agent
served and BOS's record otherwise. Tool calls are in neither: they stream to an
event_sink while the turn runs.
Mode 2 — mount the gateway¶
GatewayMount gives you the whole BOS runtime as an ASGI application you mount
inside your own web app. Your host keeps its socket, its middleware and its auth;
BOS never binds a port and never authenticates.
from contextlib import asynccontextmanager
from fastapi import FastAPI
from bos.config import Workspace
from bos.runner import GatewayMount
CONFIG = {...} # the same dict shape as mode 1, plus [runtime] actors/channels
def make_app() -> FastAPI:
mount = GatewayMount(
lambda: Workspace(".", "/var/lib/myapp/.bos", CONFIG),
public_base_url="https://myapp.example.com/bos",
)
@asynccontextmanager
async def lifespan(_: FastAPI):
await mount.start()
try:
yield
finally:
await mount.stop()
app = FastAPI(lifespan=lifespan)
app.mount("/bos", mount.build_app())
return app
The first argument is a factory, not a workspace. A restart must re-read
configuration, which means building a new Workspace — so GatewayMount is
handed a callable it can call again.
build_app() is mounted once, before start(), and kept across restarts. The
Gateway object behind it is replaced wholesale by a restart; the ASGI app in
front is stable, so your host mounts it at import time and never re-mounts.
start() and stop() belong in your host's lifespan. They are what acquire
and release the singleton lock and bring the actors and channels up and down.
public_base_url is yours to supply. You chose the socket and the mount path,
so the gateway cannot discover its own URL. Passing it is what lets the status
snapshot carry a usable base_url for clients and for boscli gateway restart.
bos_dir must be a real, writable path. The singleton lock, gateway.state
and the channel cursors live in <bos_dir>/run/, so a mounted gateway cannot be
entirely in memory even when the chat store is.
Mounted at /bos, you get GET /bos/api/status, GET /bos/api/actors,
POST /bos/api/restart, the upload endpoints and WS /bos/ws. POST
/api/restart re-reads configuration and rebuilds the runtime in place — new
channels go live, a deleted agent stops being callable — without dropping your
host process. mount.status() returns the same snapshot that route serves, so
your own health endpoint can expose it directly.
examples/embed_gateway_fastapi.py
is a complete host, runnable offline:
pip install bos-ai[gateway] fastapi uvicorn
uvicorn --factory examples.embed_gateway_fastapi:make_app --port 8123
curl -s http://127.0.0.1:8123/bos/api/status
curl -s -X POST http://127.0.0.1:8123/bos/api/restart
BOS performs no authentication
A mounted gateway is protected by whatever the host puts in front of it. A
standalone gateway binds 127.0.0.1 by default; if you bind elsewhere, you
front it yourself.
What both modes share¶
Almost everything. The two modes differ only in who drives a turn — below that line, it is one system:
- Configuration — the same schema in both, dict or TOML. See the Configuration reference.
- Agents — the same
[agents.<kind>]tables and the same@ep_agentfactories. Mode 2 additionally maps agents to named actors; mode 1 never touches the actor table. - Tools, plugins, providers, stores — the same
@ep_tool,@ep_plugin,@ep_provider,@ep_chat_storeregistrations, resolved the same way. See Extending BOS. - Memory and skills — the same consolidation and the same skill directories. See Memory & skills.
So an extension you write for a mode-1 embed keeps working unchanged if you later mount the gateway. What you would rewrite is your own integration layer — the code that decides when a turn happens.
The contract¶
bos.sdk.__all__ is the promise. If a name is in it, it is supported; if it is
not, it may move:
It holds 41 names: BosApp, ModelInfo (what list_models returns) and open_harness; the agent surface (AgentPort,
Agent, AgentHarness, AgentResult, Message, TurnContext, and AbortTurn, which
stops a turn); the ports you
can implement (LLM, ChatStore, Consolidator, ToolSet, TurnInterceptor,
PromptProvider, TurnEventSink) together with every type those ports' own
methods take or return, including the message-content types (MessageContent,
TextPart, ImagePart, FilePart); the nine ep_* extension points; and
Workspace, RootConfig, validate_config.
AgentPort is what the agent-returning calls are typed as — the four members a
host actually uses (name, request_stop, ask, run). Agent is BOS's own
implementation of it and satisfies it structurally; an agent backed by an
external runtime implements the port without being an Agent (BEP 19 §3.3).
That last point is enforced, not aspirational: a test walks every Protocol in
__all__ and fails if any type in one of its method signatures is unpromised. So
you can implement any port in bos.sdk using only names from bos.sdk — writing
a ChatStore never sends you hunting in bos.core for ChatCommit or
TokenEstimate.
bos.sdk re-exports; it does not redefine. bos.sdk.Agent is
bos.core.Agent, one class and one isinstance answer.
Everything else remains importable — including the _-prefixed helpers
bos.core re-exports for extension authors — and is explicitly unstable.
GatewayMount is mode 2's entry point and lives in bos.runner, outside the
contract, because it depends on the [gateway] extra that bos.sdk must not
require.
Where to go next¶
- Configuration — every key the dict accepts.
- Extending BOS — tools, plugins, channels, providers.
- Runtime & gateway — what mode 2 is actually running.
- CLI —
boscli askand the gateway commands, for driving the same project from a terminal.