#AI assistant

alepha/ai puts a model behind one provider contract and gives your app an assistant that knows the page the user is on and can run your features for them. There is no vendor SDK: every provider is plain fetch and ReadableStream, so the same code runs on Node, Bun and Cloudflare Workers.

1import { $ai, AiProvider } from "alepha/ai";

#Choosing a provider

The vendor is the module you register, never an env string. Without one, MemoryAiProvider answers: specs script it, and in development it replies that no vendor module is registered, so a page with a chat renders and names its own cause.

Start with Workers AI, and fall back to Anthropic where it cannot do the job (a model without function calling, or answers that are not good enough). The choice is per app; there is no automatic failover between vendors.

AlephaAiCloudflare only substitutes the provider on Workers, so an app deployed there can register it unconditionally and still run yarn dev on Node. Register AlephaAiOpenai after it to reach the same Workers AI models over REST locally and in Docker:

1import { Alepha, run } from "alepha";2import { AlephaAiCloudflare } from "alepha/ai/cloudflare";3import { AlephaAiOpenai } from "alepha/ai/openai";4 5run(Alepha.create().with(AlephaAiCloudflare).with(AlephaAiOpenai));

The first substitution wins, so the order matters: on Workers the binding provider substitutes first and answers; elsewhere it substitutes nothing and the REST provider answers. The Cloudflare build declares the ai binding when the module is registered.

Every failure is a typed error from one family, whatever the vendor: AiAuthError, AiRateLimitError (with retryAfter), AiOverloadedError, AiContextLengthError, AiToolsUnsupportedError (the model cannot call tools), all extending AiError.

#Calling a model directly

AiProvider streams one response as typed events. complete() collects them:

 1import { $inject } from "alepha"; 2import { AiProvider } from "alepha/ai"; 3  4class Summary { 5  ai = $inject(AiProvider); 6  7  async summarize(text: string) { 8    const reply = await this.ai.complete({ 9      system: "Summarize in one sentence.",10      messages: [{ role: "user", content: text }],11    });12    return reply.text;13  }14}

#An assistant: $ai

$ai declares an assistant whose tools are your $actions:

 1import { $inject, z } from "alepha"; 2import { $ai, type AiMessage } from "alepha/ai"; 3import { $secure, type UserAccountToken } from "alepha/security"; 4import { $action } from "alepha/server"; 5  6class InvoiceApi { 7  list = $action({ 8    method: "GET", 9    path: "/invoices",10    description: "List the invoices of the signed-in user",11    use: [$secure()],12    schema: {13      query: z.object({ status: z.enum(["open", "paid"]).optional() }),14      response: z.array(z.object({ id: z.integer(), status: z.text() })),15    },16    handler: () => [],17  });18 19  create = $action({20    method: "POST",21    path: "/invoices",22    description: "Create an invoice",23    use: [$secure()],24    schema: {25      body: z.object({ amount: z.number() }),26      response: z.object({ id: z.integer() }),27    },28    handler: () => ({ id: 1 }),29  });30}31 32class Support {33  invoices = $inject(InvoiceApi);34 35  assistant = $ai({36    system: "You help with invoices. Be brief.",37    tools: [this.invoices.list, this.invoices.create],38  });39 40  async answer(messages: AiMessage[], user: UserAccountToken) {41    const result = await this.assistant.run({ messages, user });42    return result.text;43  }44}

• A tool is an action. Its name is the action's name (letters, digits, _ and -, at most 64), its description the action's description (required: it is how the model knows when to call it), and its input an object with params, query and body for the parts the action declares. A bad name or a missing description fails at start.
• It runs as the user. Each call goes through action.run(input, { user }) with the user you pass, so the action's own use: [$secure(...)] decides: the assistant can never do more than the user could. Without a user the call runs anonymous and a secured action refuses; there is no "system" fallback. A refusal, a validation error or a thrown error goes back to the model as a tool error, never to the browser as a 403.
• Bounds. maxSteps (10 model calls by default) ends the turn with end: length; timeout and the ambient abort signal cancel the call in flight.

#Confirmation and resume

A call to an action whose method is not GET (or whatever confirm says) does not run on its own. The turn pauses: stream() emits a confirm event with the call id and its input, then end: paused. A client tool, declared with clientTool(name, { description, schema }), pauses the same way with a client_tool event, because the browser runs it (navigate, fillForm).

The loop is re-entrant: its whole state is the message list. Persist every message event as it comes; resuming is a new call with the stored messages and the user's answer:

 1await assistant.run({ 2  messages: stored, 3  user, 4  resume: { callId, decision: "approve" }, 5}); 6await assistant.run({ 7  messages: stored, 8  user, 9  resume: { callId, result: { ok: true } },10});

An approved call runs the input stored in the messages, never input sent back by the browser, and a call id nothing waits on is refused. A denial reaches the model as the tool error "Denied by the user.". Nothing lives in memory between the two requests, which is the shape that works on Workers.

#Page context

Pass context: { route, page, data } with a new turn and it rides on the user's message as a delimited <page-context> block, capped at 8 KB and truncated with a marker past it. It never goes into the system prompt: it changes every turn, which would void the vendor's prompt cache, and it is client-sent data, which the model must read as the user's. The augmented message is emitted as a message event, so what you store is what the model saw; AiContextBlock.strip() removes the block for display.

#The server side: alepha/api/ai

AlephaApiAi stores conversations per user and serves them, on top of your $ai:

Every endpoint is $secure() and scoped to the caller: another user's conversation is a 404, never a 403. On the streams, every refusal happens before the stream opens, so it is the response's status: 401 signed out, 404 not yours, 409 not waiting on that call, 429 over the turn rate or the token budget (AiBudgetExceededError). Each message is stored as it completes, tokens on the assistant rows; a pause stores the call, and a resume claims it once.

1import { Alepha, run } from "alepha";2import { AlephaAiCloudflare } from "alepha/ai/cloudflare";3import { AlephaApiAi } from "alepha/api/ai";4 5// The vendor module FIRST: AlephaApiAi registers alepha/ai, and the first6// provider substituted is the one used.7run(Alepha.create().with(AlephaAiCloudflare).with(AlephaApiAi).with(MyApp));

The module ships its entities (ai_conversations, ai_messages), not migrations: generate them with the app's (alepha db migrations create), as for every alepha/api/* module. With several $ais in the app, a chat names the one it talks to (assistant).

#Budget, rate and retention: the api.ai parameter

Runtime settings, editable by an operator without a redeploy (alepha/api/parameters):

The budget is checked before a turn from the stored token counts, so a turn can overshoot by its own size. Retention is a daily $job that reads the parameter at every tick. Page data and tool results live in the messages, so retention is a privacy setting as much as a storage one.

#The client side: @alepha/ui/ai

 1import { AiChat, useAiContext } from "@alepha/ui/ai"; 2  3const InvoicePage = (props: InvoicePageProps) => { 4  const form = useForm({ schema: invoiceSchema, handler: save }); 5  // What the page shows, and the form the assistant may fill (never submit). 6  useAiContext({ page: "invoice", data: props.invoice, form }); 7  const [open, setOpen] = useState(false); 8  return ( 9    <>10      <Button onClick={() => setOpen(true)}>Ask</Button>11      <AiChat open={open} onOpenChange={setOpen} invalidates={[["invoices"]]} />12    </>13  );14};

• <AiChat /> is a drawer: streamed markdown replies, the conversation history, and a confirmation card for every mutation, which shows the input the server stored and answers with the call id only.
• useAiChat() is the same state and actions without the UI: items, streaming, pending, send, approve, deny, open, each resolving a RunResult. invalidates lists the query keys to refresh after a turn ran a tool, so the page shows what the assistant changed.
• useAiContext({ page, data, form }) declares what a page shows. Every turn carries the current route and the innermost declared context.
• Client tools run in the browser and the turn resumes with their result on its own: navigate (by page name, through the typed router) and fillForm (only the form a page offered, never submitted). Declare them on your $ai with aiNavigateTool and aiFillFormTool from alepha/ai; your own go in useAiChat({ clientTools }).

The browser opens a $sse through useClient() like any action: a refusal rejects with its typed error, the events arrive as they are sent.

#One build, Node and Workers

apps/examples/ai is a notes app with one read action, one write action and an assistant, built once with alepha build --runtime node,workerd:

• on Workers, AlephaAiCloudflare answers through the env.AI binding, which the build declares in wrangler.jsonc (and the Lore deploy uploads);
• in Docker (alepha image), the same build runs on Node, and AlephaAiOpenai reaches the same Workers AI models over REST with AI_BASE_URL=https://api.cloudflare.com/client/v4/accounts/<id>/ai/v1 and a Cloudflare API token as AI_API_KEY.

Only the environment differs. Its e2e runs the Node build against a scripted MemoryAiProvider: ask, confirm the write, see the page change.

#Testing

MemoryAiProvider is the default under test. Script one response per model call:

1import { Alepha } from "alepha";2import { AlephaAi, MemoryAiProvider } from "alepha/ai";3 4const alepha = Alepha.create().with(AlephaAi);5const ai = alepha.inject(MemoryAiProvider);6ai.script([{ type: "tool_call", id: "c1", name: "list", input: {} }]);7ai.script([{ type: "text", text: "You have two invoices." }]);

ai.requests keeps every request, so a spec asserts what the model would have been sent.