AI
Use Workers AI with streaming chat, file summaries, credit reservations, queues, retries, and recovery.
Source ownership
| Module | Responsibility |
|---|---|
packages/ai | Workers AI calls and model input |
packages/db/src/store.ts | Ledger, reservations, settlement, and compensation |
apps/web/src/server/jobs.ts / packages/db/src/index.ts | Task dispatch, outbox, leases, and retries |
apps/web/src/server/api.ts | Session, input, and ownership checks |
apps/web/src/server.ts | Queue consumption and scheduled recovery |
The current model is @cf/meta/llama-3.1-8b-instruct through the AI binding. Switching models requires checking input/output contracts, stream format, cost, and tests; the starter does not automatically configure every AI provider.
Local versus real inference
Vite defaults to remoteBindings: false so local startup does not require a remote account. This does not emulate an AI model. A visible chat screen does not prove inference is available.
For real acceptance, configure a Workers AI binding and quota in your own account, then deliberately use remote bindings or your deployment. Calls can consume service resources. Do not use the public demo for load or cost tests, and never paste credentials into issue reports.
Streaming chat
POST /api/chat accepts:
{
"prompt": "Explain the uploaded release notes",
"requestId": "e60b2e4b-32b1-42a5-bfd0-dbb20ecb37e9"
}Prompt length is 1–16,000 JavaScript characters. requestId is optional but must be a UUID when provided. Generate a fresh ID for a new operation rather than reusing this example. Consume text/event-stream progressively and handle interrupted streams and errors; do not call response.json().
Chat defaults to one credit, controlled by product.quotas.chatCredits. The server rejects insufficient balance. Disabling a button is not a balance check.
File summaries
- Upload your own text/plain or text/markdown file.
- POST
{ "fileId": "your-owned-file-uuid" }to /api/summary. - Inspect the created background task instead of expecting a synchronous full-text response.
- Read the result or inspect failure state and use existing retry behavior.
Summary cost defaults to five credits through summaryCredits. Although uploads can reach 1 MiB, summaries use only the first 16,000 characters. There is no automatic long-document chunking, image OCR, or arbitrary PDF ingestion.
Credits and asynchronous consistency
Welcome credits default to 30 through welcomeCredits. Operations reserve credits atomically and settle or compensate when the result is known. Task/outbox constraints and leases coordinate execution. Do not deduct in the client or charge again on queue retry.
JOBS drives background work. The Worker has primary and dead-letter consumers, with three retries in the current configuration. A five-minute cron recovers task/outbox work requiring attention. Duplicate delivery, retries, and process interruption must use the same idempotent state machine rather than direct UI balance edits.
Administration and recovery
Users see their own tasks and usage. Administrators inspect tasks, audit, and dead letters. Recovery must identify the exact task and respect its state, permissions, and lease. For long-running pending work, inspect queues, consumers, and outbox records before creating duplicates.
Verify failure paths
- Empty/oversized prompts, insufficient balance, and another user's file must fail without duplicate charges.
- For chat, check progressive output and ledger settlement; for interruption, check the visible error and credit state.
- For summaries, test success, model failure, duplicate delivery, expired leases, and dead-letter recovery.
- No local output: verify whether AI can actually run with the selected binding.
- Stuck pending: inspect queue bindings, consumers, cron, and recoverable outbox state.
Unit and Workers tests exercise logic boundaries. Real inference, production quota, and result quality require separate acceptance. See testing and deployment.