ai
@rtorcato/cf-common/ai wraps the Workers AI binding's run(). You pass the
model name and its inputs, and the return type follows from the model. It adds
one thing on top: a failed call becomes a CloudflareError instead of a raw
upstream error.
import { runAi } from '@rtorcato/cf-common/ai'
import { getBinding } from '@rtorcato/cf-common/env'
const ai = getBinding<Ai>(env, 'AI')
Run a model
const { response } = await runAi(ai, '@cf/meta/llama-3.2-3b-instruct', {
prompt: 'Name one Cloudflare product.',
})
// response: string | undefined
The model name is checked against the Workers AI model map in
@cloudflare/workers-types. Pass a name that isn't there, or inputs in the wrong
shape, and the compiler stops you before deploy.
Options
The third argument is passed to run() as is, so gateway settings and the other
binding options work the same way:
await runAi(ai, '@cf/meta/llama-3.2-3b-instruct', { prompt: 'hi' }, {
gateway: { id: 'my-gateway' },
})
When the call fails
Quota limits, model errors and upstream outages throw a CloudflareError with
status 502, code AI_RUN_FAILED and the original error as cause. The
message is not exposed to clients, so under defineFetch the response is a
generic Internal error. Log cause if you need the detail. See
errors for how to catch it.
Good to know
runAicovers the normalrun()call. Streaming and WebSocket responses come back asReadableStreamandResponse, so use the binding'srun()directly for those.- A missing
AIbinding is a deploy misconfiguration.getBindingthrows a non-exposed500for it, the same as the other bindings.
For a runnable walkthrough, see the /ai route in apps/example. See the
ai API reference for every option.