Skip to main content

pipelines

@rtorcato/cf-common/pipelines wraps a Pipeline binding, Cloudflare's streaming ingestion into R2. The binding's send() takes an array of untyped records; createPipelineProducer gives you a typed send(record) and a chunked sendBatch(records).

import { createPipelineProducer } from '@rtorcato/cf-common/pipelines'
import { getBinding } from '@rtorcato/cf-common/env'
import type { Pipeline } from 'cloudflare:pipelines'

interface PageView extends Record<string, unknown> {
userId: string
path: string
at: number
}

const views = createPipelineProducer<PageView>(getBinding<Pipeline<PageView>>(env, 'VIEWS'))

Bind it​

Create the pipeline with Wrangler, then bind it:

{
"pipelines": [{ "pipeline": "page-views", "binding": "VIEWS" }]
}

Send​

await views.send({ userId, path: url.pathname, at: Date.now() })

The record type is asserted, not validated: pair it with your own schema if the data comes from outside.

Send in batches​

await views.sendBatch(rows.map((r) => ({ userId: r.id, path: r.path, at: r.at })))

sendBatch splits the records into send calls of 100 and sends them in order. Pass { chunkSize } to change that; the default is this package's choice, not a limit Cloudflare publishes for the binding. It stops at the first chunk that fails, so earlier chunks stay delivered. A chunkSize that isn't a positive integer throws a CloudflareError with status 500 and code PIPELINE_INVALID_CHUNK_SIZE before anything is sent.

When a send fails​

A rejected send becomes a CloudflareError with status 502, code PIPELINE_SEND_FAILED and the original error as cause. The message is not exposed, so defineFetch returns a generic error to the client. See errors for how to catch it.

Use ctx.waitUntil(views.send(...)) if the response shouldn't wait on the pipeline, and catch the rejection yourself in that case.

See the pipelines API reference for the full signatures.