DataEgressGet started

Data delivery for B2B SaaS

Your customers want their data in S3.
Stop building the pipeline yourself.

Ship reliable scheduled CSV and Parquet exports to customer-owned storage, with retries, manifests, checksums and monitoring built in.

Get startedView docs

Your query. Their bucket. We handle the delivery. Or try the live demo →

The problem

One enterprise request, six pieces of infrastructure.

When an enterprise customer asks for a daily data feed, your team ends up building background jobs, pagination for large tables, a Parquet writer, multipart S3 uploads, retry logic, and failure monitoring — for a feature that isn’t your core product. It gets built once, badly, under deadline pressure, and then nobody wants to touch it again.

What this replaces

Build it yourself, or don’t.

Build it yourself

  • – Background jobs
  • – Pagination
  • – Streaming
  • – CSV / Parquet encoding
  • – Multipart uploads
  • – Retries
  • – Idempotency
  • – Checksums
  • – Manifests
  • – Monitoring
  • – Failure recovery

DataEgress

Included.

One SDK call and a destination — the rest is handled.

How it fits into your product

You keep the query. We run the pipeline.

Your SaaSdefineDataset()DataEgressCSV / ParquetCustomer S3

DataEgress never connects to your database. Your own fetchPage function is the only thing that ever queries your data — see why that’s the whole architecture.

How it works

Three steps, no queue to operate.

01

Define your dataset

Wrap your own query in a fetchPage function — tenant isolation and column selection stay entirely yours.

02

Choose schedule and destination

Point it at a signed URL or a customer-owned S3/R2 bucket, and set a cron — daily, hourly, whatever fits.

03

We deliver and verify every export

Streaming, multipart uploads, retries, and a checksummed manifest — on every single run.

Verifiable delivery

Every delivery is verifiable.

You can prove what was delivered. Every completed export produces a manifest with the exact row count, a checksum computed from the bytes actually written, and a schema version — not a status you have to take on faith.

  • ✓ Row count
  • ✓ Checksum
  • ✓ Schema version
  • ✓ Delivery status
  • ✓ Idempotency key
{
  "export_id": "exp_8f2c1a9b4d3e",
  "status": "completed",
  "rows": 200000,
  "schema_version": "1",
  "checksum": "3a7f9e1c...b6d2",
  "generated_at": "2026-08-27T02:00:00.000Z",
  "completed_at": "2026-08-27T02:00:04.200Z",
  "files": [
    {
      "path": "acme/events/2026-08-27.parquet",
      "rows": 200000,
      "bytes": 14184320,
      "checksum": "3a7f9e1c...b6d2"
    }
  ]
}

Reliability

Built for the failure cases, not just the happy path.

✓

Streaming, not buffering

Built and tested for large exports — memory stays bounded by page size, not total row count.

✓

Multipart uploads

Correct past S3's 5GB single-PUT limit — a naive implementation fails silently there.

✓

Retries without duplicate delivery

Deterministic destination keys mean a retry overwrites, never duplicates.

✓

Row counts & checksums

Every file's SHA-256 is computed by streaming the actual bytes written, not asserted.

✓

Verifiable manifests

Schema version, row count, checksum, per-file breakdown — on every completed run.

✓

Real cron scheduling

Checked every minute, evaluated in UTC, same code path as a manual export.

✓

Honest failure states

Which stage failed and how many rows were processed before it did.

✓

Customer-owned storage

Delivers to S3/R2 buckets your customer controls — tested against real Cloudflare R2.

Validated with 1M-row streaming exports. Run the live demo to see it end to end.

Developer experience

This is the whole integration.

No boilerplate, no config file. Your query, exposed as one HTTP handler.

import { defineDataset } from "@dataegress/sdk";

const dataset = defineDataset({
  id: "transactions",
  fetchPage: async ({ tenantId, cursor, limit }) => {
    const rows = await yourOwnQuery({ tenantId, cursor, limit });
    return { rows, nextCursor: rows.length < limit ? null : rows.at(-1).id };
  },
});

export const POST = dataset.handler(); // mount anywhere, point DataEgress at its URL

Full walkthrough in Getting started.

Your data stays under your control

Trust comes from the architecture, not a badge.

✓

No direct database access

DataEgress only calls the HTTP endpoint you expose.

✓

Customer-controlled queries

Your fetchPage function decides what leaves your database.

✓

Customer-owned storage destinations

Buckets your customer owns, not ours.

✓

Encrypted destination credentials

Encrypted at rest, never echoed back by the API.

✓

Streaming processing

Every export is processed as a stream, never loaded into memory as a whole.

✓

Retry-safe delivery

Deterministic destination keys mean a retry overwrites, never duplicates.

✓

Verifiable manifests

Row count, checksum and schema version on every completed run.

✓

Deterministic checksums

Computed from the bytes actually written, not asserted.

Full detail in Security and Limits.

Use cases

Built for the ask your enterprise deals keep surfacing.

Enterprise customer data feeds

Give your biggest customers a nightly feed instead of a one-off export call.

Scheduled S3 exports

Cron-driven delivery to a bucket your customer owns — checked every minute, no drift.

Large Parquet exports

Typed, compressed files your customer's data team can load straight into DuckDB or Spark.

Customer data portability

A structured, verifiable export path for customers who want their data out.

Analytics / product data delivery

Stream millions of event rows out without loading them into memory.

Your next enterprise customer shouldn’t require another custom export pipeline.

Get startedContact us