Getting started

From zero to your first export.

Five steps. Everything below is real, copyable code against the actual API — not pseudocode. Full reference in the docs.

Install the SDK

One package, no framework lock-in.

npm install @dataegress/sdk

Define your dataset

A fetchPage function — your own query, your own tenant isolation. DataEgress calls this over HTTP; it never connects to your database.

import { defineDataset } from "@dataegress/sdk";

const transactions = defineDataset({
  id: "transactions",
  fetchPage: async ({ tenantId, cursor, limit }) => {
    const rows = await db.transaction.findMany({
      where: { tenantId, id: { gt: cursor ? Number(cursor) : 0 } },
      orderBy: { id: "asc" },
      take: limit,
    });
    return {
      rows,
      nextCursor: rows.length < limit ? null : String(rows.at(-1)!.id),
    };
  },
});

export const POST = transactions.handler(); // mount at e.g. /api/dataegress/transactions

Run your first export

curl -X POST https://app.dataegress.dev/api/datasets \
  -H 'content-type: application/json' \
  -d '{"id": "transactions", "fetchUrl": "https://your-app.com/api/dataegress/transactions"}'

curl -X POST https://app.dataegress.dev/api/exports \
  -H 'content-type: application/json' \
  -d '{"tenantId": "acme", "dataset": "transactions", "destination": "acme-s3", "format": "parquet"}'

Configure S3 (or R2)

Point a destination at a bucket your tenant owns. Credentials are encrypted at rest and never echoed back by the API.

import { DataEgress } from "@dataegress/sdk";

const dataEgress = new DataEgress({ baseUrl: "https://app.dataegress.dev" });

await dataEgress.createDestination({
  tenantId: "acme",
  id: "acme-s3",
  type: "s3",
  s3: {
    bucket: "acme-data-lake",
    region: "us-east-1", // "auto" for Cloudflare R2
    accessKeyId: "...",
    secretAccessKey: "...",
  },
});

Add a schedule

Standard 5-field cron, evaluated in UTC. Checked every minute — same code path as a manual export.

await dataEgress.createSchedule({
  tenantId: "acme",
  dataset: "transactions",
  destination: "acme-s3",
  format: "parquet",
  cron: "0 2 * * *", // every day at 02:00 UTC
});

That’s the whole integration.

DataEgress runs the schedule, delivery, retries and manifests — there’s no queue or worker to operate on your side. Questions, or need a custom limit or BYOC setup?

View docs Contact us