FlowBridge overview
FlowBridge is the runtime, governance, and delivery surface of
DataChord. It takes the DAGDocument compiled by MapCraft and gives you
everything you need to run it safely on a schedule, ship its output to a
warehouse or partner, and publish it as a serverless package.
If MapCraft asks "what should this data become?", FlowBridge asks "how, where, and how often does it get there — and what guard-rails apply when it does?".
What you can do
- Compile a DAG from MapCraft mappings and trigger it (manual, scheduled, drift-fix, or backfill).
- Apply policies — mask, tokenize, or redact at the field level — that run at compile time and runtime.
- Configure Extract & Load for warehouse sources (Postgres, Snowflake) with CDC modes and high-water tracking.
- Configure Delivery Adapters to drop documents over SFTP, HTTP, file, or webhook.
- Watch for drift and accept (or reject) auto-proposed fixes.
- Schedule runs with cron or interval cadences.
- Build serverless packages for AWS Lambda, GCP Cloud Run, or GCP Cloud Function.
The six orchestration surfaces
Each surface is independent — you can use FlowBridge for just Delivery, just Schedules, or all six. They share the same DAGDocument so you do not have to reconfigure anything twice.
Core concepts
| Concept | What it is |
|---|---|
DAGDocument | The compiled IR (JSON) of a MapCraft project: nodes, edges, metadata. Versioned per compile. |
DAGRun | One execution of a DAG. Has a status, a trigger, a records_processed count, and an optional error message. |
DAGSchedule | A cron or interval cadence attached to a DAG. |
DeliveryAdapter | A configured output channel — type (SFTP / HTTP / FILE / WEBHOOK) plus credentials reference. |
ExtractLoadState | High-water mark state for CDC. Tracks the last extracted value of the watermark column. |
Policy | A field-level mask / tokenize / redact rule. Enforced when the DAG is compiled and when it runs. |
OAuthGrant | A stored OAuth grant for a sync-to-app target. |
PackageArtifact | A built serverless bundle — Lambda zip, Cloud Run image manifest, or Cloud Function bundle — with a SHA-256 and a publish state. |
DriftEvent | A detected change in the source (schema, freshness, validator) with a fix proposal. |
DAGRun status
A run starts in pending as soon as it is queued. Once a worker picks it
up, it moves to running. It ends in success (with records_processed)
or failed (with error_message and a stack-trace reference).
Trigger model
A DAGRun can be started in four ways.
The trigger is recorded on the DAGRun itself, so you can filter run
history by how it was started — useful when a scheduled run fails and you
need to confirm whether your manual re-run succeeded.
Policy-as-Code
Policies attach to target fields and have three actions:
| Action | What it does | Typical use |
|---|---|---|
| Mask | Replace identifying parts of a value with a placeholder. | customer_email → ***@domain.tld for marketing exports. |
| Tokenize | Replace with a deterministic token that round-trips through a vault. | PII that must be re-identifiable by a privileged service. |
| Redact | Drop the field entirely from the output. | Fields a partner explicitly must not see. |
Policies are scoped to a destination — a specific delivery adapter, a specific warehouse target, or "all destinations". A single field can carry different policies for different destinations.
Policies are validated when the DAG is compiled and re-validated at run time. A misconfigured policy (unknown field, missing vault binding) will fail the run, by design. The alternative — silently emitting unmasked data — is never the default. If you see a policy validation error, fix the policy; do not bypass it.
Where to go next
- Hands-on tutorial: FlowBridge walkthrough — take the compiled DAG from MapCraft, apply a policy, schedule it, deliver it, and build a Lambda package.
- Previous stage: MapCraft overview.
- Cross-suite map: Product flow & architecture.