Data Flow

How data moves through ReplayCI — from CLI execution to dashboard display.

---

Overview

ReplayCI processes LLM test data through a strict pipeline. Every stage enforces security checks, and no data reaches storage without passing through SecurityGate.

CLI → SecurityGate scan → HTTPS POST → API validation → SecurityGate scan
→ Filesystem (encrypted) + PostgreSQL (RLS) → Dashboard (session-authenticated)

---

1. CLI execution

When you run npx replayci, the CLI:

1. Loads config from .replayci.yml (merged with CLI flags)
2. Reads contract YAML files from the specified pack directory
3. Executes contracts against the configured LLM provider
4. Builds a RunArtifactV02 envelope containing all step results, traceability metadata, and fingerprints

No data leaves your machine unless REPLAYCI_API_KEY is set. The --persist flag writes artifacts to local disk only.

What's in a run artifact

Field	Description
run_id	Unique identifier (r_ prefix + UUID)
tenant_id	Your team identifier
provider / model	Which LLM was tested
steps[]	Per-contract results with fingerprints
traceability	Runner version, corpus hash, timestamps
determinism_proof	Cross-run fingerprint comparison (if --repeat used)

---

2. API submission

When REPLAYCI_API_KEY is set, the CLI pushes the run artifact to the hosted API:

POST https://app.replayci.com/api/v1/runs
Authorization: Bearer rci_live_...
Content-Type: application/json

Validation checks

1. API key authentication — middleware validates the Bearer token and resolves the tenant
2. Email verification — unverified accounts are rejected
3. Rate limiting — per-tenant monthly run budget (rolling 30-day window)
4. Payload size — maximum size enforced
5. Schema validation — required fields are checked
6. SecurityGate scan — full artifact is scanned for secrets before any persistence

What happens on failure

Every validation failure returns an appropriate HTTP status and halts processing. The CLI never throws on push failure — errors are reported to stderr as warnings.

---

3. SecurityGate scanning

SecurityGate runs on every persist, export, upload, log, and render path (fail-closed). It scans for known secret patterns including API keys, authentication tokens, credentials, and accidental PII.

If any pattern matches, the entire operation is denied — there is no bypass flag.

Redaction happens before hashing, so fingerprints are computed on clean data.

---

4. Persistence (two-phase)

Run artifacts are persisted in two systems simultaneously:

Filesystem (encrypted blobs)

Artifacts are written to the filesystem under a namespaced path:

/{tenantId}/runs/{runId}/blobs/{contentHash}

Each blob is encrypted with AES-256-GCM using envelope encryption (V2). Encryption keys are derived per-tenant via HKDF from a root secret. See Infrastructure Security for encryption details.

PostgreSQL (structured metadata)

Run metadata is written to PostgreSQL within a database transaction, covering run records, per-contract results, evaluation outcomes, and argument checks.

All tables enforce row-level security (RLS) scoped by tenant_id. The persistence layer uses idempotent writes for crash safety.

Artifact deduplication

Before writing a blob, the system checks the database for an existing record with the same content hash. Duplicate blobs are skipped — only the metadata reference is created.

---

5. Dashboard display

The dashboard at app.replayci.com is a Next.js application with session-based authentication:

1. User authenticates via email + password → receives a signed session cookie
2. Middleware validates the session token on every request
3. All database queries are scoped by tenant_id via ORM client extensions
4. Encrypted blobs are decrypted on-demand with the tenant's derived key

What you can see

• Run history with pass/fail status and fingerprints
• Per-contract step details and failure categories
• Baseline lifecycle (CANDIDATE → ACTIVE → STALE → RETIRED)
• Drift detection results
• API key management

What you can't see

• Other tenants' data (RLS enforced at the database level)
• Raw encryption keys (derived per-request, never stored in application memory)
• Decrypted artifacts from other tenants (cryptographic isolation via HKDF)

---

Data at rest vs. in transit

State	Protection
In transit	TLS 1.2+ (Cloudflare edge → origin), HTTPS enforced
At rest (blobs)	AES-256-GCM envelope encryption, per-tenant key derivation
At rest (database)	PostgreSQL with RLS, encrypted connection strings
In logs	Structured logger with automatic secret redaction

---

Network boundaries

Boundary	What crosses it
CLI → API	Run artifact JSON (over HTTPS)
API → Filesystem	Encrypted blob bytes
API → PostgreSQL	Structured metadata (tenant-scoped)
Dashboard → Browser	Rendered HTML + decrypted artifact data (over HTTPS)
CLI → LLM Provider	Contract prompts + tool schemas (your responsibility to review)

The CLI communicates with LLM providers directly — ReplayCI does not proxy or intercept these calls. Provider API keys (REPLAYCI_PROVIDER_KEY) are used locally and never sent to the ReplayCI API.