Context & Scope

System Context (C4 Level 1)

The system context shows Akili and the external actors and systems it interacts with.

C4Context
    title System Context — Akili Platform

    Person(dev, "Data Product Developer", "Writes YAML manifests + SQL/Python logic")
    Person(ops, "Platform Operator", "Monitors health, manages tenants, handles incidents")
    Person(consumer, "Data Consumer", "Queries data products via API or dashboards")

    System(akili, "Akili Platform", "Data Product PaaS — orchestration, execution, quality, serving, governance")

    System_Ext(pg, "PostgreSQL (CNPG)", "OLTP store — registry, state, RLS tenant isolation")
    System_Ext(redpanda, "Redpanda", "Event streaming — domain events, per-tenant topics")
    System_Ext(ceph, "Ceph RGW", "S3-compatible object storage — data lake")
    System_Ext(starrocks, "StarRocks", "OLAP analytics — Iceberg federation")
    System_Ext(dagster, "Dagster", "Orchestration engine — asset graph, sensors, IO managers")
    System_Ext(authentik, "Authentik", "Identity provider — OIDC, SSO, RBAC groups")
    System_Ext(argocd, "ArgoCD", "GitOps — App-of-Apps deployment pattern")
    System_Ext(gitlab, "GitLab", "Source control, CI/CD pipelines, container registry")

    Rel(dev, akili, "Declares products via CLI/API", "HTTPS")
    Rel(ops, akili, "Monitors via Portal", "HTTPS")
    Rel(consumer, akili, "Queries data via API", "HTTPS")
    Rel(akili, pg, "Reads/writes state", "TCP/5432")
    Rel(akili, redpanda, "Publishes/consumes events", "TCP/9092")
    Rel(akili, ceph, "Stores data lake files", "S3 API")
    Rel(akili, starrocks, "Analytical queries", "MySQL protocol")
    Rel(akili, dagster, "Triggers executions", "gRPC/HTTP")
    Rel(akili, authentik, "Authenticates users", "OIDC")
    Rel(argocd, gitlab, "Syncs manifests", "HTTPS")

Business Context

Interface	Partner	Protocol	Purpose
CLI / REST API	Developers	HTTPS	Manifest submission, validation, deployment
Portal	Operators	HTTPS	Monitoring, tenant management, dashboards
Query API	Consumers	HTTPS	Data access with tenant-scoped RLS
Event bus	Internal services	Redpanda (TCP)	Domain events (product registered, execution completed, quality checked)
Object storage	Dagster IO managers	S3 API (Ceph RGW)	Data lake reads/writes
GitOps	ArgoCD	Git (HTTPS)	Infrastructure deployment and drift detection

Technical Context

Network Boundaries

All services run in a single K3s cluster on Hetzner bare-metal servers. External access is through Cloudflare (DNS + WAF) and Traefik (ingress controller). Internal services communicate over the cluster network with namespace-scoped NetworkPolicies.

Zone	Access	Services
Public (Cloudflare → Traefik)	External HTTPS	Portal, API, docs
Internal (cluster network)	Pod-to-pod	All backend services
Data plane (dedicated nodes)	Internal only	Dagster, Spark, StarRocks
Storage plane (Ceph)	Internal only	Ceph OSDs, RGW

Authentication Flow

1. User authenticates via Authentik (OIDC)
2. Portal receives JWT token via NextAuth
3. Portal proxies API calls with JWT (BFF pattern)
4. Control-plane validates JWT and extracts tenant_id
5. Every database query is scoped by tenant_id (RLS)