Phantom — Technical Architecture

Phantom Architecture

How It Works

Phantom is a Kubernetes operator that injects a sidecar via mutating webhook. The sidecar fetches secrets from an EU-hosted OpenBao/Vault instance directly into process memory — secrets never touch etcd, never enter Kubernetes Secrets, and the cloud provider never holds the keys.

Key Strengths

• Secrets never touch etcd. Eliminates an entire class of attacks (etcd dump, backup exfiltration, KMS compulsion). The correct approach for managed Kubernetes where you have zero control over the control plane.
• Three-tier caching is well-designed. Hot cache → sealed local cache → grace period progression is operationally sound. Sealed cache key derivation from SA token + cluster HMAC is reasonable.
• Circuit breaker on the webhook is the right pattern for fail-closed security products. The override escape hatch (namespace label) is correctly positioned as an auditable last resort.
• Canary injection via namespace labels is operationally mature thinking for a product that modifies every pod in the cluster.

Known Concerns

• gVisor as “optional lightweight sandbox” is undersold. Without it, a root-level attacker on the node can read process memory via /proc/[pid]/mem. The “optionally” qualifier weakens the story.
• The sidecar is a single point of failure per pod. If phantom-proxy crashes and the sealed cache is expired, the application loses access to all secrets. Consider a direct (attested) fallback path to OpenBao.
• Env var patching requires applications to use environment variables or a specific socket protocol. Applications that read secrets from files need a different mechanism — solvable but not addressed.

Trust Model

Technical Feasibility

Phantom Complexity Breakdown

• Mutating webhook: well-understood pattern, excellent Go libraries. 2-3 weeks for a senior Go engineer.
• OpenBao integration (secret fetch, caching, renewal): 3-4 weeks. Three-tier cache adds complexity but is well-scoped.
• Sidecar injection with mesh awareness: 4-6 weeks. Compatibility matrix (Istio, Linkerd, OTel, Dapr, GKE FUSE) is the time sink.
• Circuit breaker + operator lifecycle: 2-3 weeks.
• Cross-provider testing matrix: 4-6 weeks. The hidden cost — testing on GKE Standard, GKE Autopilot, EKS (EC2 + Fargate), and AKS.
• Attestation (SEV-SNP/TDX): 6-8 weeks. Requires specialized knowledge.
• Total: ~5-7 months for production-ready Phantom with attestation.

What Needs More Research

1. Nitro Enclaves integration — fundamentally different from SEV-SNP/TDX. Needs PoC before committing.
2. eBPF memory-access monitoring — what can eBPF detect that’s actionable? Detect-and-alert vs. detect-and-block?

Key Technical Limitations

What Phantom Cannot Do

1. Cannot protect data processed in cleartext. Once the app decrypts a secret, data exists in cleartext in application memory. TEE mitigates this but isn’t universal.
2. Cannot protect against a compromised application. If the application is malicious (supply chain attack), it has legitimate access to decrypted secrets.
3. Cannot protect Kubernetes metadata. Pod names, labels, annotations, network policies — all visible to the cloud provider.
4. Cannot protect against hardware-level attacks on TEEs. AMD SEV-SNP and Intel TDX have had side-channel vulnerabilities (CacheWarp, speculative execution).
5. Cannot enforce key sovereignty after key release. Once a secret is released into the sidecar’s memory, it’s in the cloud provider’s infrastructure.
6. Cannot protect against legal coercion of the customer. This product protects against US extraterritorial reach, not all legal compulsion.

Protection Model Breakdown

Attacks NOT Defended Against

• Supply chain attacks on application container images
• Side-channel attacks on TEE implementations (timing, power analysis, cache-based)
• Social engineering of personnel with access to OpenBao
• Insider threats from the customer’s own team
• Network-level DDoS preventing connectivity to OpenBao
• Container escape followed by host memory access (without TEE)
• Coerced firmware updates on TEE hardware by cloud provider at government request

Cross-Provider Compatibility

Phantom Cross-Provider Status

Phantom’s core architecture (webhook + sidecar + external secrets) works across all three providers with provider-specific code paths for networking, identity, and attestation.

Undocumented Issues to Address

1. GKE Workload Identity Federation — default SA token behavior changes may affect sealed cache key derivation.
2. EKS Pod Identity — sidecar’s OpenBao auth must support both traditional K8s auth and provider-specific identity federation.
3. AKS Node Auto-Provisioning (Karpenter) — eBPF programs must handle different kernel versions within the same cluster.
4. GKE Gateway API migration — network policies may need to understand Gateway API resources.
5. EKS Access Entries — changes who can interact with the API server and bypass webhooks.
6. Multi-tenant GKE clusters (GKE Enterprise) — fleet-level policies can override per-cluster webhook configurations.
7. ARM / Graviton nodes — eBPF programs, sidecar images, and crypto must be multi-arch.
8. Windows node pools — current architecture is Linux-only. Should be explicitly documented as unsupported.
9. Spot / preemptible node eviction — sidecar must handle SIGTERM gracefully and clean up sealed cache.
10. Network policies (Calico/Cilium) — sidecar needs explicit NetworkPolicy rules to reach OpenBao.

Scalability Analysis

Pod Scale Assessment

OpenBao — Biggest Scalability Risk

Missing Mitigations

• Request coalescing — if 50 pods request the same secret simultaneously, OpenBao should be hit once, not 50 times.
• Batch secret fetch — 3 secrets in one API call instead of 3 sequential calls reduces connection overhead 3x.
• Staggered renewal — add jitter to TTL to spread renewal load.

eBPF Overhead at Scale

At 100 pods per node, a memory-access tracepoint on sys_read/sys_write could fire millions of times per second. Even incrementing a counter adds 50-200ns per syscall.

Sidecar Resource Overhead

Tech Stack

Go — Right Choice for Phantom

OpenBao vs Alternatives

eBPF vs Alternatives for Monitoring

MVP Scope — “Secrets That Never Touch etcd”

Phantom Core Components

1. Mutating admission webhook that injects phantom-proxy sidecar into labeled pods
2. Sidecar that fetches secrets from external OpenBao and exposes them via: environment variables, Unix domain socket, and mounted tmpfs file
3. In-memory cache with TTL-based renewal (skip sealed local cache for MVP)
4. Pre-flight connectivity check (Job-based, writes to ConfigMap)
5. Helm chart designed for EKS add-on constraints (no hooks, no lookup)
6. Single-provider launch: GKE Standard (simplest webhook behavior)

What to Cut from v1

Critical Path to First Deployable Version

Comparison to Alternatives

Alt 1: Full Confidential Computing (Just Use TEEs)

Trade-off: Full CC is simpler but more expensive and less available. Phantom works on standard VMs and adds CC as optional enhancement — correct positioning for reaching the broadest market.

Alt 2: Sovereign Cloud (Use EU Providers)

Alt 3: Client-Side Encryption Libraries

Alt 4: VPN to On-Premises HSM

Technically works but adds significant operational complexity (VPN management, on-premises infrastructure, latency). Phantom’s managed OpenBao is operationally simpler. However, for customers with existing on-prem HSMs (banks, defense), this should be a supported deployment mode.

Technical Risks

High-Impact Risks

Dependency Risks

Architecture Improvements

Concrete changes that raise the architecture score to 8.5/10.

┌──────────────────────────────────────┐
│  Node                                      │
│  [Pod A] [Pod B] [Pod C] [Pod D]           │
│     │       │       │       │              │
│     └───────┴───┬───────┘  UDS         │
│                │                           │
│       [Phantom DaemonSet, ~80MB]             │
│       [    Shared Cache         ]             │
│                │ mTLS                      │
└────────────────┴─────────────────────┘
                 │
          [  OpenBao EU  ]

type SecretProvider interface {
    GetSecret(ctx context.Context, path string, identity PodIdentity) (*Secret, error)
    WatchSecret(ctx context.Context, path string) (<-chan SecretEvent, error)
    RevokeLeases(ctx context.Context, identity PodIdentity) error
    HealthCheck(ctx context.Context) error
}

# compatibility-db/charts/bitnami/postgresql/16.4.0.yaml
chart:
  repository: bitnami
  name: postgresql
  versions_tested: ["16.4.0", "16.3.x", "15.x"]

injection:
  status: "compatible"    # compatible | partial | incompatible | untested
  mode: "sidecar"         # sidecar | daemonset | both

testing:
  method: "automated"
  platform: "gke-standard"
  k8s_versions: ["1.29", "1.30", "1.31"]

# Cache key derivation for StatefulSets
cache_key = HKDF-SHA256(
    ikm:  service_account_token,
    salt: cluster_hmac_key,
    info: "phantom-statefulset:" + statefulset_name + ":" + pod_ordinal
)

Score Impact Summary

Verdict

Recommendations

1. Ship Phantom alone. Nothing else in v1. Phantom on GKE Standard is the MVP. Add EKS in v1.1, AKS in v1.2.
2. Add a DaemonSet mode as alternative to per-pod sidecars for customers with 1,000+ pods.
3. Implement request coalescing and batch secret fetching to mitigate OpenBao bottleneck risk.
4. Abstract the OpenBao dependency behind a SecretProvider interface from day one.
5. Invest in a testing strategy proportional to security claims: fuzzing, property-based testing, integration tests on all providers.
6. Be explicit about what’s not protected. Build honest security documentation that CISOs will trust.