OCI L4 Relay Specification
1. Problem Statement
Public services that require direct L4/TLS ingress use a stable cloud endpoint without publishing or accepting service traffic on the residential WAN address. Tier R preserves original client addresses and terminates TLS inside the cluster while keeping the home firewall outside the public ingress data path.
2. Deployed Design
A single-purpose OCI VM provides Tier R L4 ingress. A service assigned to Tier R uses a Cloudflare gray-cloud A record for the OCI reserved IPv4 address. The VM forwards TCP/80 and TCP/443 over WireGuard to the dedicated in-cluster Cilium Gateway identity referred to as Gateway R. It forwards TCP/25, TCP/465, and TCP/993 to the mail LoadBalancer.
The OCI VM provides transparent L4 forwarding. TLS terminates at the declared cluster targets.
Internet -> OCI public IP -> DNAT -> WireGuard -> Gateway R or L4 service
OPNsense is not in the Tier R ingress data path. For Tier R, it passes only outbound WireGuard traffic from the Kubernetes-scheduled K3s node to OCI. Its WAN address is control-plane input used to constrain OCI WireGuard ingress to one IPv4 /32.
| Identity |
Meaning |
| Tier R |
Public relay ingress through OCI over WireGuard |
| Gateway R |
Dedicated Cilium Gateway used by Tier R |
gateway-relay |
Kubernetes object name for Gateway R |
3. Architecture
%%{init: {'theme': 'neutral'}}%%
flowchart LR
subgraph internet["Internet"]
Client["External Client"]
DNS["Cloudflare DNS\nA records"]
end
subgraph oci["OCI"]
NSG["Relay NSG\nservice admission"]
VM["Relay VM\nnftables + WireGuard"]
OCISecrets["OCI Secrets\nkey seed + public key"]
OCIAPI["OCI API\nrestricted NSG update"]
end
subgraph home["Home LAN"]
OPN["OPNsense\nWAN status API"]
Controller["WAN Controller\n10s poll + 2 observations"]
Node["K3s Nodes\nactive owner + standby"]
Cilium["Cilium\nLB + Gateway API"]
GW["Gateway R\ngateway-relay"]
Mail["Mail\nLoadBalancer"]
SVC["Public Services"]
Vault["Vault\npeer, PKI, API material"]
end
DNS --> Client
Client -->|"TCP 25/80/443/465/993"| NSG --> VM
VM -->|"DNAT, no SNAT"| Node
VM <-->|"WireGuard"| Node
Node -->|"routed to Gateway R"| GW
Node -->|"routed to mail address"| Mail
GW --> SVC
OCISecrets --> VM
Vault --> Node
Vault --> Controller
Controller -->|"PKI-validated HTTPS"| OPN
Controller -->|"replace exact UDP source rule"| OCIAPI --> NSG
Node -. "outbound UDP only" .-> OPN
3.1 Data Path
| Step |
Source |
Destination |
Notes |
| 1 |
External client |
OCI public service port |
Cloudflare DNS returns the reserved OCI address. |
| 2a |
OCI relay |
Gateway R:80/443 |
OCI applies DNAT and forwards web traffic over WireGuard. |
| 2b |
OCI relay |
Mail target:25/465/993 |
OCI applies DNAT and forwards mail traffic over WireGuard. |
| 3 |
K3s node |
Selected LoadBalancer |
Cilium delivers the packet to Gateway R or the mail target without another DNAT layer. |
| 4a |
Cilium Gateway |
Web listener |
Relay forwarding ends at the declared Gateway listener. |
| 4b |
Mail target |
Mail protocol listener |
Relay forwarding ends at the declared mail listener. |
| 5 |
Cluster service |
External client |
Replies route back through the OCI WireGuard tunnel. |
The relay must preserve the original client source address. Therefore the OCI VM performs DNAT only and does not SNAT traffic forwarded to the cluster.
3.2 Control Plane Ownership
| Component |
Owner |
Management model |
| OCI infrastructure, IAM, monitoring, budget |
OpenTofu |
Declarative OCI provider resources |
| Cloudflare service DNS |
OpenTofu |
Gray-cloud records consume the reserved address from OCI remote state |
| Sealed relay image |
Packer and pinned OCI SDK |
Digest-identified build from a pinned Oracle Linux platform image |
| OCI relay key seed and public key |
OCI Secrets |
Deterministic private-key derivation; change-aware public-key publication |
| Home-side WireGuard key material |
Vault |
Private key consumed by Kubernetes; public key rendered into relay configuration |
| Admission source cache and publishers |
Argo Workflows and Longhorn |
Independent web-allow and SMTP-deny compilation, validation, and publication |
| Admission bundles |
OCI Object Storage |
Policy-specific validation, caching, and atomic activation |
| Cilium Gateway identity |
Kubernetes manifests |
Dedicated gateway-relay / Gateway R |
| Home-side tunnel and policy routing |
Kubernetes host-level workload |
Two anti-affined hostNetwork replicas; an exact-name Lease admits one active owner |
| WireGuard source restriction |
Kubernetes controller |
Polls OPNsense WAN over Vault-PKI TLS and updates one OCI NSG rule |
| OPNsense API identity and trust root |
Vault and Vault PKI |
VSO projects a read-only API credential and the dedicated OPNsense root CA |
| Guest logs |
VictoriaLogs |
Authenticated bounded shipping outside the relay availability path |
4. Network Design
4.1 Identities and Addressing
Operational design refers to Gateway identities while the implementation renders their assigned addresses into OCI nftables, WireGuard, and home-side routing.
| Identity or Network |
Purpose |
Requirement |
| OCI public address |
Public DNS target |
Reserved IPv4; IPv6 is out of scope for v1 |
| OPNsense WAN address |
WireGuard NSG source |
Current public IPv4 represented as one exact /32 |
| WireGuard transfer CIDR |
Relay tunnel |
10.151.47.238/30; does not overlap pod, service, LAN, or LB ranges |
Gateway R / gateway-relay |
Relay-specific public Gateway |
Dedicated Cilium LoadBalancer address 10.104.40.127 |
| Mail LoadBalancer |
Direct mail protocol target |
Dedicated Cilium LoadBalancer address 10.66.153.202 |
The WireGuard transfer CIDR must not overlap:
| Existing Range |
Purpose |
10.90.141.119/16 |
Cilium LoadBalancer IP pool |
172.19.28.14/12 |
Cilium pod CIDR |
| Home LAN ranges |
Internal routing and OPNsense policy |
4.2 OCI Relay Rules
The OCI VM owns public DNAT.
| Rule |
Requirement |
| Public TCP/80 |
DNAT to Gateway R TCP/80 over WireGuard |
| Public TCP/443 |
DNAT to Gateway R TCP/443 over WireGuard |
| Public TCP/25 |
DNAT to the mail target TCP/25 over WireGuard |
| Public TCP/465 |
DNAT to the mail target TCP/465 over WireGuard |
| Public TCP/993 |
DNAT to the mail target TCP/993 over WireGuard |
| SNAT |
Disabled for forwarded client traffic |
| Forwarding |
Enabled only between public interface and wg-oci0 for approved ports |
| Rate limiting |
Enforced before traffic enters WireGuard to protect the relay path |
| SSH |
No OCI network security group ingress or authorized keys; sshd.service and sshd.socket are masked in the sealed image |
| WireGuard UDP |
Open only on the relay port from the current OPNsense WAN IPv4 /32 |
The relay uses an atomically loaded declarative nftables ruleset and does not mutate iptables.
4.3 Relay Rate Limiting
The OCI VM enforces L4 ingress limits before conntrack, DNAT, and WireGuard forwarding. The controls preserve service availability under hostile ingress; their numeric ceilings are containment settings, not throughput objectives.
Rate limiting is not a substitute for application-level quotas or authentication. It is a transport guardrail for the relay path.
| Control |
Deployed contract |
| Traffic-control shaping |
Shape traffic toward home at 150 Mbit/s |
| Web SYN controls |
120/minute with burst 120 per source; 1,000/second with burst 1,000 aggregate |
| SMTP SYN controls |
60/minute with burst 20 per source; 500/second with burst 500 aggregate |
| Authenticated-mail controls |
30/minute with burst 15 per source; 200/second with burst 200 aggregate |
| Aggregate packet ceilings |
20,000/second web, 10,000/second SMTP, and 5,000/second authenticated mail, each with an equal burst |
| HTTP-only quarantine |
After 30 unmatched TCP/80 attempts/hour with a five-packet burst, deny TCP/80 for 30 minutes |
| HTTPS invariant |
A TCP/443 attempt records a ten-minute exception, clears HTTP-only quarantine, and remains outside it |
| State bounds |
Each dynamic per-source tracker holds at most 65,536 entries and has an explicit timeout |
| Protocol boundary |
WireGuard handshakes are outside web, SMTP, authenticated-mail, quarantine, and shaping admission |
| Observability |
Named aggregate counters and per-element SMTP quarantine counters expose relay drops |
| Configuration |
OpenTofu renders every limit into the sealed guest configuration |
4.4 Address Admission
flowchart LR
Geography[MaxMind and IPLocate\nJP KR SG US Europe] --> Candidate[Geographic consensus]
VPNGeo[MaxMind or IPLocate\nJP KR] --> VPNException[Configured VPN ASN exception]
VPNASN[Explicit operator ASNs] --> VPNException
Infrastructure[Cloud CDN and VPS ranges] --> Deny[Selective deny union]
Abuse[Tor Spamhaus selective IP2Proxy] --> Deny
Candidate --> Difference[Admitted IPv4 set]
VPNException --> Difference
Deny --> Difference
Difference --> Version["web-allow/v1\nimmutable version"]
Version --> Current[Atomic current promotion]
Current --> Guest[Guest refresh]
Guest --> Rules[Atomic web admission table]
Guest --> LastGood[Validated last-known-good cache]
| Boundary |
Contract |
| Purpose |
Reduce hosting, anonymizer, and known-abuse exposure as defense in depth; never treat source classification as human identity |
| Geographic base |
Intersect MaxMind and IPLocate country data for Japan, Korea, Singapore, the United States, and Europe |
| VPN operator exception |
Intersect explicitly configured VPN ASNs with the union of MaxMind and IPLocate Japan/Korea ranges, exclude the geographic base, and subtract the selective deny union |
| Selective deny |
Subtract published cloud, CDN, VPS, Tor, Spamhaus DROP, and fresh selective IP2Proxy ranges |
| Publisher |
Run the multi-source fetch and compile DAG in-cluster with provider cadence, validators, bounded retry, and a durable private cache |
| Publication |
Publish immutable versions and current state under web-allow/v1; retain legacy admission/v1 only during consumer migration |
| Guest refresh |
Read the current private object through the instance principal every six hours, validate it independently, and install it atomically |
| Failure behavior |
Retain the validated last-known-good bundle; a sealed guest without a valid fresh or cached bundle fails closed |
| Monitoring exceptions |
Admit only the exact external health-check source addresses rendered by OpenTofu ahead of geographic filtering |
| IPv6 |
Outside the Tier R v1 admission boundary |
TCP/25 uses a separate smtp-deny/v1 transport policy.
| Boundary |
Contract |
| Evidence |
Relay-observed IPv4 source, transport reason, observation window, count, and expiry only |
| Excluded evidence |
Message content, recipients, DNSBL, SPF, DKIM, DMARC, authentication, and spam classification |
| Bounds |
At most 4,096 covered addresses, 1,024 intervals, a 10-minute observation window, 20 observations, and a 24-hour lifetime |
| Failure behavior |
Invalid, absent, expired, oversized, or unreachable current and cached state installs an empty deny set |
| Decay |
Each element expires no later than its declared expiry; omission from the next authorized publication removes it promptly |
| Isolation |
Separate prefix, schema, publisher identity, cache, helper, timer, nftables table, counters, and alarms from web-allow |
syn-rate evidence is a 60-second local snapshot of canonical IPv4, cumulative drop count, and expiry from per-element counters on the existing ten-minute smtp_syn_quarantine_v4; the reader is unprivileged with only CAP_NET_ADMIN, and the 65,536-source enforcement capacity remains independent from bundle caps.
4.5 Home-Side Routing
Because client source addresses are preserved, the home-side node must return relay ingress responses through the OCI tunnel. Replies must not use the normal LAN default route through OPNsense.
The active home-side workload routes configured LAN and WireGuard client CIDRs through the main table before selecting table 51820 for other replies sourced from Gateway R at 10.104.40.127 or the mail target at 10.66.153.202. It reconciles stale local state before competing for ownership and removes the link, routes, rules, and relay-owned nftables table when ownership ends. This keeps preserved public client destinations on the OCI return path without capturing internal connections to the same Gateway identity.
4.6 WireGuard Source Reconciliation
stateDiagram-v2
[*] --> Poll
Poll --> Retain: API, TLS, or address validation failure
Poll --> Observe: public WAN IPv4 differs
Observe --> Poll: candidate changes
Observe --> Update: same candidate observed twice
Update --> Retain: OCI update or response validation failure
Update --> Probe: exact NSG rule replaced
Probe --> Poll: relay TCP/443 recovers within 90s
Probe --> Retain: recovery probe fails
| Control |
Contract |
| Runtime |
One digest-pinned, non-root controller on a control-plane node with no service-account token |
| OPNsense polling |
Read the interface overview every ten seconds and select the interface named WAN |
| Trust |
Validate HTTPS against the dedicated pki_opnsense root; never disable certificate or hostname verification |
| Credential scope |
Use the read-only OPNsense interface-watcher API identity projected by Vault Secrets Operator |
| Address validation |
Accept only a public, unicast IPv4 address |
| Debounce |
Require two consecutive observations of the same new address |
| OCI mutation |
Replace only the identified stateful UDP WireGuard NSG rule while preserving its ID, port, protocol, and description |
| OCI response check |
Require OCI to return exactly that rule with the requested /32 and expected immutable fields |
| Recovery check |
Require TCP/443 on the relay address to recover within 90 seconds |
| Failure behavior |
Retain the current rule on polling or validation failure; retry a failed candidate without widening the source boundary |
| OpenTofu ownership |
Declare the rule and ignore only controller-owned source drift |
5. Cilium Interaction Requirements
The home-side tunnel must avoid conflicting with Cilium-managed networking.
| Requirement |
Rationale |
| Do not modify Cilium-owned iptables or nftables chains |
Cilium owns service, LB, host firewall, and proxy packet paths. |
| Do not flush host firewall rulesets |
Flushing can remove Cilium-managed policy state. |
Use the dedicated WireGuard interface wg-oci0 |
Avoids collision with Cilium WireGuard encryption. |
| Use a dedicated routing table and priority |
Keeps relay return routing separate from node default routing. |
Decouple AllowedIPs from route installation |
Preserved client sources require 0.0.0.0/0 peer association without a tunnel default route. |
| Do not perform cluster-side DNAT |
OCI owns the only DNAT layer and leaves Cilium service handling unchanged. |
| Preserve packet-capture access for diagnostics |
Cilium Gateway traffic can traverse host, TPROXY, and Envoy paths in non-obvious ways. |
flowchart LR
Lease["Lease\n30s duration"] --> Active["Active replica\nwg-oci0 owner"]
Lease -.-> Standby["Prepared standby\nno network state"]
Active --> State["wg-oci0 + table 51820\nlocal return rules + nftables"]
Active -->|"refresh every 5s"| Fence["Outer UDP fence\n15s timeout"]
Active -->|"loss"| Expiry["Fence expires"]
Expiry --> Blocked["Stale peer blocked"]
Blocked -->|"Lease release or expiry"| Standby
Standby --> Active
| Control |
Contract |
| Runtime |
Two anti-affined hostNetwork replicas; digest-pinned static scratch image |
| Privilege |
Root + NET_ADMIN; read-only root filesystem; private key mode 0400 |
| Ownership |
Exact-name Lease; RBAC permits only get, update, and patch on that Lease |
| Fencing |
Outer UDP requires a 15-second nftables set element refreshed only by the owner |
| Reconciliation |
Startup removes stale local relay state before leader election |
| Unhealthy-term backoff |
Consecutive unhealthy ownership terms retry exponentially from five seconds to a 60-second cap |
| Reverse-path filtering |
All K3s hosts persist net.ipv4.conf.all.rp_filter=0 and net.ipv4.conf.default.rp_filter=2 before the relay starts |
| Inner ingress |
wg-oci0 admits Gateway R TCP/80 and TCP/443 plus the mail target TCP/25, TCP/465, and TCP/993; all else drops |
| Peer identity |
One shared identity and one active endpoint; active-active is prohibited |
AllowedIPs |
0.0.0.0/0 on both peers; automatic route installation disabled |
| Routes |
Configured internal destinations use the main table; other replies from Gateway R or the mail target select table 51820 |
6. Bootstrap and Secret Management
6.1 OCI VM Bootstrap
The live relay uses a digest-identified custom image built from a pinned Oracle Linux 10.1 AArch64 platform image. Packer owns the disposable builder and provisioning lifecycle. A lockfile-pinned OCI Python SDK seals the provisioned instance. The relay runs on VM.Standard.A1.Flex with 1 OCPU, 4 GB RAM, and a 50 GB boot volume. OpenTofu-rendered cloud-init writes only the runtime configuration and starts the preinstalled bootstrap unit.
| Task |
Requirement |
| Build runtime packages |
Packer upgrades the base and installs pinned WireGuard, nftables, traffic-control, and journal-upload packages |
| Build relay binaries |
Packer installs digest-recorded guest, web admission, SMTP deny, SMTP evidence reader, and vlagent binaries |
| Enable IP forwarding |
Required for L4 relay behavior |
| Initialize relay key |
Read the OCI-generated seed and derive the same private key on every boot |
| Publish public key |
Update its OCI secret only when the current value differs |
| Render WireGuard config |
Use OCI private key and embedded home-side public key |
| Render nftables config |
DNAT web ports to Gateway R and mail ports to the mail target over WireGuard |
| Start services |
Enable WireGuard and nftables through systemd |
| Package provenance |
Record exact installed package NEVRAs, binary SHA-256 digests, base-image OCID, and image-input revision digest |
| Image sealing |
Record a pending image identity before waiting; publish the final manifest only after OCI reports AVAILABLE |
| Firewall switch |
Validate nftables before stopping firewalld; load the ruleset atomically |
| Shaping lifecycle |
WireGuard PostUp and PreDown own traffic-control restoration |
| Remote diagnostics |
OCI Run Command remains self-scoped; SSH is not a recovery dependency |
| Admission refresh |
Independent helpers refetch, validate, cache, and atomically install web-allow and smtp-deny |
| Failure behavior |
Bootstrap and web admission fail closed; invalid SMTP-deny state installs an empty deny set |
6.2 Secret Boundaries
| Material |
Authoritative storage |
OpenTofu state contract |
| OCI relay WireGuard seed |
OCI Secrets |
Must not be stored in state |
| OCI relay WireGuard public key |
OCI Secrets publication bundle |
Permitted in state |
| Home-side WireGuard private key |
Vault |
Must not be stored in state |
| Home-side WireGuard public key |
Vault relay configuration rendered into cloud-init |
Permitted in state |
| OPNsense interface API key |
Vault KV |
Must not be stored in state |
| OPNsense HTTPS root CA |
Vault pki_opnsense |
Public trust material |
| WAN controller OCI private key |
Vault KV |
Must not be stored in state |
| WAN controller OCI public key |
Vault relay configuration |
Permitted in state |
OpenTofu creates the seed, public-key secret, and IAM policy. The relay derives the private key in memory and publishes only its public key. Private key bytes do not enter Terraform state or OCI secret versions.
6.3 OCI IAM
The relay instance-principal dynamic groups require both the relay compartment and their exact quorate_relay authorization tags.
| Principal |
Permission |
Scope |
| Relay instance |
Read secret bundles |
Exact WireGuard seed, published-key, and log-ingest secrets |
| Relay instance |
Update secret |
SECRET_UPDATE on the published-key secret only |
| Relay instance |
Read objects |
Validated admission bucket only |
| Relay instance |
Execute Run Command |
Self-scoped instance-agent command execution only |
| Web-allow publisher |
Create and read immutable objects |
web-allow/v1/versions/* and migration-only admission/v1/versions/* |
| Web-allow publisher |
Create, overwrite, and read current objects |
Exact web-allow and migration-only legacy current objects |
| SMTP-deny publisher |
Create and read immutable objects |
smtp-deny/v1/versions/* only |
| SMTP-deny publisher |
Create, overwrite, and read current object |
Exact SMTP-deny current object only |
| WAN controller |
Update network security group security rules |
Exact operation in the relay compartment |
| Object Storage service |
Delete retained object versions and incomplete uploads |
Admission bucket lifecycle only |
| All relay principals |
No secret listing or KMS key management |
Prevent lateral access to unrelated secrets and keys |
6.4 Observability and Diagnostics
flowchart LR
Guest[Sealed guest units] --> Journal[Dedicated systemd journal namespace]
Journal --> Upload[systemd-journal-upload]
Upload --> Vlagent[vlagent bounded buffer]
Vlagent --> VictoriaLogs[Authenticated VictoriaLogs ingestion]
VictoriaLogs --> Dashboard[Grafana Tier R dashboard]
Guest --> Metrics[WireGuard-only Prometheus endpoint]
Metrics --> VictoriaMetrics
VictoriaMetrics --> Dashboard
Health[OCI external health checks] --> Alarms[OCI alarms and notifications]
Console[OCI serial console capture] -. manual break glass .-> Guest
Run[OCI Run Command] -. control-plane break glass .-> Guest
| Control |
Contract |
| Guest logs |
Send the dedicated relay journal through bounded vlagent to VictoriaLogs |
| Guest metrics |
Expose relay health, freshness, capacity, drop, evidence, and observer metrics to VictoriaMetrics over WireGuard |
| Conntrack records |
Log sampled transport-only TCP completions matching a declared port and post-DNAT target; cap emission rate and burst |
| Rejected traffic |
Keep nftables drop counters; do not retain OCI NSG rejects |
| Buffer bounds |
Limit vlagent to 64 MiB memory, 512 MiB disk, and 1 MiB/s upload |
| Availability boundary |
Observability failure must not affect relay forwarding |
| External monitoring |
Use OCI Health Checks, Monitoring alarms, and Notifications for public web and enabled mail reachability |
| Break glass |
Keep OCI serial-console capture and self-scoped Run Command without SSH |
7. Cloud Requirements
| Resource |
Requirement |
| Compute |
VM.Standard.A1.Flex, 1 OCPU, 4 GB RAM, and 50 GB boot volume |
| OS |
Sealed custom image built from a pinned Oracle Linux 10.1 AArch64 platform image |
| Public IP |
Static reserved public IPv4 |
| Firewall |
TCP/80 and TCP/443 are global; mail ports use explicit source admissions; WireGuard UDP uses the current OPNsense WAN /32 |
| Rate limit |
Pre-conntrack guards and traffic-control shaping protect the relay-to-home path |
| Admission storage |
Private versioned Object Storage prefixes with independent web and SMTP-deny ownership and lifecycle |
| Monitoring |
OCI monitors external reachability; VictoriaLogs and VictoriaMetrics receive guest telemetry; OCI tracks cost |
| DNS |
Services assigned to Tier R use Cloudflare gray-cloud A records for the OCI reserved address |
The relay VM is single-purpose and must not host application logic, terminate TLS, or provide general-purpose bastion access.
8. Kubernetes Requirements
| Resource |
Requirement |
| Gateway |
Dedicated gateway-relay / Gateway R at 10.104.40.127 |
| Mail forwarding target |
Source-preserving LoadBalancer at 10.66.153.202 for TCP/25, TCP/465, and TCP/993 |
| TLS transparency |
Preserve the TLS byte stream through the declared target |
| Network policy |
Gateway ingress policy must allow preserved client source traffic |
| Host policy |
Every schedulable node permits the relay tunnel and Gateway R packet path |
| Scheduling |
Two tunnel replicas use node anti-affinity |
| Ownership |
Exact-name Lease selects one active owner; the standby holds no relay network state |
| Confinement |
Relay-owned nftables admits only declared Gateway R and mail destination/port pairs from wg-oci0 |
| WAN controller |
One control-plane replica, read-only root filesystem, no ingress, and egress to cluster DNS, exact OPNsense HTTPS, and world TCP/443 for OCI API |
| Secret sync |
Vault Secrets Operator projects OPNsense credentials and CA, controller OCI credentials, and the relay health endpoint |
| Observability |
Lease state, nftables counters, Hubble, relay journals, VictoriaLogs, controller logs, and packet captures support diagnostics |
9. Acceptance Checks
| Test |
Expected Result |
| Public DNS resolves service to OCI IP |
Client connects to OCI, not residential WAN |
| TCP/80 redirect |
Gateway HTTP listener returns the expected HTTPS redirect |
| TCP/443 TLS |
Certificate is served by Cilium Gateway, not OCI |
| Mail TLS pass-through |
TCP/25, TCP/465, and TCP/993 TLS reaches the mail target unchanged |
| Port isolation |
Undeclared destination and port combinations drop before forwarding |
| Source IP preservation |
Application or Gateway logs observe real client IP |
| Return path |
Node replies traverse wg-oci0 to OCI |
| Rate limiting |
Sustained ingress above the configured cap is shaped or dropped before saturating WireGuard |
| OPNsense visibility |
OPNsense observes only outbound WireGuard traffic, not public inbound service traffic |
| Cilium stability |
Cilium services, Gateway, host firewall, and pod networking remain healthy |
| Relay reboot |
VM re-derives its key, restores services and shaping, and resumes forwarding |
| Node reboot |
Home-side tunnel and policy routing are restored declaratively |
| Abrupt owner loss |
Fence precedes ownership transfer; public service resumes through the new owner |
| Old-owner rejoin |
Startup reconciliation removes stale state before leader election |
| Tunnel negative traffic |
Undeclared Gateway R and mail destination/port pairs are dropped on wg-oci0 |
| Admission failure |
Web surfaces fail closed while TCP/25 installs an empty fail-open SMTP deny set |
| Pressure isolation |
Web, SMTP, authenticated mail, and WireGuard control retain independent budgets |
| OPNsense API failure |
The controller retains the existing WireGuard source rule and retries without widening it |
| WAN address change |
Two matching observations replace the NSG source with only the new public IPv4 /32 |
| OCI response mismatch |
The controller rejects the reconciliation and does not mark the candidate applied |
| WAN recovery |
Relay TCP/443 becomes reachable after the exact NSG rule replacement |
10. Non-Goals
| Non-Goal |
Reason |
| L7 proxying on OCI |
The relay forwards L4 traffic without owning service TLS or HTTP routing. |
| Cloud WAF |
This design is an L4 relay, not a Cloudflare Tunnel replacement for WAF-managed services. |
| General VPN access into the LAN |
The relay tunnel reaches only the ingress target required for public services. |
| Replacing Tier O |
Cloudflare Tunnel remains the path for services that terminate at Cloudflare. |
| Replacing internal Tier I or Direct Tier D |
Internal LAN and WireGuard access remain separate paths. |
11. Implementation Decisions
| Area |
Decision |
| Gateway identity |
A dedicated gateway-relay / Gateway R owns 10.104.40.127. |
| Home-side ownership |
Two anti-affined replicas use one exact-name Lease and an expiring outer-UDP fence. |
| Node selection |
Kubernetes schedules the replicas; the Lease selects one active owner. |
| Return routing |
Configured LAN and WireGuard destinations use the main table; other replies from 10.104.40.127/32 or 10.66.153.202/32 select table 51820. |
| Cluster-side NAT |
OCI performs DNAT; the cluster does not add another DNAT layer. |
| WireGuard exposure |
The in-cluster WAN controller keeps OCI ingress restricted to the current home WAN /32. |
| DNS ownership |
Cloudflare records consume the OCI reserved address from relay remote state. |
| IPv6 |
IPv6 is outside the v1 relay boundary. |
12. Provider and Cost Boundary
| Control |
Contract |
| Provider and region |
OCI ******** (********) |
| Compute allocation |
VM.Standard.A1.Flex, 1 OCPU, 4 GB RAM |
| Storage allocation |
One 50 GB boot volume, one final sealed custom image, and the private admission bucket |
| Public address |
One reserved IPv4 address |
| Cost scope |
All relay OCI resources remain in the dedicated relay compartment or are explicitly linked tenancy-level IAM resources |
| Budget |
A declarative monthly 35-unit compartment budget alerts at actual 50%, 80%, and 100%, and forecast 100% |
| Budget semantics |
Alerts provide visibility and do not enforce a hard spending limit |
| Storage lifecycle |
Web-allow and SMTP-deny versions expire after 30 days; incomplete multipart uploads abort after one day |
| Excluded resources |
No managed load balancer, block volume, volume backup, or boot-volume backup |
