feat(gke.mk): replace kubescape with gke-security-posture-enable/disable targets

# GKE Security Posture — Migration from Kubescape

## Summary

In July 2026 we removed Kubescape from `prod-services` and `stage-services` and replaced it

with GKE's built-in Security Posture feature. This document records why, what failed, and

how to manage the replacement going forward.

## What Failed (and Why)

Kubescape's `node-agent` DaemonSet was `0/1 Ready` across all three prod nodes for 26+ days

before it was investigated.

**Root cause chain:**

1. `capabilities.continuousScan: enable` (the eBPF runtime profiling feature) causes node-agents

   to continuously write container behavior data (`containerprofile` objects) into the kubescape

   storage aggregated API server.

2. The storage backend uses a 5 Gi PVC (`standard` StorageClass, no expansion support). After

   ~180 days of operation the `containerprofile` directory alone consumed **4.7 Gi** — 96% of the

   volume.

3. With the PVC nearly full, write I/O in the storage server deadlocked. All data endpoints

   (`applicationprofiles`, `networkneighborhoods`, etc.) began timing out; the kube-apiserver

   returned HTTP 429 to node-agents querying the storage readiness check.

4. Node-agents could never pass their readiness probe (`HTTP 500`), and kept restart-looping

   indefinitely (131 restarts on one pod over 26 days).

**Why it went undetected:** Kubescape was deployed with `keepLocal: true` — all findings stayed

inside the cluster with no export to Slack, SCC, or the ARMO cloud dashboard. Nobody was

reading the security data, so the broken state had no observable effect on operations.

## Why Kubescape Was Removed

- Security findings were not consumed by anyone (no alerting, no dashboard wired up).

- `continuousScan` generated data that filled the PVC and would recur every ~10 days after a wipe.

- Fixing it properly (PVC migration to an expandable StorageClass, upgrade to 1.40.x, resize to

  20+ Gi) was pure ops cost for zero delivered value.

- GKE Security Posture covers the primary use cases (CIS checks, image CVE scanning) with zero

  in-cluster overhead, integrated directly into the GCP Console and SCC.

## Replacement: GKE Security Posture (Standard)

**Enabled on:** `prod-services`, `stage-services` (both `us-west1`).

**What it provides (standard tier, free):**

| Feature | Details |

|---------|---------|

| Workload configuration audit | CIS Benchmark, NSA/CISA Kubernetes Hardening checks |

| Image vulnerability scanning | OS-level CVE scanning for running containers |

| Security dashboard | GCP Console → Kubernetes Engine → Security tab |

| SCC integration | Findings flow automatically into Security Command Center |

**What it does NOT provide (vs. Kubescape):**

- eBPF runtime threat detection (syscall anomalies, network behavior profiling) — this requires

  GKE Enterprise or Container Threat Detection (paid add-on).

- Admission webhook policy enforcement — use OPA/Gatekeeper or Binary Authorization for that.

## How to Manage

```bash

# Enable on current cluster context

make gke-security-posture-enable

# Disable

make gke-security-posture-disable

```

These targets are defined in `~/.otica/makefiles/gke.mk` and require `GKE_CLUSTER_NAME`,

`GCP_REGION`, and `GCP_PROJECT_ID`.

## If You Need Runtime Threat Detection

Options in order of cost:

1. **GKE Container Threat Detection** — GKE Enterprise add-on, eBPF-based, managed by Google.

2. **Falco** — open-source, runs as a DaemonSet, can alert to Slack/PagerDuty. Lower ops burden

   than Kubescape because it doesn't accumulate persistent data.

3. **Kubescape** — re-install with `continuousScan: disable` (posture + vuln scanning only),

   larger PVC (`standard-rwo`, 20 Gi+), wired to Slack or the ARMO platform. Only justified if

   someone commits to reading the alerts.

		--filter="name ~ ${GCP_ENVIRONMENT}" \

		--format='get(name,zone,scheduling.preemptible,creationTimestamp)'|sed 's%https://.*/%%' | sort -k1

.PHONY: install-kubescape

install-kubescape: kc-config ## Install Kubescape operator for continuous cluster scanning

	helm repo add kubescape https://kubescape.github.io/helm-charts/

	helm repo update

	helm upgrade --install kubescape kubescape/kubescape-operator \

		-n kubescape \

		--create-namespace \

		--set clusterName=${GKE_CLUSTER_NAME} \

		--set capabilities.continuousScan=enable

.PHONY: upgrade-kubescape

upgrade-kubescape: kc-config ## Upgrade Kubescape operator to latest chart version

	helm repo update

	helm upgrade kubescape kubescape/kubescape-operator \

		-n kubescape \

		--set clusterName=${GKE_CLUSTER_NAME} \

		--set capabilities.continuousScan=enable

.PHONY: uninstall-kubescape

uninstall-kubescape: kc-config ## Uninstall Kubescape operator

	@confirm.sh "Uninstall Kubescape from cluster ${GKE_CLUSTER_NAME}?"

	helm uninstall kubescape -n kubescape

.PHONY: gke-security-posture-enable

gke-security-posture-enable: gcp-login ## Enable GKE Security Posture (standard tier: CIS checks + image vuln scanning)

	gcloud container clusters update ${GKE_CLUSTER_NAME} \

		--region=${GCP_REGION} --project=${GCP_PROJECT_ID} \

		--security-posture=standard

	gcloud container clusters update ${GKE_CLUSTER_NAME} \

		--region=${GCP_REGION} --project=${GCP_PROJECT_ID} \

		--workload-vulnerability-scanning=standard

.PHONY: gke-security-posture-disable

gke-security-posture-disable: gcp-login ## Disable GKE Security Posture

	@confirm.sh "Disable GKE Security Posture on cluster ${GKE_CLUSTER_NAME}?"

	gcloud container clusters update ${GKE_CLUSTER_NAME} \

		--region=${GCP_REGION} --project=${GCP_PROJECT_ID} \

		--security-posture=disabled

	gcloud container clusters update ${GKE_CLUSTER_NAME} \

		--region=${GCP_REGION} --project=${GCP_PROJECT_ID} \

		--workload-vulnerability-scanning=disabled

# Exposes cluster object metrics scrapeable by Prometheus / visualizable in Grafana

.PHONY: install-kube-state-metrics