Commit 5e651dd0 authored by xuwang's avatar xuwang
Browse files

otica-migrate-kube: rule for rendering vault-bearing properties into Secret/ConfigMap



- §6: new "Properties-file-into-Secret (update-runtime-properties)" subsection —
  embed via tmpl.Inline(file.Read ...)|base64.Encode (inherits vault-kv + .Env,
  verified gomplate 4.x), eliminating the recipe-built RUNTIME_PROPERTIES handoff;
  plus pre-render and stringData fallbacks, keystore-ConfigMap, namespace collapse,
  target-mapping table, malformed-shebang sweep
- §4: cover all #!vault2* shebangs (vault2properties/vault2chart) + note
  vault2properties comment/blank-line stripping

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 5afd19eb
Loading
Loading
Loading
Loading
+87 −4
Original line number Diff line number Diff line
@@ -159,17 +159,19 @@ Templates use a first-line shebang. OTICA's `render.sh` keys on it:
|---|---|
| `#!gomplate` | renders via gomplate ✅ |
| `#!envsubst` | renders via envsubst ✅ |
| `#!vault2text` / `#!vault2kube` | **ERRORS** ❌ |
| `#!vault2text` / `#!vault2kube` / `#!vault2properties` / `#!vault2chart` | **ERRORS** ❌ |

`#!vault2text` and `#!vault2kube` files MUST be converted to `#!gomplate`. Sweep every template dir:
All `#!vault2*` files MUST be converted to `#!gomplate` — they all resolve `%%path%%` vault refs the same way (decode to original format), so the §4 syntax table applies uniformly. Sweep every template dir:

```bash
find . -type f \( -name '*.yml' -o -name '*.yaml' -o -name '*.tmpl*' \) \
find . -type f \( -name '*.yml' -o -name '*.yaml' -o -name '*.tmpl*' -o -name '*.properties' \) \
  -not -path '*/.git/*' -print0 \
  | xargs -0 -I{} sh -c 'printf "%-55s %s\n" "{}" "$(head -1 "{}")"' \
  | grep -E 'vault2text|vault2kube|envsubst'
  | grep -E 'vault2text|vault2kube|vault2properties|vault2chart|envsubst'
```

**`#!vault2properties` strips comment (`^#`) and blank lines; `#!vault2text` keeps them.** gomplate renders verbatim, so a converted `#!vault2properties` file will *keep* its comments/blanks — harmless for a Java `.properties` file (it ignores `#` and blank lines), but expect a non-empty `diff` vs the legacy render. If a byte-clean render matters, pipe the artifact through `grep -v '^\s*#' | grep -v '^\s*$'` in the pre-render recipe (see §6).

**Syntax conversion table:**

| Concern | vault2text/envsubst | gomplate |
@@ -296,6 +298,87 @@ Vault binary-blob note: Vault stores binary keytabs as `{format: base64, value:

**Render payloads verbatim — never `| strings.TrimSpace` before `base64.Encode`.** PEM keys and certs are commonly stored with a significant trailing newline; trimming it drops a byte from the decoded payload, so the rendered Secret no longer matches the original and a workload may fail later when it loads a key/cert missing its final `\n`. This mirrors the `vault-read.sh` verbatim-output fix.

### Properties-file-into-Secret (`update-runtime-properties` pattern)

Some apps render a whole **properties/config file** that itself contains vault refs, base64 it, and inject it into a Secret. The ps-cloud-framework shape builds the value in the recipe and `envsubst`s it into the template:

```makefile
# legacy — has NO OTICA equivalent (no kube_apply.sh; no "export VAR=$(render) ; subst" flow)
update-runtime-properties: vault-login create-namespace ## update runtime properties secret
	export RUNTIME_PROPERTIES=`cat ${ARTIFACTS}/runtime.properties | render.sh | base64` ; \
	kube_apply.sh ${TEMPLATES}/secret.yml ${TEMPLATES}/cm-authority-manager-keystore.yml
```

Where the rendered file (`#!vault2text`, or commonly `#!vault2properties` for `.properties` files) mixes env refs and vault refs:

```
org...url=https://${APP_FQDN}/
spring...client-secret=%%${SEC_PATH}/${APP}/${GCP_ENVIRONMENT}/rhsso/client-secret%%
```

**Do NOT port the recipe.** Convert the file to gomplate and let a single `templates-sec/` Secret render + base64 it — `kc-apply-sec` applies it. No custom target, no `kube_apply.sh`.

This **eliminates the `export RUNTIME_PROPERTIES=\`... | render.sh | base64\`** recipe handoff and the matching `${RUNTIME_PROPERTIES}` placeholder in the Secret: the render + base64 now happen *inside* the template, so there is no env var to build, export, or `envsubst`. Delete that line and the `data: ...: ${RUNTIME_PROPERTIES}` placeholder.

1. **Convert the properties file** refs per the §4 table (`${X}``{{.Env.X}}`, `%%path%%``{{ tmpl.Exec "vault-kv" "path" }}`). Vault path with interpolation:
   ```diff
   - spring...client-secret=%%${SEC_PATH}/${APP}/${GCP_ENVIRONMENT}/rhsso/client-secret%%
   + spring...client-secret={{ tmpl.Exec "vault-kv" (printf "%s/%s/%s/rhsso/client-secret" .Env.SEC_PATH .Env.APP .Env.GCP_ENVIRONMENT) }}
   ```
   **Drop the shebang line** (`#!vault2text` / `#!vault2properties`) — the file is no longer rendered standalone; it becomes a gomplate *partial* embedded by the Secret template (a leftover `#!...` line would land verbatim in the payload). Keep it under `common/` (e.g. `artifacts/runtime.properties`), not in a `templates*/` dir, so `render-all.sh` doesn't try to render it on its own.

2. **Embed it directly in the Secret** (`templates-sec/secret.yml`, `#!gomplate`) with `file.Read` + `tmpl.Inline` + `base64.Encode` — one step, no pre-render artifact:
   ```yaml
   #!gomplate
   apiVersion: v1
   kind: Secret
   metadata:
     namespace: {{.Env.KUBE_NAMESPACE}}
     name: {{.Env.APP}}-secrets
   type: Opaque
   data:
     local.runtime.properties: {{ tmpl.Inline (file.Read (filepath.Join .Env.ARTIFACTS "runtime.properties")) | base64.Encode }}
   ```
   `tmpl.Inline` renders the embedded file's `{{.Env.* }}` and `{{ tmpl.Exec "vault-kv" ... }}` refs **in the parent scope** — it inherits render.sh's `vault-kv` define and the environment (verified gomplate 4.x; needs ≥ 3.3 for `tmpl.Inline`). `file.Read` paths are relative to the dir `make` runs in, so anchor with `.Env.ARTIFACTS` (an absolute realpath).
   **No `| strings.TrimSpace`** before `base64.Encode` — the legacy `... | base64` trimmed nothing; trimming drops the file's trailing newline and breaks parity (see the verbatim note above).

   > `include "alias"` returns a datasource's **raw** content *without* rendering it — embedding a template needs `tmpl.Inline` (or pipe `include` through `tmpl.Inline`). Plain `file.Read | base64.Encode` (no `tmpl.Inline`) base64s the unrendered text and ships literal `{{...}}` into the Secret — only use it for a file with no template refs.

**Fallback — pre-render to an artifact** (only if you're on gomplate < 3.3, or want `#!vault2properties` comment/blank stripping preserved): keep the file's `#!gomplate` shebang, render it to a gitignored artifact wired before `kc-render-sec`, then `file.Read` (no `tmpl.Inline`) that artifact:
   ```makefile
   .PHONY: render-runtime-properties
   render-runtime-properties: vault-login
   	@cat ${ARTIFACTS}/runtime.properties | render.sh > ${ARTIFACTS}/.runtime.properties.rendered
   kc-render-sec: render-runtime-properties
   ```
   Add `.runtime.properties.rendered` to `.gitignore` (resolved secrets). For `#!vault2properties` strip parity append `| grep -v '^\s*#' | grep -v '^\s*$'` (optional — Java ignores comments/blanks).

**Simplest of all** — if the file feeds only this one Secret and isn't separately maintained, paste the body straight into the Secret as a `stringData:` block scalar (k8s base64s it; no `file.Read`, no artifact, single source of truth).

**Sidecar ConfigMaps holding secret blobs (e.g. a keystore) also go to `templates-sec/`.** A `#!vault2kube` ConfigMap with `binaryData` from a vault blob becomes `#!gomplate` with `base64.Encode` (vault stores keystores `format:base64`; `vault-kv` decodes, `base64.Encode` re-encodes → byte-identical, same as the keytab note above):
```yaml
binaryData:
  authority-manager.jks: {{ tmpl.Exec "vault-kv" (printf "%s/common/authority-manager/%s/keystore" .Env.SEC_PATH .Env.AM_ENV) | base64.Encode }}
```
Put it in `templates-sec/` so `kc-apply-sec` wipes the rendered bytes after apply and `kc-diff` never prints them.

**Namespace prereq.** `create-namespace` (applying `ns.yml`) is redundant — `kc-init`/`kc-force-init` auto-create `KUBE_NAMESPACE`. Set `KUBE_NAMESPACE := ${APP_NAMESPACE}`, drop `create-namespace` and `ns.yml`.

**Target collapse:**

| legacy | OTICA |
|---|---|
| `update-runtime-properties` | `kc-apply-sec` |
| `render-runtime-properties` (debug-only render) | `kc-render-sec` |
| `deploy: … update-runtime-properties …` | `deploy: kc-config kc-apply-sec kc-apply` |

**Malformed shebangs.** This pattern's `secret.yml` shipped a typo line 1: `#!subenvst`. `render.sh` only recognizes `#!gomplate` / `#!envsubst` — any other shebang falls through to non-template passthrough and is applied **unrendered** (silent). Sweep for unknown shebangs alongside the §4 sweep:

```bash
find . -type f \( -name '*.yml' -o -name '*.yaml' -o -name '*.tmpl*' \) -not -path '*/.git/*' -print0 \
  | xargs -0 -I{} sh -c 'h=$(head -1 "{}"); case "$h" in \#!*) echo "$h" | grep -qE "gomplate|envsubst|vault2text|vault2kube" || printf "%-55s %s\n" "{}" "$h";; esac'
```

### Validate

```bash