Commit a8de2699 authored by xuwang's avatar xuwang
Browse files

renovate: add dependency-update module, scripts, CI template, docs 



Wrap self-hosted Renovate for GitLab the way opa.mk wraps OPA:

- makefiles/renovate.mk: project-scoped check/init/validate/dryrun/run targets
  (run is confirm.sh-guarded). Repo, endpoint, and token are all derived or
  defaulted, so the module needs no required vars.
- makefiles/renovate-admin.mk: org-wide autodiscover dry-run/run, reusing the
  base module's settings + renovate-check.
- scripts/renovate-run.sh: dispatcher that auto-detects the runtime
  (renovate -> npx -> docker), resolves the GitLab token/endpoint inline
  (gitlab.sh hard-exits without a token, which would break validate), and
  assembles each mode's invocation.
- scripts/renovate-init.sh: scaffold a starter renovate.json (clobber-guarded).
- .gitlab/renovate-ci.yml: includable, schedule-only CI job for unattended runs.
- docs/renovate.md + README link.

Changelog: added
Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 9347bc59

.gitlab/renovate-ci.yml

0 → 100644
+37 −0
Original line number Diff line number Diff line 
# OTICA Renovate CI template — schedulable self-hosted Renovate for GitLab.

#

# Usage in a consumer .gitlab-ci.yml:

#

#   include:

#     - project: 'iac/cloud-framework'        # OTICA repo on your GitLab server

#       file: '.gitlab/renovate-ci.yml'

#       ref: 'v1.2.1'                          # pin to an OTICA release tag

#

# Then:

#   1. Add a *masked* CI/CD variable RENOVATE_TOKEN — a GitLab PAT or

#      group/project access token with the `api` scope and at least Developer

#      role (so Renovate can open MRs).

#   2. Create a pipeline schedule (CI/CD > Schedules), e.g. hourly or nightly.

#

# The job only runs on scheduled pipelines (and manual web runs), never on every

# push. By default it processes just this project; for org-wide scanning set

# RENOVATE_AUTODISCOVER: "true" (and drop RENOVATE_REPOSITORIES) with a token

# that can see the target group/instance.



renovate:

  stage: .pre

  image: renovate/renovate:latest

  variables:

    RENOVATE_PLATFORM: gitlab

    RENOVATE_ENDPOINT: $CI_API_V4_URL

    RENOVATE_AUTODISCOVER: "false"

    RENOVATE_REPOSITORIES: $CI_PROJECT_PATH

    LOG_LEVEL: info

    FORCE_COLOR: "3"

  rules:

    - if: '$CI_PIPELINE_SOURCE == "schedule"'

    - if: '$CI_PIPELINE_SOURCE == "web"'

      when: manual

      allow_failure: true

  script:

    - renovate

README.md

+5 −0
Original line number Diff line number Diff line
@@ -220,6 +220,11 @@ authoritative, always-current list of targets and descriptions. For per-module i 
(variable contracts, design notes), see

[docs/OTICA_TECHNICAL_REFERENCE.md](docs/OTICA_TECHNICAL_REFERENCE.md#module-reference).



A few modules have dedicated guides:



- **Renovate** (automated dependency updates) — `renovate.mk` / `renovate-admin.mk` and a

  schedulable GitLab CI template: see **[docs/renovate.md](docs/renovate.md)**.



## Common Workflows



Task recipes for the most common operations — Terraform deploy, OPA validation, Vault, Azure

docs/renovate.md

0 → 100644
+210 −0
Original line number Diff line number Diff line 
# Renovate — Automated Dependency Updates



[Renovate](https://github.com/renovatebot/renovate) keeps a project's pinned

dependencies (Terraform providers/modules, Docker base images, Helm charts,

GitHub Actions, npm, …) current by opening merge requests as new versions ship.

OTICA wraps the self-hosted Renovate runtime for self-managed GitLab via two

modules and a schedulable CI template:



| File | Scope |

| --- | --- |

| [`makefiles/renovate.mk`](../makefiles/renovate.mk) | This project (repo derived from its git remote) |

| [`makefiles/renovate-admin.mk`](../makefiles/renovate-admin.mk) | Org-wide autodiscover across many repos |

| [`.gitlab/renovate-ci.yml`](../.gitlab/renovate-ci.yml) | Includable GitLab CI job for scheduled runs |



## How it works



Renovate is a **stateless batch bot**, not a daemon. Each run it:



1. **Clones** the target repo via the GitLab API (using its token).

2. **Scans** for dependencies across managers — Terraform providers/modules,

   Dockerfiles, Helm charts, `package.json`, GitLab CI `include`s, …

3. **Compares** each against upstream for newer versions.

4. **Opens / updates** a merge request per update (or per group) and maintains a

   **Dependency Dashboard** issue summarizing everything.

5. **Exits.** It does nothing between runs, so something must *trigger* it on a

   cadence.



That last point drives the design: Renovate is something you *schedule*, not run

continuously. OTICA exposes two trigger paths to the same engine — one for

humans, one for automation:



```

   LOCAL (developer, ad-hoc)              CI (scheduled, unattended)

   ─────────────────────────             ──────────────────────────

   make renovate-dryrun / -run           GitLab pipeline schedule (e.g. nightly)

        │                                       │

        ▼                                       ▼

   scripts/renovate-run.sh                .gitlab-ci.yml includes a

        │  auto-detect runtime            renovate job (in-runner, or a

        │  renovate → npx → docker        Cloud Build variant for heavy runs)

        ▼                                       │

   ┌──────────────────────────────────────────▼───────────┐

   │                renovate (the engine)                   │

   │   reads renovate.json + RENOVATE_* env, hits GitLab    │

   └───────────────────────────────────────────────────────┘

                          │

                          ▼

            opens/updates dependency MRs + Dependency Dashboard

```



- **Local path** — `make renovate-*` (the `renovate.mk` / `renovate-admin.mk`

  targets below). For trying it out or running on demand from your machine.

- **CI path** — include [`.gitlab/renovate-ci.yml`](../.gitlab/renovate-ci.yml)

  and create a pipeline schedule. The job's `rules` fire **only** on scheduled

  (and manual web) pipelines, so it runs alone and never on branch/tag pushes.

  CI-config catalogs may also offer a managed **Cloud Build** variant

  (`gbuild-renovate.yml`) that offloads heavy runs to a bigger worker.



Both paths run the *same* engine, read the *same* per-repo `renovate.json`, and

authenticate with the *same* kind of scoped GitLab token — they differ only in

**what kicks Renovate off** and **where it executes**.



### Update lifecycle



1. A run opens MRs like *"Update terraform google to v5.x"* plus a Dependency

   Dashboard issue.

2. Your normal pipeline runs on each MR → you review and merge.

3. The next run **reconciles** — rebases open MRs, closes ones for versions you

   skipped, opens new ones. It is convergent: each run moves the repo toward

   current and cleans up after itself.



## Prerequisites



**Runtime — auto-detected**, no configuration needed. The wrapper

([`renovate-run.sh`](../scripts/renovate-run.sh)) picks the first available of:



1. `renovate` on `PATH` (npm global install; needs **Node 20+**)

2. `npx --yes renovate` (also Node 20+)

3. the `renovate/renovate` **Docker** image



Override with `RENOVATE_CMD` (e.g. `RENOVATE_CMD="npx --yes renovate"`, a custom

docker wrapper, or `RENOVATE_CMD=echo` to print the assembled command without

running it).



**Token.** Renovate authenticates with the consumer's GitLab token, resolved

like the rest of OTICA: `RENOVATE_TOKEN` → `GITLAB_TOKEN` → `~/.gitlab-token`.

Scope depends on the action:



| Action | Required token scope |

| --- | --- |

| `renovate-validate` | _(none — config-only)_ |

| `renovate-dryrun`, `renovate-admin-dryrun` | `read_api` |

| `renovate-run`, `renovate-admin-run` | `api` (+ Developer role to open MRs) |



The endpoint defaults to `${GITLAB_SERVER}/api/v4` (`GITLAB_SERVER` defaults to

`https://code.stanford.edu`).



## Project targets (`renovate.mk`)



```makefile

include ${OTICA_DIR}/makefiles/renovate.mk

```



| Target | What it does |

| --- | --- |

| `renovate-check` | Verify a runtime is available (`renovate` \| `npx` \| `docker`) |

| `renovate-init` | Scaffold a starter `renovate.json` into this project |

| `renovate-validate` | Validate `${RENOVATE_CONFIG}` with `renovate-config-validator` |

| `renovate-dryrun` | Dry-run this repo — logs proposed updates, opens **no** MRs |

| `renovate-run` | Run Renovate on this repo — opens update MRs (`confirm.sh`-guarded) |



Typical first run:



```bash

make renovate-init        # writes renovate.json (won't clobber an existing one)

make renovate-validate    # check the config parses

make renovate-dryrun      # see what Renovate WOULD do (needs read_api token)

make renovate-run         # open the MRs (needs api token; prompts to confirm)

```



### Settings (override in `env.mk`)



| Variable | Default | Purpose |

| --- | --- | --- |

| `RENOVATE_CMD` | _(auto)_ | Force a specific runtime |

| `RENOVATE_PLATFORM` | `gitlab` | Renovate platform |

| `RENOVATE_ENDPOINT` | `${GITLAB_SERVER}/api/v4` | GitLab API endpoint |

| `RENOVATE_CONFIG` | `renovate.json` | Config file for `renovate-validate` / `renovate-init` |

| `RENOVATE_REPO` | _(derived from git remote)_ | `namespace/project` to operate on |

| `RENOVATE_ARGS` | _(empty)_ | Extra args appended to the Renovate invocation |



The starter `renovate.json` extends `config:recommended` with a dependency

dashboard, semantic commits, a weekly schedule, and grouped minor+patch updates

for Terraform / Docker / Helm — edit it to taste.



## Autodiscover / admin targets (`renovate-admin.mk`)



```makefile

include ${OTICA_DIR}/makefiles/renovate-admin.mk

```



| Target | What it does |

| --- | --- |

| `renovate-admin-dryrun` | Autodiscover dry-run across every repo the token can see |

| `renovate-admin-run` | Autodiscover real run across matching repos (`confirm.sh`-guarded) |



Narrow the scope with `RENOVATE_AUTODISCOVER_FILTER` (a glob, e.g. `mygroup/**`):



```bash

make renovate-admin-dryrun RENOVATE_AUTODISCOVER_FILTER='iac/**'

```



This needs a **group or instance** token that can see the target repos.



## Scheduled runs in GitLab CI (`.gitlab/renovate-ci.yml`)



For unattended updates, include the template and schedule a pipeline:



```yaml

# consumer .gitlab-ci.yml

include:

  - project: 'iac/cloud-framework'

    file: '.gitlab/renovate-ci.yml'

    ref: 'v1.2.1'                      # pin to an OTICA release tag

```



Then:



1. **CI/CD → Variables**: add a *masked* `RENOVATE_TOKEN` (PAT or group/project

   access token, `api` scope, ≥ Developer role).

2. **CI/CD → Schedules**: create a schedule (hourly/nightly).



The job runs only on scheduled pipelines (and manual web runs), processing just

the current project by default. For org-wide scanning set `RENOVATE_AUTODISCOVER:

"true"` and drop `RENOVATE_REPOSITORIES`, with a token that can see the group.



### Shared CI/CD catalog templates



The `.gitlab/renovate-ci.yml` above is the framework's self-contained copy. The

[gitlab-platform CI/CD catalog](https://code.stanford.edu/iac/examples/platforms/gitlab-platform/-/tree/main/cicd)

also ships catalog-style templates that match its other `cicd/*.yml` building

blocks (shared `stages`/`workflow`, runner tags, Workload Identity):



| Template | Runs on | Token source |

| --- | --- | --- |

| [`cicd/renovate.yml`](https://code.stanford.edu/iac/examples/platforms/gitlab-platform/-/blob/main/cicd/renovate.yml) | the GitLab runner (in-runner) | masked CI variable `RENOVATE_TOKEN` |

| [`cicd/gbuild-renovate.yml`](https://code.stanford.edu/iac/examples/platforms/gitlab-platform/-/blob/main/cicd/gbuild-renovate.yml) | managed **Google Cloud Build** (for heavy runs) | Secret Manager, key-less via Workload Identity |



See that catalog's [README](https://code.stanford.edu/iac/examples/platforms/gitlab-platform/-/blob/main/cicd/README.md#scheduled-dependency-updates-renovate)

for setup. Either way, scaffold the per-repo config with `make renovate-init`.



> **These are example templates.** They carry placeholders (e.g.

> `<NAMESPACE>/<CI_CONFIG_REPO>`, `<RUNNER_TAG>`, `<PROJECT_ID>`) and default

> values (machine type, logs bucket, Secret Manager path) that you must replace

> with your project's specifics before use — see the catalog README's

> "Placeholders to replace" table.



## How it fits OTICA



`renovate-run.sh` resolves the token/endpoint inline (rather than sourcing

`gitlab.sh`, which is built for API calls and hard-exits without a token) so that

`renovate-validate` and command-assembly checks work with no token. Real runs are

`confirm.sh`-guarded and honor `NONINTERACTIVE=1` for CI/automation — the same

conventions as the rest of the framework.



To preview the exact command without a runtime or network:



```bash

RENOVATE_CMD=echo RENOVATE_TOKEN=dummy make renovate-dryrun

```

makefiles/renovate-admin.mk

0 → 100644
+41 −0
Original line number Diff line number Diff line 
ifndef RENOVATE_ADMIN_MK_INCLUDED

	RENOVATE_ADMIN_MK_INCLUDED := 1

unexport RENOVATE_ADMIN_MK_INCLUDED



####################################

# Renovate (admin) — org-wide autodiscover across many repos

# Ref: https://docs.renovatebot.com/self-hosted-configuration/#autodiscover

#

# Usage in project Makefile:

#   include ${OTICA_DIR}/makefiles/renovate-admin.mk

#

# Runs Renovate with autodiscover so it processes every repo the token can see,

# optionally narrowed by RENOVATE_AUTODISCOVER_FILTER (e.g. "mygroup/**"). Needs

# a group/instance token: read_api for dry-runs, api for real runs.

#

# Targets:

#   renovate-admin-dryrun  autodiscover dry-run across accessible repos

#   renovate-admin-run     autodiscover real run across accessible repos (confirm)

#

# See docs/renovate.md.

####################################



# Reuse RENOVATE_CMD/PLATFORM/ENDPOINT settings + the renovate-check target.

ifndef RENOVATE_MK_INCLUDED

include $(dir $(lastword $(MAKEFILE_LIST)))renovate.mk

endif



RENOVATE_AUTODISCOVER_FILTER ?=

export RENOVATE_AUTODISCOVER_FILTER



.PHONY: renovate-admin-dryrun

renovate-admin-dryrun: renovate-check ## autodiscover dry-run across accessible repos (read_api)

	@renovate-run.sh autodiscover-dryrun



.PHONY: renovate-admin-run

renovate-admin-run: renovate-check ## autodiscover real run across accessible repos (api)

	@if confirm.sh "Run Renovate autodiscover$${RENOVATE_AUTODISCOVER_FILTER:+ (filter: $$RENOVATE_AUTODISCOVER_FILTER)} and open MRs across ALL matching repos?"; then \

		renovate-run.sh autodiscover-run ; \

	fi



endif # RENOVATE_ADMIN_MK_INCLUDED

makefiles/renovate.mk

0 → 100644
+64 −0
Original line number Diff line number Diff line 
ifndef RENOVATE_MK_INCLUDED

	RENOVATE_MK_INCLUDED := 1

unexport RENOVATE_MK_INCLUDED



####################################

# Renovate — automated dependency updates (self-hosted, GitLab)

# Ref: https://github.com/renovatebot/renovate  https://docs.renovatebot.com

#

# Usage in project Makefile:

#   include ${OTICA_DIR}/makefiles/renovate.mk

#

# Runtime is auto-detected: an installed `renovate`, else `npx --yes renovate`,

# else the renovate/renovate Docker image. Override with RENOVATE_CMD.

# Auth uses the consumer GitLab token (GITLAB_TOKEN / ~/.gitlab-token):

# read_api scope for dry-runs, api scope for real runs.

#

# Targets:

#   renovate-check     verify a Renovate runtime is available

#   renovate-init      scaffold a starter renovate.json into this project

#   renovate-validate  validate ${RENOVATE_CONFIG} (renovate-config-validator)

#   renovate-dryrun    dry-run this repo; logs proposed updates, opens no MRs

#   renovate-run       run Renovate on this repo; opens update MRs (confirm)

#

# See docs/renovate.md.

####################################



# ── Settings (override in env.mk) ──────────────────────────────────────────────

RENOVATE_CMD       ?=

RENOVATE_PLATFORM  ?= gitlab

RENOVATE_ENDPOINT  ?=

RENOVATE_CONFIG    ?= renovate.json

RENOVATE_REPO      ?=

RENOVATE_ARGS      ?=



export RENOVATE_CMD RENOVATE_PLATFORM RENOVATE_ENDPOINT \

       RENOVATE_CONFIG RENOVATE_REPO RENOVATE_ARGS



.PHONY: renovate-check

renovate-check: ## verify a Renovate runtime is available (renovate | npx | docker)

	@if [ -n "${RENOVATE_CMD}" ]; then exit 0; fi; \

	command -v renovate >/dev/null 2>&1 \

	  || command -v npx >/dev/null 2>&1 \

	  || command -v docker >/dev/null 2>&1 \

	  || { echo "No Renovate runtime: install Node 20+ (renovate/npx) or Docker, or set RENOVATE_CMD."; exit 1; }



.PHONY: renovate-init

renovate-init: ## scaffold a starter renovate.json into this project

	@renovate-init.sh



.PHONY: renovate-validate

renovate-validate: renovate-check ## validate ${RENOVATE_CONFIG} with renovate-config-validator

	@renovate-run.sh validate



.PHONY: renovate-dryrun

renovate-dryrun: renovate-check ## dry-run Renovate on this repo (opens no MRs; read_api)

	@renovate-run.sh dryrun



.PHONY: renovate-run

renovate-run: renovate-check ## run Renovate on this repo; opens update MRs (api)

	@if confirm.sh "Run Renovate on '$${RENOVATE_REPO:-this repo}' and open update MRs?"; then \

		renovate-run.sh run ; \

	fi



endif # RENOVATE_MK_INCLUDED