Commit fc5e4dd6 authored by xuwang's avatar xuwang
Browse files

refactor(renovate): drop in-tree bot management; identity is provisioned out-of-band 



Renovate now only consumes a token (RENOVATE_TOKEN[_FILE|_VAULT_PATH]); creating
the bot identity is the job of a Terraform SA-provisioning project (e.g.
provision-gitlab-sa / the gitlab-renovate-bot example), per the established
handoff. Remove scripts/renovate-bot.sh and the renovate-bot-* targets +
RENOVATE_BOT_* vars from renovate-admin.mk (sa-create was also dead on GitLab CE).
renovate-admin keeps update/dryrun/run; docs rewritten to the Terraform pattern.

Changelog: removed
Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 1f372ca0

docs/renovate.md

+26 −131
Original line number Diff line number Diff line
@@ -86,7 +86,7 @@ running it). 
server**, resolved by precedence:

`RENOVATE_TOKEN` → `RENOVATE_TOKEN_FILE` → `RENOVATE_TOKEN_VAULT_PATH` (read via

`vault-read.sh`) → `GITLAB_TOKEN` → `~/.gitlab-token`. The Vault path reads back

exactly what `renovate-bot-sa-create` stored (same `RENOVATE_TOKEN_VAULT_PATH`),

exactly what the Terraform-provisioned bot stored (same `RENOVATE_TOKEN_VAULT_PATH`),

so a fleet can run with no token files on disk. Scope depends on the action:



| Action | Required token scope |
@@ -212,150 +212,45 @@ OTICA's `make renovate-admin-update` writes the explicit `repositories` list; pi 
one of the rows above for `onboarding`/`requireConfig` depending on whether you

want central control (Centralized) or explicit per-repo opt-in (Per-repo opt-in).



## Dedicated bot identity (optional, `renovate-admin.mk`)

## Bot identity (provisioned outside Renovate)



By default scans/CI authenticate as a person's PAT. For unattended use, provision

a dedicated **Renovate bot** (a GitLab access token whose bot user owns the MRs).

The bot is managed with an **operator token** on the repo's own server (Owner/

Maintainer + `api`; instance admin for `sa-create`). Resolution:

`RENOVATE_BOT_ADMIN_TOKEN` → `RENOVATE_BOT_ADMIN_TOKEN_FILE` → ambient

`GITLAB_TOKEN` → `~/.gitlab-token`. Set `RENOVATE_BOT_ADMIN_TOKEN_FILE` when the

project's `GITLAB_TOKEN` targets a different server than the repo (e.g. a

code.stanford.edu Terraform backend on a gitlab.med repo) — it wins over that

ambient token.

For unattended runs, authenticate as a dedicated **bot** rather than a person's

PAT. OTICA keeps identity *creation* out of Renovate: Renovate only **reads** the

token (`RENOVATE_TOKEN_VAULT_PATH` / `RENOVATE_TOKEN_FILE`), and the bot is

provisioned declaratively — recommended with Terraform.



| Target | What it does |

| --- | --- |

| `renovate-bot-create` | Create the bot token — a **group** token if `RENOVATE_BOT_GROUP` is set (one identity for the whole multi-repo scan), else a **project** token (one repo's CI). Stored at `RENOVATE_BOT_TOKEN_FILE`. |

| `renovate-bot-list` | List `renovate-bot` tokens on the target group/project. |

| `renovate-bot-rotate` | Atomically rotate the token (GitLab revokes the old, issues a new one with the same name/scopes) and rewrite the file + Vault (`confirm.sh`-guarded). |

| `renovate-bot-revoke` | Revoke the token and remove the local file (`confirm.sh`-guarded). |

| `renovate-bot-ci-set` | Set the masked `RENOVATE_TOKEN` CI/CD variable on a project (`REPO=ns/proj`) from the stored token. |



| Variable | Default | Purpose |

| --- | --- | --- |

| `RENOVATE_BOT_NAME` | `renovate-bot` | Bot identifier — token name + default file/Vault path. Set distinct names to run **multiple bots** (see below) |

| `RENOVATE_BOT_GROUP` | _(empty)_ | Group full path → create a group token (else project) |

| `RENOVATE_BOT_REPO` | origin | Project for project-token / `ci-set` |

| `RENOVATE_BOT_TOKEN_FILE` | `~/.otica-tokens/<name>.token` | Where the token is stored |

| `RENOVATE_BOT_ACCESS_LEVEL` | `40` (Maintainer) | `30` (Developer) for least privilege; Maintainer is needed to automerge protected branches |

| `RENOVATE_BOT_EXPIRES` | +365 days | Token expiry (`YYYY-MM-DD`); used for new and rotated tokens |

### Terraform-provisioned bot (recommended)



### Multiple bots (one per repo tree)



A group/project token only covers one tree, so for several trees run a bot **per

tree**, distinguished by `RENOVATE_BOT_NAME` (each gets its own token, file, and

Vault path). Pair each with its group and admin-config subset:



```bash

# bot + scan for the gcp tree

make renovate-bot-create  RENOVATE_BOT_NAME=renovate-bot-gcp     RENOVATE_BOT_GROUP=gcp

make renovate-admin-run   RENOVATE_BOT_NAME=renovate-bot-gcp \

                          RENOVATE_TOKEN_FILE=~/.otica-tokens/renovate-bot-gcp.token \

                          RENOVATE_ADMIN_CONFIG=renovate-admin.gcp.json



# bot + scan for the irt-dcs tree

make renovate-bot-create  RENOVATE_BOT_NAME=renovate-bot-irt-dcs RENOVATE_BOT_GROUP=irt-dcs

make renovate-admin-run   RENOVATE_BOT_NAME=renovate-bot-irt-dcs \

                          RENOVATE_TOKEN_FILE=~/.otica-tokens/renovate-bot-irt-dcs.token \

                          RENOVATE_ADMIN_CONFIG=renovate-admin.irt-dcs.json

```

Let your GitLab-identity Terraform own the bot and have Renovate consume only a

Vault token path — clean separation, and it works on GitLab **CE** (where the

Enterprise service-accounts API isn't available) by using an **external user**.

Use a SA-provisioning project (e.g. `provision-gitlab-sa`) or the

[`gitlab-renovate-bot`](https://code.stanford.edu/iac/examples/platforms/gitlab-platform/-/tree/main/terraform/modules/gitlab-renovate-bot)

example module:



`list` / `rotate` / `revoke` all act on the bot named by `RENOVATE_BOT_NAME`, so

each tree's bot is managed independently. (For sub-repos spanning groups with **no**

common parent, the single cross-group service account below is usually simpler than

one bot per group.)



**Use it for the scan** — point Renovate at the bot token (it outranks the ambient

`GITLAB_TOKEN`):



```bash

# one group bot that can see every repo in the group

make renovate-bot-create RENOVATE_BOT_GROUP=mygroup

make renovate-admin-run RENOVATE_TOKEN_FILE=~/.otica-tokens/renovate.token

# (or set RENOVATE_TOKEN_FILE in env.mk so it's the default)

```



**Use it for CI/CD** — create a project bot and push it into the project's variable

the `renovate-ci.yml` template reads:



```bash

make renovate-bot-create        # project token (origin)

make renovate-bot-ci-set        # sets masked RENOVATE_TOKEN on the project

```



### Rotation



`make renovate-bot-rotate` rotates **atomically** via GitLab's rotate API — it

revokes the old token and issues a new one with the same name/scopes and a fresh

`RENOVATE_BOT_EXPIRES`, then rewrites `RENOVATE_BOT_TOKEN_FILE` and the Vault path.

No duplicate tokens, no manual revoke.



- group/project token → rotated by id with your creator token;

- service-account token → self-rotated using the bot token itself (no admin).



It's `confirm.sh`-guarded, so it's cron-friendly with `NONINTERACTIVE=1` — schedule

it ahead of `RENOVATE_BOT_EXPIRES`. If the token feeds a CI variable, re-run

`renovate-bot-ci-set` after rotating (or read it from Vault in CI so there's

nothing to update). Tear down entirely with `renovate-bot-revoke`.



### Terraform-provisioned bot (recommended for managed orgs)



If you already manage GitLab identities with Terraform, let *that* own the bot and

have Renovate consume only a Vault token path — clean separation, and it works on

GitLab **CE** (where service accounts aren't available) by using an **external

user**. Example with a SA-provisioning project (`provision-gitlab-sa`):



1. Add a service-account entry (creates the user + token, adds it to the groups

   Renovate scans, stores the token in Vault):

1. Declare the bot — creates an external-user SA + `api`/`write_repository` token,

   adds it to the groups Renovate scans, and stores the token in Vault:

   ```hcl

   { name = "sa_renovate", note = "Renovate bot",

   { name = "sa-renovate-<proj>", note = "Renovate bot",

     vault_path_prefix = "secret/projects/<proj>/gitlab/sa",

     token_scopes = ["api", "write_repository"],

     groups = [ { full_path = "gcp",     access_level = "maintainer" },

                { full_path = "irt-dcs", access_level = "maintainer" } ] }

   ```

   `terraform apply` → token at `…/sa/sa_renovate/access_token`.

   `terraform apply` → token at `…/sa/sa-renovate-<proj>/access_token`.

2. Point Renovate at it — nothing else needed:

   ```make

   RENOVATE_TOKEN_VAULT_PATH = secret/projects/<proj>/gitlab/sa/sa_renovate/access_token

   ```

The `renovate-bot-*` targets below are the alternative for consumers *without* such

a project.



### Cross-group bot (sub-repos in several top-level groups)



A group/project access token only covers **one** group, and Renovate uses **one

token per run** — so if your scan set spans multiple top-level groups (e.g.

`gcp/*`, `irt-dcs/*`, `irt-db/*`, with no common parent) a single group token

won't reach them all. Use **one bot user (service account)** added to each group:



| Target | Who runs it | What it does |

| --- | --- | --- |

| `renovate-bot-sa-create` | an **instance admin** | Creates a GitLab service account + a PAT (api, write_repository) via the admin-only `/service_accounts` API; stores the PAT at `RENOVATE_BOT_TOKEN_FILE`. Needs `RENOVATE_BOT_ADMIN_TOKEN_FILE`. |

| `renovate-bot-add-groups` | a **group Owner** (you) | Adds `RENOVATE_BOT_USER` as a member (Developer) of every group in the scan set — taken from `RENOVATE_BOT_GROUPS`, else the distinct top-level groups in `renovate-admin.json`. |



```bash

# 1. admin (once): mint the service account + token

make renovate-bot-sa-create RENOVATE_BOT_ADMIN_TOKEN_FILE=~/.gitlab-admin-token RENOVATE_BOT_USER=renovate-bot



# 2. you (group owner): grant it the groups it must scan

make renovate-bot-add-groups RENOVATE_BOT_USER=<sa-username>     # groups inferred from renovate-admin.json



# 3. point scans at the bot

make renovate-admin-run RENOVATE_TOKEN_FILE=~/.otica-tokens/renovate.token

   RENOVATE_TOKEN_VAULT_PATH = secret/projects/<proj>/gitlab/sa/sa-renovate-<proj>/access_token

   ```



`sa-create` also stores the token in **Vault** when `RENOVATE_TOKEN_VAULT_PATH`

is set — or automatically at `${SEC_PATH}/gitlab/renovate_bot/<user>/token` when

`SEC_PATH` is defined (via `vault-write.sh`). It's the **same** variable the scan

reads from, so set it once and the token never lands on disk: `sa-create` writes

it, `renovate-admin-run` reads it back with `vault-read.sh`.

**Cross-group fleets** (sub-repos in several top-level groups with no common

parent) are covered the same way: one external-user bot added to each group sees

them all in a single run — no per-group tokens needed. **Rotate/revoke** by

re-applying or tainting the token resource in the Terraform project.



If you can't create a service account (the API is admin-only — it returns 404 to

non-admins), the equivalent is any **dedicated user** added to those groups; just

store its PAT at `RENOVATE_BOT_TOKEN_FILE`. Either way, one identity that's a

member of all the groups is what lets a single Renovate run see every sub-repo.

Without Terraform, point `RENOVATE_TOKEN_FILE` (or `RENOVATE_TOKEN_VAULT_PATH`) at

any token you mint by hand with `api` + `write_repository` scope, made a member of

the groups it must scan.



## Scheduled runs in GitLab CI (`.gitlab/renovate-ci.yml`)

makefiles/renovate-admin.mk

+13 −65
Original line number Diff line number Diff line
@@ -3,25 +3,22 @@ ifndef RENOVATE_ADMIN_MK_INCLUDED 
unexport RENOVATE_ADMIN_MK_INCLUDED



####################################

# Renovate (admin) — org-wide autodiscover across many repos

# Renovate (admin) — scan a controlled set of repos from one place

# Ref: https://docs.renovatebot.com/self-hosted-configuration/#autodiscover

#

# Usage in project Makefile:

#   include ${OTICA_DIR}/makefiles/renovate-admin.mk

#

# Runs Renovate with autodiscover so it processes every repo the token can see,

# optionally narrowed by RENOVATE_AUTODISCOVER_FILTER (e.g. "mygroup/**"). Needs

# a group/instance token: read_api for dry-runs, api for real runs.

#

# Centralize control in one place with RENOVATE_ADMIN_CONFIG: a global config

# file (default renovate-admin.json) holding autodiscover / autodiscoverFilter /

# onboarding / requireConfig. Keep onboarding=false + requireConfig=required so

# only repos that already have their own renovate.json are managed — editing this

# one file (or onboarding a repo) controls the whole fleet.

# Drives Renovate over the repos in RENOVATE_ADMIN_CONFIG (default

# renovate-admin.json) — an explicit `repositories` list (recommended) or

# autodiscover + RENOVATE_AUTODISCOVER_FILTER. The identity Renovate runs as is

# provisioned out-of-band (e.g. a Terraform SA project) and consumed via

# RENOVATE_TOKEN_VAULT_PATH — see renovate.mk and docs/renovate.md.

#

# Targets:

#   renovate-admin-dryrun  autodiscover dry-run across accessible repos

#   renovate-admin-run     autodiscover real run across accessible repos (confirm)

#   renovate-admin-update  rebuild the repo list in RENOVATE_ADMIN_CONFIG from repos.txt

#   renovate-admin-dryrun  dry-run across the configured repo set

#   renovate-admin-run     real run across the configured repo set (confirm)

#

# See docs/renovate.md.

####################################
@@ -33,68 +30,19 @@ endif 


RENOVATE_AUTODISCOVER_FILTER ?=

RENOVATE_ADMIN_CONFIG        ?= renovate-admin.json

# Optional dedicated Renovate bot identity (access token). Group token if

# RENOVATE_BOT_GROUP is set (multi-repo scan), else a project token (per-repo CI).

# RENOVATE_BOT_NAME identifies the bot — use distinct names to run several bots

# for different repo trees (each gets its own token, file, and Vault path).

RENOVATE_BOT_NAME            ?= renovate-bot

RENOVATE_BOT_GROUP           ?=

RENOVATE_BOT_REPO            ?=

RENOVATE_BOT_TOKEN_FILE      ?= ${HOME}/.otica-tokens/${RENOVATE_BOT_NAME}.token

RENOVATE_BOT_ACCESS_LEVEL    ?=

RENOVATE_BOT_EXPIRES         ?=

# Cross-group service-account bot (sub-repos spanning several top-level groups).

RENOVATE_BOT_USER            ?=

RENOVATE_BOT_GROUPS          ?=

RENOVATE_BOT_ADMIN_TOKEN_FILE ?=

# sa-create can also store the token in Vault via RENOVATE_TOKEN_VAULT_PATH (the

# same var the scan reads from) — declared in renovate.mk.

export RENOVATE_AUTODISCOVER_FILTER RENOVATE_ADMIN_CONFIG \

       RENOVATE_BOT_NAME RENOVATE_BOT_GROUP RENOVATE_BOT_REPO RENOVATE_BOT_TOKEN_FILE \

       RENOVATE_BOT_ACCESS_LEVEL RENOVATE_BOT_EXPIRES \

       RENOVATE_BOT_USER RENOVATE_BOT_GROUPS RENOVATE_BOT_ADMIN_TOKEN_FILE

export RENOVATE_AUTODISCOVER_FILTER RENOVATE_ADMIN_CONFIG



.PHONY: renovate-admin-update

renovate-admin-update: ## regenerate ${RENOVATE_ADMIN_CONFIG} repos from ${SUB_REPOS_FILE} (tf/docker only)

	@renovate-admin-update.sh



# ── Optional Renovate bot identity ──────────────────────────────────────────────

.PHONY: renovate-bot-create

renovate-bot-create: ## create a dedicated Renovate bot token (group if RENOVATE_BOT_GROUP set, else project)

	@renovate-bot.sh create



.PHONY: renovate-bot-list

renovate-bot-list: ## list renovate-bot tokens on the target group/project

	@renovate-bot.sh list



.PHONY: renovate-bot-rotate

renovate-bot-rotate: ## rotate the renovate-bot token (revoke old + issue new); updates file/Vault

	@if confirm.sh "Rotate the renovate-bot token (the old token will be revoked)?"; then renovate-bot.sh rotate ; fi



.PHONY: renovate-bot-revoke

renovate-bot-revoke: ## revoke the renovate-bot token and remove the local file

	@if confirm.sh "Revoke the renovate-bot token?"; then renovate-bot.sh revoke ; fi



.PHONY: renovate-bot-ci-set

renovate-bot-ci-set: ## set the masked RENOVATE_TOKEN CI/CD variable from the bot token (REPO=ns/proj)

	@renovate-bot.sh ci-set ${REPO}



# Cross-group: one bot user across several top-level groups.

.PHONY: renovate-bot-sa-create

renovate-bot-sa-create: ## create a service-account bot + token (needs RENOVATE_BOT_ADMIN_TOKEN_FILE, instance admin)

	@renovate-bot.sh sa-create



.PHONY: renovate-bot-add-groups

renovate-bot-add-groups: ## add RENOVATE_BOT_USER to each group it must scan (group owner; groups from admin config)

	@renovate-bot.sh add-groups



.PHONY: renovate-admin-dryrun

renovate-admin-dryrun: renovate-check ## autodiscover dry-run across accessible repos (read_api)

renovate-admin-dryrun: renovate-check ## dry-run across the configured repo set (read_api)

	@renovate-run.sh autodiscover-dryrun



.PHONY: renovate-admin-run

renovate-admin-run: renovate-check ## autodiscover real run across accessible repos (api)

	@if confirm.sh "Run Renovate autodiscover$${RENOVATE_AUTODISCOVER_FILTER:+ (filter: $$RENOVATE_AUTODISCOVER_FILTER)} and open MRs across ALL matching repos?"; then \

renovate-admin-run: renovate-check ## real run across the configured repo set (api)

	@if confirm.sh "Run Renovate across the configured repo set and open MRs?"; then \

		renovate-run.sh autodiscover-run ; \

	fi

makefiles/renovate.mk

+1 −1
Original line number Diff line number Diff line
@@ -35,7 +35,7 @@ RENOVATE_ARGS ?= 
# points at a different GitLab (e.g. a code.stanford.edu Terraform backend) than

# where this repo lives. Precedence: RENOVATE_TOKEN > RENOVATE_TOKEN_FILE >

# RENOVATE_TOKEN_VAULT_PATH > GITLAB_TOKEN > ~/.gitlab-token. The Vault path reads

# back what renovate-bot sa-create stored (same RENOVATE_TOKEN_VAULT_PATH).

# back the token a SA-provisioning project (e.g. provision-gitlab-sa) wrote there.

RENOVATE_TOKEN_FILE       ?=

RENOVATE_TOKEN_VAULT_PATH ?=

scripts/renovate-bot.sh

deleted100755 → 0
+0 −258

File deleted.

Preview size limit exceeded, changes collapsed.

scripts/renovate-run.sh

+2 −2
Original line number Diff line number Diff line
@@ -26,7 +26,7 @@ 
#   RENOVATE_TOKEN_FILE   file holding that token; use when the project's

#                         GITLAB_TOKEN targets a different server than this repo

#   RENOVATE_TOKEN_VAULT_PATH  Vault path to read the token from (vault-read.sh);

#                         the same path renovate-bot sa-create writes it to

#                         where a SA-provisioning project writes the bot token

#   GITLAB_SERVER         default: the origin remote's host (else code.stanford.edu)

#   GITLAB_TOKEN[_FILE]   consumer GitLab token / file (default ~/.gitlab-token)

#
@@ -83,7 +83,7 @@ case "${GITLAB_SERVER}" in *://*) ;; *) GITLAB_SERVER="https://${GITLAB_SERVER}" 
# A project's GITLAB_TOKEN may target a DIFFERENT GitLab server than this repo

# (e.g. a code.stanford.edu Terraform backend while the code lives on gitlab.med),

# so a dedicated Renovate token for the repo's own server wins over it. The Vault

# path is the same one renovate-bot sa-create writes the token to (write -> read).

# path is where a SA-provisioning project (e.g. provision-gitlab-sa) writes it.

RENOVATE_TOKEN="${RENOVATE_TOKEN:-}"

RENOVATE_TOKEN_FILE="${RENOVATE_TOKEN_FILE:-}"

RENOVATE_TOKEN_VAULT_PATH="${RENOVATE_TOKEN_VAULT_PATH:-}"