Commit e22eaae5 authored by xuwang's avatar xuwang
Browse files

feat(release): generalize release module for any GitLab repo



Rename release.mk -> gitlab-release.mk and the otica-* release scripts to
generic/gitlab- names, and decouple the release from the OTICA install: the
scripts now act on RELEASE_REPO_DIR (the dir make runs in), with VERSION_FILE,
CHANGELOG_FILE, RELEASE_REMOTE, RELEASE_BRANCH, RELEASE_CHANGELOG_MARKER and
RELEASE_VALIDATE_CMD overridable. Any GitLab repo can now include the module to
cut its own releases; OTICA preserves its behavior via RELEASE_VALIDATE_CMD.

Add a `make gitlab-release-onboarding` target (scaffolds VERSION, CHANGELOG.md,
.gitlab/changelog_config.yml; seeds VERSION from the latest vX.Y.Z tag) and a
RELEASE_NOTES_EXTRA_CMD hook that appends command output to the notes (e.g. a
docker image's bundled component versions). Documented in docs/gitlab-release.md.

Changelog: added
Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
parent a24ae0f4
Loading
Loading
Loading
Loading
+2 −2
Original line number Diff line number Diff line
@@ -75,11 +75,11 @@ The `MAKELEVEL` guard prevents re-running on recursive make calls. There is **no

### Releases & Versioning

`VERSION` (repo root) is the semver source of truth; releases are git tags `vX.Y.Z` with a generated `CHANGELOG.md` and a GitLab Release. `makefiles/release.mk` (included by the top-level `Makefile`, developer-only) drives `version` / `release-preview` / `release` / `release-{patch,minor,major}` via the `otica-version.sh`, `otica-changelog.sh`, `otica-release.sh` scripts. Releases publish to the canonical remote (`OTICA_RELEASE_REMOTE`, default `origin` = code.stanford.edu) and authenticate to the **OTICA hosting server** with `OTICA_SERVER_TOKEN[_FILE]` — distinct from a consumer's `GITLAB_TOKEN`.
`VERSION` (repo root) is the semver source of truth; releases are git tags `vX.Y.Z` with a generated `CHANGELOG.md` and a GitLab Release. `makefiles/gitlab-release.mk` (included by the top-level `Makefile`) drives `version` / `release-preview` / `release` / `release-{patch,minor,major}` via the `release-version.sh`, `gitlab-changelog.sh`, `gitlab-release.sh` scripts. The module is GitLab-specific but repo-agnostic: it operates on `RELEASE_REPO_DIR` (default: the dir `make` runs in, so OTICA releases itself), with `VERSION_FILE` / `CHANGELOG_FILE` / `RELEASE_CHANGELOG_MARKER` / `RELEASE_VALIDATE_CMD` overridable — so any GitLab repo (e.g. a docker-xxxx project) can include it. OTICA's top Makefile sets `RELEASE_VALIDATE_CMD = tests/validate.sh all` to gate its own releases. Releases publish to the canonical remote (`RELEASE_REMOTE`, default `origin` = code.stanford.edu; legacy `OTICA_RELEASE_REMOTE` still honored) and authenticate to the **release host** with `OTICA_SERVER_TOKEN[_FILE]` (falls back to `GITLAB_TOKEN`).

**Mirroring:** code.stanford.edu is canonical; gitlab.med is an automatic **push mirror** (branches + tags), so a release pushed to `origin` reaches med on the next mirror sync (GitLab rate-limits push mirrors to about once every 5 minutes). Never release to or push directly at med — it's overwritten to match code. GitLab Release *objects* aren't mirrored (only tags), which is all consumers need to pin.

`CHANGELOG.md` is generated from `Changelog:` git trailers (config in `.gitlab/changelog_config.yml`). Consumers move to a release with `make update-otica` and are nudged to upgrade by `otica-upgrade-check.sh` (release-tag-aware). See the [README](README.md) "Releases & Versioning" and [CONTRIBUTING.md](CONTRIBUTING.md).
`CHANGELOG.md` is generated from `Changelog:` git trailers (config in `.gitlab/changelog_config.yml`). Consumers move to a release with `make update-otica` and are nudged to upgrade by `otica-upgrade-check.sh` (release-tag-aware). See the [README](README.md) "Releases & Versioning", [docs/gitlab-release.md](docs/gitlab-release.md) (full module reference: targets, variables, `make gitlab-release-onboarding`, `RELEASE_NOTES_EXTRA_CMD`, docker CI integration), and [CONTRIBUTING.md](CONTRIBUTING.md).

### Agentic Workflow (developer)

+5 −1
Original line number Diff line number Diff line
@@ -7,8 +7,12 @@
OTICA_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST))))
OTICA_REPO ?= $(shell git -C $(OTICA_DIR) config --get remote.origin.url 2>/dev/null)

# Gate OTICA's own releases on the framework validator. gitlab-release.mk runs
# this in the repo before tagging; consumers without a validator leave it unset.
export RELEASE_VALIDATE_CMD ?= tests/validate.sh all

include $(OTICA_DIR)/makefiles/help.mk
include $(OTICA_DIR)/makefiles/release.mk
include $(OTICA_DIR)/makefiles/gitlab-release.mk

.PHONY: validate
validate: ## run all validation checks (parse + deps + scripts + changelog)
+23 −2
Original line number Diff line number Diff line
@@ -118,8 +118,29 @@ make release VERSION=1.3.0 # or cut an explicit version

`make release` runs validation, renders the changelog, updates `VERSION` +
`CHANGELOG.md`, commits, tags `vX.Y.Z`, pushes to the canonical remote
(`OTICA_RELEASE_REMOTE`, default `origin`), and publishes a GitLab Release —
prompting for confirmation before any mutation.
(`RELEASE_REMOTE`, default `origin`; legacy `OTICA_RELEASE_REMOTE` still honored),
and publishes a GitLab Release — prompting for confirmation before any mutation.

The release module ([`makefiles/gitlab-release.mk`](makefiles/gitlab-release.mk))
is GitLab-specific but repo-agnostic: it acts on `RELEASE_REPO_DIR` (the dir
`make` runs in), so any GitLab repo can `include` it and cut releases the same
way. Repo-specific knobs (`VERSION_FILE`, `CHANGELOG_FILE`,
`RELEASE_CHANGELOG_MARKER`, `RELEASE_VALIDATE_CMD`) default to OTICA's layout.

**Onboarding a new repo:** `include` the module, then run
`make gitlab-release-onboarding` to scaffold `VERSION` (seeded from the latest
`vX.Y.Z` tag, else `0.1.0`), `CHANGELOG.md` (with the prepend marker), and
`.gitlab/changelog_config.yml`. The target is idempotent — it never clobbers an
existing file. Set `RELEASE_BRANCH` in `env.mk` if the default branch isn't
`main`.

**Customizing release notes:** notes come from `Changelog:` git trailers. To add
content that isn't in the commits — e.g. a list of bundled component versions —
set `RELEASE_NOTES_EXTRA_CMD` to a command whose stdout is appended under the
release header (flows into the preview, `CHANGELOG.md`, and the GitLab Release).

Full module reference (targets, variables, docker CI integration, onboarding):
[docs/gitlab-release.md](docs/gitlab-release.md).

### Mirroring topology

docs/gitlab-release.md

0 → 100644
+257 −0
Original line number Diff line number Diff line
# GitLab Releases & Versioning

How any GitLab-hosted repo cuts versioned releases with OTICA: bump a `VERSION`,
generate a `CHANGELOG.md` from `Changelog:` git trailers, tag `vX.Y.Z`, push, and
publish a **GitLab Release** — all from `make`.

This is the OTICA framework reference for the
[`gitlab-release.mk`](../makefiles/gitlab-release.mk) module (helper scripts:
[`release-version.sh`](../scripts/release-version.sh),
[`gitlab-changelog.sh`](../scripts/gitlab-changelog.sh),
[`gitlab-release.sh`](../scripts/gitlab-release.sh),
[`gitlab-release-onboard.sh`](../scripts/gitlab-release-onboard.sh)). OTICA uses
it for its own framework releases, and it is reusable by any GitLab repo. Working
docker examples live in the demo project under
`demo-project/sub-projects/docker-toolchain` and `…/docker-demo-app`.

> **GitLab-specific.** Changelog rendering and the Release object use the GitLab
> REST API. The module is not portable to GitHub/Gitea as-is.

---

## The release model

```
make release-minor
   │  bump VERSION (2.3.0 → 2.4.0)
   │  render notes from `Changelog:` trailers  (GitLab changelog API)
   │  prepend notes to CHANGELOG.md
   │  commit "Release v2.4.0"  +  annotated tag v2.4.0
   │  push  <branch> + tag  →  origin

git tag v2.4.0  ──────────────►  GitLab Release v2.4.0  (notes as description)

        └──► CI pipeline on the tag  ──►  builds & publishes image :v2.4.0
                                           (for docker repos — see "CI integration")
```

Key ideas:

- **`VERSION`** (repo root) is the single source of truth — plain `X.Y.Z` semver.
  `gitlab-release.sh` is the only writer, so a failed release never leaves it
  dirty.
- **The git tag is the release.** `vX.Y.Z` is what consumers pin to and what a
  docker CI pipeline builds from.
- **The repo being released is the directory `make` runs in** (`RELEASE_REPO_DIR`,
  default `$(CURDIR)`) — *not* the OTICA install. That decoupling is what makes
  the module reusable across repos.

---

## Quick start (onboard a repo)

1. Include the module in your `Makefile` (after `help.mk`):

   ```makefile
   include ${OTICA_DIR}/makefiles/help.mk
   include ${OTICA_DIR}/makefiles/docker.mk
   include ${OTICA_DIR}/makefiles/gitlab-release.mk
   ```

2. Scaffold the release artifacts:

   ```bash
   make gitlab-release-onboarding
   ```

   This creates (idempotently — never clobbering existing files):

   | File | Purpose |
   |------|---------|
   | `VERSION` | semver source of truth (seeded from the latest `vX.Y.Z` tag, else `0.1.0`) |
   | `CHANGELOG.md` | generated changelog, with the prepend marker line |
   | `.gitlab/changelog_config.yml` | category map + render template for trailers |

3. If your default branch isn't `main`, set it in `env.mk` (the onboarding
   target reminds you):

   ```makefile
   RELEASE_BRANCH=master
   ```

4. Review, commit, and push the scaffolding. Then cut a release:

   ```bash
   make release-preview     # dry-run the notes; writes nothing
   make release-patch       # 2.3.0 → 2.3.1, tag, push, publish
   ```

---

## Targets

| Target | Description |
|--------|-------------|
| `make version` | print the current `VERSION` |
| `make gitlab-release-onboarding` | scaffold `VERSION`, `CHANGELOG.md`, `.gitlab/changelog_config.yml` |
| `make release-preview` | preview the rendered notes for the current `VERSION` (writes nothing) |
| `make release VERSION=x.y.z` | cut an explicit version |
| `make release-patch` | bump patch, then release |
| `make release-minor` | bump minor, then release |
| `make release-major` | bump major, then release |

Mutating targets gate on: clean working tree, on `RELEASE_BRANCH`, local branch
matches the remote, the tag doesn't already exist, the new version sorts strictly
above the latest `vX.Y.Z` tag, an optional validation hook, and a confirm prompt.

---

## Variables

| Variable | Default | Purpose |
|----------|---------|---------|
| `RELEASE_REPO_DIR` | `$(CURDIR)` | repo to release |
| `RELEASE_REMOTE` | `origin` (legacy `OTICA_RELEASE_REMOTE` honored) | canonical git remote |
| `RELEASE_BRANCH` | `main` | release branch |
| `VERSION_FILE` | `<repo>/VERSION` | version file |
| `CHANGELOG_FILE` | `<repo>/CHANGELOG.md` | changelog file |
| `RELEASE_CHANGELOG_MARKER` | `<!-- New release sections are prepended below this line` | line after which notes are inserted |
| `RELEASE_VALIDATE_CMD` | _(unset → skip)_ | pre-release check run in the repo |
| `RELEASE_NOTES_EXTRA_CMD` | _(unset)_ | command whose stdout is appended to the notes |
| `VERSION` | — | explicit version for `make release` |

Set repo-specific values in `env.mk` (which the consumer `Makefile` `export`s).

---

## Changelog from `Changelog:` trailers

Notes are collected from commit messages carrying a `Changelog:` git trailer.
Commits without one are ignored, so routine commits stay out of the changelog.

```
Pin base image to bookworm-slim

Changelog: changed
```

The trailer value selects the section (`.gitlab/changelog_config.yml`):
`breaking` · `added` · `changed` · `deprecated` · `removed` · `fixed` ·
`security` · `performance` · `other`.

> **Trailer gotcha.** The `Changelog:` line must sit in the contiguous trailer
> block at the **end** of the message (next to any `Co-Authored-By:`), or git
> won't parse it and the entry is dropped. `make release` warns about
> `feat/fix/perf` commits in range that have no trailer before you confirm.

---

## Customizing release notes

Trailers cover "what changed in the code." For content that isn't derivable from
commits — most commonly **the versions of tools bundled in a docker image**
set `RELEASE_NOTES_EXTRA_CMD`. Its stdout is appended under the release header, so
it shows up in `release-preview`, `CHANGELOG.md`, and the GitLab Release alike.

In `env.mk`:

```makefile
RELEASE_NOTES_EXTRA_CMD=bash .gitlab/release-components.sh
```

A repo-local extractor that reads the pinned versions out of the `Dockerfile`
(place it outside any gitignored `scripts/` dir, e.g. in `.gitlab/`):

```bash
#!/bin/bash -eu
# .gitlab/release-components.sh — emit a Components section from the Dockerfile.
df=${1:-Dockerfile}
echo "### Components"
echo
awk '
    /get\.helm\.sh\/helm-v/ { if (match($0,/v[0-9]+\.[0-9]+\.[0-9]+/)) printf "- helm %s\n",      substr($0,RSTART,RLENGTH) }
    /AZ_VERSION=/            { if (match($0,/[0-9]+\.[0-9]+\.[0-9]+/))  printf "- azure-cli %s\n", substr($0,RSTART,RLENGTH) }
' "${df}"
```

Resulting notes:

```markdown
## 2.87.1 (2026-06-18)

#### Added

- Adopt OTICA release flow (abc1234)

### Components

- helm v4.2.0
- azure-cli 2.87.0
```

For one-off, hand-written notes instead of an automated list, point
`RELEASE_NOTES_EXTRA_CMD` at `cat RELEASE_NOTES_EXTRA.md` and edit that file
before releasing.

---

## Authentication

The release talks to the GitLab REST API (changelog + Release object) and needs
an **api-scoped token for the release host**. Resolution order
([`otica-server-token.sh`](../scripts/otica-server-token.sh)):

1. `OTICA_SERVER_TOKEN` (env)
2. `OTICA_SERVER_TOKEN_FILE` (default `~/.otica-token`)
3. `GITLAB_TOKEN`

> **Host-shadowing gotcha.** Because the default file `~/.otica-token` usually
> exists (the OTICA-host token), a consumer releasing on a *different* GitLab
> host who sets only `GITLAB_TOKEN` gets it **shadowed** → `401`. On a consumer
> host, set the token explicitly:
>
> ```bash
> export OTICA_SERVER_TOKEN_FILE=~/.gitlab-token   # or OTICA_SERVER_TOKEN=…
> make release-patch
> ```

---

## CI integration (docker images)

For docker repos the git tag is the build trigger. With the Cloud Build template
(`cicd/gbuild-image.yml`, used by `docker-toolchain`), the `build-image` job runs
only on a tag:

```yaml
build-image:
  rules:
    - if: $CI_COMMIT_TAG
  # builds & pushes  :latest  :prod-latest  :$CI_COMMIT_TAG
```

So `make release-patch` → pushes `v2.4.0` → the tag pipeline builds and publishes
image `:v2.4.0`. The release commit on the branch does **not** build an image
(the job is tag-only). No CI changes are needed to adopt the release flow — the
module only adds the tagging/notes/Release layer on top of the existing
tag-triggered build.

---

## Rollout ordering

A repo that `include`s `gitlab-release.mk` depends on that file existing in the
`~/.otica` checkout. CI is unaffected (the docker build doesn't use the
`Makefile`), but **local `make`** breaks for anyone whose `~/.otica` predates the
module. Roll out in this order:

1. Release OTICA with `gitlab-release.mk` and have consumers `make update-otica`.
2. Then add the `include` (and `env.mk` release vars) to consumer repos.

---

## See also

- [README → Releases & Versioning](../README.md) — OTICA's own release workflow
- [CONTRIBUTING.md](../CONTRIBUTING.md)`Changelog:` trailer conventions
- `demo-project/sub-projects/docker-toolchain` — Cloud Build (tag-triggered) example
+41 −25
Original line number Diff line number Diff line
ifndef RELEASE_MK_INCLUDED
	RELEASE_MK_INCLUDED := 1
unexport RELEASE_MK_INCLUDED
ifndef GITLAB_RELEASE_MK_INCLUDED
	GITLAB_RELEASE_MK_INCLUDED := 1
unexport GITLAB_RELEASE_MK_INCLUDED

## release.mk
## gitlab-release.mk
##
## Maintainer targets for cutting OTICA framework releases: bump the version,
## Targets for cutting a release of a GitLab-hosted repo: bump the version,
## render a changelog from `Changelog:` git trailers, tag, push, and publish a
## GitLab Release on the canonical remote.
## GitLab Release on the canonical remote. GitLab-specific (uses the GitLab
## changelog + releases REST API); not portable to other forges as-is.
##
## This module is for OTICA *developers* (included by the repo's top-level
## Makefile) — consumer projects do not use it. Consumers instead move to a
## release with `make update-otica` (see envs/otica-env.mk).
## OTICA uses this for its own framework releases (included by the repo's
## top-level Makefile). It is also reusable by any GitLab repo (e.g. a
## docker-xxxx project): include this file and set the variables below — the
## release operates on RELEASE_REPO_DIR (the directory make runs in), not on the
## OTICA install. Consumers do not use it to *consume* OTICA; that is
## `make update-otica` (see envs/otica-env.mk).
##
## Implementation: helper scripts in OTICA_SCRIPTS_DIR talk to the GitLab REST
## API via curl (see scripts/gitlab.sh). Mutating targets gate on a clean tree,
## the release branch, validation, and a confirm prompt.
## the release branch, optional validation, and a confirm prompt.
##
## Variables:
##   OTICA_RELEASE_REMOTE       canonical git remote (default: origin)
##   RELEASE_REPO_DIR           repo to release (default: the make dir, $(CURDIR))
##   RELEASE_REMOTE             canonical git remote (default: origin; honors
##                              legacy OTICA_RELEASE_REMOTE)
##   RELEASE_BRANCH             release branch (default: main)
##   VERSION_FILE               version file (default: <repo>/VERSION)
##   CHANGELOG_FILE             changelog file (default: <repo>/CHANGELOG.md)
##   RELEASE_CHANGELOG_MARKER   line after which new notes are inserted
##   RELEASE_VALIDATE_CMD       pre-release check run in the repo (default: none)
##   RELEASE_NOTES_EXTRA_CMD    command whose stdout is appended to the notes
##                              (e.g. a bundled-component version list)
##   VERSION                    explicit version for `release` (e.g. VERSION=1.2.0)
## Required: a token for the OTICA *hosting* server (which may differ from a
## consumer project's GitLab server) via OTICA_SERVER_TOKEN or
## OTICA_SERVER_TOKEN_FILE (default: ~/.otica-token).
## Required: an API token for the release host via OTICA_SERVER_TOKEN or
## OTICA_SERVER_TOKEN_FILE (default: ~/.otica-token), falling back to GITLAB_TOKEN.

# Work standalone (dev Makefile) or alongside help.mk (sets these already).
OTICA_DIR ?= $(abspath $(dir $(lastword $(MAKEFILE_LIST)))..)
OTICA_SCRIPTS_DIR ?= ${OTICA_DIR}/scripts
OTICA_RELEASE_REMOTE ?= origin
RELEASE_REPO_DIR ?= $(CURDIR)
RELEASE_REMOTE ?= $(or $(OTICA_RELEASE_REMOTE),origin)
RELEASE_BRANCH ?= main
export

.PHONY: gitlab-release-onboarding
gitlab-release-onboarding: ## scaffold release artifacts (VERSION, CHANGELOG.md, changelog config)
	@${OTICA_SCRIPTS_DIR}/gitlab-release-onboard.sh

.PHONY: version
version: ## print the current framework version
	@${OTICA_SCRIPTS_DIR}/otica-version.sh
version: ## print the current project version
	@${OTICA_SCRIPTS_DIR}/release-version.sh

.PHONY: release-preview
release-preview: ## preview changelog notes for VERSION (default: current). Writes nothing
	@v="${VERSION}"; [ -n "$$v" ] || v=$$(${OTICA_SCRIPTS_DIR}/otica-version.sh); \
	@v="${VERSION}"; [ -n "$$v" ] || v=$$(${OTICA_SCRIPTS_DIR}/release-version.sh); \
		echo "Changelog preview for v$$v:"; echo; \
		${OTICA_SCRIPTS_DIR}/otica-changelog.sh "$$v"
		${OTICA_SCRIPTS_DIR}/gitlab-changelog.sh "$$v"

.PHONY: release
release: ## cut a release: make release VERSION=x.y.z (validate, changelog, tag, push, publish)
@@ -47,20 +63,20 @@ release: ## cut a release: make release VERSION=x.y.z (validate, changelog, tag,
		echo "Usage: make release VERSION=x.y.z (or use release-patch|release-minor|release-major)"; \
		exit 1; \
	fi
	@${OTICA_SCRIPTS_DIR}/otica-release.sh "${VERSION}"
	@${OTICA_SCRIPTS_DIR}/gitlab-release.sh "${VERSION}"

.PHONY: release-patch
release-patch: ## bump the patch version and cut a release
	@$(MAKE) --no-print-directory release VERSION=$$(${OTICA_SCRIPTS_DIR}/otica-version.sh patch)
	@$(MAKE) --no-print-directory release VERSION=$$(${OTICA_SCRIPTS_DIR}/release-version.sh patch)

.PHONY: release-minor
release-minor: ## bump the minor version and cut a release
	@$(MAKE) --no-print-directory release VERSION=$$(${OTICA_SCRIPTS_DIR}/otica-version.sh minor)
	@$(MAKE) --no-print-directory release VERSION=$$(${OTICA_SCRIPTS_DIR}/release-version.sh minor)

.PHONY: release-major
release-major: ## bump the major version and cut a release
	@$(MAKE) --no-print-directory release VERSION=$$(${OTICA_SCRIPTS_DIR}/otica-version.sh major)
	@$(MAKE) --no-print-directory release VERSION=$$(${OTICA_SCRIPTS_DIR}/release-version.sh major)

## end of release.mk
## end of gitlab-release.mk

endif # RELEASE_MK_INCLUDED
endif # GITLAB_RELEASE_MK_INCLUDED
Loading