Commit 0cbb9d6c authored by Adam Lewenberg's avatar Adam Lewenberg
Browse files

much work on README

parent 5d8ebdbb
Pipeline #31877 failed with stages
in 6 seconds
[[_TOC_]]
# Build a Debian package using GitLab CI/CD
## Quick Start
This repository has quite a bit of technical detail. If all you want are the
instructions to build and upload your Debian package to Stanford's Debian
repository start with ["Build and upload a Debian package to Stanford's
repository"][1].
## Overview
This project contains GitLab CI/CD scripts used by _other_ GitLab projects
to build and (optionally) upload a Debian package. That is, this project
does not do any CI/CD itself, rather, it provides scripts and files for
_other_ GitLab projects to use for building/uploading a Debian package.
To see an example of another GitLab project using the scripts in this
repository see the [gitlab-package-build-stanford GitLab project][1].
(Note: the [gitlab-package-build-stanford][1] project is access-restricted
to Stanford.)
At the heart of the biuld scripts this repository provides is the
[`debian-package` Docker container][2].
Depending on the value of certain environment variables this is what
the CI/CD does: For each Debian release listed in the environment variable
`DEBIAN_RELEASES`
1. The package is built against that Debian release.
1. The `lintian` utility is run against the results.
1. [Optional] The Debian package is uploaded using `dput`.
## Build configuration
* `DEBIAN_RELEASES`: a space-delimited list of Debian distributions you
want to build for. Example: `sid bullseye`. Default: `sid buster
bullseye`.
## `dput` configuration
Most of the complexity of the configuration centers around the `dput`
portion. We provide details on how to configure `dput` in this section.
### The `.dput.cf` file
The GitLab CI/CD creates a `.dput.cf` file that it hands off to the
`debian-package` Docker container. This file has four sections:
* `[stanford]`: upload to the `stanford` archive of `debian-repo.stanford.edu`.
* `[local]`: upload to the `local` archive of `debian-repo.stanford.edu`.
* `[stanford-dev]`: upload to the `stanford` archive of `debian-repo-dev.stanford.edu`.
* `[local-dev]`: upload to the `local` archive of `debian-repo-dev.stanford.edu`.
The generated `.dput.cf` file also configures the SSH options to use the
private SSH key pointed to by the file path in the `SSH_PRIVATE_KEY_PATH`.
You need to set `SSH_PRIVATE_KEY_PATH` yourself for each GitLab project.
See also the "Requirements" section below.
### Configuring `dput`
## Requirements
The CI/CD will only attempt to `dput` the Debian package if both
`DPUT_HOST` and `DPUT_CONF` are set.
* `DPUT_HOST`: If this is set to one of the sections of the `.dput.cf`
file then `dput` will upload the Debian build files to the server and
archive specified by that secton (see section "The `.dput.cf` file" for
more details.) Example: `stanford-dev`. Default: none.
* `DPUT_USER`: the username to use when `dput` uploads the Debian packages.
The server uploaded to must already have this account set up.
Example: `auto-build`. Default: none.
### Requirements
1. If you are using the `dput` portion of the `debian-package` Docker
image to upload the package build you will need an account on the target
Debian repository servers. This will be a local account that gains access
using an ssh key-pair.
Debian repository servers. This will be a local account that used
an ssh key-pair to gain access.
1. If uploading using `dput` you must create a project CI/CD File variable
`SSH_PRIVATE_KEY_PATH`. Its content should be the _private_ key part of
the SSH key-pair used by the login credentials you want to use for
accessing the Debian repository server. Note that this value is a _secret_
so be sure that the only people how have access to maintain your
repository have permission to see this value.
### Debian repository server local account
When using `dput` to upload the Debian package you will need to provide user
credentials to `dput` so it can access the server it uploads to. To do
this you must contact the managers of the Debian package repository
servers and ask them to set you up a local account on the server. You will
also need to provide them the public portion of an SSH key-pair. The
private portion you put in the `SSH_PRIVATE_KEY_PATH` GitLab CI/CD File
variable.
## CI/CD details
The file [`.gitlab-ci.yml`](.gitlab-ci.yml) defines what happens
when GitLab runs the project's CI/CD. There are two stages.
The first stage's purpose is to generate
a CI/CD YAML file used by the second stage. The second stage actually does
the Debian package build and `dput` upload. We are forced to use this
two-stage method because GitLab's CI/CD [DSL][3] does not support loops.
### Stage 1: (`generate-config`)
The first stage of [`.gitlab-ci.yml`](.gitlab-ci.yml) generates two file
[artifacts][4] to be used by the second stage. This first file is a YAML
file `generated-config.yml` that is itself a set of CI/CD instructions.
This file will be used as the set of instructions for the second stage.
The second file artifact generated is a `.dput.cf` file used when/if the
Debian package is uploaded via `dput`.
The artifact file `generated-config.yml` consists of one stage for each
Debian distribution enumerated in `DEBIAN_RELEASES`. Each of these
substages will do the following:
1. Fix the permissions on the file pointed to by `SSH_PRIVATE_KEY_PATH`.
1. Set the environment variable `DEBEMAIL` to the value of the author of
the latest commit.
1. If building for a Debian distribution other than `sid` add a new
entry to `debian/changelog` that sets the version (see versions below).
1. Do the Debian package build and upload using the [`debian-package`][2]
Docker image.
See the file [`gitlab-ci.yml.template`](gitlab-ci.yml.template) for
details on the generated YAML file.
### Stage 2: (`child-pipeline`)
The second stage of [`.gitlab-ci.yml`](.gitlab-ci.yml) is a [_child
pipeline_][5] that uses the instructions in the artifact file
`generated-config.yml` generated by `generate-config`. This second stage
is a child pipeline because (of this writing) using a child pipeline is
the only way to get GitLab to use a [dynamically-generated][6] CI/CD
instruction file.
[1]: https://code.stanford.edu/et-iedo-public/debian-packages/gitlab-package-build-stanford
[2]: https://github.com/macrotex/debian-package-docker
[3]: https://en.wikipedia.org/wiki/Domain-specific_language
## Building for distributions
[4]: https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html
If you are planning to upload the builds, yo uneed to do the builds
separately
[5]: https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html
1. Set `DEBIAN_DISTROS` to the distribution
[6]: https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html#dynamic-child-pipelines
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment