Gitea Generic Registry Automatic Update

What is a generic registry?

A generic registry is a registry without specific purpose. For example: container registry store containers, pypi registry store python packages, and so on… A generic registry can be used to store compiled binaries, configuration files. For example, I created a repository to myself where I want to store my shell setup (includes: zsh, tmux and neovim). If I deploy a new server, instead of install it from scratch, I just would like to download and uncompress a file.

It is very easy to upload into Gitea Generic registry, based on their document:

curl --user your_username:your_password_or_token \
     --upload-file path/to/file.bin \
     https://gitea.example.com/api/packages/testuser/generic/test_package/1.0.0/file.bin

The plan

The whole solution can be found in this repository.

To automate it, make sense a script has to be written. But furthermore, I want to put this script into a container. Why? Container can run on anywhere, so I am not tied for any server. I am not tied for any server dependency as well. Workflow of the container is pretty simple:

Checking input variables.
Checking version number that compatible with semver.
Git clone from the registry based on provided version number (so repository and package version will be match).
Run upload.sh script from the cloned repository.
Upload the result into Gitea Generic Registry.

Let’s make it

This section is split into two part:

Build the image itself.
Explain the builder script.

Build the image

I use Rocky Linux as base, the reason I also use it on my server. I hardcode the digest because I monitor my repositories using Renovate in systemd timers, see previous post. Further of this image is very straightforward: copy the builder script, install git, set some label.

Entrypoint will be builder script itself and as parameter it get build_and_push. The reason, because of some CI/CD reason, I want to have option to build without putting it into Gitea, so just let it there on a temporary volume.

The Container file looks like the following:

FROM quay.io/rockylinux/rockylinux:9-ubi-init@sha256:a3a9c6bdcbb848a0de2ce2fb02ec5ba2505f1c4b764c4dd570c760e34226aaab

RUN mkdir /app
WORKDIR /app

COPY ./build.sh .
RUN chmod +x ./build.sh

RUN dnf install -y git \
    && dnf clean all \
    && rm -rf /var/cache /var/log/dnf* /var/log/yum.*

LABEL summary="Generic package registry uploaded" \
    usage="Use it based on the README.md file in code repository" \
    name="generic-builder" \
    org.opencontainers.image.authors="Attila Molnar <onlyati@pm.me>" \
    org.opencontainers.image.description="This image is made to create and uplaod generic packages into Gitea generic regsitry" \
    org.opencontainers.image.licenses="MIT" \
    org.opencontainers.image.url="https://git.thinkaboutit.tech/pandora/generic-builder" \
    org.opencontainers.image.vendor="thinkaboutit.tech"

ENTRYPOINT [ "./build.sh" ]
CMD [ "build_and_push" ]

Builder script

The whole builder script can be found here.

Validate the input data

I start in the script with validation:

If not HEPHA_DEBUG specified, then default value is no.
If HEPHA_DEBUG value is yes, then set debug (set -x).
Write out the parameter that got and validate that.

#!/bin/bash

HEPHA_DEBUG="${HEPHA_DEBUG:=no}"

if [[ "$HEPHA_DEBUG" == "yes" ]]; then
    set -eux
else
    set -eu
fi

echo "script is started with parameter: [$1]"

if [ "$1" != "build" ] && [ "$1" != "build_and_push" ]; then
    echo "failed: invalid parameter"
    exit 1
fi

The next part validate the all variable has been passed.

# Set default values if they are not provided
HEPHA_REGISTRY="${HEPHA_REGISTRY:=git.thinkaboutit.tech}"
HEPHA_ORGNAME="${HEPHA_ORGNAME:=pandora}"

# Check for mandatory environment variables
: "${HEPHA_PACKAGE_NAME:?missing environment variable}"
: "${HEPHA_PACKAGE_VERSION:?missing environment variable}"
: "${HEPHA_GIT_USER:?missing environment variable}"
: "${HEPHA_GIT_PAT:?missing environment variable}"
: "${HEPHA_GIT_LINK:?missing environment variable}"

I verify that the version number that has been passed is compatible with semver. So something like 1.2.3.

# Verify semantical versioning
if [[ ! "$HEPHA_PACKAGE_VERSION" =~ ^[0-9]+(\.[0-9]+){2}$ ]]; then
    echo "HEPHA_PACKAGE_VERSION expects version in semver format, e.g.: 1.2.3"
    exit 1
fi

It depends that it is a build or build_and_push action further variables must present.

if [[ "$1" == "build_and_push" ]]; then
    HEPHA_DEPLOY_REGISTRY="${HEPHA_DEPLOY_REGISTRY:=git.thinkaboutit.tech}"
    : "${HEPHA_DEPLOY_PAT:?missing environment variable}"
    : "${HEPHA_DEPLOY_USER:?missing environment variable}"
fi

if [[ "$1" == "build" ]]; then
    : "${HEPHA_EXPORT_PATH:?missing environment variable}"
fi

Run the action

Next part is git clone the specified tag with a v prefix. A file, called upload.sh must present in the target repository. This script get a file name as parameter and must compress everything into that file. See below for an example.

git_tag="v${HEPHA_PACKAGE_VERSION}"
echo "clone ${git_tag} from $HEPHA_GIT_LINK repository..."
git clone \
    --depth 1 \
    --branch $git_tag \
    https://${HEPHA_GIT_USER}:${HEPHA_GIT_PAT}@${HEPHA_GIT_LINK} work
cd work

file_name=${HEPHA_PACKAGE_NAME}_${HEPHA_PACKAGE_VERSION}.tar.gz
./upload.sh $file_name

The final part depends on which kind of action is running.

# If this is just build, then export package
if [[ "$1" == "build" ]]; then
    archive_name="$HEPHA_EXPORT_PATH/${file_name}"
    cp $file_name $archive_name
    ls -l $archive_name
    exit 0
fi

# If this is build and push then upload to generic registry
curl --user $HEPHA_DEPLOY_USER:$HEPHA_DEPLOY_PAT \
    --upload-file ./$file_name \
    https://${HEPHA_REGISTRY}/api/packages/${HEPHA_ORGNAME}/generic/${HEPHA_PACKAGE_NAME}/${HEPHA_PACKAGE_VERSION}/${file_name}

Build the container

If everything at place, container can be build with command like:

podman build -t git.thinkaboutit.tech/pandora/generic-builder .

Let’s try it

I mentioned a repository above, called shell-config@v1.0.0. I will run the container that is built above.

podman run --rm --dns 1.1.1.1 \
    --env HEPHA_PACKAGE_NAME=shell-config \
    --env HEPHA_PACKAGE_VERSION=1.0.0 \
    --env HEPHA_GIT_USER=ci-user \
    --env HEPHA_GIT_LINK=git.thinkaboutit.tech/pandora/shell-config \
    --secret hepha-git-token,type=env,target=HEPHA_GIT_PATH \
    --env HEPHA_DEPLOY_USER=ci-user \
    --secret hepha-deploy-pat,type=env,target=HEPHA_DEPLOY_PAT \
    git.thinkaboutit.tech/pandora/generic-builder

Summary

So, why am I doing it? I like tinkering with things, and I know that there are a lot of open source solution for thing like this, but it is very different if I make it to happen. I learn a lot and I am planning to make a platform for myself (mostly for fun and learning). Builder image like this are some pillar of my future CI/CD solution that I am doing in the background.

There is a lot of extra feature that can be handy, like adding extra tooling option for builder script, more flexible approach. By time, they will also happen of course.

What is a generic registry?#

The plan#

Let’s make it#

Build the image#

Builder script#

Validate the input data#

Run the action#

Build the container#

Let’s try it#

Summary#