Release 25.03 (G3v7)

Author: accetto

CHANGELOG.md

@@ -6,6 +6,36 @@
 
 ***
 
+### Release 25.03 (G3v7)
+
+This is the first `G3v7` release, bringing an improved building pipeline.
+
+The helper script `ci-builder.sh` can build final images significantly faster, because the temporary helper images are used as external caches.
+
+Internally, the helper image is built by the `pre_build` hook script and then used by the `build` hook script.
+
+The helper image is now deleted by the `build` hook script and not the `pre_build` hook script as before.
+
+The `Dockerfiles` got a new metadata label `any.accetto.built-by="docker"`.
+
+#### Remarks
+
+If you would build a final image without building also the helper image (e.g. by executing `builder.sh latest build`), then there could be an error message about trying to remove the non-existing helper image.
+You can safely ignore the message.
+
+For example:
+
+```shell
+### The next line would build the helper image, but it was not executed.
+#./build.sh latest pre_build
+
+./build.sh latest build
+
+### then somewhere near the end of the log
+Removing helper image
+Error response from daemon: No such image: accetto/devops-headless-coding-g3_nvm-helper:latest
+```
+
 ### Release 25.01
 
 This is a maintenance release.

README.md

@@ -2,7 +2,7 @@
 
 ## Project `accetto/headless-coding-g3`
 
-Version: G3v6
+Version: G3v7
 
 ***
 
@@ -26,8 +26,10 @@ Version: G3v6
     - [Building images](#building-images)
     - [Image generations](#image-generations)
     - [Project versions](#project-versions)
+      - [Previous versions](#previous-versions)
     - [Project goals](#project-goals)
     - [Project features](#project-features)
+    - [How to fork](#how-to-fork)
     - [Getting help](#getting-help)
     - [Credits](#credits)
 
@@ -95,16 +97,26 @@ The **first generation** (G1) contains the GitHub repository [accetto/ubuntu-vnc
 
 ### Project versions
 
-This file describes the **sixth version** (G3v6) of the project.
+This file describes the **seventh version** (G3v7) of the project.
+
+This version brings an improved building pipeline.
+
+The helper script `ci-builder.sh` can build final images significantly faster, because the temporary helper images are used as external caches.
+
+Internally, the helper image is built by the `pre_build` hook script and then used by the `build` hook script.
+
+The helper image is now removed by the `build` hook script and not the `pre_build` hook script.
 
 However, also this version keeps evolving.
 Please check the [CHANGELOG][this-changelog] for more information about the changes.
 
+#### Previous versions
+
 The previous versions are still available in this **GitHub** repository as the branches named as `archived-generation-g3v{d}`.
 
 *Remark*: The version number `G3v4` has been skipped, to align the numbering with the **sibling project** [accetto/ubuntu-vnc-xfce-g3][accetto-github-ubuntu-vnc-xfce-g3].
 
-The main purpose of the version `G3v6` is to keep the project and the images uniform with the ones from the sibling `Ubuntu` projects.
+The main purpose of the version `G3v6` was to keep the project and the images uniform with the ones from the sibling `Ubuntu` projects.
 
 The version `G3v5` has brought only one significant change comparing to the previous version `G3v3` and it also introduces the [portable Visual Studio Code][vscode-portable] installation.
 
@@ -154,6 +166,10 @@ Please refer to the [sibling project][accetto-github-ubuntu-vnc-xfce-g3_project-
 
 Please refer to the [sibling project][accetto-github-ubuntu-vnc-xfce-g3_project-features] to learn more about the project features.
 
+### How to fork
+
+If you want to fork this project, then please check the page [How to fork this repository][sibling-wiki-how-to-fork] in the sibling [Wiki][sibling-wiki].
+
 ### Getting help
 
 If you have found a problem or you just have a question, please check the [User guide][this-user-guide], [Issues][this-issues] and the [sibling Wiki][sibling-wiki] first.
@@ -220,6 +236,8 @@ Credit goes to all the countless people and companies, who contribute to open so
 
 [sibling-wiki]: https://github.com/accetto/ubuntu-vnc-xfce-g3/wiki
 
+[sibling-wiki-how-to-fork]: https://github.com/accetto/ubuntu-vnc-xfce-g3/wiki/How-to-fork
+
 [sibling-discussions]: https://github.com/accetto/ubuntu-vnc-xfce-g3/discussions
 
 [accetto-github-ubuntu-vnc-xfce-g3_project-versions]: https://github.com/accetto/ubuntu-vnc-xfce-g3#project-versions

docker/Dockerfile.xfce.nodejs

@@ -350,7 +350,7 @@ RUN \
     NODEJS_PATH="/usr/local" \
     &&  if [ ! -s "/tmp/g3-cache/nodejs/node-v${NODEJS_VERSION}-${NODEJS_DISTRO}.tar.xz" ] ; then \
             wget --show-progress --progress=bar:force:noscroll \
-                -q "https://nodejs.org/dist/${NODEJS_VERSION}/node-v${NODEJS_VERSION}-${NODEJS_DISTRO}.tar.xz" \
+                -q "https://nodejs.org/dist/v${NODEJS_VERSION}/node-v${NODEJS_VERSION}-${NODEJS_DISTRO}.tar.xz" \
                 -P "/tmp/g3-cache/nodejs" ; \
         fi \
     &&  tar xJf "/tmp/g3-cache/nodejs/node-v${NODEJS_VERSION}-${NODEJS_DISTRO}.tar.xz" --strip 1 -C "${NODEJS_PATH}" \
@@ -453,6 +453,7 @@ RUN \
     &&  echo "${HEADLESS_USER_NAME}:$(cat ${STARTUPDIR}/.initial_sudo_password)" | chpasswd \
     &&  ln -s "${HOME}"/readme.md "${HOME}"/Desktop/README \
     &&  envsubst <"${HOME}"/Desktop/versionsticker.desktop >/tmp/esub.tmp && mv -f /tmp/esub.tmp "${HOME}"/Desktop/versionsticker.desktop \
+    &&  chmod +x "${STARTUPDIR}"/set_user_permissions.sh \
     && "${STARTUPDIR}"/set_user_permissions.sh "${STARTUPDIR}" "${HOME}"
 
 USER "${HEADLESS_USER_ID}"
@@ -489,4 +490,5 @@ LABEL \
     org.label-schema.vcs-ref="${ARG_VCS_REF}"
 
 LABEL \
-    any.accetto.version-sticker="${ARG_VERSION_STICKER}"
+    any.accetto.version-sticker="${ARG_VERSION_STICKER}" \
+    any.accetto.built-by="docker"

docker/Dockerfile.xfce.nvm

@@ -435,6 +435,7 @@ RUN \
     &&  echo "${HEADLESS_USER_NAME}:$(cat ${STARTUPDIR}/.initial_sudo_password)" | chpasswd \
     &&  ln -s "${HOME}"/readme.md "${HOME}"/Desktop/README \
     &&  envsubst <"${HOME}"/Desktop/versionsticker.desktop >/tmp/esub.tmp && mv -f /tmp/esub.tmp "${HOME}"/Desktop/versionsticker.desktop \
+    &&  chmod +x "${STARTUPDIR}"/set_user_permissions.sh \
     && "${STARTUPDIR}"/set_user_permissions.sh "${STARTUPDIR}" "${HOME}"
 
 USER "${HEADLESS_USER_ID}"
@@ -471,4 +472,5 @@ LABEL \
     org.label-schema.vcs-ref="${ARG_VCS_REF}"
 
 LABEL \
-    any.accetto.version-sticker="${ARG_VERSION_STICKER}"
+    any.accetto.version-sticker="${ARG_VERSION_STICKER}" \
+    any.accetto.built-by="docker"

docker/Dockerfile.xfce.postman

@@ -401,6 +401,7 @@ RUN \
     &&  echo "${HEADLESS_USER_NAME}:$(cat ${STARTUPDIR}/.initial_sudo_password)" | chpasswd \
     &&  ln -s "${HOME}"/readme.md "${HOME}"/Desktop/README \
     &&  envsubst <"${HOME}"/Desktop/versionsticker.desktop >/tmp/esub.tmp && mv -f /tmp/esub.tmp "${HOME}"/Desktop/versionsticker.desktop \
+    &&  chmod +x "${STARTUPDIR}"/set_user_permissions.sh \
     && "${STARTUPDIR}"/set_user_permissions.sh "${STARTUPDIR}" "${HOME}"
 
 USER "${HEADLESS_USER_ID}"
@@ -437,4 +438,5 @@ LABEL \
     org.label-schema.vcs-ref="${ARG_VCS_REF}"
 
 LABEL \
-    any.accetto.version-sticker="${ARG_VERSION_STICKER}"
+    any.accetto.version-sticker="${ARG_VERSION_STICKER}" \
+    any.accetto.built-by="docker"

docker/Dockerfile.xfce.python

@@ -427,6 +427,7 @@ RUN \
     &&  echo "${HEADLESS_USER_NAME}:$(cat ${STARTUPDIR}/.initial_sudo_password)" | chpasswd \
     &&  ln -s "${HOME}"/readme.md "${HOME}"/Desktop/README \
     &&  envsubst <"${HOME}"/Desktop/versionsticker.desktop >/tmp/esub.tmp && mv -f /tmp/esub.tmp "${HOME}"/Desktop/versionsticker.desktop \
+    &&  chmod +x "${STARTUPDIR}"/set_user_permissions.sh \
     && "${STARTUPDIR}"/set_user_permissions.sh "${STARTUPDIR}" "${HOME}"
 
 USER "${HEADLESS_USER_ID}"
@@ -463,4 +464,5 @@ LABEL \
     org.label-schema.vcs-ref="${ARG_VCS_REF}"
 
 LABEL \
-    any.accetto.version-sticker="${ARG_VERSION_STICKER}"
+    any.accetto.version-sticker="${ARG_VERSION_STICKER}" \
+    any.accetto.built-by="docker"

docker/Dockerfile.xfce.vscode

@@ -400,6 +400,7 @@ RUN \
     &&  echo "${HEADLESS_USER_NAME}:$(cat ${STARTUPDIR}/.initial_sudo_password)" | chpasswd \
     &&  ln -s "${HOME}"/readme.md "${HOME}"/Desktop/README \
     &&  envsubst <"${HOME}"/Desktop/versionsticker.desktop >/tmp/esub.tmp && mv -f /tmp/esub.tmp "${HOME}"/Desktop/versionsticker.desktop \
+    &&  chmod +x "${STARTUPDIR}"/set_user_permissions.sh \
     && "${STARTUPDIR}"/set_user_permissions.sh "${STARTUPDIR}" "${HOME}"
 
 USER "${HEADLESS_USER_ID}"
@@ -436,4 +437,5 @@ LABEL \
     org.label-schema.vcs-ref="${ARG_VCS_REF}"
 
 LABEL \
-    any.accetto.version-sticker="${ARG_VERSION_STICKER}"
+    any.accetto.version-sticker="${ARG_VERSION_STICKER}" \
+    any.accetto.built-by="docker"

docker/hooks/build

@@ -14,6 +14,9 @@ main() {
     local -i exit_code=0
     local target_stage
 
+    ### helper image is also used as a cache and therefore it must not use a tag
+    local helper_image="${DOCKER_REPO}"_"${DOCKER_TAG}${helper_suffix}"
+
     if [[ -f "${_build_context}/${_scrap_demand_stop_building}" ]] ; then
 
         echo "Skipping build on demand."
@@ -23,7 +26,9 @@ main() {
     ### refresh local cache for the case the 'pre_build' was not run
     "${_mydir}/${cache_script}" "${_branch}" "${_blend}"
     exit_code=$?
-    if [[ ${exit_code} -ne 0 ]] ; then die "Script '${cache_script}' failed with code ${exit_code}." ${exit_code} ; fi
+    if [[ ${exit_code} -ne 0 ]] ; then
+        die "Script '${cache_script}' failed with code ${exit_code}." ${exit_code} 
+    fi
 
     ### get current version sticker
     if [[ -f "${_build_context}/${_scrap_version_sticker_current}" ]] ; then
@@ -73,6 +78,7 @@ main() {
             ### no quotes around ${params}!
             set -x
             docker build ${params} \
+                --cache-from=${helper_image} \
                 -f "${DOCKERFILE_PATH}" \
                     --build-arg BASEIMAGE="${BASEIMAGE}" \
                     --build-arg BASETAG="${BASETAG}" \
@@ -112,7 +118,13 @@ main() {
             exit_code=$?
             set +x
 
-            if [[ ${exit_code} -ne 0 ]] ; then die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code} ; fi
+            if [[ ${exit_code} -ne 0 ]] ; then
+                die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code}
+            fi
+
+            ### delete the helper image in any case
+            echo; echo "Removing helper image"
+            ${CONTAINER_MANAGER:-docker} rmi "${helper_image}"
             ;;
 
         ### -------------------------
@@ -124,6 +136,7 @@ main() {
             ### no quotes around ${params}!
             set -x
             docker build ${params} \
+                --cache-from=${helper_image} \
                 -f "${DOCKERFILE_PATH}" \
                     --build-arg BASEIMAGE="${BASEIMAGE}" \
                     --build-arg BASETAG="${BASETAG}" \
@@ -162,7 +175,13 @@ main() {
             exit_code=$?
             set +x
 
-            if [[ ${exit_code} -ne 0 ]] ; then die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code} ; fi
+            if [[ ${exit_code} -ne 0 ]] ; then
+                die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code}
+            fi
+
+            ### delete the helper image in any case
+            echo; echo "Removing helper image"
+            ${CONTAINER_MANAGER:-docker} rmi "${helper_image}"
             ;;
 
         ### --------------------------
@@ -174,6 +193,7 @@ main() {
             ### no quotes around ${params}!
             set -x
             docker build ${params} \
+                --cache-from=${helper_image} \
                 -f "${DOCKERFILE_PATH}" \
                     --build-arg BASEIMAGE="${BASEIMAGE}" \
                     --build-arg BASETAG="${BASETAG}" \
@@ -208,7 +228,13 @@ main() {
             exit_code=$?
             set +x
 
-            if [[ ${exit_code} -ne 0 ]] ; then die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code} ; fi
+            if [[ ${exit_code} -ne 0 ]] ; then
+                die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code}
+            fi
+
+            ### delete the helper image in any case
+            echo; echo "Removing helper image"
+            ${CONTAINER_MANAGER:-docker} rmi "${helper_image}"
             ;;
 
         ### -------------------------
@@ -220,6 +246,7 @@ main() {
             ### no quotes around ${params}!
             set -x
             docker build ${params} \
+                --cache-from=${helper_image} \
                 -f "${DOCKERFILE_PATH}" \
                     --build-arg BASEIMAGE="${BASEIMAGE}" \
                     --build-arg BASETAG="${BASETAG}" \
@@ -257,7 +284,13 @@ main() {
             exit_code=$?
             set +x
 
-            if [[ ${exit_code} -ne 0 ]] ; then die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code} ; fi
+            if [[ ${exit_code} -ne 0 ]] ; then
+                die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code}
+            fi
+
+            ### delete the helper image in any case
+            echo; echo "Removing helper image"
+            ${CONTAINER_MANAGER:-docker} rmi "${helper_image}"
             ;;
 
         ### -------------------------
@@ -269,6 +302,7 @@ main() {
             ### no quotes around ${params}!
             set -x
             docker build ${params} \
+                --cache-from=${helper_image} \
                 -f "${DOCKERFILE_PATH}" \
                     --build-arg BASEIMAGE="${BASEIMAGE}" \
                     --build-arg BASETAG="${BASETAG}" \
@@ -304,7 +338,13 @@ main() {
             exit_code=$?
             set +x
 
-            if [[ ${exit_code} -ne 0 ]] ; then die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code} ; fi
+            if [[ ${exit_code} -ne 0 ]] ; then
+                die "Failed to build image '${DOCKER_REPO}:${DOCKER_TAG}'" ${exit_code}
+            fi
+
+            ### delete the helper image in any case
+            echo; echo "Removing helper image"
+            ${CONTAINER_MANAGER:-docker} rmi "${helper_image}"
             ;;
 
         * )

docker/hooks/env.rc

@@ -61,6 +61,9 @@ declare _readme_context
 # VERSION_STICKER_PREFIX=${VERSION_STICKER_SUFFIX:-"BETA-"}
 # VERSION_STICKER_SUFFIX=${VERSION_STICKER_SUFFIX:-"-BETA"}
 
+### suffix added to the helper image name
+declare helper_suffix="-helper"
+
 ### be careful with moving this statement
 ### remove the first two command line arguments (<branch> <blend>)
 if [[ $# -ge 2 ]] ; then shift 2 ; fi