Skip to content

Comments

Build docker images through CI#207

Merged
jking-aus merged 17 commits intosigp:unstablefrom
antondlr:build-docker
Mar 25, 2025
Merged

Build docker images through CI#207
jking-aus merged 17 commits intosigp:unstablefrom
antondlr:build-docker

Conversation

@antondlr
Copy link
Member

Issue Addressed

https://github.com/sigp/devops-pm/issues/63

Additional Info

  • The anchor binary is now defined in the ENTRYPOINT, meaning we don't need to specify it anymore when running the container: docker run sigp/anchor --help vs. docker run sigp/anchor anchor --help.

  • ARM builds are disabled for now as they take [over three hours to complete]. (https://github.com/antondlr/anchor/actions/runs/14017403755/job/39244825764) .
    We can revisit this once we have our first ARM user :-)

  • Assumes the secrets DOCKER_USERNAME and DOCKER_PASSWORD are defined. Pulling this from vault is commented out (and untested).

@realbigsean
Copy link
Member

One more ask!

there is an interop1 branch that is going to be used for interop, can we also chuck in an auto build for sigp/anchor:interop1

- name: Dockerhub login
run: |
echo "${DOCKER_PASSWORD}" | docker login --username ${DOCKER_USERNAME} --password-stdin
- name: Create and push multiarch manifest
Copy link
Contributor

@magick93 magick93 Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure cross does this already - cross uses buildx internally.
You can try pulling the manifest from any of the images from https://hub.docker.com/repository/docker/magick93/anchor.

So I'm not sure if docker-multiarch-manifest is needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it is, cross does indeed publish with a manifest but we want the regular (no arch defined) image tag to contain both architectures eventually.
in this stripped down, x86_only version we can skip it indeed and just add the tag. but this works for multiarch, just needs some commenting out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah or do you mean you can push to the same tag and it will append instead of replace, if it's a different arch?
I'll play with it a bit, would be good to get rid of it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested it out; it overwrites the tag altogether.
so for multiarch there are two valid approaches:

  • build multiarch manifest manually (what we do here)
  • build all architectures in one task and push (with: platforms: linux/arm64,linux/amd64). this slows everything down even more though as both will be built on the same runner instance.

So yeah we can get rid of it, at a cost for actual multiarch builds.
I'm fine with both, do we aim for simplicity or speed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If "simplicity" means 3hr builds, i'd vote for speed, at least for now :)

Copy link
Contributor

@magick93 magick93 Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dknopik

If we consider purely from speed, for now, would the following be an improvement?

  • only build docker images on a tag
  • only run tests by default

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antondlr - It looks like I'm mistaken. The manifest is not implicitly appended. I've been experimenting on eleel using bake which is basically a wrapper around buildx and depending on which iteration on the matrix finishes last is what is written (overwritten) in the manifest. So I think explicitly updating or appending the manifest would be the reliable way to go.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is exactly what I found in testing, too.
also no appropriate config in docker/build-push-action to amend instead of replace :/
luckily it's a pretty fast task to execute!

@jking-aus
Copy link
Member

thanks for sorting this out so quick guys -- really appreciate it

@jking-aus jking-aus merged commit a89fda2 into sigp:unstable Mar 25, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants