Enable Docker CI by AshwinHIBM · Pull Request #517 · ppc64le-cloud/test-infra

AshwinHIBM · 2024-12-10T06:26:06Z

The purpose of each job:

periodic-config-docker: Check the configuration of the kernel for suitability to run Docker.
periodic-build-dev-image-docker: Build a minimal Docker development image.
postsubmit-unit-test-docker: Run unit tests against https://github.com/moby/moby
postsubmit-integration-test-docker: Run integration tests against https://github.com/moby/moby
postsubmit-integration-cli-docker: Run the end-to-end test suite from https://github.com/docker/cli

ppc64le-cloud-bot · 2024-12-10T06:26:16Z

Hi @AshwinHIBM. Thanks for your PR.

I'm waiting for a ppc64le-cloud member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

config/jobs/periodic/docker/periodic-ci-docker.yaml

Rajalakshmi-Girish · 2024-12-11T09:35:30Z

/ok-to-test

clnperez · 2025-01-03T20:33:07Z

@Rajalakshmi-Girish has some good comments/requests so I'll approve if she does. Thanks Raji!

mkumatag · 2025-03-27T06:21:20Z

@Rajalakshmi-Girish lets review this PR

Rajalakshmi-Girish · 2025-03-28T16:18:53Z

@Rajalakshmi-Girish lets review this PR

I shall review this in a day.

Rajalakshmi-Girish

@AshwinHIBM Let us schedule a call tomorrow. Please help me understand better.

Rajalakshmi-Girish · 2025-04-01T11:32:28Z

config/jobs/periodic/docker/periodic-ci-docker.yaml

+
+            echo "* Start prow-info-docker *"
+            chmod ug+x $PWD/upstream-master-ci/prow-info-docker.sh
+            $PWD/upstream-master-ci/prow-info-docker.sh


@AshwinHIBM You mentioned periodic-config-docker is to Check the configuration of the kernel for suitability to run Docker., where as line here https://github.com/ppc64le-cloud/docker-ce-build/blob/main/upstream-master-ci/prow-info-docker.sh#L14 seems to display a text Prow Job to run CI tests on the Docker packages, Seems some discrepancy.

I see DATE being directed to /home/prow/go/src/github.com/ppc64le-cloud/docker-ce-build/env/date.list at https://github.com/ppc64le-cloud/docker-ce-build/blob/main/upstream-master-ci/prow-info-docker.sh#L12
But there doesn't seem to be a file date.list in https://github.com/ppc64le-cloud/docker-ce-build/tree/main/env!
@AshwinHIBM Can we connect to understand better?

@AshwinHIBM You mentioned periodic-config-docker is to Check the configuration of the kernel for suitability to run Docker., where as line here https://github.com/ppc64le-cloud/docker-ce-build/blob/main/upstream-master-ci/prow-info-docker.sh#L14 seems to display a text Prow Job to run CI tests on the Docker packages, Seems some discrepancy.

The statement "Prow Job to run CI tests...." is just a generic statement used in all the jobs, but agreed, it can be more specific.

@AshwinHIBM You mentioned periodic-config-docker is to Check the configuration of the kernel for suitability to run Docker., where as line here https://github.com/ppc64le-cloud/docker-ce-build/blob/main/upstream-master-ci/prow-info-docker.sh#L14 seems to display a text Prow Job to run CI tests on the Docker packages, Seems some discrepancy.

ppc64le-cloud/docker-ce-build#271

config/jobs/periodic/docker/periodic-ci-docker.yaml

ppc64le-cloud-bot · 2025-04-01T11:52:58Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AshwinHIBM
Once this PR has been reviewed and has the lgtm label, please assign rajalakshmi-girish for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

config/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Rajalakshmi-Girish

Do all jobs postsubmit-unit-test-docker, postsubmit-integration-test-docker, postsubmit-integration-test-docker require dind image?

Also, here are my comments for your scripts at https://github.com/ppc64le-cloud/docker-ce-build/tree/main/upstream-master-ci - just sharing my thoughts!

Why the mention of adding a git commit at https://github.com/ppc64le-cloud/docker-ce-build/blob/main/setup-pj-trigger.sh#L20? I see only date info being added to folder under https://github.com/ppc64le-cloud/docker-ce-build/tree/prow-job-tracking/job
Why the intermediate https://github.com/ppc64le-cloud/docker-ce-build/blob/main/setup-pj-trigger.sh? Why not directly call https://github.com/ppc64le-cloud/docker-ce-build/blob/main/trigger-prow-job-from-git.sh from https://github.com/ppc64le-cloud/docker-ce-build/blob/main/upstream-master-ci/prow-info-docker.sh
How is moby dynamic here? https://github.com/ppc64le-cloud/docker-ce-build/blob/main/upstream-master-ci/build-dev-image.sh#L1 Where is it cloned?
Why two files prow-unit-test-docker.sh unit-tests.sh? Can't the common part from files of first kind be clubbed at one place?

Rajalakshmi-Girish · 2025-04-04T06:51:37Z

config/jobs/ppc64le-cloud/build-docker/postsubmit-ci-docker.yaml

+            #Set this to your dev branch for testing
+            WORK_BRANCH=main
+            git fetch https://github.com/${REPO_OWNER}/${REPO_NAME} ${WORK_BRANCH}
+            git checkout ${WORK_BRANCH}


@AshwinHIBM Will just git checkout main be not enough in place of above three commands in all jobs?

Changing https://github.com/ppc64le-cloud/test-infra/pull/517/files#diff-7cc77aecddebbf7c3478065e05b998c98df879793b4afa288afb1781bd4c34e8R2 to AshwinHIBM/docker-ce-build automatically sets REPO_OWNER to AshwinHIBM and REPO_NAME to docker-ce-build which lets me test changes to the CI scripts which are in my fork before pushing to main. Having the WORK_BRANCH parameter on L37 prevents writing it twice on the next 2 lines. git fetch is the only line when the repository is fetched from the internet (as we're not using git clone here). This is the style we use for the existing jobs too. If there's a better style to provide the above functionality, I can change to it.

Rajalakshmi-Girish · 2025-04-04T06:54:24Z

config/jobs/periodic/docker/periodic-ci-docker.yaml

+
+            echo "* Start prow-info-docker *"
+            chmod ug+x $PWD/upstream-master-ci/prow-info-docker.sh
+            $PWD/upstream-master-ci/prow-info-docker.sh


What Kernel settings are checked by periodic-config-docker job? is it required to check everyday?
Can we not verify it withe current underlying build cluster and take it for granted?

It runs https://github.com/moby/moby/blob/master/contrib/check-config.sh
checks cgroups, apparmor, etc. Are you suggesting we remove it from CI and only run it when something changes with the underlying cluster like the OS version? I'm fine with that idea (we probably don't need to run it daily) but is there a way to know something changed with the cluster? Or instead changing the frequency to once a week/month?

Rajalakshmi-Girish · 2025-04-04T07:01:26Z

config/prow/config.yaml

+      - error
+    channel: prow-job-notifications
+    report_template: 'Job {{.Spec.Job}} of type {{.Spec.Type}} ended with state {{.Status.State}}. <!subteam^S02N6DWBX0F> <{{.Status.URL}}|View logs>'



As discussed, shall we first try switching for scripts in https://github.com/ppc64le-cloud/test-infra/blob/master/config/jobs/ppc64le-cloud/build-docker/postsubmit-build-docker.yaml

AshwinHIBM · 2025-04-04T08:50:27Z

Squashing 7dece64 and 114c053.

ppc64le-cloud-bot requested review from Rajalakshmi-Girish and mkumatag December 10, 2024 06:26

ppc64le-cloud-bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 10, 2024

ppc64le-cloud-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 10, 2024

AshwinHIBM mentioned this pull request Dec 10, 2024

Add docker ci jobs scripts for test-infra repo ppc64le-cloud/docker-ce-build#244

Merged

Rajalakshmi-Girish reviewed Dec 11, 2024

View reviewed changes

config/jobs/periodic/docker/periodic-ci-docker.yaml Show resolved Hide resolved

Rajalakshmi-Girish requested changes Dec 11, 2024

View reviewed changes

Rajalakshmi-Girish reviewed Dec 11, 2024

View reviewed changes

config/jobs/periodic/docker/periodic-ci-docker.yaml Show resolved Hide resolved

Rajalakshmi-Girish requested changes Dec 11, 2024

View reviewed changes

config/jobs/periodic/docker/periodic-ci-docker.yaml Outdated Show resolved Hide resolved

Rajalakshmi-Girish added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Dec 11, 2024

AshwinHIBM force-pushed the enable-docker-ci branch from 6c2c9b3 to 02afba6 Compare December 16, 2024 12:51

ppc64le-cloud-bot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 16, 2024

AshwinHIBM force-pushed the enable-docker-ci branch from 02afba6 to af3b2ad Compare January 6, 2025 05:40

AshwinHIBM force-pushed the enable-docker-ci branch 4 times, most recently from 0e8af61 to 23a11d9 Compare January 28, 2025 17:45

AshwinHIBM force-pushed the enable-docker-ci branch from 23a11d9 to fb05c55 Compare February 4, 2025 10:35

AshwinHIBM marked this pull request as draft February 4, 2025 10:36

ppc64le-cloud-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 4, 2025

AshwinHIBM force-pushed the enable-docker-ci branch 2 times, most recently from b4056e0 to 114c053 Compare February 5, 2025 13:19

AshwinHIBM marked this pull request as ready for review March 24, 2025 12:32

ppc64le-cloud-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 24, 2025

ppc64le-cloud-bot requested a review from Rajalakshmi-Girish March 24, 2025 12:32

Rajalakshmi-Girish requested changes Apr 1, 2025

View reviewed changes

Rajalakshmi-Girish requested changes Apr 4, 2025

View reviewed changes

AshwinHIBM added 2 commits April 4, 2025 14:21

Enable Docker CI

dfbf6a9

Centralize reporter_config for CI jobs

70c06ed

AshwinHIBM force-pushed the enable-docker-ci branch from fcfd159 to 70c06ed Compare April 4, 2025 08:52

Conversation

AshwinHIBM commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ppc64le-cloud-bot commented Dec 10, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Rajalakshmi-Girish commented Dec 11, 2024

Uh oh!

clnperez commented Jan 3, 2025

Uh oh!

mkumatag commented Mar 27, 2025

Uh oh!

Rajalakshmi-Girish commented Mar 28, 2025

Uh oh!

Rajalakshmi-Girish left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ppc64le-cloud-bot commented Apr 1, 2025

Uh oh!

Rajalakshmi-Girish left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AshwinHIBM Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AshwinHIBM commented Apr 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

AshwinHIBM commented Dec 10, 2024 •

edited

Loading

Rajalakshmi-Girish left a comment •

edited

Loading

AshwinHIBM Apr 7, 2025 •

edited

Loading