-
Notifications
You must be signed in to change notification settings - Fork 7.2k
feat(build): wanda ray image builds #59936
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| name: "ray-extra-py$PYTHON_VERSION-cpu$ARCH_SUFFIX" | ||
| disable_caching: true | ||
| froms: | ||
| - "cr.ray.io/rayproject/ray-py$PYTHON_VERSION-cpu-base-extra$ARCH_SUFFIX" # CPU base-extra image | ||
| - "cr.ray.io/rayproject/ray-wheel-py$PYTHON_VERSION$ARCH_SUFFIX" # Ray wheel | ||
| dockerfile: ci/docker/ray-image.Dockerfile | ||
| build_args: | ||
| - PYTHON_VERSION | ||
| - ARCH_SUFFIX | ||
| - IMAGE_TYPE=ray | ||
| - PLATFORM=cpu | ||
| - BASE_VARIANT=base-extra | ||
| - RAY_COMMIT=$BUILDKITE_COMMIT | ||
| - RAY_VERSION | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| name: "ray-extra-py$PYTHON_VERSION-cu$CUDA_VERSION$ARCH_SUFFIX" | ||
| disable_caching: true | ||
| froms: | ||
| - "cr.ray.io/rayproject/ray-py$PYTHON_VERSION-cu$CUDA_VERSION-base-extra$ARCH_SUFFIX" # CUDA base-extra image | ||
| - "cr.ray.io/rayproject/ray-wheel-py$PYTHON_VERSION$ARCH_SUFFIX" # Ray wheel | ||
| dockerfile: ci/docker/ray-image.Dockerfile | ||
| build_args: | ||
| - PYTHON_VERSION | ||
| - ARCH_SUFFIX | ||
| - IMAGE_TYPE=ray | ||
| - PLATFORM=cu$CUDA_VERSION | ||
| - BASE_VARIANT=base-extra | ||
| - RAY_COMMIT=$BUILDKITE_COMMIT | ||
| - RAY_VERSION |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| name: "ray-py$PYTHON_VERSION-cpu$ARCH_SUFFIX" | ||
| disable_caching: true | ||
| froms: | ||
| - "cr.ray.io/rayproject/ray-py$PYTHON_VERSION-cpu-base$ARCH_SUFFIX" # CPU base image with Python + deps | ||
| - "cr.ray.io/rayproject/ray-wheel-py$PYTHON_VERSION$ARCH_SUFFIX" # Ray wheel | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wanda froms order reversed from Dockerfile FROM statementsHigh Severity The Additional Locations (2) |
||
| dockerfile: ci/docker/ray-image.Dockerfile | ||
| build_args: | ||
| - PYTHON_VERSION | ||
| - ARCH_SUFFIX | ||
| - PLATFORM=cpu | ||
| - RAY_COMMIT=$BUILDKITE_COMMIT | ||
| - RAY_VERSION | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| name: "ray-py$PYTHON_VERSION-cu$CUDA_VERSION$ARCH_SUFFIX" | ||
| disable_caching: true | ||
| froms: | ||
| - "cr.ray.io/rayproject/ray-py$PYTHON_VERSION-cu$CUDA_VERSION-base$ARCH_SUFFIX" # CUDA base image | ||
| - "cr.ray.io/rayproject/ray-wheel-py$PYTHON_VERSION$ARCH_SUFFIX" # Ray wheel | ||
| dockerfile: ci/docker/ray-image.Dockerfile | ||
| build_args: | ||
| - PYTHON_VERSION | ||
| - ARCH_SUFFIX | ||
| - PLATFORM=cu$CUDA_VERSION | ||
| - RAY_COMMIT=$BUILDKITE_COMMIT | ||
| - RAY_VERSION |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # syntax=docker/dockerfile:1.3-labs | ||
| # | ||
| # Ray Image Builder | ||
| # ============================== | ||
| # Installs the Ray wheel into a base image (CPU or CUDA), includes | ||
| # pip freeze output for reproducibility. | ||
| # | ||
| ARG PYTHON_VERSION=3.10 | ||
| ARG PLATFORM=cpu | ||
| ARG ARCH_SUFFIX= | ||
| ARG IMAGE_TYPE=ray | ||
| ARG BASE_VARIANT=base | ||
| ARG BASE_IMAGE=cr.ray.io/rayproject/${IMAGE_TYPE}-py${PYTHON_VERSION}-${PLATFORM}-${BASE_VARIANT}${ARCH_SUFFIX} | ||
| ARG RAY_WHEEL_IMAGE=cr.ray.io/rayproject/ray-wheel-py${PYTHON_VERSION}${ARCH_SUFFIX} | ||
|
|
||
| FROM ${RAY_WHEEL_IMAGE} AS wheel-source | ||
| FROM ${BASE_IMAGE} | ||
|
|
||
| ARG RAY_COMMIT=unknown-commit | ||
| ARG RAY_VERSION=3.0.0.dev0 | ||
|
|
||
| LABEL io.ray.ray-commit="${RAY_COMMIT}" | ||
| LABEL io.ray.ray-version="${RAY_VERSION}" | ||
|
|
||
| COPY --from=wheel-source /*.whl /home/ray/ | ||
|
|
||
| # Install Ray wheel with all extras | ||
| # Uses requirements_compiled.txt from base image (already at /home/ray/) | ||
| RUN <<EOF | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| WHEEL_FILES=(/home/ray/ray-*.whl) | ||
| if [[ ${#WHEEL_FILES[@]} -ne 1 ]]; then | ||
| echo "Error: Expected 1 ray wheel file, but found ${#WHEEL_FILES[@]} in /home/ray/." >&2 | ||
| ls -l /home/ray/*.whl >&2 | ||
| exit 1 | ||
| fi | ||
andrew-anyscale marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| WHEEL_FILE="${WHEEL_FILES[0]}" | ||
|
|
||
| echo "Installing wheel: $WHEEL_FILE" | ||
|
|
||
| $HOME/anaconda3/bin/pip --no-cache-dir install \ | ||
| -c /home/ray/requirements_compiled.txt \ | ||
| "${WHEEL_FILE}[all]" | ||
|
|
||
| $HOME/anaconda3/bin/pip freeze > /home/ray/pip-freeze.txt | ||
|
|
||
| echo "Ray version: $($HOME/anaconda3/bin/python -c 'import ray; print(ray.__version__)')" | ||
| EOF | ||
|
|
||
| CMD ["/bin/bash"] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| name: "ray-llm-extra-py$PYTHON_VERSION-cu$CUDA_VERSION$ARCH_SUFFIX" | ||
| disable_caching: true | ||
| froms: | ||
| - "cr.ray.io/rayproject/ray-llm-py$PYTHON_VERSION-cu$CUDA_VERSION-base-extra$ARCH_SUFFIX" # LLM base-extra image | ||
| - "cr.ray.io/rayproject/ray-wheel-py$PYTHON_VERSION$ARCH_SUFFIX" # Ray wheel | ||
| dockerfile: ci/docker/ray-image.Dockerfile | ||
| build_args: | ||
| - PYTHON_VERSION | ||
| - ARCH_SUFFIX | ||
| - IMAGE_TYPE=ray-llm | ||
| - PLATFORM=cu$CUDA_VERSION | ||
| - BASE_VARIANT=base-extra | ||
| - RAY_COMMIT=$BUILDKITE_COMMIT | ||
| - RAY_VERSION |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| name: "ray-llm-py$PYTHON_VERSION-cu$CUDA_VERSION$ARCH_SUFFIX" | ||
| disable_caching: true | ||
| froms: | ||
| - "cr.ray.io/rayproject/ray-llm-py$PYTHON_VERSION-cu$CUDA_VERSION-base$ARCH_SUFFIX" # LLM base image | ||
| - "cr.ray.io/rayproject/ray-wheel-py$PYTHON_VERSION$ARCH_SUFFIX" # Ray wheel | ||
| dockerfile: ci/docker/ray-image.Dockerfile | ||
| build_args: | ||
| - PYTHON_VERSION | ||
| - ARCH_SUFFIX | ||
| - IMAGE_TYPE=ray-llm | ||
| - PLATFORM=cu$CUDA_VERSION | ||
| - RAY_COMMIT=$BUILDKITE_COMMIT | ||
| - RAY_VERSION |
Uh oh!
There was an error while loading. Please reload this page.