CLI tool binary, no deps & Generate manifest from existing config.yaml

[adnanrahic]

CLI tool binary, no deps & Generate manifest from existing config.yaml

Context

• Checked out from feature/config-to-manifest. Includes all code from feat: generate manifest from existing collector config.yaml #49. Merging this will close both [Feature Request] Build distro with an existing config.yaml file #33 and [Feature Request] Package otel-distro-builder as installable CLI (no Docker required) #51 . Deleting feat: generate manifest from existing collector config.yaml #49 in favor of this.

Summary

Implements the "Package Python app as CLI" plan: the builder is now an installable CLI (otel-distro-builder) that runs on the host with no Docker required. For full distribution builds, Go is the only required dependency; OCB is downloaded by the CLI. Manifest generation (--from-config, --generate-only) has no runtime dependencies when using the standalone binary or pip install.

Changes

Packaging & CLI

• pyproject.toml at repo root: installable package otel-distro-builder with console_scripts entry, dependencies from builder/requirements.txt, and package data (templates, versions.yaml, components.yaml, bindplane_components.yaml).
• Package-relative paths: New builder/src/resources.py uses importlib.resources and sys._MEIPASS (PyInstaller) so templates and YAML resolve correctly when installed or frozen. All references in version.py, main.py, build.py, component_registry.py, manifest_generator.py use these helpers.
• Host-first build: Removed hardcoded BUILD_DIR = "/build". Build and artifact directories are host paths (e.g. --artifacts ./artifacts); BuildContext takes a build_dir argument. Default artifacts dir is ./artifacts (cwd-based).

Standalone binary & release

• PyInstaller: otel-distro-builder.spec produces a one-file binary; make build-cli builds it locally. Optional dependency: pip install pyinstaller.
• CI: .github/workflows/build-cli-binaries.yml on tag v* builds binaries for linux/amd64, linux/arm64 (ubuntu-24.04-arm), and darwin/arm64, runs in-job tests and a no-Python job to validate the binary, then uploads tarballs to the release.
• Release: release.yml builds the Python package and attaches dist/* to the release; binary workflow attaches platform tarballs.

Homebrew & docs

• Formula: Formula/otel-distro-builder.rb with version and OS/arch-specific URLs (sha256 placeholders; update on first release).
• README: Install section (pip, Homebrew, standalone binary), dependencies note (Go only for full builds; none for generate-only), and updated Prerequisites.

Scripts & tests

• Scripts: generate_manifest.sh and run_local_build.sh prefer otel-distro-builder on PATH, then python -m builder.src.main, with host paths. Docker remains optional.
• CI: Base tests install the package and run otel-distro-builder --help and a generate-only smoke test.
• Make: script-test, build-cli, test-standalone-binary (validates binary with no Python at runtime).

Config-to-manifest (from existing branch)

• Config-to-manifest support: --from-config, --output-manifest, --generate-only, --no-bindplane, etc., with components.yaml / bindplane_components.yaml, and scripts/docs aligned.

Testing

• make unit-test — 108 unit tests passed
• make script-test — script smoke tests (generate_manifest, run_local_build, build_from_config) passed
• make test-standalone-binary — run after make build-cli to confirm the binary works with no Python at runtime
• CI: base-tests (unit + install package + CLI smoke), build-cli-binaries (build + per-platform test + no-Python test)

Notes

• ARM runners: ubuntu-24.04-arm is used for linux/arm64. If your GitHub plan doesn't include ARM runners, remove that matrix entry from build-cli-binaries.yml.
• Homebrew: Update Formula/otel-distro-builder.rb with real sha256 values after the first release that publishes binary tarballs.
• Pylint: Minor style warnings (e.g. global-statement in resources, import-outside-toplevel in main); score 9.94/10. Can be cleaned in a follow-up if desired.

Checklist

• Unit tests pass
• Script smoke tests pass
• README and docs updated (Install, Dependencies, Prerequisites)
• CI: package install + CLI smoke in base-tests
• CI: build-cli-binaries (multi-platform + standalone binary test)
• Go documented as the only dependency for full builds; OCB downloaded by CLI

Made with Cursor

• In general, to fix this class of problem you add an explicit permissions block either at the workflow root (applies to all jobs) or per job, giving only the scopes and access levels needed (e.g., contents: read). Jobs that need broader permissions (like publishing a release) can override with their own permissions blocks.
• For this workflow, the upload-to-release job already has permissions: contents: write, which is appropriate. The other jobs (build-binary and test-standalone-no-python) only need to read the repository contents and use artifacts; they do not push commits or modify issues, so they can safely run with contents: read. The least intrusive and clearest fix is to add a workflow-level permissions: block setting contents: read. This will apply to all jobs by default, while the existing permissions block in upload-to-release will override it for that job, preserving its necessary write access.
• Concretely, in .github/workflows/build-cli-binaries.yml, insert a permissions: section near the top of the file, after the name: declaration and before on:. No imports or additional methods are needed; this is purely a YAML configuration change for the workflow.

To fix the problem, explicitly declare a minimal permissions block for the test-standalone-no-python job so it does not inherit broader default permissions. This job only needs to download an artifact and read repository contents at most, so contents: read is sufficient.

Concretely, in .github/workflows/build-cli-binaries.yml, under the test-standalone-no-python job (around line 91), add a permissions: section at the same indentation level as runs-on and needs. The block should specify contents: read. No additional imports or methods are needed; this is a pure YAML configuration change and does not alter existing functional behavior of the workflow.