Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
kernels/src/kernels/cli.py
Outdated
| delete_patterns=["benchmark*.py"], | ||
| commit_message="Benchmarks uploaded using `kernels`.", | ||
| allow_patterns=["benchmark*.py"], |
There was a problem hiding this comment.
small nit: delete_patterns are the same as allow_patterns, is this expected
There was a problem hiding this comment.
ah yea this is strange looking syntax but I believe its correct, since I think this is how to specify which files to overwrite/remove if they don't exist in the latest local copy of the dir.
There was a problem hiding this comment.
makes sense! thanks for clarifying
kernels/src/kernels/cli.py
Outdated
| repo_id = create_repo( | ||
| repo_id=args.repo_id, private=args.private, exist_ok=True | ||
| ).repo_id | ||
|
|
||
| if args.branch is not None: | ||
| create_branch(repo_id=repo_id, branch=args.branch, exist_ok=True) | ||
|
|
There was a problem hiding this comment.
With the versions PR merged, this needs a sync against the version handling.
drbh
force-pushed
the
benchmark-command-fixes
branch
from
a7d7242 to
e3ee2af
Compare
drbh
force-pushed
the
benchmark-command-fixes
branch
from
e3ee2af to
41dff23
Compare
|
updates the upload command now uses the version in two cases
the benchmark command now requires a branch or version to be specified. the following are valid commands
note that
|
| if benchmarks_only: | ||
| metadata = Metadata.load_from_build_toml(kernel_dir / "build.toml") | ||
| if metadata.version is None: | ||
| raise ValueError( | ||
| f"Cannot upload benchmarks only without a version specified in build.toml at: {(kernel_dir / 'build.toml').absolute()}" | ||
| ) | ||
|
|
||
| branch = f"v{metadata.version}" | ||
|
|
||
| upload_folder( | ||
| repo_id=repo_id, | ||
| folder_path=kernel_dir / "benchmarks", | ||
| revision=branch, | ||
| path_in_repo="benchmarks", | ||
| delete_patterns=["benchmark*.py"], | ||
| commit_message="Benchmarks uploaded using `kernels`.", | ||
| allow_patterns=["benchmark*.py"], | ||
| ) | ||
|
|
||
| print( | ||
| f"✅ Benchmarks upload successful. Find the kernel in: https://hf.co/{repo_id}" | ||
| ) | ||
| return # Exit if only benchmarks are to be uploaded |
There was a problem hiding this comment.
I think we should completely remove the benchmark_only case:
- It would only be useful in the case one also has built kernels, but only wants to upload the benchmark, which seems very rare. (And if no kernels were built, there is no additional cost, since existing variants will not be uploaded again).
- It relies on the existence of
build.toml, but this file may not be present (e.g. if someone has their own build infra). - I makes the upload logic quite a bit more complex.
In the very rare case where the first point happens, someone could use the huggingface CLI to upload.
|
closing in favor of #229 |
4 participants
This PR
get_kernel_sha_from_build_nameas the name parsing was not working with arbitrary kernels._synchronizethat had a typo from the previous changes--benchmarksand--benchmarks-onlyflag to theuploadsubcommand to upload the benchmark directory