filter pods by node name and improve oldest pod selection logic#37
Open
cybergeek2077 wants to merge 2 commits intoProject-HAMi:mainfrom
Open
filter pods by node name and improve oldest pod selection logic#37cybergeek2077 wants to merge 2 commits intoProject-HAMi:mainfrom
cybergeek2077 wants to merge 2 commits intoProject-HAMi:mainfrom
Conversation
Signed-off-by: cybergeek2077 <zhaoshen@buaa.edu.cn>
Contributor
Author
|
@archlitchi The build failed with an error saying there's no space left on the device, but I can't find a way to restart it. |
Member
|
hi, could you resolve this conflict? you can leave the rest to me |
Contributor
Author
|
I have resolved the conflict, also added filter the AssignedNodeAnnotations in my PR. This pull request includes changes to the Improvements to pod filtering and selection:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The device plugin allocates a device by selecting the oldest pod, but it did not filter pods by node. This caused a bug if different nodes' pods have the same time assigned by Volcano (e.g., a Gang scheduler). It may get a pod on another node and so assign another node's GPU.
This PR fixes that bug by filtering pods by node name and also improves the oldest pod selection logic by filtering pods that are pending and have the right Volcano annotations.
Hami implement reference: Project-HAMi/HAMi#340