feat: manifest schema and generation #77
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Using CUE, we now provide a schema for manifest validation and generation based on GPU type. This can be utilised via the `nccl.nu` script that supports running a `sync` against a kubeconfig to create or verify GPU information from a remote cluster. You can also use `generate` with flags to get a dynamic Kubernetes manifest for running the nccl tests
There was a problem hiding this comment.
Pull request overview
This PR introduces CUE-based schema validation and manifest generation for NCCL tests. It adds a Nushell CLI script (nccl.nu) that can sync GPU configurations from a Kubernetes cluster and generate dynamic Kubernetes manifests for running NCCL tests based on GPU type and scale parameters.
Changes:
- Added CUE schema for GPU configuration validation
- Created Nushell CLI script for cluster synchronization and manifest generation
- Added GPU configuration files for multiple GPU types (A100, H100, A40, GB200, RTX P6000)
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| schema/gpu.cue | Defines the GPU configuration schema with validation constraints |
| utils/new_gpu.cue | Provides template for generating new GPU configurations from cluster data |
| gpus/*.cue | Configuration files for specific GPU types with hardware specifications |
| generate.cue | Main manifest generation logic with conditional feature flags |
| nccl.nu | Nushell CLI script for sync and generate commands |
| cue.mod/module.cue | CUE module definition with Kubernetes schema dependencies |
| README.md | Documentation for the new manifest generation feature |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Author
|
Example of a |
|
I have basically zero experience in this codebase, but is this the first time we're introducing both CUE and nushell? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using CUE, we now provide a schema for manifest validation and generation based on GPU type.
This can be utilised via the
nccl.nuscript that supports running asyncagainst a kubeconfig tocreate or verify GPU information from a remote
cluster.
You can also use
generatewith flags to get adynamic Kubernetes manifest for running the nccl tests