-
Notifications
You must be signed in to change notification settings - Fork 1.1k
synchronize kubeflow workspaces manifests v2.0.0 alpha.1 #3430
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
christian-heusel
wants to merge
7
commits into
kubeflow:master
Choose a base branch
from
christian-heusel:synchronize-notebooks-v2-manifests-v2.0.0-alpha.0
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
a5d6458
Update kubeflow/notebooks manifests from v2.0.0-alpha.0
christian-heusel 6df53e3
chore: Add sidebar link for Workspaces
christian-heusel 48fd0d9
Update kubeflow/notebooks manifests from v2.0.0-alpha.1
christian-heusel 062f746
ci: Add integration test for Kubeflow Workspaces
christian-heusel b085210
chore: Drop the helm charts for notebooks-v1
christian-heusel 55b8e82
ci: Add pipeline integration test for Kubeflow Workspaces
christian-heusel 351a016
fix: Add clusterrole patch to fix auth issue
christian-heusel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| name: Test Workspaces Controller | ||
| on: | ||
| pull_request: | ||
| paths: | ||
| - tests/install_KinD_create_KinD_cluster_install_kustomize.sh | ||
| - .github/workflows/workspaces_controller_test.yaml | ||
| - applications/workspaces/** | ||
| - common/cert-manager/** | ||
| - common/oauth2-proxy/** | ||
| - common/istio*/** | ||
| - tests/istio* | ||
| - tests/cert_manager_install.sh | ||
| - tests/oauth2-proxy_install.sh | ||
| - tests/multi_tenancy_install.sh | ||
|
|
||
| permissions: | ||
| contents: read | ||
| actions: read | ||
|
|
||
| jobs: | ||
| workspaces_controller_test: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1 | ||
|
|
||
| - name: Install KinD, Create KinD cluster and Install kustomize | ||
| run: ./tests/install_KinD_create_KinD_cluster_install_kustomize.sh | ||
|
|
||
| - name: Create kubeflow namespace | ||
| run: kustomize build common/kubeflow-namespace/base | kubectl apply -f - | ||
|
|
||
| - name: Install cert-manager | ||
| run: ./tests/cert_manager_install.sh | ||
|
|
||
| - name: Install Istio | ||
| run: ./tests/istio-cni_install.sh | ||
|
|
||
| - name: Install oauth2-proxy | ||
| run: ./tests/oauth2-proxy_install.sh | ||
|
|
||
| - name: Install kubeflow-istio-resources | ||
| run: kustomize build common/istio/kubeflow-istio-resources/base | kubectl apply -f - | ||
|
|
||
| - name: Install KF Multi Tenancy | ||
| run: ./tests/multi_tenancy_install.sh | ||
|
|
||
| - name: Build and apply workspaces manifests | ||
| run: | | ||
| kustomize build applications/workspaces/overlays/istio | kubectl apply -f - | ||
| kubectl wait --for=condition=Ready pods -n cert-manager --all --timeout=120s | ||
| kubectl wait --for=condition=Ready pods \ | ||
| -n kubeflow-workspaces --all \ | ||
| --timeout=600s | ||
|
|
||
| - name: Check rollout status of workspaces deployments | ||
| run: | | ||
| kubectl rollout status deployment/workspaces-controller -n kubeflow-workspaces | ||
| kubectl rollout status deployment/workspaces-backend -n kubeflow-workspaces | ||
| kubectl rollout status deployment/workspaces-frontend -n kubeflow-workspaces | ||
|
|
||
| - name: Create KF Profile | ||
| run: kustomize build common/user-namespace/base | kubectl apply -f - | ||
|
|
||
| - name: Port forward | ||
| run: | | ||
| INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}') | ||
| nohup kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80 & | ||
| while ! curl localhost:8080; do echo waiting for port-forwarding; sleep 1; done; echo port-forwarding ready | ||
|
|
||
| - name: List workspaces over API with authorized SA Token | ||
| run: | | ||
| KF_PROFILE=kubeflow-user-example-com | ||
| TOKEN="$(kubectl -n ${KF_PROFILE} create token default-editor)" | ||
|
|
||
| STATUS_CODE=$(curl -v \ | ||
| --silent --output /dev/stderr --write-out "%{http_code}" \ | ||
| "localhost:8080/workspaces/api/v1/workspaces/${KF_PROFILE}" \ | ||
| -H "Authorization: Bearer ${TOKEN}") | ||
|
|
||
| if test "${STATUS_CODE}" -ne 200; then | ||
| echo "Error, this call should be authorized to list workspaces in namespace ${KF_PROFILE}." | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: List workspaces over API with unauthorized SA Token | ||
| run: | | ||
| KF_PROFILE=kubeflow-user-example-com | ||
| TOKEN="$(kubectl -n default create token default)" | ||
|
|
||
| STATUS_CODE=$(curl -v \ | ||
| --silent --output /dev/stderr --write-out "%{http_code}" \ | ||
| "localhost:8080/workspaces/api/v1/workspaces/${KF_PROFILE}" \ | ||
| -H "Authorization: Bearer ${TOKEN}") | ||
|
|
||
| if test "${STATUS_CODE}" -ne 403; then | ||
| echo "Error, this call should fail to list workspaces in namespace ${KF_PROFILE}." | ||
| exit 1 | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| name: Test Pipeline run from Workspace | ||
| on: | ||
| pull_request: | ||
| paths: | ||
| - tests/install_KinD_create_KinD_cluster_install_kustomize.sh | ||
| - .github/workflows/workspaces_pipeline_run_test.yaml | ||
| - applications/workspaces/** | ||
| - applications/pipeline/upstream/** | ||
| - tests/istio* | ||
| - common/cert-manager/** | ||
| - common/oauth2-proxy/** | ||
| - common/istio*/** | ||
| - common/kubeflow-namespace/** | ||
| - tests/cert_manager_install.sh | ||
| - tests/multi_tenancy_install.sh | ||
| - tests/pipelines_install.sh | ||
|
|
||
| permissions: | ||
| contents: read | ||
| actions: read | ||
|
|
||
| jobs: | ||
| workspaces_pipeline_run_test: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1 | ||
|
|
||
| - name: Install KinD, Create KinD cluster and Install kustomize | ||
| run: ./tests/install_KinD_create_KinD_cluster_install_kustomize.sh | ||
|
|
||
| - name: Create kubeflow namespace | ||
| run: kustomize build common/kubeflow-namespace/base | kubectl apply -f - | ||
|
|
||
| - name: Install Istio | ||
| run: ./tests/istio-cni_install.sh | ||
|
|
||
| - name: Install oauth2-proxy | ||
| run: ./tests/oauth2-proxy_install.sh | ||
|
|
||
| - name: Install cert-manager | ||
| run: ./tests/cert_manager_install.sh | ||
|
|
||
| - name: Install Kubeflow Istio Resources | ||
| run: kustomize build common/istio/kubeflow-istio-resources/base | kubectl apply -f - | ||
|
|
||
| - name: Install KF Multi Tenancy | ||
| run: ./tests/multi_tenancy_install.sh | ||
|
|
||
| - name: Install KF Pipelines | ||
| run: ./tests/pipelines_install.sh | ||
|
|
||
| - name: Build and apply workspaces manifests | ||
| run: | | ||
| kustomize build applications/workspaces/overlays/istio | kubectl apply -f - | ||
| kustomize build applications/dashboard/upstream/poddefaults-webhooks/overlays/cert-manager | kubectl apply -f - | ||
| kubectl wait --for=condition=Ready pods -n cert-manager --all --timeout=120s | ||
| kubectl wait --for=condition=Ready pods \ | ||
| -n kubeflow-workspaces --all \ | ||
| --timeout=600s | ||
|
|
||
| - name: Check rollout status of workspaces deployments | ||
| run: | | ||
| kubectl rollout status deployment/workspaces-controller -n kubeflow-workspaces | ||
| kubectl rollout status deployment/workspaces-backend -n kubeflow-workspaces | ||
| kubectl rollout status deployment/workspaces-frontend -n kubeflow-workspaces | ||
|
|
||
| - name: Create KF Profile | ||
| run: ./tests/kubeflow_profile_install.sh | ||
|
|
||
| - name: Apply PodDefaults to access ml-pipeline with projected token | ||
| run: kubectl apply -f tests/poddefaults.access-ml-pipeline.kubeflow-user-example-com.yaml | ||
|
|
||
| - name: Create WorkspaceKind and Workspace with PodDefaults | ||
| run: | | ||
| kubectl apply -f tests/workspacekind.test.yaml | ||
| kubectl apply -f tests/workspace.test.kubeflow-user-example-com.yaml | ||
| kubectl wait --for=jsonpath='{.status.state}'=Running \ | ||
| workspace/test -n kubeflow-user-example-com \ | ||
| --timeout=600s | ||
|
|
||
| - name: Copy and execute the pipeline run script in KF Workspace | ||
| run: | | ||
| WORKSPACE_POD="$(kubectl -n kubeflow-user-example-com get pods \ | ||
| -l notebooks.kubeflow.org/workspace-name=test \ | ||
| -o jsonpath='{.items[0].metadata.name}')" | ||
|
|
||
| kubectl -n kubeflow-user-example-com cp \ | ||
| ./tests/pipeline_run_and_wait_kubeflow.py \ | ||
| ${WORKSPACE_POD}:/home/jovyan/pipeline_run_and_wait_kubeflow.py | ||
|
|
||
| kubectl -n kubeflow-user-example-com exec -ti \ | ||
| ${WORKSPACE_POD} -- python /home/jovyan/pipeline_run_and_wait_kubeflow.py | ||
|
|
||
| - name: Apply Pod Security Standards restricted levels | ||
| run: ./tests/PSS_enable.sh |
147 changes: 147 additions & 0 deletions
147
applications/workspaces/components/centraldashboard/centraldashboard-config.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| apiVersion: v1 | ||
| data: | ||
| settings: |- | ||
| { | ||
| "DASHBOARD_FORCE_IFRAME": true | ||
| } | ||
| links: |- | ||
| { | ||
| "menuLinks": [ | ||
| { | ||
| "icon": "book", | ||
| "items": [ | ||
| { | ||
| "link": "/jupyter/", | ||
| "text": "Notebooks", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/tensorboards/", | ||
| "text": "TensorBoards", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/volumes/", | ||
| "text": "Volumes", | ||
| "type": "item" | ||
| } | ||
| ], | ||
| "text": "Notebooks v1", | ||
| "type": "section" | ||
| }, | ||
| { | ||
| "icon": "book", | ||
| "items": [ | ||
| { | ||
| "link": "/workspaces/", | ||
| "text": "Workspaces", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/workspaces/workspacekinds/", | ||
| "text": "WorkspaceKinds", | ||
| "type": "item" | ||
| } | ||
| ], | ||
| "text": "Notebooks v2", | ||
| "type": "section" | ||
| }, | ||
| { | ||
| "icon": "kubeflow:katib", | ||
| "link": "/katib/", | ||
| "text": "Katib Experiments", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "type": "item", | ||
| "link": "/kserve-endpoints/", | ||
| "text": "KServe Endpoints", | ||
| "icon": "kubeflow:models" | ||
| }, | ||
| { | ||
| "icon": "kubeflow:pipeline-centered", | ||
| "items": [ | ||
| { | ||
| "link": "/pipeline/#/pipelines", | ||
| "text": "Pipelines", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/pipeline/#/experiments", | ||
| "text": "Experiments", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/pipeline/#/runs", | ||
| "text": "Runs", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/pipeline/#/recurringruns", | ||
| "text": "Recurring Runs", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/pipeline/#/artifacts", | ||
| "text": "Artifacts", | ||
| "type": "item" | ||
| }, | ||
| { | ||
| "link": "/pipeline/#/executions", | ||
| "text": "Executions", | ||
| "type": "item" | ||
| } | ||
| ], | ||
| "text": "Pipelines", | ||
| "type": "section" | ||
| } | ||
| ], | ||
| "externalLinks": [], | ||
| "documentationItems": [ | ||
| { | ||
| "desc": "The Kubeflow website", | ||
| "link": "https://www.kubeflow.org/", | ||
| "text": "Kubeflow Website" | ||
| }, | ||
| { | ||
| "desc": "Documentation for Kubeflow Pipelines", | ||
| "link": "https://www.kubeflow.org/docs/components/pipelines/", | ||
| "text": "Kubeflow Pipelines Documentation" | ||
| }, | ||
| { | ||
| "desc": "Documentation for Kubeflow Notebooks", | ||
| "link": "https://www.kubeflow.org/docs/components/notebooks/", | ||
| "text": "Kubeflow Notebooks Documentation" | ||
| }, | ||
| { | ||
| "desc": "Documentation for Kubeflow Training Operator", | ||
| "link": "https://www.kubeflow.org/docs/components/training/", | ||
| "text": "Kubeflow Training Operator Documentation" | ||
| }, | ||
| { | ||
| "desc": "Documentation for Katib", | ||
| "link": "https://www.kubeflow.org/docs/components/katib/", | ||
| "text": "Katib Documentation" | ||
| } | ||
| ], | ||
| "quickLinks": [ | ||
| { | ||
| "desc": "Kubeflow Notebooks", | ||
| "link": "/jupyter/new", | ||
| "text": "Create a new Notebook" | ||
| }, | ||
| { | ||
| "desc": "Kubeflow Pipelines", | ||
| "link": "/pipeline/#/pipelines", | ||
| "text": "Upload a Pipeline" | ||
| }, | ||
| { | ||
| "desc": "Pipelines", | ||
| "link": "/pipeline/#/runs", | ||
| "text": "View Pipeline Runs" | ||
| } | ||
| ] | ||
| } | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: dashboard-config |
4 changes: 4 additions & 0 deletions
4
applications/workspaces/components/centraldashboard/kustomization.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1alpha1 | ||
| kind: Component | ||
| patches: | ||
| - path: centraldashboard-config.yaml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
|
|
||
| # --> DANGER: Workspaces (Notebooks v2) is still pre-GA <-- | ||
| # | ||
| # !!! DO NOT DEPLOY THIS TO A PRODUCTION CLUSTER !!! | ||
| # | ||
| # See this for the current status: https://www.kubeflow.org/docs/components/notebooks/notebooks-v2-pre-ga-banner | ||
| resources: | ||
| - ../../upstream/backend/overlays/istio | ||
| - ../../upstream/controller/overlays/istio | ||
| - ../../upstream/frontend/overlays/istio | ||
| - workspaces-base-edit-clusterrole.yaml | ||
|
|
||
| patches: | ||
| - patch: |- | ||
| - op: replace | ||
| path: /rules | ||
| value: [] | ||
| target: | ||
| group: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: kubeflow-workspaces-edit | ||
|
|
||
| # Can be dropped with the next release: | ||
| # https://github.com/kubeflow/notebooks/pull/1032 | ||
| images: | ||
| - name: ghcr.io/kubeflow/notebooks/workspaces-backend | ||
| newTag: 2.0.0-alpha.1 | ||
| - name: ghcr.io/kubeflow/notebooks/workspaces-controller | ||
| newTag: 2.0.0-alpha.1 | ||
| - name: ghcr.io/kubeflow/notebooks/workspaces-frontend | ||
| newTag: 2.0.0-alpha.1 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.