You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/eks-cluster-mgmt/README.md
+98-40Lines changed: 98 additions & 40 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,42 +1,53 @@
1
1
# Amazon EKS cluster management using Symphony & ACK
2
-
This example demonstrates how to manage a fleet of EKS clusters using Symphony, ACK, and ArgoCD -- it creates EKS clusters, and bootstraps them with the required add-ons
3
2
4
-
A hub-spoke model is used in this example; a management cluster (hub) is created as part of the initial setup and the controllers needed for provisioning and bootstrapping workload clusters (spokes) are installed on top.
3
+
This example demonstrates how to manage a fleet of EKS clusters using Symphony,
4
+
ACK, and ArgoCD -- it creates EKS clusters, and bootstraps them with the
5
+
required add-ons
5
6
7
+
A hub-spoke model is used in this example; a management cluster (hub) is created
8
+
as part of the initial setup and the controllers needed for provisioning and
9
+
bootstrapping workload clusters (spokes) are installed on top.
6
10
7
-
**NOTE:** As this example evolves, some of the instructions below will be detailed further (e.g. the creation of the management cluster), others (e.g. controllers installation) will be automated via the GitOps flow.
11
+
**NOTE:** As this example evolves, some of the instructions below will be
12
+
detailed further (e.g. the creation of the management cluster), others (e.g.
13
+
controllers installation) will be automated via the GitOps flow.
8
14
9
15
## Prerequisites
16
+
10
17
1. AWS account for the management cluster
11
18
2. AWS account for workload clusters; each with the following IAM roles:
12
-
-`eks-cluster-mgmt-ec2`
13
-
-`eks-cluster-mgmt-eks`
14
-
-`eks-cluster-mgmt-iam`
15
-
16
-
The permissions should be as needed for every controller.
9. Create the GitHub repo `cluster-mgmt` in your organization; it will contain the clusters definition, and it will be reconciled to the management cluster via the GitOps flow
93
+
9. Create the GitHub repo `cluster-mgmt` in your organization; it will contain
94
+
the clusters definition, and it will be reconciled to the management cluster
95
+
via the GitOps flow
73
96
74
97
**NOTE:** Until Symphony is released, make sure the repo you create is private.
75
98
76
99
10. Save the URL of the created repo in an environment variable:
The initial configuration creates one workload cluster named `workload-cluster1`.
205
+
The initial configuration creates one workload cluster named
206
+
`workload-cluster1`.
170
207
171
208
**TODO:** add steps for cluster/account mapping
172
209
173
-
18. Add a workload cluster by adding a manifest for it under `clusters/`. Refer to `clusters/workload-cluster1.yaml` as an example.
210
+
18. Add a workload cluster by adding a manifest for it under `clusters/`. Refer
211
+
to `clusters/workload-cluster1.yaml` as an example.
174
212
19. Include the new cluster manifest in `clusters/kustomization.yaml`.
175
-
20. Add the cluster name and corresponding account number in `charts-values/ack-multi-account/values.yaml`.
213
+
20. Add the cluster name and corresponding account number in
214
+
`charts-values/ack-multi-account/values.yaml`.
176
215
21. Commit/push the changes to Git.
177
216
178
-
179
217
## Known issues
180
-
1. You will need to restart the Symphony controller when you add a new workload cluster due to a bug in the controller. Once the resource group `eksclusterwithvpc` is applied, the controller is able to apply the corresponding VPC resources, but it is not able to recognize the generated ids (e.g. subnet id), and feed that into EKS resources. Refer to [this issue](https://github.com/aws-controllers-k8s/private-symphony/issues/8) for more details.
181
-
2. Deleting a cluster does not properly clean up all cluster resources i.e. subnets, routetables are left strangling. ACK EC2 controller keep reporting dependencies preventing deletion. To work around this issue, attempt restart ACK EC2 controller, and/or manually deleting the resources.
218
+
219
+
1. You will need to restart the Symphony controller when you add a new workload
220
+
cluster due to a bug in the controller. Once the resource group
221
+
`eksclusterwithvpc` is applied, the controller is able to apply the
222
+
corresponding VPC resources, but it is not able to recognize the generated
223
+
ids (e.g. subnet id), and feed that into EKS resources. Refer to
224
+
[this issue](https://github.com/awslabs/private-symphony/issues/8) for more
225
+
details.
226
+
2. Deleting a cluster does not properly clean up all cluster resources i.e.
227
+
subnets, routetables are left strangling. ACK EC2 controller keep reporting
228
+
dependencies preventing deletion. To work around this issue, attempt restart
229
+
ACK EC2 controller, and/or manually deleting the resources.
182
230
183
231
## Clean-up
184
-
1. Delete ArgoCD bootstrap application, and wait for workload clusters and hosting VPCs to be deleted:
232
+
233
+
1. Delete ArgoCD bootstrap application, and wait for workload clusters and
234
+
hosting VPCs to be deleted:
235
+
185
236
```sh
186
237
kubectl delete application bootstrap -n argocd
187
238
```
239
+
188
240
2. Uninstall ArgoCD helm chart
241
+
189
242
```sh
190
243
helm uninstall argocd -n argocd
191
244
```
245
+
192
246
3. Delete ArgoCD IAM role and policy
247
+
193
248
```sh
194
249
aws iam delete-role --role-name argocd-hub-role
195
250
```
251
+
196
252
4. Delete ArgoCD IAM policy
253
+
197
254
```sh
198
255
aws iam delete-policy --policy-arn arn:aws:iam::$ACCOUNT_ID:policy/argocd-policy
0 commit comments