Skip to content

Commit cdd809e

Browse files
kterui9019kterui9019
committed
feat: implement docker compose deployer
- Add deployWithCompose() and cleanupWithCompose() methods - Support SKAFFOLD_COMPOSE_FILE environment variable for custom compose file paths - Automatically replace image names in compose files with skaffold-built images - Add comprehensive unit tests with 12 test cases covering all functionality - Add complete working example in examples/docker-compose-deploy/ - Add testdata files for unit testing
1 parent 48bae15 commit cdd809e

File tree

24 files changed

+1133
-5
lines changed

24 files changed

+1133
-5
lines changed

‎docs-v1/content/en/docs/pipeline-stages/deployers/docker.md‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,129 @@ the application image `my-image` to the local Docker daemon:
4141
{{< alert title="Note" >}}
4242
Images listed to be deployed with the `docker` deployer **must also have a corresponding build artifact built by Skaffold.**
4343
{{< /alert >}}
44+
45+
## Deploying with Docker Compose
46+
47+
Skaffold can deploy your application using Docker Compose instead of individual containers.
48+
This is useful when your application is already configured with a `docker-compose.yml` file
49+
and you want to leverage Compose's features like service dependencies, networks, and volumes.
50+
51+
### Configuration
52+
53+
To deploy using Docker Compose, set `useCompose: true` in the `docker` deploy configuration:
54+
55+
```yaml
56+
deploy:
57+
docker:
58+
useCompose: true
59+
images:
60+
- my-app
61+
```
62+
63+
### How it works
64+
65+
When `useCompose` is enabled, Skaffold:
66+
67+
1. Reads your `docker-compose.yml` file (or a custom file specified via environment variable)
68+
2. Builds images as specified in your `build` section
69+
3. Automatically replaces image names in the compose file with the built image tags
70+
4. Creates a temporary compose file with the updated images
71+
5. Runs `docker compose up -d` with a unique project name (`skaffold-{runID}`)
72+
6. On cleanup, runs `docker compose down --volumes --remove-orphans`
73+
74+
### Image name mapping
75+
76+
**Important**: For Skaffold to correctly replace images in your compose file, the image names
77+
in your `docker-compose.yml` must match (or be contained in) the image names specified in
78+
the `build.artifacts` section.
79+
80+
For example, if your `skaffold.yaml` has:
81+
82+
```yaml
83+
build:
84+
artifacts:
85+
- image: gcr.io/my-project/frontend-app
86+
- image: gcr.io/my-project/backend-app
87+
```
88+
89+
Your `docker-compose.yml` should use matching image names:
90+
91+
```yaml
92+
version: '3.8'
93+
services:
94+
frontend:
95+
image: frontend-app # Matches the suffix of gcr.io/my-project/frontend-app
96+
backend:
97+
image: backend-app # Matches the suffix of gcr.io/my-project/backend-app
98+
```
99+
100+
Skaffold will replace `frontend-app` with `gcr.io/my-project/frontend-app:latest-abc123`
101+
and `backend-app` with `gcr.io/my-project/backend-app:latest-def456`.
102+
103+
### Custom compose file location
104+
105+
By default, Skaffold looks for `docker-compose.yml` in the current directory.
106+
You can specify a custom location using the `SKAFFOLD_COMPOSE_FILE` environment variable:
107+
108+
```bash
109+
export SKAFFOLD_COMPOSE_FILE=path/to/my-compose.yml
110+
skaffold dev
111+
```
112+
113+
Or inline:
114+
115+
```bash
116+
SKAFFOLD_COMPOSE_FILE=docker-compose.prod.yml skaffold run
117+
```
118+
119+
### Example
120+
121+
Complete example configuration:
122+
123+
**skaffold.yaml:**
124+
```yaml
125+
apiVersion: skaffold/v4beta13
126+
kind: Config
127+
build:
128+
artifacts:
129+
- image: my-web-app
130+
docker:
131+
dockerfile: Dockerfile
132+
deploy:
133+
docker:
134+
useCompose: true
135+
images:
136+
- my-web-app
137+
```
138+
139+
**docker-compose.yml:**
140+
```yaml
141+
version: '3.8'
142+
services:
143+
web:
144+
image: my-web-app
145+
ports:
146+
- "8080:8080"
147+
environment:
148+
- NODE_ENV=development
149+
redis:
150+
image: redis:7-alpine
151+
ports:
152+
- "6379:6379"
153+
```
154+
155+
When you run `skaffold dev`, Skaffold will:
156+
- Build `my-web-app` image
157+
- Replace `my-web-app` in the compose file with the built tag (e.g., `my-web-app:latest-abc123`)
158+
- Leave `redis:7-alpine` unchanged (not built by Skaffold)
159+
- Deploy both services using `docker compose up`
160+
161+
### Limitations and Notes
162+
163+
- The compose file must have a valid `services` section
164+
- Only images that are built by Skaffold will be replaced
165+
- External images (like `postgres`, `redis`, etc.) are deployed as-is
166+
- The compose project name is automatically generated as `skaffold-{runID}` to avoid conflicts
167+
- Multiple Skaffold instances can run simultaneously without interfering with each other
168+
169+
For a complete working example, see [`examples/docker-compose-deploy`](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/docker-compose-deploy).

‎docs-v1/content/en/samples/deployers/docker-compose.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
deploy:
2+
docker:
3+
useCompose: true
4+
images:
5+
- my-web-app
6+
- my-api-app

‎docs-v2/content/en/docs/deployers/docker.md‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,3 +42,129 @@ the application image `my-image` to the local Docker daemon:
4242
{{< alert title="Note" >}}
4343
Images listed to be deployed with the `docker` deployer **must also have a corresponding build artifact built by Skaffold.**
4444
{{< /alert >}}
45+
46+
## Deploying with Docker Compose
47+
48+
Skaffold can deploy your application using Docker Compose instead of individual containers.
49+
This is useful when your application is already configured with a `docker-compose.yml` file
50+
and you want to leverage Compose's features like service dependencies, networks, and volumes.
51+
52+
### Configuration
53+
54+
To deploy using Docker Compose, set `useCompose: true` in the `docker` deploy configuration:
55+
56+
```yaml
57+
deploy:
58+
docker:
59+
useCompose: true
60+
images:
61+
- my-app
62+
```
63+
64+
### How it works
65+
66+
When `useCompose` is enabled, Skaffold:
67+
68+
1. Reads your `docker-compose.yml` file (or a custom file specified via environment variable)
69+
2. Builds images as specified in your `build` section
70+
3. Automatically replaces image names in the compose file with the built image tags
71+
4. Creates a temporary compose file with the updated images
72+
5. Runs `docker compose up -d` with a unique project name (`skaffold-{runID}`)
73+
6. On cleanup, runs `docker compose down --volumes --remove-orphans`
74+
75+
### Image name mapping
76+
77+
**Important**: For Skaffold to correctly replace images in your compose file, the image names
78+
in your `docker-compose.yml` must match (or be contained in) the image names specified in
79+
the `build.artifacts` section.
80+
81+
For example, if your `skaffold.yaml` has:
82+
83+
```yaml
84+
build:
85+
artifacts:
86+
- image: gcr.io/my-project/frontend-app
87+
- image: gcr.io/my-project/backend-app
88+
```
89+
90+
Your `docker-compose.yml` should use matching image names:
91+
92+
```yaml
93+
version: '3.8'
94+
services:
95+
frontend:
96+
image: frontend-app # Matches the suffix of gcr.io/my-project/frontend-app
97+
backend:
98+
image: backend-app # Matches the suffix of gcr.io/my-project/backend-app
99+
```
100+
101+
Skaffold will replace `frontend-app` with `gcr.io/my-project/frontend-app:latest-abc123`
102+
and `backend-app` with `gcr.io/my-project/backend-app:latest-def456`.
103+
104+
### Custom compose file location
105+
106+
By default, Skaffold looks for `docker-compose.yml` in the current directory.
107+
You can specify a custom location using the `SKAFFOLD_COMPOSE_FILE` environment variable:
108+
109+
```bash
110+
export SKAFFOLD_COMPOSE_FILE=path/to/my-compose.yml
111+
skaffold dev
112+
```
113+
114+
Or inline:
115+
116+
```bash
117+
SKAFFOLD_COMPOSE_FILE=docker-compose.prod.yml skaffold run
118+
```
119+
120+
### Example
121+
122+
Complete example configuration:
123+
124+
**skaffold.yaml:**
125+
```yaml
126+
apiVersion: skaffold/v4beta13
127+
kind: Config
128+
build:
129+
artifacts:
130+
- image: my-web-app
131+
docker:
132+
dockerfile: Dockerfile
133+
deploy:
134+
docker:
135+
useCompose: true
136+
images:
137+
- my-web-app
138+
```
139+
140+
**docker-compose.yml:**
141+
```yaml
142+
version: '3.8'
143+
services:
144+
web:
145+
image: my-web-app
146+
ports:
147+
- "8080:8080"
148+
environment:
149+
- NODE_ENV=development
150+
redis:
151+
image: redis:7-alpine
152+
ports:
153+
- "6379:6379"
154+
```
155+
156+
When you run `skaffold dev`, Skaffold will:
157+
- Build `my-web-app` image
158+
- Replace `my-web-app` in the compose file with the built tag (e.g., `my-web-app:latest-abc123`)
159+
- Leave `redis:7-alpine` unchanged (not built by Skaffold)
160+
- Deploy both services using `docker compose up`
161+
162+
### Limitations and Notes
163+
164+
- The compose file must have a valid `services` section
165+
- Only images that are built by Skaffold will be replaced
166+
- External images (like `postgres`, `redis`, etc.) are deployed as-is
167+
- The compose project name is automatically generated as `skaffold-{runID}` to avoid conflicts
168+
- Multiple Skaffold instances can run simultaneously without interfering with each other
169+
170+
For a complete working example, see [`examples/docker-compose-deploy`](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/docker-compose-deploy).

‎docs-v2/content/en/samples/deployers/docker-compose.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
deploy:
2+
docker:
3+
useCompose: true
4+
images:
5+
- my-web-app
6+
- my-api-app

‎integration/delete_test.go‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,67 @@ func getContainers(ctx context.Context, t *testutil.T, deployedContainers []stri
133133
return cl
134134
}
135135

136+
func getComposeContainers(ctx context.Context, t *testutil.T, projectNamePrefix string, client docker.LocalDaemon) []types.Container {
137+
t.Helper()
138+
139+
// List all containers
140+
cl, err := client.ContainerList(ctx, container.ListOptions{
141+
All: true,
142+
})
143+
t.CheckNoError(err)
144+
145+
// Filter containers by project name prefix
146+
var result []types.Container
147+
for _, c := range cl {
148+
if project, ok := c.Labels["com.docker.compose.project"]; ok {
149+
// Check if project name starts with the prefix (e.g., "skaffold-")
150+
if len(project) > len(projectNamePrefix) && project[:len(projectNamePrefix)] == projectNamePrefix {
151+
result = append(result, c)
152+
}
153+
}
154+
}
155+
156+
return result
157+
}
158+
159+
func TestDeleteDockerComposeDeployer(t *testing.T) {
160+
tests := []struct {
161+
description string
162+
dir string
163+
args []string
164+
}{
165+
{
166+
description: "docker compose deployer",
167+
dir: "testdata/docker-compose-deploy",
168+
args: []string{},
169+
},
170+
}
171+
172+
for _, test := range tests {
173+
testutil.Run(t, test.description, func(t *testutil.T) {
174+
MarkIntegrationTest(t.T, CanRunWithoutGcp)
175+
ctx := context.Background()
176+
177+
// Run skaffold to deploy with docker compose
178+
skaffold.Run(test.args...).InDir(test.dir).RunOrFail(t.T)
179+
180+
// Verify containers are running
181+
client := SetupDockerClient(t.T)
182+
containers := getComposeContainers(ctx, t, "skaffold-", client)
183+
if len(containers) == 0 {
184+
t.T.Fatal("Expected at least one container to be deployed")
185+
}
186+
187+
// Delete the deployment
188+
skaffold.Delete(test.args...).InDir(test.dir).RunOrFail(t.T)
189+
190+
// Verify containers are deleted
191+
containers = getComposeContainers(ctx, t, "skaffold-", client)
192+
t.CheckDeepEqual(0, len(containers))
193+
})
194+
}
195+
}
196+
136197
func TestDeleteNonExistedHelmResource(t *testing.T) {
137198
var tests = []struct {
138199
description string

‎integration/dev_test.go‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,48 @@ func TestDevCancelWithDockerDeployer(t *testing.T) {
187187
}
188188
}
189189

190+
func TestDevCancelWithDockerComposeDeployer(t *testing.T) {
191+
if runtime.GOOS == "windows" {
192+
t.Skip("graceful cancel doesn't work on windows")
193+
}
194+
195+
tests := []struct {
196+
description string
197+
dir string
198+
minContainers int
199+
projectPrefix string
200+
}{
201+
{
202+
description: "interrupt dev loop in Docker Compose deployer",
203+
dir: "testdata/docker-compose-deploy",
204+
minContainers: 1,
205+
projectPrefix: "skaffold-",
206+
},
207+
}
208+
209+
for _, test := range tests {
210+
t.Run(test.description, func(t *testing.T) {
211+
MarkIntegrationTest(t, CanRunWithoutGcp)
212+
p, err := skaffold.Dev().InDir(test.dir).StartWithProcess(t)
213+
if err != nil {
214+
t.Fatalf("error starting skaffold dev process")
215+
}
216+
217+
if err = waitForComposeContainersRunning(t, test.projectPrefix, test.minContainers); err != nil {
218+
t.Fatalf("failed waiting for containers: %v", err)
219+
}
220+
221+
p.Signal(syscall.SIGINT)
222+
223+
state, _ := p.Wait()
224+
225+
if state.ExitCode() != 0 {
226+
t.Fail()
227+
}
228+
})
229+
}
230+
}
231+
190232
func TestDevAPIBuildTrigger(t *testing.T) {
191233
MarkIntegrationTest(t, CanRunWithoutGcp)
192234

0 commit comments

Comments
 (0)