Skip to content

Aliyun OSS - S3 compatible storage issue!!! #879

@shusaan

Description

@shusaan

I have configured cluster to access (Aliyun OSS) S3 compatible storage but it show this error, can you please help me how to resolve this.
Here are configuration.

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster1
spec:
  instances: 1
  startDelay: 300
  stopDelay: 300
  replicationSlots:
    highAvailability:
      enabled: true
    updateInterval: 300
  primaryUpdateStrategy: unsupervised
  storage:
    storageClass: sc-ha
    size: 20Gi
  walStorage:
    storageClass: sc-ha
    size: 20Gi
  monitoring:
    enablePodMonitor: true
  backup:
    barmanObjectStore:
      destinationPath: "s3://testbucket/"
      endpointURL: "https://cn-shanghai.oss.aliyuncs.com"
      s3Credentials:
        accessKeyId:
          name: minio-creds
          key: MINIO_ACCESS_KEY
        secretAccessKey:
          name: minio-creds
          key: MINIO_SECRET_KEY
      wal:
        compression: gzip
        maxParallel: 8
    retentionPolicy: "30d"

Here are the logs

{"level":"info","ts":"2023-10-19T14:55:21Z","logger":"barman-cloud-check-wal-archive","msg":"2023-10-19 14:55:21,576 [1568] ERROR: Barman cloud WAL archive check exception: An error occurred (403) when calling the HeadBucket operation: Forbidden","pipe":"stderr","logging_pod":"cluster1-1"}
{"level":"error","ts":"2023-10-19T14:55:21Z","logger":"wal-archive","msg":"Error invoking barman-cloud-check-wal-archive","logging_pod":"cluster1-1","currentPrimary":"cluster1-1","targetPrimary":"cluster1-1","options":["--endpoint-url","https://cn-shanghai.oss.aliyuncs.com","--cloud-provider","aws-s3","s3://testbucket/","cluster1"],"exitCode":-1,"error":"exit status 4","stacktrace":"github.com/cloudnative-pg/cloudnative-pg/pkg/management/log.(*logger).Error\n\tpkg/management/log/log.go:128\ngithub.com/cloudnative-pg/cloudnative-pg/pkg/management/barman/archiver.(*WALArchiver).CheckWalArchiveDestination\n\tpkg/management/barman/archiver/archiver.go:257\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.checkWalArchive\n\tinternal/cmd/manager/walarchive/cmd.go:373\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.run\n\tinternal/cmd/manager/walarchive/cmd.go:190\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.NewCmd.func1\n\tinternal/cmd/manager/walarchive/cmd.go:85\ngithub.com/spf13/cobra.(*Command).execute\n\tpkg/mod/github.com/spf13/[email protected]/command.go:940\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tpkg/mod/github.com/spf13/[email protected]/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\tpkg/mod/github.com/spf13/[email protected]/command.go:992\nmain.main\n\tcmd/manager/main.go:64\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/proc.go:267"}
{"level":"error","ts":"2023-10-19T14:55:21Z","msg":"while barman-cloud-check-wal-archive","logging_pod":"cluster1-1","error":"unexpected failure invoking barman-cloud-wal-archive: exit status 4","stacktrace":"github.com/cloudnative-pg/cloudnative-pg/pkg/management/log.(*logger).Error\n\tpkg/management/log/log.go:128\ngithub.com/cloudnative-pg/cloudnative-pg/pkg/management/log.Error\n\tpkg/management/log/log.go:166\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.checkWalArchive\n\tinternal/cmd/manager/walarchive/cmd.go:374\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.run\n\tinternal/cmd/manager/walarchive/cmd.go:190\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.NewCmd.func1\n\tinternal/cmd/manager/walarchive/cmd.go:85\ngithub.com/spf13/cobra.(*Command).execute\n\tpkg/mod/github.com/spf13/[email protected]/command.go:940\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tpkg/mod/github.com/spf13/[email protected]/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\tpkg/mod/github.com/spf13/[email protected]/command.go:992\nmain.main\n\tcmd/manager/main.go:64\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/proc.go:267"}

I Have also tested with user that has full permission but the error was same, is there any custom config required for ALIYUN OSS storage?
As per Document the OSS is compatible with S3.

Please advice

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions