Auto batch tuning for Postgres snapshots

[dascalescuro]

• What’s new: pgstream can now automatically tune snapshot batch size based on observed throughput (opt-in).
• Why: Static batch sizes break down across regions / variable latency. Oversized batches can trigger timeouts / buffer pressure and tank throughput.
• How it works (high level): directional search over batch sizes + stability checks; falls back to defaults if measurements remain unstable.

Enable it

target:
  postgres:
    url:"postgresql://user:password@localhost:5432/mytargetdatabase"
    batch:
      auto_tune:
        enabled:true

Version

Available in v0.9.3+.

Read more

• Blog (benchmarks + rationale): https://xata.io/blog/postgres-snapshots-with-batch-size-auto-tuning
• Docs (config reference): https://xata.io/documentation/opensource/pgstream/docs/configuration#pgstream_postgres_writer_batch_auto_tune_convergence_threshold
• Release/tag: https://github.com/xataio/pgstream/releases/tag/v0.9.3
• PR (implementation details): [Snapshot] Automatic write batch bytes #649

We’d love feedback on

• Cross-region / high-jitter networks
• Any workloads where tuning doesn’t converge well
• “Weird” timeout/batch interactions you’ve seen