v1.29.3

What's changed

Ignore unknown fields when decoding Worker Deployment memo for backwards compatibility after rolling back from v1.30

Full Changelog: v1.29.2...v1.29.3

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use the tag 1.29.3)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

v1.28.2

What's Changed

This patch includes fixes for CVE-2025-14987 and CVE-2025-14986 and a few other improvements

Full Changelog: v1.28.1...v1.28.2

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release

Server (use the tag 1.28.2)
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

v1.27.4

What's Changed

This patch includes fixes for CVE-2025-14987 and CVE-2025-14986

Full Changelog: v1.27.3...v1.27.4

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release

Server (use the tag 1.27.4)
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

v1.29.2

What's Changed

This patch includes fixes for CVE-2025-14987 and CVE-2025-14986

Full Changelog: v1.29.1...v1.29.2

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release

Server (use the tag 1.29.2)
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

v1.29.1

Release Highlights

This patch release fixes an issue in Priority and Workflow Versioning.
It also fixes a bug in Workflow Retry for versioned workflows.

Full Changelog: v1.29.0...v1.29.1

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

v1.29.0

⚠️💥 UPCOMING BREAKING CHANGES 💥⚠️

Starting from next server release 1.30.0, for security reasons, Temporal docker images will be slimmed down and we are taking away the binaries and packages that don’t strictly need to be included. This includes:

temporalio/server

• temporal CLI - included in admin-tools
• tctl and tctl-authorization-plugin - both are deprecated CLIs
• dockerize - used for templating the configuration, functionality that is now inlined in the server codebase
• curl - not part of the Temporal distribution

temporalio/admin-tools

• tctl and tctl-authorization-plugin
• python3
• libev
• curl
• jq
• yq
• mysql-client
• postgresql-client
• expat
• tini
• cqlsh

Task queue fairness - pre-release

Description:

Task queue fairness allows you to control the execution order of workflows, activities, and child workflows within a single task queue by assigning fairness keys and weights. Note that priority keys take precedence over fairness assignments.

Fairness can be attached to workflows and activities using the latest versions of most SDKs. In order for priority to take effect on the server, you need to switch to set the dynamic config matching.enableFairness to true either on specific task queues, namespaces, or globally.

⚠️ Turning the feature on/off will cause currently backlogged tasks to be lost; which can cause workflows to be stuck. This limitation will be lifted in future releases.

See more usage details here: Temporal - Task Queue Fairness Guide (Pre-Release)

Rollout operational task: (schema upgrade)

Versioning improvement

Bug fixes

• RampingVersionPercentageChangedTime is now visible from the Routing Config and can be accessed via a DescribeDeployment call (#8089)
• Correct the drainage status of a deactivated version when it gets rolled back (#8119)
• Fix max deployment count check to not block new versions (#7841)
• Do not set sticky queue if a deployment transition is ongoing (#7852)
• Do not bypass task generation when version changes during a deployment transition (#7890)
• Make unsupported and deprecated Deployment API’s return Unimplemented error (#8009)
• Update DeploymentName from Override and fix batch UpdateOptions serialization bug (#7910)

Improvements

• Queries on drained + poller-less version to respond with descriptive error message (#7946)
• SetWorkerDeploymentManager: Help coordinate multiple tenant writes to the k8s worker controller without overwriting each others changes (#8278)
• Allow people to revert a task queue back to the un-versioned state without needing to set a 0% ramped non-nil Ramping Version (#8172)
• Allow users to be able to opt-in to activating versions that have not yet had any pollers (#8254)

Cloud rollout operational task:

Task queue config

Description:

Task queues can be now configured via the new UpdateTaskQueueConfig endpoint. It allows configuring (1) the queue’s maximum requests per second and (2) default maximum requests per second for fairness keys (feature is in pre-release). For example, using the Temporal CLI 1.5.0 or later:

temporal task-queue config set --task-queue foo --task-queue-type bar --queue-rate-limit 100 --queue-rate-limit-reason "throttling"

Note that the rate limit for the task queue from the API takes precedence over the rate limit set via the worker’s TaskQueueActivitiesPerSecond option. If the API rate limit is unset again, it will fall back to the worker’s rate limit again, if set. Otherwise the system’s default limit is applied.

Authorizer

Description: Added a new dynamic config (frontend.exposeAuthorizerErrors) to control whether the frontend authorization interceptor should propagate errors returned by the Authorizer component as-is or wrap them with a PermissionDenied service error. Default is false, meaning all errors will be wrapped with PermissionDenied, which matches current behavior to avoid breaking any custom Authorizer implementers.

Eager workflow start public release (on by default)

Description: Eager workflow start is a latency optimization for worker and a starter a colocated in the same process. If the starter (client) requests eager execution, and a worker slot is available, the client will request the server to start the workflow eagerly. If permitted (the dynamic
config is on, and there's no start delay), the first workflow task will be returned inline to be processed by the colocated worker.

This feature is now on by default and the EagerExecutionAccepted flag has been added to WorkflowExecutionStartedEventAttributes for debugging purposes (#8056)

To turn this feature off set the dynamic config system.enableEagerWorkflowStart to false.

Activity and workflow metrics changes

Description: A number of activity and workflow metrics were added and activity_e2e_latency has been deprecated. (#8196, #8185)

Deprecated Metrics

• activity_e2e_latency → Deprecated, replaced with better named activity_start_to_close_latency

New Metrics

• activity_start_to_close_latency: Per attempt latency from activity start to close
• activity_schedule_to_close_latency: End-to-end duration, including retries and backoff.
• activity_success: Number of succeeded activities
• activity_fail: Number of final failures for activities
• activity_timeout: Incremented on the final activity timeout tagged by timeout_type
• activity_task_fail: Failures for activities including retries
• activity_task_timeout: Number of activity attempt timeouts, tagged by timeout_type.
• activity_cancel: Number of canceled activities
• workflow_duration: End to end latency of workflow

Batch operation improvement

Description: Removed the internal BatchParams go struct in favor of a protobuf struct for safer serialization. Optimized the batch operation processing with proactive page fetching, removing the worker wait for new pages to be completed. (#8144, #8081).

Batch Activity Reset and Update Options

Description: Support activity reset and update-options on the server side. (#8061)

Nexus

Description:

• Fixed a bug where the standby outbound task executor could sometimes return a NamespaceNotActive error.
• Fixed a data race in Nexus disallowed headers dynamic config.
• All remote frontend HTTP calls will always attempt to use HTTP2 by default.
• Changed the default to true for dynamic config component.nexusoperations.recordCancelRequestCompletionEvents . This config will be removed in a future release.
• Fixed a bug for forwarded Nexus operation completion HTTP requests that contained a failure. The failure will now be reconstructed instead of reusing the original HTTP request body.
• Added logic to forward Nexus HTTP requests using the same dispatch type (by endpoint or by namespace+task queue) as the original request. This behavior is behind a dynamic config, frontend.nexusForwardRequestUseEndpointDispatch, because endpoints currently do not support replication and therefore forwarding requests by endpoint will not work out of the box.
• The original HTTP request headers will now be passed through as-is, without sanitization, for forwarded requests.
• Bug fix: a new workflow task will be generated for NexusOperationCancelRequestCompleted|Failed events.

Visibility

Description: ScanWorkflowExecutions has been removed from Visibility. The API is still available, but it simply calls ListWorkflowExecutions.

Worker Insights

Description: Worker insights is an umbrella project that addresses the operational complexities and toil of worker management (tuning performance knobs, scaling workers). Our end goal is to automate this process for our users. As a first step, this release adds the capability to propagate the worker state (configuration + SDK metrics) to the server via a heartbeat mechanism. This state is stored in-memory on the matching servers, and can be queried by the user.

To achieve this, this release introduces 3 APIs:

1. RecordWorkerHeartbeat: To send heartbeat to the matching service. Used by SDK.
2. ListWorkers: To query the state of 1 or more workers that match a predicate.
3. DescribeWorker: To query the state of a specific worker.

These APIs are disabled by default.

Flags are: frontend.WorkerHeartbeatsEnabled and frontend.ListWorkersEnabled.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use the tag 1.29.0)

Server
Server With Auto Setup (what is Auto-Setup?)
[Admin-Tools](https://hu...

v1.27.3

Release Highlights

• Limit the number of parts allowed for auth token by @picatz in #8122

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use the tag 1.27.3)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Full Changelog: v1.27.2...v1.27.3

v1.26.3

Release Highlights

• Limit the number of parts allowed for auth token by @picatz in #8122

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use the tag 1.26.3)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Full Changelog: v1.26.2...v1.26.3

v1.28.1

This patch release fixes a few Workflow Update, Worker Deployment, Scheduler, Matching, and security bugs.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use the tag 1.28.1)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Full Changelog: v1.28.0...v1.28.1

v1.28.0

Schema changes

Before upgrading your Temporal Cluster to v1.28.0, you must upgrade your core schemas to the following:

• MySQL schema v1.17
• PostgreSQL schema v1.17
• Cassandra schema v1.12

Please see our upgrade documentation for the necessary steps to upgrade your schemas.

Deprecation Announcements

Deprecating old Versioning APIs: The following APIs related to previous versions of Worker Versioning are deprecated.

The following APIs related to the December 2024 pre-release of Worker Versioning have been deprecated and are now no longer supported:

• DescribeDeployment
• ListDeployments
• GetDeploymentReachability
• GetCurrentDeployment
• SetCurrentDeployment

The following APIs are now deprecated and will be removed once the latest APIs reach to General Availability in the coming months:

• UpdateWorkerVersioningRules
• GetWorkerVersioningRules
• UpdateWorkerBuildIdCompatibility
• GetWorkerBuildIdCompatibility
• GetWorkerTaskReachability

Release Highlights

Update-With-Start GA

Update-With-Start sends a Workflow Update that checks whether an already-running Workflow with that ID exists. If it does, the Update is processed. If not, it starts a new Workflow Execution with the supplied ID. When starting a new Workflow, it immediately processes the Update.

Update-With-Start is great for latency-sensitive use cases.

Nexus - Back Multiple Operations by Single Workflow

Now you can have multiple callers starting operations backed by a single workflow. When the handler tries to start a workflow that is already running, with the “use existing” conflict policy, the server will attach the caller’s callback to the running workflow. When the workflow completes, the server will call all attached callbacks to notify the callers with the workflow result.

Here’s an example using Go SDK (available in v1.34.0+):

import (
	"context"
	
	"github.com/nexus-rpc/sdk-go/nexus"
	enumspb "go.temporal.io/api/enums/v1"
	"go.temporal.io/sdk/client"
	"go.temporal.io/sdk/temporalnexus"
)

sampleOperation := temporalnexus.NewWorkflowRunOperation(
	"sample-operation",
	SampleWorkflow,
	func (
		ctx context.Context,
		input SampleWorkflowInput,
		options nexus.StartOperationOptions,
	) (client.StartWorkflowOptions, error) {
		return client.StartWorkflowOptions{
			// Workflow ID is used as idempotency key.
			ID: "sample-workflow-id",
			// If a workflow with same ID is already running, then it will attach the callback to the existing running workflow.
			// Otherwise, it will start a new workflow.
			WorkflowIDConflictPolicy: enumspb.WORKFLOW_ID_CONFLICT_POLICY_USE_EXISTING,
		}, nil
	},
)

The handler workflow will return a RequestIdReferenceLink to the caller. This is an indirect link to the history event that attached the callback in the handler workflow. Links can provide information about the handler workflow to the caller. In order to get the exact event history, there is now the RequestIdInfos map in the WorkflowExtendedInfo field of a DescribeWorkflowExecutionResponse. To enable RequestIdReferenceLink, you have to set the dynamic config history.enableRequestIdRefLinks to true (this might become enabled by default in a future release).

Nexus - Callback ↔ Link Association

Nexus links were previously stored in the history event not directly associated with the callback that it came together. Now, the server is storing the Nexus links together with the callback. With this direct association, you can easily find out the caller that triggered the Nexus workflow from the callback through these links for example. This feature requires the latest version of Go SDK (v1.35.0+) and Java SDK (v1.30.0+).

Nexus - Cancellation Types

The server now supports Nexus operation cancellation types. These are specified when starting an operation and indicate what should happen to Nexus operations when the parent context that started them is cancelled. To use them, you must be using an SDK version that supports them. Available cancellation types are:

• Abandon - Do not request cancellation of the operation
• TryCancel - Request cancellation of the operation and immediately report cancellation to callers
• WaitRequested - Request cancellation and wait until the cancellation request has been received by the operation handler
• WaitCompleted - Request cancellation and wait for the operation to complete. The operation may or may not complete as cancelled. Default and behavior for server versions <1.28

For the WaitRequested type to work, you must set the dynamic config component.nexusoperations.recordCancelRequestCompletionEvents to true (default false ).

Nexus - Miscellaneous

• Nexus callback request processing logic will now attempt to deserialize failure information from the body of failed callback HTTP requests. This means that Nexus operation handlers should now see more informative messages about why their callback failed to be delivered to callers.
• Links added by Nexus operations should now be appropriately propagated across continue-as-new, workflow reset, and workflow retries.

Versioning / Safe-Deploy Public Preview

The following Worker Versioning APIs graduated into Public Preview stage. Production usage is encouraged but note that limited changes might be made to the APIs before General Availability in the coming months.

• ListWorkerDeployments
• DescribeWorkerDeployment
• DescribeWorkerDeploymentVersion
• SetWorkerDeploymentCurrentVersion
• SetWorkerDeploymentRampingVersion
• UpdateWorkerVersionMetadata
• DeleteWorkerDeployment
• DeleteWorkerDeploymentVersion

Using Worker Versioning: Find instructions in https://docs.temporal.io/worker-versioning.

Operator notes:

• The following configs need to be enabled:
  • frontend.workerVersioningWorkflowAPIs (default: true)
  • system.enableDeploymentVersions (default: true)
• Knobs:
  • matching.maxDeployments controls the maximum number of worker deployments that the server allows to be registered in a single namespace (default: 100, safe to increase to much higher values)
  • matching.maxVersionsInDeployment controls the maximum number of versions that the server allows to be registered in a single worker deployments (default: 100, unsafe to increase beyond a few 100s)
  • matching.maxTaskQueuesInDeploymentVersion controls the maximum number of task queues that the server allows to be registered in a single worker deployment version (default: 100, unsafe to increase beyond a few 100s)
  • matching.wv.VersionDrainageStatusVisibilityGracePeriod systems waits for this amount of time before checking the drainage status of a version that just entered in DRAINING state (default: 3 minutes, setting a very low value might cause the status to become DRAINED incorrectly)
  • matching.wv.VersionDrainageStatusRefreshInterval interval used for checking drainage status (default: 3 minutes, lowering the value will increase load on the Visibility database)

Please see deprecation warnings regarding earlier versions of Temporal versioning APIs.

⚠️ Important for Worker Versioning users (v1.27.x → v1.28.0)

If you used Worker Versioning in v1.27.x, you must delete all Worker Deployments (via DeleteWorkerDeployment) before upgrading to v1.28.0, then recreate them after. This is due to breaking changes between v1.27.2 and v1.28.0.

If you already upgraded or need help, ask in #safe-deploys on Community Slack.

Simple priority for task queues - pre-release

Simple priority allows you to control the execution order of workflows, activities, and child workflows based on assigned priority values within a single task queue. You can select a priority level in the integer range [1,5]. A lower value implies higher priority.

Priority can be attached to workflows and activities using the latest versions of most SDKs. In order for priority to take effect on the server, you need to switch to the new implementation of the matching service: set the dynamic config matching.useNewMatcher to true either on specific task queues, namespaces, or globally. After the new matcher has been turned on for a task queue, turning it off will cause tasks with non-default priority to be temporarily lost until it’s turned on again.

When the setting is changed, the implementation will be switched immediately, which may cause a momentary disruption in task dispatch.

Besides enabling priority, the new matcher will have a different profile of persistence operations, and slightly different behavior with task forwarding and some other edge cases. If you see performance regressions or unexpected behavior with the new matcher, please let us know.

See more usage details here: Temporal - Task Queue Priority Guide (Pre-Release)

Operator commands

• Improvements to the activity in DescribeWorkflow:
  • Add activity options to the pending activity info
    • Task Queue name
    • All timeouts
    • Retry Policy
  • Add PauseInfo to the pending activity info
    • Timestamp
    • Identity
    • Pause reason
  • Add PAUSED and PAUSE_REQUESTED to activity state. This allows to distinguish between the situation when pause signal is received, but activity is still running on the worker.
• Send ActivityPause/ActivityReset flag in heartbeat response. This notifies the workers about activity state.
• Add the ability to restore activity options to its original state for ResetActivity and UpdateActivityOptions commands.
• Add a flag to CLI to restore activity options for the activity res...