Update OpenAI realtime GA API #949
Conversation
🦋 Changeset detectedLatest commit: 5c1795c The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
toubatbrian
changed the base branch from
main
to
brian/fix-realtime-otel-export
| @@ -0,0 +1,5 @@ | |||
| --- | |||
| '@livekit/agents-plugin-openai': patch | |||
There was a problem hiding this comment.
given that the now legacy AudioFormat was exposed to users and we're renaming it, I think this should be a minor version upgrade.
There was a problem hiding this comment.
I can make it backward competible tho. One thing makes it hard to do minor bump is that we recently sync all plugin packages, along with main agent package, to the same version on every release. Making a minor bump on one plugin would cause all packages to be minor bumped. Any idea on how we should properly handle this?
There was a problem hiding this comment.
Actually, while AudioFormat type changed from a string ('pcm16') to an interface ({ type: 'audio/pcm', rate: number }), users never interact with this type directly - it's only used internally for the WebSocket protocol.
The public API (constructor options) remains fully backward compatible:
Beta:
new RealtimeModel({ model: 'gpt-4o-realtime', voice: 'alloy', temperature: 0.8, inputAudioTranscription: { model: 'whisper-1' }, turnDetection: { type: 'server_vad' }, // ... other options});GA (same code still works):
new RealtimeModel({ model: 'gpt-4o-realtime', voice: 'alloy', temperature: 0.8, // kept for compat, marked @deprecated inputAudioTranscription: { model: 'whisper-1' }, turnDetection: { type: 'server_vad' }, // ... other options // NEW optional: inputAudioNoiseReduction, tracing});All existing constructor options are preserved, temperature is kept (just deprecated), and new options (inputAudioNoiseReduction, tracing) are additive. Keeping it as a patch should be fine since no user-facing breaking changes.
Let me know how you think @lukasIO
There was a problem hiding this comment.
I just verified, both
llm: new openai.realtime.beta.RealtimeModel(),and
llm: new openai.realtime.RealtimeModel(),are working properly.
There was a problem hiding this comment.
Actually, while AudioFormat type changed from a string ('pcm16') to an interface ({ type: 'audio/pcm', rate: number }), users never interact with this type directly - it's only used internally for the WebSocket protocol.
if that's the case then we should think about not exporting it from index.ts in the first place going forward. Right now it's available for users to interact with as a type even if they don't directly use it.
One thing makes it hard to do minor bump is that we recently sync all plugin packages, along with main agent package, to the same version on every release. Making a minor bump on one plugin would cause all packages to be minor bumped.
yeah, this situation is less than ideal imo. Let's discuss this in our sync next week.
No description provided.