User Timestamp Parsing by chenosaurus · Pull Request #1812 · livekit/client-sdk-js

chenosaurus · 2026-02-18T06:57:44Z

No description provided.

changeset-bot · 2026-02-18T06:57:48Z

⚠️ No Changeset found

Latest commit: b37b19b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

examples/demo/demo.ts

          <span id="e2ee-${identity}" class="e2ee-on"></span>
        </div>
      </div>
+      <div id="user-ts-${identity}" class="user-ts-overlay">


In general, to fix unsafe HTML construction you should either (1) avoid innerHTML entirely and build the DOM with document.createElement and textContent/innerText, or (2) rigorously escape/sanitize any untrusted values before inserting them into HTML. Since this is demo UI code and we want to preserve functionality while minimizing changes, the best fix here is to ensure that any untrusted dynamic value interpolated into the HTML string is passed through a simple HTML-escaping helper before being inserted into innerHTML.

Concretely, in examples/demo/demo.ts we should define a small escapeHtml function near the rendering helpers that replaces &, <, >, ", and ' with their corresponding HTML entities. Then, inside renderParticipant, we should compute a safe version of the participant identity, e.g. const safeIdentity = escapeHtml(identity);, and use safeIdentity instead of identity in every interpolated position within the div.innerHTML template. This keeps the structure of the HTML unchanged (IDs, class names, etc.) but ensures that any untrusted characters are escaped and can no longer break out of their intended context to execute scripts. This one change addresses all CodeQL variants that trace through participant/track data into the innerHTML template.

…-sdk-js into dc/feature/user_timestamp

github-actions · 2026-02-22T01:58:48Z

size-limit report 📦

Path	Size
dist/livekit-client.esm.mjs	96.28 KB (+1.1% 🔺)
dist/livekit-client.umd.js	104.97 KB (+0.73% 🔺)

1egoman

It might be possible I'm reviewing this prematurely - Shijing had asked I take a look at this. That being said, IMO there's still quite a bit of work to be done here and I'm not sure I'm fully following how this all works right now.

1egoman · 2026-03-19T20:37:15Z

src/e2ee/worker/FrameCryptor.ts

+    // Always attempt to strip LKTS packet trailer before any e2ee
+    // processing. On the send side, the trailer is appended *after* encryption,
+    // so it must be removed *before* decryption.
+    if (isVideoFrame(encodedFrame) && encodedFrame.data.byteLength > 0) {
+      try {
+        const packetTrailerResult = stripPacketTrailerFromEncodedFrame(
+          encodedFrame as RTCEncodedVideoFrame,
+        );
+        if (packetTrailerResult !== undefined && this.trackId && this.participantIdentity) {
+          const msg: PacketTrailerMessage = {
+            kind: 'packetTrailer',
+            data: {
+              trackId: this.trackId,
+              participantIdentity: this.participantIdentity,
+              timestampUs: packetTrailerResult.timestampUs,
+              frameId: packetTrailerResult.frameId,
+              rtpTimestamp: packetTrailerResult.rtpTimestamp,
+            },
+          };
+          postMessage(msg);
+        }
+      } catch {
+        // Best-effort: never break media pipeline if timestamp parsing fails.
+      }
+    }
+


question: Is the FrameCryptor (which I believe runs as part of the e2ee worker, which may not be loaded if e2ee is not enabled) the best place for this logic to live?

Is there a reason it has to run in a web worker context (ie, maybe it is super computationally expensive / etc)? Is this feature supposed to work if e2ee is disabled?

1egoman · 2026-03-19T20:39:06Z

src/e2ee/E2eeManager.ts

+  private handleUserTimestamp(
+    trackId: string,
+    participantIdentity: string,
+    timestampUs: number,
+    frameId?: number,
+    rtpTimestamp?: number,
+  ) {
+    if (!this.room) {
+      return;


question: Similar to my last comment - I don't know a ton about the user timestamp feature, but my suspicion is that putting handleUserTimestamp inside the E2eeManager is mixing concerns unnecesarily.

1egoman · 2026-03-19T20:51:52Z

src/packet_trailer/packetTrailer.worker.ts

+/**
+ * Lightweight worker for stripping LKTS packet trailers from inbound
+ * encoded video frames via RTCRtpScriptTransform.
+ *
+ * When a valid trailer is found, the metadata is posted back to the main
+ * thread so the SDK can store the RTP-to-frame-metadata mapping on the
+ * corresponding RemoteVideoTrack.
+ */
+import {


question: I'm a bit confused now - so this change is introducing a second webworker exclusively for packet trailer stripping, but somehow this worker ties into the existing E2eeManager / e2ee worker? Assuming this isn't in an incomplete state mid migration from one approach to another, I might need some more background context to better understand this.

As a heads up - introducing another webworker is a fairly significant change. I think it might be viable because this packet trailer stuff is fully additive, but it's something we've used sparingly, so I just want to make sure there's a good reason to use it here.

More related background context I wrote in another context where somebody was proposing introducing a new webworker, maybe you would find it useful:

Starting a webworker requires that you pass a url to another script that you want to run to the new Worker("https://path.to/worker/script.js") (docs) constructor when creating the new worker, but because livekit is a library we don't have control over the end user's build process / have no way to force them to generate another file in their end application, let alone get a predictable path to where it might be hosted.

The one place that is using a webworker right now in the web sdk is end to end encryption, and the way it gets around this problem (and the way we'd have to do this for compression) is by bubbling up the web worker construction to the end user, forcing them to build / host a separate "worker script" containing the e2ee worker code, call new Worker("...") + pass the url to this "worker script", and then pass the fully constructed Worker object into the main Livekit SDK. For example, this is how this is done in the client-sdk-js demo app. This is super cumbersome / not an easy thing to instruct a user to do.

1egoman · 2026-03-19T20:55:56Z

src/room/track/RemoteVideoTrack.ts

+   *
+   * @internal
+   */
+  setUserTimestamp(timestampUs: number, rtpTimestamp?: number, frameId?: number) {


nitpick: As a heads up, in data tracks, the userTimestamp field is a bigint. It might make sense to use that here too.

1egoman · 2026-03-19T20:57:33Z

src/room/track/RemoteVideoTrack.ts

+  /**
+   * Look up the frame metadata associated with a given RTP timestamp.
+   * The entry is consumed (removed) after lookup, matching the Rust SDK
+   * behaviour.


nitpick: I suspect this was probably LLM generated with a prompt like "port the rust implementation over to the web sdk" - it would be good to strip out some of these egregious comparisons to the rust sdk before this would end up getting merged.

question: Also in this context, is actually the "consuming the entry like rust" type behavior a good idea? In rust there are benefits to avoid extra cloning, but it has the serious tradeoff of making requests non idempotent. Given the web doesn't have these same data ownership concerns it is a fairly serious rough edge for from what I can tell no real benefit, assuming there is some other way the entries in the userTimestampMap are cleared so it doesn't grow in an unbounded fashion.

1egoman · 2026-03-19T21:01:37Z

src/room/Room.ts

 }

+function createPacketTrailerWorker(): Worker {
+  return new Worker(new URL('./livekit-client.packet-trailer.worker.js', import.meta.url));


issue: See my comment here - this type of thing isn't viable and won't work for all downstream consumers of the package transparently.

chenosaurus · 2026-04-02T18:58:35Z

It might be possible I'm reviewing this prematurely - Shijing had asked I take a look at this. That being said, IMO there's still quite a bit of work to be done here and I'm not sure I'm fully following how this all works right now.

@1egoman sorry this was still in draft and was just a quick version to test that everything works. Still need to clean up and get it in shape for release. I will review the comments and reach out for any questions.

chenosaurus added 2 commits February 17, 2026 19:18

wip adding user timestamp extraction

41e5ec1

wip

4aab3e5

github-advanced-security bot found potential problems Feb 18, 2026

View reviewed changes

examples/demo/demo.ts Fixed Show fixed Hide fixed

add css class

e3c48d3

github-advanced-security bot found potential problems Feb 21, 2026

View reviewed changes

chenosaurus added 4 commits February 21, 2026 17:44

eslint

df6c1d8

Merge branch 'main' into dc/feature/user_timestamp

9db706b

fix worker import

aec3263

Merge branch 'dc/feature/user_timestamp' of github.com:livekit/client…

7db8729

…-sdk-js into dc/feature/user_timestamp

chenosaurus added 3 commits March 13, 2026 12:56

update to TLV packet trailer format

6e16a9d

Merge branch 'main' into dc/feature/user_timestamp

04ce0ed

rename to Packet Trailer

45820bd

1egoman reviewed Mar 19, 2026

View reviewed changes

Merge branch 'main' into dc/feature/user_timestamp

b37b19b

@@ -877,42 +877,52 @@
               })();
             }
+            function escapeHtml(value: string): string {
+              return value
+                .replace(/&/g, '&amp;')
+                .replace(/</g, '&lt;')
+                .replace(/>/g, '&gt;')
+                .replace(/"/g, '&quot;')
+                .replace(/'/g, '&#39;');
+            }
             // updates participant UI
             function renderParticipant(participant: Participant, remove: boolean = false) {
               const container = getParticipantsAreaElement();
               if (!container) return;
               const { identity } = participant;
-              let div = container.querySelector(`#participant-${identity}`);
+              const safeIdentity = escapeHtml(identity);
+              let div = container.querySelector(`#participant-${safeIdentity}`);
               if (!div && !remove) {
                 div = document.createElement('div');
-                div.id = `participant-${identity}`;
+                div.id = `participant-${safeIdentity}`;
                 div.className = 'participant';
                 div.innerHTML = `
-                  <video id="video-${identity}"></video>
-                  <audio id="audio-${identity}"></audio>
+                  <video id="video-${safeIdentity}"></video>
+                  <audio id="audio-${safeIdentity}"></audio>
                   <div class="info-bar">
-                    <div id="name-${identity}" class="name">
+                    <div id="name-${safeIdentity}" class="name">
                     </div>
                     <div style="text-align: center;">
-                      <span id="codec-${identity}" class="codec">
+                      <span id="codec-${safeIdentity}" class="codec">
                       </span>
-                      <span id="size-${identity}" class="size">
+                      <span id="size-${safeIdentity}" class="size">
                       </span>
-                      <span id="bitrate-${identity}" class="bitrate">
+                      <span id="bitrate-${safeIdentity}" class="bitrate">
                       </span>
                     </div>
                     <div class="right">
-                      <span id="signal-${identity}"></span>
-                      <span id="mic-${identity}" class="mic-on"></span>
-                      <span id="e2ee-${identity}" class="e2ee-on"></span>
+                      <span id="signal-${safeIdentity}"></span>
+                      <span id="mic-${safeIdentity}" class="mic-on"></span>
+                      <span id="e2ee-${safeIdentity}" class="e2ee-on"></span>
                     </div>
                   </div>
-                  <div id="user-ts-${identity}" class="user-ts-overlay">
+                  <div id="user-ts-${safeIdentity}" class="user-ts-overlay">
                   </div>
                   ${
                     !isLocalParticipant(participant)
                       ? `<div class="volume-control">
-                    <input id="volume-${identity}" type="range" min="0" max="1" step="0.1" value="1" orient="vertical" />
+                    <input id="volume-${safeIdentity}" type="range" min="0" max="1" step="0.1" value="1" orient="vertical" />
                   </div>`
                       : `<progress id="local-volume" max="1" value="0" />`
                   }
@@ -920,8 +907,8 @@
                 `;
                 container.appendChild(div);
-                const sizeElm = container.querySelector(`#size-${identity}`);
-                const videoElm = <HTMLVideoElement>container.querySelector(`#video-${identity}`);
+                const sizeElm = container.querySelector(`#size-${safeIdentity}`);
+                const videoElm = <HTMLVideoElement>container.querySelector(`#video-${safeIdentity}`);
                 videoElm.onresize = () => {
                   updateVideoSize(videoElm!, sizeElm!);
                 };

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

User Timestamp Parsing#1812

User Timestamp Parsing#1812
chenosaurus wants to merge 11 commits intomainfrom
dc/feature/user_timestamp

chenosaurus commented Feb 18, 2026

Uh oh!

changeset-bot bot commented Feb 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Check warning

Copilot Autofix

github-actions bot commented Feb 22, 2026 •

edited

Loading

Uh oh!

1egoman left a comment

Uh oh!

1egoman Mar 19, 2026

Uh oh!

1egoman Mar 19, 2026

Uh oh!

1egoman Mar 19, 2026 •

edited

Loading

Uh oh!

1egoman Mar 19, 2026 •

edited

Loading

Uh oh!

1egoman Mar 19, 2026 •

edited

Loading

Uh oh!

1egoman Mar 19, 2026

Uh oh!

chenosaurus commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

chenosaurus commented Feb 18, 2026

Uh oh!

changeset-bot bot commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

Uh oh!

Check warning

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot Autofix

github-actions bot commented Feb 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

size-limit report 📦

Uh oh!

1egoman left a comment

Choose a reason for hiding this comment

Uh oh!

1egoman Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

1egoman Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

1egoman Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

1egoman Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

changeset-bot bot commented Feb 18, 2026 •

edited

Loading

github-actions bot commented Feb 22, 2026 •

edited

Loading

1egoman Mar 19, 2026 •

edited

Loading

1egoman Mar 19, 2026 •

edited

Loading

1egoman Mar 19, 2026 •

edited

Loading