Improve UX for reconnecting after server restart by Darshan808 · Pull Request #548 · jupyterlab/jupyter-collaboration

Darshan808 · 2026-02-23T10:07:16Z

Fixes #535

Description

Jupyter_server_ydoc:

Store session IDs and collaboration version in .jupyter/collaboration_sessions.json on each connection
On reconnect with an old session ID, check compatibility before rejecting, allow silent reconnect if the root directory and package version are unchanged
Only close with 1003 when reconnection would be unsafe (different directory or version mismatch)

User Facing Changes

Most server restarts (same directory, same version) no longer interrupt the user at all.

github-actions · 2026-02-23T10:07:26Z

👈 Launch a Binder on branch Darshan808/jupyter-collaboration/reconnect-automatically

krassowski

I think this goes in the right direction. The frontend changes fallback to unknown_session so it should be a fine upgrade experience for existing serves (whether jupyter-server-ydoc of jupyverse)

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py

packages/docprovider/src/tokens.ts

packages/docprovider/src/__tests__/sessionClose.spec.ts

Copilot

Pull request overview

This PR improves real-time collaboration reconnection behavior after a server restart by persisting recent collaboration session IDs/versions server-side and by showing a translated, actionable client dialog only when reconnection is unsafe.

Changes:

Persist recent server session IDs (and package version) under .jupyter/collaboration_sessions.json and use them to decide whether an old client sessionId can safely reconnect.
Update the WebSocket close payload for unsafe reconnects / initialization errors to structured JSON and handle it client-side with translated dialogs and an optional “Reload” action.
Add unit tests for session-store compatibility logic (Python) and close-payload handling (TS).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py`	Adds session store + compatibility check helpers and exposes `SERVER_SESSION`/`YDOC_SERVER_VERSION`.
`projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py`	Persists current session on connect and conditionally closes with structured JSON payload when reconnect is unsafe.
`packages/docprovider/src/yprovider.ts`	Parses 1003 close payload JSON and presents translated dialogs with optional reload.
`packages/docprovider/src/tokens.ts`	Defines `ISessionClosePayload` payload shape for close-reason handling.
`tests/test_session_store.py`	Adds Python unit tests for session compatibility decisions.
`packages/docprovider/src/__tests__/sessionClose.spec.ts`	Adds Jest tests validating dialog + reload behavior for 1003 close payloads.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py

packages/docprovider/src/yprovider.ts

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

Co-authored-by: Michał Krassowski <[email protected]>

krassowski

This makes sense to me. The UI test failures are unrelated, due to a new JupyterLab version release which fixed incorrect fonts in some input elements.

krassowski · 2026-03-12T14:26:21Z

@davidbrochart does it look good to you too?

davidbrochart

Would it be better to use an SQLite database to store the sessions?

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

krassowski · 2026-03-12T15:29:22Z

Would it be better to use an SQLite database to store the sessions?

On one hand SQLite is already a dependency because pycrdt-store and jupyter-server-file-id use it, on the other hand this is such a simple structure that I am not sure if it is better. No strong opinion here.

davidbrochart

It seems that this PR introduces a new .jupyter directory. I'm wondering if we could take advantage of the YStore, which is by default an SQLite database in .jupyter_ystore.db?

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py

davidbrochart · 2026-03-13T07:46:59Z

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

+import json
+import uuid
+from datetime import datetime, timezone
+from ._version import __version__  # noqa
+import asyncio
+import os


Since these imports are not ordered, it looks like the linter doesn't run on this file?

I think the linter isn't configured for these. I just raised a PR #553

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

davidbrochart · 2026-03-13T07:57:38Z

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

+
+    # Collaboration package version changed → reject
+    if previous_version != current_version:
+        return False, "version_mismatch"


We already have shared document versions, e.g. the YNotebook version. Should we use it or is it the jupyter-collaboration package version that we want to check?

It is definitely jupyter-collaboration (specifically jupyter-server-ydoc and jupyter-docprovider but collaboration metapackage is a good proxy) that we want to check as changes sometimes come with changes to custom messages sent yjs protocol or can come with changes to yjs version itself.

Maybe we should also consider versions of YNotebook and other documents. I am not sure how easy it would be to add here.

We can get the document version
document_version = getattr(self.room._document, "version", None).
I made it optional so the connection is still allowed if the previous session didn’t store a document version.

krassowski · 2026-03-13T09:03:43Z

It seems that this PR introduces a new .jupyter directory.

It is not new in the sense that both JupyterLab Desktop and jupyter-ai use .jupyter already. I am also planning to align usage of hidden files with this pattern in jupyter-lsp It is new use in this package, yes.

I'm wondering if we could take advantage of the YStore, which is by default an SQLite database in .jupyter_ystore.db?

YStore is configurable and some deployments do not use the SQLite database variant.

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

krassowski · 2026-03-23T18:35:52Z

Hmm the tests on macos seem to hit the limit of 20 minutes:

but I did not see that on recent merge to main:

I see that the test itself "passes":

but then it hangs. This hints at a potential issue with asyncio cleanup of tasks in this PR

krassowski

It feels like AnyioPath or Locking mechanism could be to blame. I will ask copilot for a pass.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

krassowski · 2026-03-25T07:58:12Z

Hmm with the latest change the tests are failing and timing out. I guess it tells us that the locking is problematic.

Darshan808 · 2026-03-25T08:14:21Z

Not sure where things are going wrong!
These tests are stalling locally as well.

jupyter-collaboration/tests/test_documents.py

Lines 52 to 101 in 0b0dd73

    
           async def test_room_concurrent_initialization( 
        
               jp_serverapp, 
        
               rtc_create_file, 
        
               rtc_connect_doc_client, 
        
           ): 
        
               file_format = "text" 
        
               file_type = "file" 
        
               file_path = "dummy.txt" 
        
               await rtc_create_file(file_path) 
        
               async def connect(file_format, file_type, file_path): 
        
                   websocket, room_name = await rtc_connect_doc_client(file_format, file_type, file_path) 
        
                   async with websocket: 
        
                       pass 
        
               t0 = time() 
        
               async with create_task_group() as tg: 
        
                   tg.start_soon(connect, file_format, file_type, file_path) 
        
                   tg.start_soon(connect, file_format, file_type, file_path) 
        
               t1 = time() 
        
               delta = t1 - t0 
        
               assert delta < 0.6 
        
               await cleanup(jp_serverapp) 
        
           async def test_room_sequential_opening( 
        
               jp_serverapp, 
        
               rtc_create_file, 
        
               rtc_connect_doc_client, 
        
           ): 
        
               file_format = "text" 
        
               file_type = "file" 
        
               file_path = "dummy.txt" 
        
               await rtc_create_file(file_path) 
        
               async def connect(file_format, file_type, file_path): 
        
                   t0 = time() 
        
                   websocket, room_name = await rtc_connect_doc_client(file_format, file_type, file_path) 
        
                   async with websocket: 
        
                       pass 
        
                   t1 = time() 
        
                   return t1 - t0 
        
               dt = await connect(file_format, file_type, file_path) 
        
               assert dt < 1 
        
               dt = await connect(file_format, file_type, file_path) 
        
               assert dt < 1 
        
               await cleanup(jp_serverapp)

Looking into it 👀

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py

davidbrochart · 2026-03-25T09:59:44Z

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

    return encoded_path.split("/")[-1]
+
+
+async def _get_jupyter_session_store(root_dir: str) -> Path:


This function is not async. Could you remind me why we don't use async file operations?

This was changed back in 14e7ae3 because the tests are failing and timing out. It appears that AnyioPath does not play well with the event loop here. If there is an async implementation that can work here I see no reason to not use it but given that we are looking at a tiny JSON file I do not consider it blocking myself as the blocking time would be minimal, right?

(I think this might be down to interaction between AnyioPath and locking, especially problematic in pytest context)

It appears that AnyioPath does not play well with the event loop here.

Maybe because we didn't use AnyIO's pytest plugin?

Usually all file operations in a server should be async, but if you don't want to do that here then at least the added async functions that are not actually async (they don't await) should be converted back to sync.

Yes, that makes sense. @Darshan808 can you explore if anyio pytest plugin can help here and if not just drop the async?

check-for-compatibility

7fa5116

Darshan808 added the enhancement New feature or request label Feb 23, 2026

show-better-dialog

a74792f

Darshan808 requested a review from krassowski February 23, 2026 10:56

krassowski reviewed Feb 23, 2026

View reviewed changes

Darshan808 added 3 commits February 27, 2026 13:48

apply-feedbacks

df6b7ba

Merge branch 'main' into reconnect-automatically

5feab5b

run-precommit

99171b5

krassowski reviewed Mar 4, 2026

View reviewed changes

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py Outdated Show resolved Hide resolved

packages/docprovider/src/tokens.ts Outdated Show resolved Hide resolved

krassowski mentioned this pull request Mar 4, 2026

Fix error handling on corrupted files #550

Merged

2 tasks

Darshan808 and others added 4 commits March 11, 2026 09:32

Merge branch 'main' into reconnect-automatically

fd0ee27

fix-ci

ddb7b31

add-tests

57269d7

Automatic application of license header

2a8f90c

Darshan808 closed this Mar 11, 2026

Darshan808 reopened this Mar 11, 2026

Darshan808 marked this pull request as ready for review March 11, 2026 07:20

Darshan808 added 2 commits March 11, 2026 13:12

fix-lint

bc101c4

run-precommit

d205a05

krassowski reviewed Mar 11, 2026

View reviewed changes

packages/docprovider/src/__tests__/sessionClose.spec.ts Outdated Show resolved Hide resolved

krassowski requested a review from Copilot March 11, 2026 08:06

Copilot started reviewing on behalf of krassowski March 11, 2026 08:06 View session

Copilot AI reviewed Mar 11, 2026

View reviewed changes

Darshan808 added 2 commits March 11, 2026 16:16

address-feedbacks

1d08ff0

modify-tests

3469669

krassowski reviewed Mar 11, 2026

View reviewed changes

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py Outdated Show resolved Hide resolved

Darshan808 and others added 3 commits March 11, 2026 21:11

Update projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py

763ca5f

Co-authored-by: Michał Krassowski <[email protected]>

transfer-tests

ec074c5

use-sleep-directly

8797100

krassowski approved these changes Mar 11, 2026

View reviewed changes

Merge branch 'main' into reconnect-automatically

b099fcb

davidbrochart reviewed Mar 12, 2026

View reviewed changes

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py Outdated Show resolved Hide resolved

use-os.devnull

4a263bc

davidbrochart reviewed Mar 13, 2026

View reviewed changes

Darshan808 added 5 commits March 15, 2026 22:01

address-feedbacks

0030681

include-document-version-too

c38fc7d

reuse-version-mismatch-reason

4a0bb16

Merge branch 'main' into reconnect-automatically

aad8368

run-precommit

a48ea99

krassowski reviewed Mar 23, 2026

View reviewed changes

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py Outdated Show resolved Hide resolved

Remove trailing space

e3baeab

krassowski reviewed Mar 24, 2026

View reviewed changes

krassowski requested a review from Copilot March 24, 2026 09:20

Copilot started reviewing on behalf of krassowski March 24, 2026 09:20 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py Show resolved Hide resolved

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py Outdated Show resolved Hide resolved

projects/jupyter-server-ydoc/jupyter_server_ydoc/utils.py Outdated Show resolved Hide resolved

apply-feedbacks

4d91e47

use-Path-instead-of-AnyioPath

14e7ae3

krassowski reviewed Mar 25, 2026

View reviewed changes

projects/jupyter-server-ydoc/jupyter_server_ydoc/handlers.py Outdated Show resolved Hide resolved

krassowski requested a review from davidbrochart March 25, 2026 09:46

Remove commented out line

6adc1f9

davidbrochart reviewed Mar 25, 2026

View reviewed changes

		return encoded_path.split("/")[-1]


		async def _get_jupyter_session_store(root_dir: str) -> Path:

Uh oh!

Conversation

Darshan808 commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes #535

Description

User Facing Changes

Uh oh!

github-actions bot commented Feb 23, 2026

Uh oh!

krassowski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

krassowski left a comment

Choose a reason for hiding this comment

Uh oh!

krassowski commented Mar 12, 2026

Uh oh!

davidbrochart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

krassowski commented Mar 12, 2026

Uh oh!

davidbrochart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krassowski commented Mar 13, 2026

Uh oh!

Uh oh!

krassowski commented Mar 23, 2026

Uh oh!

krassowski left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

krassowski commented Mar 25, 2026

Uh oh!

Darshan808 commented Mar 25, 2026

Uh oh!

Darshan808 commented Feb 23, 2026 •

edited

Loading

krassowski Mar 25, 2026 •

edited

Loading