Skip to content

Update integration tests#1450

Open
psilabs-dev wants to merge 4 commits intoDifegue:devfrom
psilabs-dev:dev-integration-tests
Open

Update integration tests#1450
psilabs-dev wants to merge 4 commits intoDifegue:devfrom
psilabs-dev:dev-integration-tests

Conversation

@psilabs-dev
Copy link
Contributor

No new tests planned atm, instead I've stabilized the existing test cases and make conditions stricter.

  • Redis/docker connections will be explicitly closed on teardown when applicable.

The following updates apply to all Playwright UI tests:

  • All tests now explicitly create a browser context, and explicitly close context/browser within the scope of a test case.
  • All tests assert no errors when checking browser network inspector. This means that a UI test case will fail if the network inspector finds any LRR-side status code that isn't sub-400 or 401. Non-LRR origin URLs are excluded (e.g. Github rate limit errors will not trigger test case failure). This is a stricter client-side condition than expecting no server-side error logs, as 4xx validation errors and silent, server-side error responses will fail the test case. This will address a subset of upcoming OpenAPI integration testing failures, including the missing "/search" endpoint issue that escaped testing from earlier.

Log health assertions are occasionally capturing Shinobu error logs ("giving up adding it to the filesystem"); this is an existing race condition which can be ignored, or we can suppress it in the future test-side or LRR-side.

@psilabs-dev psilabs-dev force-pushed the dev-integration-tests branch from 7d1b138 to 1f09287 Compare January 23, 2026 05:12
@psilabs-dev
Copy link
Contributor Author

Added a logic fix: previous docker deployment integration test used essentially vanilla redis.conf configuration; this is now fixed so that we will load the LRR redis config into the redis container instead.

  • In the scenario that we are testing a docker image: we will actually create an ephemeral container, pull the conf file out and use it to operate our test-time redis instance.

Added client-side optimizations. Mainly moved everything (including all of my other Python projects) over to uv; it (by it I mean my build times) is generally 20-30% faster. It's been pretty good.

Another client-side optimization is the request-handling, previously validation was unnecessarily done in Python, now Pydantic (with rust core) handles it, which also made request handling faster (though it's not a big improvement considering it's not a bottleneck in testing).

And some windows test-side improvements, such as not spawning console windows, which (heuristically) have alleviated the test pressure problems?

Unrelated: tests can be run with a different dockerfile now, if you ever need that for some fork-side reason.

@psilabs-dev psilabs-dev marked this pull request as draft February 26, 2026 06:55
@psilabs-dev psilabs-dev force-pushed the dev-integration-tests branch from 31d5186 to e07cd4b Compare March 6, 2026 01:21
@psilabs-dev psilabs-dev marked this pull request as ready for review March 6, 2026 05:44
@psilabs-dev
Copy link
Contributor Author

Thanks to some clankRCA™, instrumentation-spamming, hypothesis testing and process of elimination, the cause of Windows test-side failures and slowness were (practically) identified; see psilabs-dev/aio-lanraragi#173 and psilabs-dev/aio-lanraragi#172.

But anyways, the tests (ignoring Shinobu) should be more stable and faster now.

Also added/extended test coverage:

  • test_category includes a regression behavior check against OpenAPI URL encoding
  • added a double page navigation behavioral test case
  • added a reader-mode resource handler management test case
  • test_tankoubon includes similar regression check as test_category
  • added two local progress-related test cases, previously uncovered
  • added a basic plugin functionality test case
  • a plugin not available test case
  • an invalid plugin type test case (OpenAPI regression)

Added various other tests which don't run by default (OpenAPI feature + bypass, preloading check, etc.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant