-
Notifications
You must be signed in to change notification settings - Fork 5.4k
[Regression]: Specific tests are failing locally on Playwright version 1.59.0 #40009
Copy link
Copy link
Open
Labels
Description
Version
1.59.0
Steps to reproduce
- Update Playwright dependency to the latest via
yarn add @playwright/test@latest - Update Playwright browser/dependencies to the latest via
yarn playwright install - Run tests that were passing in the previous version ``
- See failing tests
Expected behavior
Subset of tests should pass as expected with version 1.59.0
Actual behavior
Subset of tests are failing as expected with version 1.59.0
➜ desktop git:(main) ✗ yarn playwright:web:test test/playwright/tests/web/share_availability.spec.ts --retries=0
yarn run v1.22.22
$ npx playwright test --config=playwright.web.config.ts test/playwright/tests/web/share_availability.spec.ts --retries=0
$ TEST_PORT=8081 NODE_ENV=test APP_ENV=test SKIP_TYPE_CHECK=true MANIFEST_V3=true PLAYWRIGHT=true node --no-deprecation server
[WebServer] VERSION:2026-04-01T17:35:12.687Z
[WebServer] EXTENSION_VERSION:3.1.60401.1735
[WebServer] GIT_BRANCH:undefined
[WebServer] GIT_REVISION:undefined
[WebServer] [HPM] Proxy created: [Function: path] -> http://localhost:6001
[WebServer] ℹ 「wds」: Project is running at http://localhost:8081/webpack-dev-server/
[WebServer] ℹ 「wds」: webpack output is served from http://localhost:8081/~backend/build/
[WebServer] ℹ 「wds」: Content not from webpack is served from /~backend/build
[WebServer] ℹ 「wds」: 404s will fallback to /index.html
[WebServer] Listening at http://localhost:8081
[WebServer] ⚡️ Starting Webpack Server... ⚡️
[WebServer] ℹ 「wdm」: wait until bundle finished: /~backend/build/index.html
[WebServer]
[WebServer] Rebuilding extension/manifest.json
[WebServer] ℹ 「wdm」: Time: 16652ms
[WebServer] ℹ 「wdm」: Compiled successfully.
Running 6 tests using 6 workers
✘ 1 … test/playwright/tests/web/share_availability.spec.ts:72:11 › Share availability › (google) › time slot selection › should not let a user start dragging from a time in the past (30.1s)
✘ 2 [chromium] › test/playwright/tests/web/share_availability.spec.ts:99:11 › Share availability › (google) › sidebar › clicking Cancel closes the sidebar (30.1s)
✘ 3 [chromium] › test/playwright/tests/web/share_availability.spec.ts:104:11 › Share availability › (google) › sidebar › typing in title does not trigger global shortcuts (30.1s)
✘ 4 …m] › test/playwright/tests/web/share_availability.spec.ts:115:11 › Share availability › (google) › sidebar › Tab moves focus from title through sidebar fields to footer buttons (30.1s)
✘ 5 [chromium] › test/playwright/tests/web/share_availability.spec.ts:94:11 › Share availability › (google) › sidebar › Escape closes the sidebar (30.1s)
✘ 6 [chromium] › test/playwright/tests/web/share_availability.spec.ts:85:11 › Share availability › (google) › sidebar › title input is focused when the sidebar opens (30.1s)
1) [chromium] › test/playwright/tests/web/share_availability.spec.ts:72:11 › Share availability › (google) › time slot selection › should not let a user start dragging from a time in the past
Test timeout of 30000ms exceeded while running "beforeEach" hook.
60 |
61 | describe('time slot selection', () => {
> 62 | beforeEach(async ({ page, pageHelper, commandHelper, flagsHelper }) => {
| ^
63 | await flagsHelper.disable(FlagNames.RECURRING_AVAILABILITY)
64 | await shareAvailabilityOnNewDraftByProvider({
65 | provider: 'google',
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:62:17
Error: locator.waitFor: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.Command') to be visible
at ../../pages/command_palette_page.ts:15
13 | await this._page.keyboard.press(key)
14 |
> 15 | await this._page.locator('.Command').waitFor({ state: 'visible' })
| ^
16 |
17 | return this.in()
18 | }
at CommandPalettePage.open (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:15:42)
at CommandPalettePage.run (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:21:5)
at shareAvailabilityOnNewDraftByProvider (/Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:221:3)
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:64:9
attachment #1: screenshot (image/png) ──────────────────────────────────────────────────────────
playwright-test-results/share_availability-Share-a-dd035-ing-from-a-time-in-the-past-chromium/test-failed-1.png
────────────────────────────────────────────────────────────────────────────────────────────────
Error Context: playwright-test-results/share_availability-Share-a-dd035-ing-from-a-time-in-the-past-chromium/error-context.md
2) [chromium] › test/playwright/tests/web/share_availability.spec.ts:85:11 › Share availability › (google) › sidebar › title input is focused when the sidebar opens
Test timeout of 30000ms exceeded while running "beforeEach" hook.
79 |
80 | describe('sidebar', () => {
> 81 | beforeEach(async ({ page, pageHelper, commandHelper, flagsHelper }) => {
| ^
82 | await openShareAvailabilitySidebar({ pageHelper, page, commandHelper, flagsHelper })
83 | })
84 |
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:81:17
Error: locator.waitFor: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.Command') to be visible
at ../../pages/command_palette_page.ts:15
13 | await this._page.keyboard.press(key)
14 |
> 15 | await this._page.locator('.Command').waitFor({ state: 'visible' })
| ^
16 |
17 | return this.in()
18 | }
at CommandPalettePage.open (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:15:42)
at CommandPalettePage.run (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:21:5)
at openShareAvailabilitySidebar (/Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:251:3)
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:82:9
attachment #1: screenshot (image/png) ──────────────────────────────────────────────────────────
playwright-test-results/share_availability-Share-a-0682b-used-when-the-sidebar-opens-chromium/test-failed-1.png
────────────────────────────────────────────────────────────────────────────────────────────────
Error Context: playwright-test-results/share_availability-Share-a-0682b-used-when-the-sidebar-opens-chromium/error-context.md
3) [chromium] › test/playwright/tests/web/share_availability.spec.ts:94:11 › Share availability › (google) › sidebar › Escape closes the sidebar
Test timeout of 30000ms exceeded while running "beforeEach" hook.
79 |
80 | describe('sidebar', () => {
> 81 | beforeEach(async ({ page, pageHelper, commandHelper, flagsHelper }) => {
| ^
82 | await openShareAvailabilitySidebar({ pageHelper, page, commandHelper, flagsHelper })
83 | })
84 |
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:81:17
Error: locator.waitFor: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.Command') to be visible
at ../../pages/command_palette_page.ts:15
13 | await this._page.keyboard.press(key)
14 |
> 15 | await this._page.locator('.Command').waitFor({ state: 'visible' })
| ^
16 |
17 | return this.in()
18 | }
at CommandPalettePage.open (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:15:42)
at CommandPalettePage.run (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:21:5)
at openShareAvailabilitySidebar (/Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:251:3)
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:82:9
attachment #1: screenshot (image/png) ──────────────────────────────────────────────────────────
playwright-test-results/share_availability-Share-a-c9151-r-Escape-closes-the-sidebar-chromium/test-failed-1.png
────────────────────────────────────────────────────────────────────────────────────────────────
Error Context: playwright-test-results/share_availability-Share-a-c9151-r-Escape-closes-the-sidebar-chromium/error-context.md
4) [chromium] › test/playwright/tests/web/share_availability.spec.ts:99:11 › Share availability › (google) › sidebar › clicking Cancel closes the sidebar
Test timeout of 30000ms exceeded while running "beforeEach" hook.
79 |
80 | describe('sidebar', () => {
> 81 | beforeEach(async ({ page, pageHelper, commandHelper, flagsHelper }) => {
| ^
82 | await openShareAvailabilitySidebar({ pageHelper, page, commandHelper, flagsHelper })
83 | })
84 |
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:81:17
Error: locator.waitFor: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.Command') to be visible
at ../../pages/command_palette_page.ts:15
13 | await this._page.keyboard.press(key)
14 |
> 15 | await this._page.locator('.Command').waitFor({ state: 'visible' })
| ^
16 |
17 | return this.in()
18 | }
at CommandPalettePage.open (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:15:42)
at CommandPalettePage.run (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:21:5)
at openShareAvailabilitySidebar (/Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:251:3)
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:82:9
attachment #1: screenshot (image/png) ──────────────────────────────────────────────────────────
playwright-test-results/share_availability-Share-a-95aba-g-Cancel-closes-the-sidebar-chromium/test-failed-1.png
────────────────────────────────────────────────────────────────────────────────────────────────
Error Context: playwright-test-results/share_availability-Share-a-95aba-g-Cancel-closes-the-sidebar-chromium/error-context.md
5) [chromium] › test/playwright/tests/web/share_availability.spec.ts:104:11 › Share availability › (google) › sidebar › typing in title does not trigger global shortcuts
Test timeout of 30000ms exceeded while running "beforeEach" hook.
79 |
80 | describe('sidebar', () => {
> 81 | beforeEach(async ({ page, pageHelper, commandHelper, flagsHelper }) => {
| ^
82 | await openShareAvailabilitySidebar({ pageHelper, page, commandHelper, flagsHelper })
83 | })
84 |
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:81:17
Error: locator.waitFor: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.Command') to be visible
at ../../pages/command_palette_page.ts:15
13 | await this._page.keyboard.press(key)
14 |
> 15 | await this._page.locator('.Command').waitFor({ state: 'visible' })
| ^
16 |
17 | return this.in()
18 | }
at CommandPalettePage.open (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:15:42)
at CommandPalettePage.run (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:21:5)
at openShareAvailabilitySidebar (/Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:251:3)
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:82:9
attachment #1: screenshot (image/png) ──────────────────────────────────────────────────────────
playwright-test-results/share_availability-Share-a-af914-ot-trigger-global-shortcuts-chromium/test-failed-1.png
────────────────────────────────────────────────────────────────────────────────────────────────
Error Context: playwright-test-results/share_availability-Share-a-af914-ot-trigger-global-shortcuts-chromium/error-context.md
6) [chromium] › test/playwright/tests/web/share_availability.spec.ts:115:11 › Share availability › (google) › sidebar › Tab moves focus from title through sidebar fields to footer buttons
Test timeout of 30000ms exceeded while running "beforeEach" hook.
79 |
80 | describe('sidebar', () => {
> 81 | beforeEach(async ({ page, pageHelper, commandHelper, flagsHelper }) => {
| ^
82 | await openShareAvailabilitySidebar({ pageHelper, page, commandHelper, flagsHelper })
83 | })
84 |
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:81:17
Error: locator.waitFor: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.Command') to be visible
at ../../pages/command_palette_page.ts:15
13 | await this._page.keyboard.press(key)
14 |
> 15 | await this._page.locator('.Command').waitFor({ state: 'visible' })
| ^
16 |
17 | return this.in()
18 | }
at CommandPalettePage.open (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:15:42)
at CommandPalettePage.run (/Users/mahtab/superhuman/desktop/test/playwright/pages/command_palette_page.ts:21:5)
at openShareAvailabilitySidebar (/Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:251:3)
at /Users/mahtab/superhuman/desktop/test/playwright/tests/web/share_availability.spec.ts:82:9
attachment #1: screenshot (image/png) ──────────────────────────────────────────────────────────
playwright-test-results/share_availability-Share-a-ba853-ar-fields-to-footer-buttons-chromium/test-failed-1.png
────────────────────────────────────────────────────────────────────────────────────────────────
Error Context: playwright-test-results/share_availability-Share-a-ba853-ar-fields-to-footer-buttons-chromium/error-context.md
6 failed
[chromium] › test/playwright/tests/web/share_availability.spec.ts:72:11 › Share availability › (google) › time slot selection › should not let a user start dragging from a time in the past
[chromium] › test/playwright/tests/web/share_availability.spec.ts:85:11 › Share availability › (google) › sidebar › title input is focused when the sidebar opens
[chromium] › test/playwright/tests/web/share_availability.spec.ts:94:11 › Share availability › (google) › sidebar › Escape closes the sidebar
[chromium] › test/playwright/tests/web/share_availability.spec.ts:99:11 › Share availability › (google) › sidebar › clicking Cancel closes the sidebar
[chromium] › test/playwright/tests/web/share_availability.spec.ts:104:11 › Share availability › (google) › sidebar › typing in title does not trigger global shortcuts
[chromium] › test/playwright/tests/web/share_availability.spec.ts:115:11 › Share availability › (google) › sidebar › Tab moves focus from title through sidebar fields to footer buttons Additional context
I uploaded a loom explaining the situation in more detail:
https://www.loom.com/share/b4e8c9db3c2d4135bd222fb2101457ef
This issue is presenting itself in local environment, I couldn't reproduce it in our CI environment in our semaphore CI environment. See screenshots passing in the CI as well as CI env info
Environment
System:
OS: macOS 26.4
CPU: (16) arm64 Apple M4 Max
Memory: 52.41 GB / 128.00 GB
Binaries:
Node: 20.19.6 - ~/.nvm/versions/node/v20.19.6/bin/node
Yarn: 1.22.22 - ~/.nvm/versions/node/v20.19.6/bin/yarn
npm: 10.8.2 - ~/.nvm/versions/node/v20.19.6/bin/npm
IDEs:
VSCode: 1.114.0 - /usr/local/bin/code
Languages:
Bash: 3.2.57 - /bin/bash
npmPackages:
@playwright/test: 1.59.0 => 1.59.0Reactions are currently unavailable