[1.17] pubsub.in-memory conformance test flaky

[sicoyle]

Run set -e
Running tests for TestPubsubConformance/pubsub/in-memory ... 
Source Package:  github.com/dapr/components-contrib/pubsub/in-memory
=== RUN   TestPubsubConformance
=== RUN   TestPubsubConformance/in-memory
=== RUN   TestPubsubConformance/in-memory/init
--- PASS: TestPubsubConformance/in-memory/init (0.00s)
=== RUN   TestPubsubConformance/in-memory/ping
--- PASS: TestPubsubConformance/in-memory/ping (0.00s)
=== RUN   TestPubsubConformance/in-memory/subscribe
--- PASS: TestPubsubConformance/in-memory/subscribe (0.00s)
=== RUN   TestPubsubConformance/in-memory/publish
time="2026-01-21T23:03:24.00240775Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
time="2026-01-21T23:03:25.002810877Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
time="2026-01-21T23:03:26.003389986Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
--- PASS: TestPubsubConformance/in-memory/publish (8.00s)
=== RUN   TestPubsubConformance/in-memory/verify_read
    pubsub.go:410: waiting for 1m0s to complete read
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-3 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-4 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-6 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-7 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-8 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-9 processed message
time="2026-01-21T23:03:27.003834529Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
    pubsub.go:425: 
        	Error Trace:	/home/runner/work/components-contrib/components-contrib/tests/conformance/pubsub/pubsub.go:425
        	Error:      	Should be empty, but was map[message-281de97d-5ec2-44e4-ac2a-92f64491701d-1:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-10:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-2:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-5:{}]
        	Test:       	TestPubsubConformance/in-memory/verify_read
        	Messages:   	expected to read 10 messages
--- FAIL: TestPubsubConformance/in-memory/verify_read (60.00s)
=== RUN   TestPubsubConformance/in-memory/multiple_handlers
=== RUN   TestPubsubConformance/in-memory/multiple_handlers/mutiple_handlers
    pubsub.go:499: waiting for 1m0s to complete read
--- PASS: TestPubsubConformance/in-memory/multiple_handlers/mutiple_handlers (0.00s)
=== RUN   TestPubsubConformance/in-memory/multiple_handlers/stop_subscribers
    pubsub.go:512: Starting iteration 0
    pubsub.go:553: Waiting for 1m0s to complete read
    pubsub.go:512: Starting iteration 1
    pubsub.go:553: Waiting for 1m0s to complete read
    pubsub.go:512: Starting iteration 2
    pubsub.go:553: Waiting for 1m0s to complete read
--- PASS: TestPubsubConformance/in-memory/multiple_handlers/stop_subscribers (10.00s)
--- PASS: TestPubsubConformance/in-memory/multiple_handlers (10.00s)
--- FAIL: TestPubsubConformance/in-memory (78.01s)
    pubsub.go:199: Simulating subscriber error
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error
--- FAIL: TestPubsubConformance (78.01s)
FAIL
coverage: 80.0% of statements in github.com/dapr/components-contrib/pubsub/in-memory
FAIL	github.com/dapr/components-contrib/tests/conformance	78.039s

=== Failed
=== FAIL: tests/conformance TestPubsubConformance/in-memory/verify_read (60.00s)
    pubsub.go:410: waiting for 1m0s to complete read
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-3 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-4 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-6 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-7 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-8 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-9 processed message
time="2026-01-21T23:03:27.003834529Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
    pubsub.go:425: 
        	Error Trace:	/home/runner/work/components-contrib/components-contrib/tests/conformance/pubsub/pubsub.go:425
        	Error:      	Should be empty, but was map[message-281de97d-5ec2-44e4-ac2a-92f64491701d-1:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-10:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-2:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-5:{}]
        	Test:       	TestPubsubConformance/in-memory/verify_read
        	Messages:   	expected to read 10 messages

=== FAIL: tests/conformance TestPubsubConformance/in-memory (78.01s)
    pubsub.go:199: Simulating subscriber error
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error

Expected Behavior

test should pass

Actual Behavior

test fails

Steps to Reproduce the Problem

run ci

Release Note

RELEASE NOTE: FIX: flakey pubsub in mem conformance test

[sicoyle]

also

Run set -e
Running tests for TestPubsubConformance/pubsub/in-memory ... 
Source Package:  github.com/dapr/components-contrib/pubsub/in-memory
=== RUN   TestPubsubConformance
=== RUN   TestPubsubConformance/in-memory
=== RUN   TestPubsubConformance/in-memory/init
--- PASS: TestPubsubConformance/in-memory/init (0.00s)
=== RUN   TestPubsubConformance/in-memory/ping
--- PASS: TestPubsubConformance/in-memory/ping (0.00s)
=== RUN   TestPubsubConformance/in-memory/subscribe
--- PASS: TestPubsubConformance/in-memory/subscribe (0.00s)
=== RUN   TestPubsubConformance/in-memory/publish
time="2026-01-21T23:03:24.00240775Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
time="2026-01-21T23:03:25.002810877Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
time="2026-01-21T23:03:26.003389986Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
--- PASS: TestPubsubConformance/in-memory/publish (8.00s)
=== RUN   TestPubsubConformance/in-memory/verify_read
    pubsub.go:410: waiting for 1m0s to complete read
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-3 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-4 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-6 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-7 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-8 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-9 processed message
time="2026-01-21T23:03:27.003834529Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
    pubsub.go:425: 
        	Error Trace:	/home/runner/work/components-contrib/components-contrib/tests/conformance/pubsub/pubsub.go:425
        	Error:      	Should be empty, but was map[message-281de97d-5ec2-44e4-ac2a-92f64491701d-1:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-10:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-2:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-5:{}]
        	Test:       	TestPubsubConformance/in-memory/verify_read
        	Messages:   	expected to read 10 messages
--- FAIL: TestPubsubConformance/in-memory/verify_read (60.00s)
=== RUN   TestPubsubConformance/in-memory/multiple_handlers
=== RUN   TestPubsubConformance/in-memory/multiple_handlers/mutiple_handlers
    pubsub.go:499: waiting for 1m0s to complete read
--- PASS: TestPubsubConformance/in-memory/multiple_handlers/mutiple_handlers (0.00s)
=== RUN   TestPubsubConformance/in-memory/multiple_handlers/stop_subscribers
    pubsub.go:512: Starting iteration 0
    pubsub.go:553: Waiting for 1m0s to complete read
    pubsub.go:512: Starting iteration 1
    pubsub.go:553: Waiting for 1m0s to complete read
    pubsub.go:512: Starting iteration 2
    pubsub.go:553: Waiting for 1m0s to complete read
--- PASS: TestPubsubConformance/in-memory/multiple_handlers/stop_subscribers (10.00s)
--- PASS: TestPubsubConformance/in-memory/multiple_handlers (10.00s)
--- FAIL: TestPubsubConformance/in-memory (78.01s)
    pubsub.go:199: Simulating subscriber error
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error
--- FAIL: TestPubsubConformance (78.01s)
FAIL
coverage: 80.0% of statements in github.com/dapr/components-contrib/pubsub/in-memory
FAIL	github.com/dapr/components-contrib/tests/conformance	78.039s

=== Failed
=== FAIL: tests/conformance TestPubsubConformance/in-memory/verify_read (60.00s)
    pubsub.go:410: waiting for 1m0s to complete read
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-3 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-4 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-6 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-7 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-8 processed message
    pubsub.go:416: deleting message-281de97d-5ec2-44e4-ac2a-92f64491701d-9 processed message
time="2026-01-21T23:03:27.003834529Z" level=error msg="conf test simulated error" instance=runnervmvatxw scope=testLogger type=log ver=unknown
    pubsub.go:425: 
        	Error Trace:	/home/runner/work/components-contrib/components-contrib/tests/conformance/pubsub/pubsub.go:425
        	Error:      	Should be empty, but was map[message-281de97d-5ec2-44e4-ac2a-92f64491701d-1:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-10:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-2:{} message-281de97d-5ec2-44e4-ac2a-92f64491701d-5:{}]
        	Test:       	TestPubsubConformance/in-memory/verify_read
        	Messages:   	expected to read 10 messages

=== FAIL: tests/conformance TestPubsubConformance/in-memory (78.01s)
    pubsub.go:199: Simulating subscriber error
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:203: Simulating subscriber success
    pubsub.go:199: Simulating subscriber error

=== FAIL: tests/conformance TestPubsubConformance (78.01s)

DONE 10 tests, 3 failures in 244.181s
Completed tests for TestPubsubConformance/pubsub/in-memory ... 
Setting CONFORMANCE_FAILURE

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions.