Add full GossipSub v1.3 support (Extensions Control Message)

[Winter-Soren]

py-libp2p currently lacks full implementation of the GossipSub v1.3 specification (Extensions Control Message). The protocol ID /meshsub/1.3.0 is present and some extension plumbing exists, but the wire format, first-message semantics, and “at most once” rules from the spec are not satisfied. This issue tracks implementing v1.3 and the Topic Observation extension (from the ethresear.ch proposal) so that py-libp2p can interoperate with other implementations and support observing nodes that receive notifications without downloading full message copies.

Background

What v1.3 specifies

• Spec: gossipsub-v1.3.md — “GossipSub v1.3: Extensions Control Message” (Candidate Recommendation, r0 2025-06-23).
• Protocol ID: /meshsub/1.3.0.
• Core mechanism: Peers advertise supported extensions via an Extensions control message so that:
  • New protocol behaviour can be added without a new protocol ID per extension.
  • Extensions can be combined and versioned independently of GossipSub.

Key spec rules:

1. First message: If a peer supports any extension, the Extensions control message MUST be included in the first message on the stream. It MUST NOT be sent more than once.
2. Protobuf: Implementations MUST use the messages defined in the spec’s extensions.proto:
   • ControlMessage has optional ControlExtensions extensions = 6 (a single message describing which extensions the sender supports).
   • ControlExtensions is a message with optional fields per extension (e.g. optional bool testExtension = 6492434).
   • Extension-specific RPC messages are optional top-level fields on RPC (e.g. optional TestExtension testExtension = 6492434).
3. Unknown extensions: Peers MUST ignore unknown extensions.
4. No negotiation: Extensions describe the sending peer’s characteristics; each peer combines both sides’ advertised sets to define behaviour (per-extension spec).

Reference implementations

• go-libp2p: PR #630 – Gossipsub Extensions (merged).
  • Router’s AddPeer(peer, protocol, helloPacket *RPC) *RPC receives the hello packet and returns the packet to send; extensions are added to that first message in extensions.AddPeer().
  • Uses ControlExtensions (single message) and tracks sentExtensions so the Extensions control message is only sent once per peer.
  • Treats a second Extensions control message from the same peer as misbehaviour (downscore).

Topic Observation extension (current implementation)

As part of this work we will implement the Topic Observation extension described in the GossipSub Topic Observation (proposed GossipSub 1.3) research post. This is not a future extension—it is in scope for the current implementation.

Motivation (from the research):

• Subscribing to a topic implies bandwidth amplification: you download or forward on the order of the mesh degree (e.g. ~D copies with mesh degree D). GossipSub 1.2's IDONTWANT reduces copies but does not guarantee exactly how many.
• Observing a topic means you do not receive full messages; you only get notifications when there is a new message (e.g. message IDs). If you need the payload, you can request it from the notifying peer once, so you download at most one copy. The actual message-request step can be defined separately; this extension deals with notifications.

Design (high level):

• Observing nodes tell subscribing peers (in the topic) that they want to observe the topic. Subscribing peers then send IHAVE to those observers when new messages arrive (IHAVE acts as a notification; observers are not expected to send IWANT in this flow).
• The relationship is unidirectional: observers receive notifications but do not send notifications or forward messages; they sit at the "border" of the network. Churn of observing nodes does not affect mesh stability.
• Protocol messages: OBSERVE (request notifications for a topic) and UNOBSERVE (stop observing). After OBSERVE, the peer sends IHAVE to the observer when there is a new message in the topic. In this use, IHAVE can be sent right after receiving a message, not only at heartbeats, and serves as a notification (no IWANT expected from the observer).

Implementing full v1.3 (Extensions Control Message) is required for Topic Observation, since it is an extension that uses the v1.3 extension mechanism. We will add both in this effort.

Current state in py-libp2p

• Protocol ID: PROTOCOL_ID_V13 = TProtocol("/meshsub/1.3.0") is defined and accepted in add_peer().
• Extension plumbing: There is an “extensions” feature check, extension_handlers, send_extension(), and handle_extension() that parse and emit a custom extension format.

Gaps vs the spec:

1. Protobuf format

   • Spec: ControlMessage has optional ControlExtensions extensions = 6 (one message with optional fields per extension). Extension RPCs are optional fields on RPC.
   • Current: libp2p/pubsub/pb/rpc.proto uses repeated ControlExtension extensions = 6 with ControlExtension having name + data (opaque bytes). This does not match the spec’s extensions.proto and prevents wire-level interoperability with implementations that use the spec format.

2. First message

   • Spec: Extensions control message must be in the first message on the stream.
   • Current: The first message is built in Pubsub.get_hello_packet() (subscriptions + senderRecord only). The router is not given the chance to add an Extensions control message to that packet, and the hello is sent without any control message. So the “first message” never carries Extensions.

3. At most once

   • Spec: Extensions control message must not be sent more than once per peer.
   • Current: There is no tracking of “already sent Extensions for this peer”; extensions can be sent later via emit_control_message() (e.g. from send_extension()), so the “at most once” rule is not enforced.

4. Misbehaviour on duplicate

   • Spec (implied): Sending Extensions more than once is invalid; implementations like go-libp2p downscore.
   • Current: Duplicate Extensions from a peer are not detected or penalised.

Proposed scope for a PR

1. Adopt spec protobuf

   • Align with the spec’s extensions.proto: introduce or mirror ControlExtensions and the RPC extension fields (e.g. TestExtension for interop). Either:
     • Integrate the spec’s extensions.proto (e.g. import or copy) and generate Python bindings, or
     • Update rpc.proto so that ControlMessage has optional ControlExtensions extensions = 6 and RPC has the same optional extension fields as the spec, keeping compatibility with existing non-extension behaviour.
   • Ensure “ignore unknown extensions” when reading ControlExtensions.

2. First message

   • Allow the router to contribute to the first message (e.g. extend the hello packet with the Extensions control message when the router supports v1.3 and has any extensions). Options:
     • Change the API so that the router’s add_peer (or a new method) receives the hello RPC and returns the RPC to send (similar to go-libp2p’s AddPeer(peer, protocol, hello) *RPC), or
     • Have Pubsub.get_hello_packet() (or the code that builds the first message) call into the router to attach Extensions when applicable.
   • Only include ControlMessage.extensions when the selected protocol is v1.3+ and the node supports at least one extension.

3. At most once

   • Track per peer whether we have already sent the Extensions control message (e.g. a set or map of peer IDs). When building the first message, add Extensions only once; for all later messages to that peer, do not add Extensions again.

4. Duplicate Extensions from peer

   • On handling an incoming RPC: if we have already recorded Extensions for this peer and the new RPC again contains control.extensions, treat as protocol violation (e.g. downscore or ignore and log). Only parse and store the peer’s extensions from the first message where they appear.

5. Topic Observation extension (current implementation)

   • Implement the Topic Observation extension from the ethresear.ch proposal: register the extension in the v1.3 extensions mechanism, add OBSERVE and UNOBSERVE control messages (or the wire format agreed for this extension), and have subscribing peers send IHAVE to observing peers when new messages arrive (as notifications). Ensure observers do not receive full message payloads unless they explicitly request them (if that path is in scope).
   • Add tests and docs for Topic Observation (observing vs subscribing, at-most-once notifications, UNOBSERVE cleanup).

References

• GossipSub v1.3 spec
• extensions.proto (spec)
• go-libp2p-pubsub PR #630 – Gossipsub Extensions
• GossipSub Topic Observation (ethresear.ch proposal, in scope for this implementation)

[Lynndabel]

I am working on this