Unable to bind to an ip address on a virtual, tap, interface

[rigdent]

Hello,

I am having an issue with an app that uses JGroups binding to an ip address on a virtual, tap, interface.
I get the error

java.net.BindException: [TCP_NIO2] /<ip address> is not a valid address on any local network interface
        at org.jgroups.util.Util.checkIfValidAddress(Util.java:4415) ~[jgroups.jar:?]

Is there any issues w/ JGroups binding to virtual tap interfaces?

I can bind to that ip address fine using netcat, nc, fine

The only thing I could see is maybe

It seems to fail the checkIfValidAddress() call which I see calls getAllAvailableAddresses()
https://github.com/belaban/JGroups/blob/master/src/org/jgroups/util/Util.java#L4901

And I see that getAllAvailableAddresses() checks to see if an address is up or not before including it
https://github.com/belaban/JGroups/blob/master/src/org/jgroups/util/Util.java#L4887

But ip addr shows that that ip's interface is marked state down.

10: tap0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc fq_codel state DOWN group default qlen 1000

I wasn't able to determine how isUp() really checks if something is up or not. Seems like always true if no socket exception? But unclear.
https://github.com/belaban/JGroups/blob/master/src/org/jgroups/util/Util.java#L4755

But since netcat can bind to it, it seems like the interface is fine.

Any thoughts would be appreciated. Thanks

[rigdent]

It seems the state DOWN was the issue even though the interface is marked UP in ifconfig and in 10: tap0: <NO-CARRIER,BROADCAST,MULTICAST,UP>

Is that something that can be changed? nc is able to bind fine on the state DOWN interface.

[nsturmwind]

The infrastructure we have essentially uses a tap interface as a bridge. The tap interface hosts the primary IP address for the server. BIRD is used to set up routes between the tap interface and the physical ethernet interfaces.

Per https://dshcherb.github.io/2016/05/10/libvirt-virbr0-nic-tap-interface.html this seems to be an acknowledged limitation of using tap interfaces as bridges without a user space program holding a file descriptor for the tap device. (We are not using libvirt, but the effect is the same.)

Java considers an interface up if it has flags IFF_UP (administrative state) and IFF_RUNNING (link level carrier state) (NetworkInterface.c).

This state can be reproduced (without bothering with the routes) with the following:

Dockerfile:

FROM registry.access.redhat.com/ubi8/ubi:latest

RUN dnf install -y \
    iproute \
    iptables \
    iputils \
    && dnf clean all

COPY setup-routing.sh /usr/local/bin/setup-routing.sh
RUN chmod +x /usr/local/bin/setup-routing.sh
ENTRYPOINT ["/usr/local/bin/setup-routing.sh"]

setup-routing.sh:

#!/bin/bash
ip tuntap add dev tap0 mode tap
ip addr add 10.0.1.1/24 dev tap0
ip link set tap0 up
ip addr show

docker build -t tap . && docker run --cap-add=NET_ADMIN --device /dev/net/tun --sysctl net.ipv4.ip_forward=1 tap

...

12: tap0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 22:46:26:0b:81:1f brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.1/24 scope global tap0
       valid_lft forever preferred_lft forever

What are your thoughts of extending JGroups to support interfaces that are administratively up but have no carrier to support cases like this?

[belaban]

I can add a flag (attribute in the transport, or system property) which overrides the down check of an interface. I don't want to remove the check because it reduces the risk of binding to NICs which are down. If you set that flag to true, it means you know what you're doing.
OK?

[nsturmwind]

That makes sense to me. Would it be possible to specify this as part of the tcp.xml config?

[belaban]

It's probably simpler to define a system property: an attribute in the transport is initialized much later in the process, when the transport has been created.

[nsturmwind]

Got it.

(We use JGroups as part of commercial third-party software, so we will need to work with them to expose this setting. I was thinking of tcp.xml because that is something that they already allow to be customized.)

[belaban]

If you can replace the version of JGroups used in the commercial product, then it doesn't matter that you use a system variable; JGroups will recognize it and not skip interfaces which are down.
Which version of JGroups is currently used?
I know that some vendors forbid changing/upgrading a component such as JGroups, or else all support is void...

[belaban]

If we go down the route of using a system property (or env variable), the question is whether we should skip the down check for all interfaces which are down, or whether you want to set a set of interfaces where the down check should be skipped.
Alternative: skip the down check only on virtual interfaces.

[nsturmwind]

The software is using JGroups 4.2.20.Final.

I think it's probably better to have a feature where only specific interfaces would have the down check overridden, to avoid unintended impact.

I don't think checking for virtual interfaces will work. From what I found, Java on *nix considers an interface virtual only if has a colon in its name, e.g. eth0:1 (NetworkInterface.c) so tap interfaces are not virtual. Testing confirms that NetworkInterface.isVirtual() returns false for tap interfaces.

[belaban]

4.2.20 is quite old... Usually I don't support such an old version. I'm making an exception in this case, but this is a one time backport from master (5.5.5). I'll create a JIRA shortly.
Do we agree that the system property will look like this?

-Dskip_interfaces_down_check=tap0,eth7

Need to find a better name. Also, I'm thinking of using regexps for the interface list. Should we also allow for IP addresses there?

[nsturmwind]

It's probably not worth adding this to 4.2.20, I assume the vendor product has upgraded since then (we are not yet on their newest version). I can find out which JGroups version they are using now. I'll also have to check with them on the structure for the system property. Hopefully one of their developers can join this conversation.

I don't think we have use case for IP addresses.

[belaban]

The last 4.2.x release is 4.2.31, so the backport would be to 4.2.32. Even the better if they used a 5.x release...

[nsturmwind]

From what I can tell, the latest version of the vendor product is using 4.2.28.Final (Julier)

[belaban]

I created https://redhat.atlassian.net/browse/JGRP-2994 to keep track of this

[belaban]

I fixed this on the master and 4.x branches. Please create a JAR from 4.x, replace the vendor's JAR and set -Dskip-up-check-for=NIC1,NIC2 where the elements can be regular expressions, e.g. "tap.*,utun1".
Let me know if this works, then I can release 4.2.32 (if needed).

[belaban]

Any news on this? @nsturmwind have you tested that this works?

[nsturmwind]

We'll need a couple of weeks to test this, we are working with the vendor to ensure we aren't violating any contract terms by swapping out the jar. Sent from Bloomberg Professional for Android

…

----- Original Message ----- From: Bela Ban ***@***.***> To: ***@***.*** CC: NOAM STURMWIND, ***@***.*** At: 03/26/26 07:46:20 UTC-04:00 Any news on this? @nsturmwind have you tested that this works? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[belaban]

Ok, let me know when you have resultsVon meinem/meiner Galaxy gesendet -------- Ursprüngliche Nachricht --------Von: Noam Sturmwind ***@***.***> Datum: 26.03.26 14:21 (GMT+01:00) An: belaban/JGroups ***@***.***> Cc: Bela Ban ***@***.***>, Comment ***@***.***> Betreff: Re: [belaban/JGroups] Unable to bind to an ip address on a virtual, tap, interface (Discussion #897) We'll need a couple of weeks to test this, we are working with the vendor to ensure we aren't violating any contract terms by swapping out the jar. Sent from Bloomberg Professional for Android

…

----- Original Message ----- From: Bela Ban ***@***.***> To: ***@***.*** CC: NOAM STURMWIND, ***@***.*** At: 03/26/26 07:46:20 UTC-04:00 Any news on this? @nsturmwind have you tested that this works? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***> —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>