Skip to content

[v3.29] backport NetworkManagerHook#932

Open
aritrbas wants to merge 1 commit intorelease/v3.29.0from
abasu-backport-nw-mgr-hook
Open

[v3.29] backport NetworkManagerHook#932
aritrbas wants to merge 1 commit intorelease/v3.29.0from
abasu-backport-nw-mgr-hook

Conversation

@aritrbas
Copy link
Collaborator


Overview

The network hook system provides two options for network configuration at each hook point:

  1. Native Go hooks (default) - Automatically handle DNS, network services, and udev ID_NET_NAME_* restoration
  2. User scripts (optional override) - Completely replace native Go hooks for custom environments

At each hook point, EITHER the native Go hook OR a user script runs, never both.
Specialized environments (like EKS and OpenShift) use custom scripts to fully override default behavior.


Execution Flow

        ┌─────────────────────────────────┐
        │ Is CALICOVPP_HOOK_* configured? │
        └────────────┬────────────────────┘
                     │
        ┌────────────┴───────────┐
       YES                      NO
        │                        │
        ▼                        ▼
   ┌──────────┐    ┌────────────────────────────────┐
   │ Run User │    │ Is CALICOVPP_ENABLE_NETWORK_   │
   │  Script  │    │ MANAGER_HOOK = true (default)? │
   │   Only   │    └──────────────┬─────────────────┘
   └──────────┘                   │
                                  │
                         ┌────────┴───────┐
                        YES              NO
                         │                │
                         ▼                ▼
                    ┌─────────┐    ┌─────────────┐
                    │  Run    │    │     Run     │
                    │ Native  │    │default_hook │
                    │ Go Hook │    │.sh (legacy) │
                    └─────────┘    └─────────────┘

Four Scenarios:

Scenario ENABLE_HOOK HOOK_SCRIPT What Runs
1. Default (Native Go Hooks) true (default) empty (default) Native Go Hook
2. Custom Script true (default) configured User Script Only
3. Legacy Fallback false empty (default) default_hook.sh
4. Legacy Fallback + Custom Script false configured User Script Only

Hook Points

There are 5 hook points:

Hook Point When Native Go Actions User Script Env Var
BEFORE_IF_READ Early init, before VPP takes over uplink Capture host udev ID_NET_NAME_* + MAC for each uplink CALICOVPP_HOOK_BEFORE_IF_READ
BEFORE_VPP_RUN Before VPP starts Fix DNS (NetworkManager), create udev restore rules CALICOVPP_HOOK_BEFORE_VPP_RUN
VPP_RUNNING After VPP starts Restart network service (restores udev ID_NET_NAME_*) CALICOVPP_HOOK_VPP_RUNNING
VPP_DONE_OK VPP stops (success) Undo DNS fix, remove udev rules, restart network service CALICOVPP_HOOK_VPP_DONE_OK
VPP_ERRORED VPP stops (error) Same cleanup as VPP_DONE_OK CALICOVPP_HOOK_VPP_ERRORED

Optional User Hook Scripts

All default to empty (no scripts by default):

CALICOVPP_HOOK_BEFORE_IF_READ=""   # Run before interface read
CALICOVPP_HOOK_BEFORE_VPP_RUN=""   # Run before VPP starts  
CALICOVPP_HOOK_VPP_RUNNING=""      # Run after VPP starts
CALICOVPP_HOOK_VPP_DONE_OK=""      # Run when VPP stops normally
CALICOVPP_HOOK_VPP_ERRORED=""      # Run when VPP errors

To add custom logic, set the env var:

# Example: On OpenShift platform
export CALICOVPP_HOOK_VPP_RUNNING='echo \'systemctl restart NetworkManager; sleep 5; nmcli con mod ens5 ipv4.dhcp-timeout infinity\' | chroot /host'
# Example: On EKS platform
export CALICOVPP_HOOK_VPP_DONE_OK='echo \'sudo rm /etc/systemd/network/ens5.network; systemctl daemon-reload; sudo systemctl restart systemd-networkd\' | chroot /host'

Native Go Hooks

The NetworkManagerHook Go module runs when no user script is configured and automatically handles:

1. System Detection

  • Detects: systemd, systemd-networkd, NetworkManager, other network services
  • Runs once on initialization and logs the detected configuration

2. DNS Management (NetworkManager)

  • Adds dns=none to /etc/NetworkManager/NetworkManager.conf before VPP starts in BEFORE_VPP_RUN
  • Removes it when VPP stops in VPP_DONE_OK/VPP_ERRORED
  • Prevents NetworkManager's DNS management from interfering with VPP

3. Network Service Restart

  • Automatically restarts the detected network service after VPP starts (VPP_RUNNING) and after VPP stops (VPP_DONE_OK/VPP_ERRORED)
  • Runs systemctl restart systemd-udev-trigger to restore udev ID_NET_NAME_* in VPP_RUNNING or remove them in VPP_DONE_OK/VPP_ERRORED

4. udev ID_NET_NAME_* Restoration

  • Captures original host-side ID_NET_NAME_ONBOARD/SLOT/PATH/MAC and interface MAC
  • Creates /host/etc/udev/rules.d/99-vpp-restore-id_net_name.rules before VPP creates taps (BEFORE_VPP_RUN)
  • Reloads udev rules so restored properties are immediately applied on new tap events
  • Removes the rule file on VPP_DONE_OK/VPP_ERRORED, reloads rules, and triggers udev change events for cleanup.
  • Keeps systemd-networkd DHCPv6 IAID behavior stable across VPP interface transitions

Controlling Hook Behavior

To use native Go hooks (default behavior):

CALICOVPP_ENABLE_NETWORK_MANAGER_HOOK=true (already set by default)

To fallback to legacy default_hook.sh:

export CALICOVPP_ENABLE_NETWORK_MANAGER_HOOK=false

To disable udev restoration sub-feature:

# Debug config (CalicoVppDebug.enableUdevNetNameRules), default=true
enableUdevNetNameRules: false

@aritrbas aritrbas self-assigned this Feb 28, 2026
Signed-off-by: Aritra Basu <aritrbas@cisco.com>
@aritrbas aritrbas force-pushed the abasu-backport-nw-mgr-hook branch from 282f4e3 to d62e62d Compare March 1, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant