Skip to content

Clean known_hosts to avoid SSH host key conflicts in dynamic environments #3000

@BurlyLuo

Description

@BurlyLuo
[root@rowan> 01-rdma-env]# cat 1-setup-env.sh 
cat <<EOF>clab.yaml | clab deploy -t clab.yaml -
name: rdma
prefix: ""
topology:
  nodes:
    r1:
      kind: generic_vm
      image: 192.168.2.100:5000/debian13
      env:
        QEMU_SMP: 6
        QEMU_MEMORY: 8192

    r2:
      kind: generic_vm
      image: 192.168.2.100:5000/debian13
      env:
        QEMU_SMP: 6
        QEMU_MEMORY: 8192

  links:
    - endpoints: ["r1:eth1", "r2:eth1"]
EOF

USER="root"
PASSWD="hive"
NODES=("r1" "r2")
IF_NAME="ens2"

wait_for_ready() {
    local container=$1
    local max_attempts=20
    local attempt=1
    
    while [ $attempt -le $max_attempts ]; do
        health_status=$(docker inspect --format='{{.State.Health.Status}}' "$container" 2>/dev/null)
        echo "[$(date '+%Y-%m-%d %H:%M:%S')] [$container] Waiting for ready... (status: $health_status)"
        if [ "$health_status" = "healthy" ]; then
            echo "[$(date '+%Y-%m-%d %H:%M:%S')] [$container] is healthy."
            return 0
        fi
        sleep 10
        ((attempt++))
    done
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] [$container] did not become healthy after $max_attempts attempts."
    return 1
}

declare -A config_cmds=(
    ["r1"]="apt update
apt -y install libibverbs1 ibverbs-utils librdmacm1 libibumad3 ibverbs-providers rdma-core net-tools iproute2 perftest 
modprobe rdma_rxe
ip a a 10.1.5.10/24 dev $IF_NAME
ip l s $IF_NAME up
rdma link add rxe1 type rxe netdev $IF_NAME
ibv_devinfo -d rxe1
"

    ["r2"]="apt update
apt -y install libibverbs1 ibverbs-utils librdmacm1 libibumad3 ibverbs-providers rdma-core net-tools iproute2 perftest
modprobe rdma_rxe
ip a a 10.1.5.11/24 dev $IF_NAME
ip l s $IF_NAME up
rdma link add rxe1 type rxe netdev $IF_NAME
ibv_devinfo -d rxe1
"
)

for node in "${NODES[@]}"; do
    container="$node"
    if wait_for_ready "$container"; then
        echo "[$(date '+%Y-%m-%d %H:%M:%S')] Applying configuration for $container..."
        ssh-keygen -f "/root/.ssh/known_hosts" -R "$container"
        if sshpass -p "$PASSWD" ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 "$USER@$container" "
            set -x
            ${config_cmds[$node]}
        " 2>&1; then
            echo "$node Apply configuration success"
        else
            echo "$node Apply configuration failed"
        fi

        sshpass -p "$PASSWD" ssh-copy-id "$USER@$container"
    fi
done
[root@rowan> 01-rdma-env]# 

if redeploy same evn. will meet this issue:

        ssh-keygen -f "/root/.ssh/known_hosts" -R "$container"
[2026-01-21 09:54:35] Applying configuration for r1...
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:/DkC/kVl8gxs0cgVB2oDp2Lg9btuBuizBwcN7882opk.
Please contact your system administrator.
Add correct host key in /root/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /root/.ssh/known_hosts:23
  remove with:
  ssh-keygen -f "/root/.ssh/known_hosts" -R "r1"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions