Adding multiple (~100000) routes via NDB is slow

[synclpz]

Using pyroute2==0.9.5 trying to add multiple routes:

with ndb.begin() as tx:
        for p in px.getCidrs():
            tx.push(ndb.routes.create(dst=p, gateway="192.168.191.1", table=101)) 

I end up this loop adding like ~1000 routes in 12 minutes (I see them hitting ip route show table 101 despite transaction is not yet committed).
I need to set up ~100 000 routes, thus it might take ~20 hours, which is unacceptable.

Is it something I miss in config/code or it's just intended to be this slow?

[svinota]

It tries to run one transaction per route, which is not a good idea.

Let me see if we can merge these in one transaction, and maybe even use less messages.

[svinota]

A kind of an optimization you can use for now:

for p in px.getCidrs():
    ndb.routes.create(dst=p, gateway="192.168.191.1", table=101).commit()

In this case, all the routes will be committed separately, no common rollback.

But it works significantly faster.

A proper fix is being implemented now.

[synclpz]

Yep, I wanted to make one transaction to update all routes at once, because they should be updated hourly. And the idea was to delete all routes from table 101, then add new routes and then commit to atomically switch routing rules on host.

I found out that this is not the best way to achieve split routing so I ended up making a file with list of prefixes and the load them into IP list of nftables, it takes less than second for 100 000 prefixes. Also nftables allows to do ip list swap so I can atomically replace one list with another.