Increase TX_QUEUE_SIZE #749
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Testing this is a bit complicated, because VM is the weakest link usually, so it requires many many VMs, or alternatively, one can send the packets out 10 times. This of course breaks communication, however in local testing works as intended.
I was able to reach 200-300 kp/s and then the error messages start appearing.
Increasing the size of TX queue by a factor of 4 makes my local tests start failing at 1 Mp/s, and in our real-life scenario makes all problems go away, only the occasional VM unable to cope with the traffic.
In real-life deployment, factor of 4 was still occasionally producing an error, factor of 8 did not. The memory allocation is not that big, so I would recommend using the larger factor for future-proofing. (130-200 MB increase in
alloc_size, but still fits into 1GiB total heap size, OSC reserves 2GiB for dpservice anyway).Increasing more is not really possible because we run into some DPDK limit, fortunately this seems to be enough.
Fixes #748