Multithreading optimization

[master255]

@arvidn I found a lot of std::thread in the code.

Creating a thread is a very expensive command. You need to optimize the creation of all threads (except for disk_io_thread_pool) using thread_pool.

Like this:

ThreadPool.hpp:
#pragma once
#include <boost/asio/thread_pool.hpp>

extern boost::asio::thread_pool mainPool;

ThreadPool.cpp:
unsigned n = std::thread::hardware_concurrency();
if (n == 0) n = 4; 

boost::asio::thread_pool pool(n);

And after:

#include "ThreadPool.hpp"
#include <boost/asio/post.hpp>

void doTask() {
    boost::asio::post(pool, []{
        ....
    });
}

This can significantly increase the speed of the library. And any of your programs.

[xavier2k6]

@master255 why not just create a Pull Request?

[master255]

@xavier2k6 I am an application programmer, not a library programmer. And as practice has shown, Arvin is afraid to accept my pull requests.

[arvidn]

Creating a thread is a very expensive command.

Do you have any evidence of this?

You need to optimize the creation of all threads

Have you measured how often threads are created?

[master255]

@arvidn Use this code to get results:

#include <boost/asio/thread_pool.hpp>
#include <boost/asio/post.hpp>
#include <chrono>
#include <iostream>
#include <thread>

using bench_clock = std::chrono::high_resolution_clock;

void bench_std_thread(int iters) {
    auto start = bench_clock::now();

    for (int i = 0; i < iters; ++i) {
        std::thread([]{}).join();
    }

    auto end = bench_clock::now();
    auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count();

    std::cout << "std::thread avg ns = " << (ns / iters) << std::endl;
}

void bench_asio_post(int iters) {
    boost::asio::thread_pool pool(1);

    auto start = bench_clock::now();

    for (int i = 0; i < iters; ++i) {
        boost::asio::post(pool, [] {});
    }

    pool.join();

    auto end = bench_clock::now();
    auto ns = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count();

    std::cout << "asio::post avg ns = " << (ns / iters) << std::endl;
}

int main() {
    const int iters = 100000;

    bench_std_thread(1000);
    bench_asio_post(1000);

    bench_std_thread(iters);
    bench_asio_post(iters);
}

Results:
std::thread avg ns = 110819
asio::post avg ns = 406

Because std::thread are processor threads. They are always expensive. All other languages use multithreading optimizations for this.

Have you measured how often threads are created?

It doesn't matter. Because delays will be noticeable, even with infrequent thread creation.

[master255]

@gubatron @proninyaroslav your applications have the same problem. But it's easier for you to optimize it.
Here's how:
public static final ThreadPoolExecutor mainExecutor = new ThreadPoolExecutor(5, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue());

And use this whenever you create threads. This can be used in absolutely all scenarios!

[gubatron]

Oh we have plenty of different threadpool setups depending on what we do for FrostWire, some similar to that with blocking queues and what not. thanks

…

-- https://x.com/gubatron <https://twitter.com/gubatron>

On Mon, Dec 29, 2025, 11:12 AM Oleg ***@***.***> wrote: *master255* left a comment (arvidn/libtorrent#8086) <#8086 (comment)> @gubatron <https://github.com/gubatron> @proninyaroslav <https://github.com/proninyaroslav> your applications have the same problem. But it's easier for you to optimize it. Here's how: public static final ThreadPoolExecutor mainExecutor = new ThreadPoolExecutor(5, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue()); And use this whenever you create threads. This can be used in absolutely all scenarios! — Reply to this email directly, view it on GitHub <#8086 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABIBCNT3BQO6IWLPMX3SXT4EFVITAVCNFSM6AAAAACQHNXVA2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOJXGE3TIOBUGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[master255]

@gubatron Yes, indeed. But you are using too few threads at the start. Only one (https://github.com/frostwire/frostwire/blob/bf100ac443f4aa615f4aa6c947b1406913a665f6/android/src/main/java/com/frostwire/android/gui/services/EngineThreadPool.java#L42). Their lifetime is too short—two seconds. This leads to additional load.
And LinkedBlockingQueue means limiting parallel threads. But in your program, this makes no sense.
You have several ExecutorServices, which is not optimal. It is best to use only one.
And for some reason, you call executor.submit when you need to call executor.execute, which is also a small thing, but it reduces performance.

And you mistakenly think that the maximum number of threads is used with LinkedBlockingQueue. In fact, only 1 thread is ever created. You never create more than one thread! Correct that. 3-

[arvidn]

this seems a bit off-topic

[master255]

@arvidn Ok, ok. Let's get back to C++

[arvidn]

Have you measured how often threads are created?

It doesn't matter. Because delays will be noticeable, even with infrequent thread creation.

If it doesn't matter, what's the point you're trying to make?

Your presentation of this idea makes it seem like you just learned about threads and think that it should be applied to everything. You are familiar with data races, lock contention and dead-locks, right? Allowing operations (on the same data) in multiple threads takes more effort than just running the functions in separate threads.

[master255]

@arvidn I just noticed that you use many different boost functions, but you don't use boost::asio::thread_pool.
And yet, there are many places in the application where it could be used:
peer_server
session
dht_server
udp_tracker
And most likely more. These components should use a single boost::asio::thread_pool.

This will make everything work much faster.

Just use it or not. The choice is yours.

[gubatron]

our goal is to chop down as many threads as possible, that pool is never idle. Tasks get queded. We used to have a big mess, many different pools and policies. Just trying to simplify, and it's working well for us

…

-- https://x.com/gubatron <https://twitter.com/gubatron>

On Mon, Dec 29, 2025, 5:33 PM Oleg ***@***.***> wrote: *master255* left a comment (arvidn/libtorrent#8086) <#8086 (comment)> @arvidn <https://github.com/arvidn> I just noticed that you use many different boost functions, but you don't use boost::asio::thread_pool. And yet, there are many places in the application where it could be used: peer_server session dht_server udp_tracker And most likely more. These components should use a single boost::asio::thread_pool. This will make everything work much faster. Just use it or not. The choice is yours. — Reply to this email directly, view it on GitHub <#8086 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABIBCKA5INFZWZQACM2KEL4EHB4FAVCNFSM6AAAAACQHNXVA2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOJXHEYTOOJSG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[master255]

@gubatron Read my message more carefully. Your ThreadPool doesn't work the way you expect it to. You are using a maximum of 1 thread. This is not multithreading, but single-threading. All tasks are executed sequentially, not in parallel. There are errors in your code! This parameter does not work!

[master255]

@arvidn I fixed the bug in the test and ran it in the online compiler. The result is fantastic! Difference 272x!!!

You can check it out for yourself online and significantly speed up your library! And all your apps!
You have: solid evidence, a way to do it, me - if you need a hint - and millions of people who need it!
Essentially, this should be refactoring, without changing the logic of the application.

@aldenml Alden, as a more influential programmer for them, tell them that I'm right. This is a very effective optimization that we all need to do! (Except for me. I've already done it)

[Emjayen]

@arvidn I just noticed that you use many different boost functions, but you don't use boost::asio::thread_pool. And yet, there are many places in the application where it could be used: peer_server session dht_server udp_tracker And most likely more. These components should use a single boost::asio::thread_pool.

This will make everything work much faster.

Just use it or not. The choice is yours.

This is painfully clueless. You do not partition threads based on logical subsystems and generic 'threadpools' are next to useless, tending to be thrown about by people who don't know what they're doing but heard that multithreading=good; most often lead to a messy web that's difficult to reason about and substantially reduced performance due to naive scheduling and consequent sharing/synchronization.

Leveraging SMP effectively requires identifying what [CPU-bound] work can be reasonably parallelized, an iterative compromise between performance gains and complexity. The vast majority of this boils down to problems of synchronization and data-layout, with ideal case being a share-nothing architecture (not realistic, but a goal).

In the case of libtorrent, aside from the rather trivial case of hashing, the primary focus would be on efficient interfacing with the kernel and its interaction with userspace, as that's where most of its time will be spent -- the kernel isn't a blackbox where performance concerns end, infact it's often where they begin.

A reasonable rough design for libtorrent would be parallelizing processing at the granularity of a flow (a "peer connection"), with a thread-per-core architecture (hard affinities), a kernel event multiplexer (kqueue/iocp) per-core, of which flows are affinitized to based on NIC RSS/MSI-X steering. This will keep the path on both rx/tx paths almost synchronization/sharing free, critically in terms of the various kernel datastructures (particularly the kernel skt and queue itself). This also means the datapath will be mostly hot; e.g, on rx, NIC DMA's to rx ring -> skbuf descriptor built -> pass up to application; data buffer already in the cache hierachy for userspace copy.

While this does have the benefit of allowing a large swathe of peer code (e.g, protocol processing) to remain synchronization-free, it would still mean considerable work to e.g, enable the piece picker to be concurrent, as there's no reasonable way to partition that problem per-core (atleast not without severely weighing against the principle of BitTorrent's rarest-first policy, but even then-). It'd also require a bunch of other synchronization, such as with the various interdependencies between peers and peers/torrent.

This would be a lot of work, regardless of how one approaches it and you have provided not even an attempt at a real solution.

I'll note the only reason I'm being a bit harsh is because I'm familiar with your history on this repo's issues, and @arvidn has, frankly, been far too tolerant of you and your nonsense.