I don't regard the lack of practical multithreading as a problem. We routinely use multiprocessing and IPC via queues for parallelism, and for me the main limitation is Python's excessive memory use: even atomic data types are objects which cause quite a bit of overhead (a memory efficient dictionary with support for a few basic data types would be great). Thankfully everything still fits in the server's memory, otherwise I would have to consider another language, such as Go.
Behind the scenes it uses multiprocessing and/or threading plus queues etc. I have a function that adds command line arguments (number of workers, use threads or processes, debug mode) and then another that returns an executor given the command line arguments.
The debug mode forces a single thread and is intended for when you want to run pdb as multiple threads/processes make things considerably more complicated.
I've been programming with import Queue, import threading, for years now (probably 8 or 9). I don't want to drop all my completely fine multithreading code for Yet Another Threading API.
Personally I think multiprocessing is just dopey. Why force me to program around two different address spaces when I can just have one?
I'd also done the Queue/threading thing for the longest time. I'd also use multiprocessing some of the time. But it was arbitrary, and I'd have to make sure that shutdown, exceptions etc are correctly handled.
> Yet Another Threading API
concurrent.futures is not yet another threading api. futures is a standard async work/result wrapper, while the process and thread executors are standard executor apis.
> Personally I think multiprocessing is just dopey
Then don't use it. But be aware that it works well for many people. Some of my code scales linearly with multiprocessing and all I had to do was use a process executor instead of a thread executor.
Well I do care. I think multiprocessing is a stupid abomination- it's like a person who never learned how to use threads found the threading library and decided to make a message-passing communication system with a thread-compatible concurrency library.
Message passing is fine (and it's the sort of thing that Go uses) but ultimately, leaving single-address-space multithreading completely on the table when it's the single most obvious and straightforward way to take advantage of modern processors is just dumb.
