問題描述
multiprocessing
模塊中的ThreadPool
和Pool
有什么區別.當我嘗試我的代碼時,這是我看到的主要區別:
Whats the difference between ThreadPool
and Pool
in multiprocessing
module. When I try my code out, this is the main difference I see:
from multiprocessing import Pool
import os, time
print("hi outside of main()")
def hello(x):
print("inside hello()")
print("Proccess id: ", os.getpid())
time.sleep(3)
return x*x
if __name__ == "__main__":
p = Pool(5)
pool_output = p.map(hello, range(3))
print(pool_output)
我看到以下輸出:
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
inside hello()
Proccess id: 13268
inside hello()
Proccess id: 11104
inside hello()
Proccess id: 13064
[0, 1, 4]
使用線程池":
from multiprocessing.pool import ThreadPool
import os, time
print("hi outside of main()")
def hello(x):
print("inside hello()")
print("Proccess id: ", os.getpid())
time.sleep(3)
return x*x
if __name__ == "__main__":
p = ThreadPool(5)
pool_output = p.map(hello, range(3))
print(pool_output)
我看到以下輸出:
hi outside of main()
inside hello()
inside hello()
Proccess id: 15204
Proccess id: 15204
inside hello()
Proccess id: 15204
[0, 1, 4]
我的問題是:
為什么每次在
Pool
中都會運行outside __main__()"?
why is the "outside __main__()" run each time in the
Pool
?
multiprocessing.pool.ThreadPool
不會產生新進程?它只是創建新線程?
multiprocessing.pool.ThreadPool
doesn't spawn new processes? It just creates new threads?
如果是這樣,使用 multiprocessing.pool.ThreadPool
與僅使用 threading
模塊有什么區別?
If so whats the difference between using multiprocessing.pool.ThreadPool
as opposed to just threading
module?
我在任何地方都沒有看到任何關于 ThreadPool
的官方文檔,有人可以幫我看看在哪里可以找到它嗎?
I don't see any official documentation for ThreadPool
anywhere, can someone help me out where I can find it?
推薦答案
multiprocessing.pool.ThreadPool
的行為與 multiprocessing.Pool
相同,唯一的區別是使用線程而不是進程來運行工作者邏輯.
The multiprocessing.pool.ThreadPool
behaves the same as the multiprocessing.Pool
with the only difference that uses threads instead of processes to run the workers logic.
你看到的原因
hi outside of main()
使用 multiprocessing.Pool
多次打印是因為池將 spawn 5 個獨立進程.每個進程都會初始化自己的 Python 解釋器并加載模塊,從而導致頂層 print
再次執行.
being printed multiple times with the multiprocessing.Pool
is due to the fact that the pool will spawn 5 independent processes. Each process will initialize its own Python interpreter and load the module resulting in the top level print
being executed again.
請注意,只有在使用 spawn
進程創建方法時才會發生這種情況(僅適用于 Windows 的方法).如果您使用 fork
之一(Unix),您將看到消息只打印一次,就像線程一樣.
Note that this happens only if the spawn
process creation method is used (only method available on Windows). If you use the fork
one (Unix), you will see the message printed only once as for the threads.
multiprocessing.pool.ThreadPool
沒有記錄,因為它的實現從未完成.它缺乏測試和文檔.你可以在源代碼中看到它的實現.
The multiprocessing.pool.ThreadPool
is not documented as its implementation has never been completed. It lacks tests and documentation. You can see its implementation in the source code.
我相信下一個自然問題是:何時使用基于線程的池以及何時使用基于進程的池?
I believe the next natural question is: when to use a thread based pool and when to use a process based one?
經驗法則是:
- IO 綁定作業 ->
multiprocessing.pool.ThreadPool
- CPU 綁定作業 ->
multiprocessing.Pool
- 混合工作 ->取決于工作量,由于進程隔離帶來的優勢,我通常更喜歡
multiprocessing.Pool
在 Python 3 上,您可能需要查看 concurrent.future.Executor
池實現.
On Python 3 you might want to take a look at the concurrent.future.Executor
pool implementations.
這篇關于多處理模塊中的 ThreadPool 與 Pool 有什么區別?的文章就介紹到這了,希望我們推薦的答案對大家有所幫助,也希望大家多多支持html5模板網!