問(wèn)題描述
multiprocessing
模塊中的ThreadPool
和Pool
有什么區(qū)別.當(dāng)我嘗試我的代碼時(shí),這是我看到的主要區(qū)別:
Whats the difference between ThreadPool
and Pool
in multiprocessing
module. When I try my code out, this is the main difference I see:
from multiprocessing import Pool
import os, time
print("hi outside of main()")
def hello(x):
print("inside hello()")
print("Proccess id: ", os.getpid())
time.sleep(3)
return x*x
if __name__ == "__main__":
p = Pool(5)
pool_output = p.map(hello, range(3))
print(pool_output)
我看到以下輸出:
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
inside hello()
Proccess id: 13268
inside hello()
Proccess id: 11104
inside hello()
Proccess id: 13064
[0, 1, 4]
使用線程池":
from multiprocessing.pool import ThreadPool
import os, time
print("hi outside of main()")
def hello(x):
print("inside hello()")
print("Proccess id: ", os.getpid())
time.sleep(3)
return x*x
if __name__ == "__main__":
p = ThreadPool(5)
pool_output = p.map(hello, range(3))
print(pool_output)
我看到以下輸出:
hi outside of main()
inside hello()
inside hello()
Proccess id: 15204
Proccess id: 15204
inside hello()
Proccess id: 15204
[0, 1, 4]
我的問(wèn)題是:
為什么每次在
Pool
中都會(huì)運(yùn)行outside __main__()"?
why is the "outside __main__()" run each time in the
Pool
?
multiprocessing.pool.ThreadPool
不會(huì)產(chǎn)生新進(jìn)程?它只是創(chuàng)建新線程?
multiprocessing.pool.ThreadPool
doesn't spawn new processes? It just creates new threads?
如果是這樣,使用 multiprocessing.pool.ThreadPool
與僅使用 threading
模塊有什么區(qū)別?
If so whats the difference between using multiprocessing.pool.ThreadPool
as opposed to just threading
module?
我在任何地方都沒(méi)有看到任何關(guān)于 ThreadPool
的官方文檔,有人可以幫我看看在哪里可以找到它嗎?
I don't see any official documentation for ThreadPool
anywhere, can someone help me out where I can find it?
推薦答案
multiprocessing.pool.ThreadPool
的行為與 multiprocessing.Pool
相同,唯一的區(qū)別是使用線程而不是進(jìn)程來(lái)運(yùn)行工作者邏輯.
The multiprocessing.pool.ThreadPool
behaves the same as the multiprocessing.Pool
with the only difference that uses threads instead of processes to run the workers logic.
你看到的原因
hi outside of main()
使用 multiprocessing.Pool
多次打印是因?yàn)槌貙?spawn 5 個(gè)獨(dú)立進(jìn)程.每個(gè)進(jìn)程都會(huì)初始化自己的 Python 解釋器并加載模塊,從而導(dǎo)致頂層 print
再次執(zhí)行.
being printed multiple times with the multiprocessing.Pool
is due to the fact that the pool will spawn 5 independent processes. Each process will initialize its own Python interpreter and load the module resulting in the top level print
being executed again.
請(qǐng)注意,只有在使用 spawn
進(jìn)程創(chuàng)建方法時(shí)才會(huì)發(fā)生這種情況(僅適用于 Windows 的方法).如果您使用 fork
之一(Unix),您將看到消息只打印一次,就像線程一樣.
Note that this happens only if the spawn
process creation method is used (only method available on Windows). If you use the fork
one (Unix), you will see the message printed only once as for the threads.
multiprocessing.pool.ThreadPool
沒(méi)有記錄,因?yàn)樗膶?shí)現(xiàn)從未完成.它缺乏測(cè)試和文檔.你可以在源代碼中看到它的實(shí)現(xiàn).
The multiprocessing.pool.ThreadPool
is not documented as its implementation has never been completed. It lacks tests and documentation. You can see its implementation in the source code.
我相信下一個(gè)自然問(wèn)題是:何時(shí)使用基于線程的池以及何時(shí)使用基于進(jìn)程的池?
I believe the next natural question is: when to use a thread based pool and when to use a process based one?
經(jīng)驗(yàn)法則是:
- IO 綁定作業(yè) ->
multiprocessing.pool.ThreadPool
- CPU 綁定作業(yè) ->
multiprocessing.Pool
- 混合工作 ->取決于工作量,由于進(jìn)程隔離帶來(lái)的優(yōu)勢(shì),我通常更喜歡
multiprocessing.Pool
在 Python 3 上,您可能需要查看 concurrent.future.Executor
池實(shí)現(xiàn).
On Python 3 you might want to take a look at the concurrent.future.Executor
pool implementations.
這篇關(guān)于多處理模塊中的 ThreadPool 與 Pool 有什么區(qū)別?的文章就介紹到這了,希望我們推薦的答案對(duì)大家有所幫助,也希望大家多多支持html5模板網(wǎng)!