問(wèn)題描述

multiprocessing 模塊中的ThreadPool 和Pool 有什么區(qū)別.當(dāng)我嘗試我的代碼時(shí)，這是我看到的主要區(qū)別:

Whats the difference between ThreadPool and Pool in multiprocessing module. When I try my code out, this is the main difference I see:

from multiprocessing import Pool
import os, time

print("hi outside of main()")

def hello(x):
    print("inside hello()")
    print("Proccess id: ", os.getpid())
    time.sleep(3)
    return x*x

if __name__ == "__main__":
    p = Pool(5)
    pool_output = p.map(hello, range(3))

    print(pool_output)

我看到以下輸出:

hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
hi outside of main()
inside hello()
Proccess id:  13268
inside hello()
Proccess id:  11104
inside hello()
Proccess id:  13064
[0, 1, 4]

使用線程池":

from multiprocessing.pool import ThreadPool
import os, time

print("hi outside of main()")

def hello(x):
    print("inside hello()")
    print("Proccess id: ", os.getpid())
    time.sleep(3)
    return x*x

if __name__ == "__main__":
    p = ThreadPool(5)
    pool_output = p.map(hello, range(3))

    print(pool_output)

我看到以下輸出:

hi outside of main()
inside hello()
inside hello()
Proccess id:  15204
Proccess id:  15204
inside hello()
Proccess id:  15204
[0, 1, 4]

我的問(wèn)題是:

• 為什么每次在Pool中都會(huì)運(yùn)行outside __main__()"?

• why is the "outside __main__()" run each time in the Pool?

multiprocessing.pool.ThreadPool 不會(huì)產(chǎn)生新進(jìn)程?它只是創(chuàng)建新線程?

multiprocessing.pool.ThreadPool doesn't spawn new processes? It just creates new threads?

如果是這樣，使用 multiprocessing.pool.ThreadPool 與僅使用 threading 模塊有什么區(qū)別?

If so whats the difference between using multiprocessing.pool.ThreadPool as opposed to just threading module?

我在任何地方都沒(méi)有看到任何關(guān)于 ThreadPool 的官方文檔，有人可以幫我看看在哪里可以找到它嗎?

I don't see any official documentation for ThreadPool anywhere, can someone help me out where I can find it?

推薦答案

multiprocessing.pool.ThreadPool 的行為與 multiprocessing.Pool 相同，唯一的區(qū)別是使用線程而不是進(jìn)程來(lái)運(yùn)行工作者邏輯.

The multiprocessing.pool.ThreadPool behaves the same as the multiprocessing.Pool with the only difference that uses threads instead of processes to run the workers logic.

你看到的原因

hi outside of main()

使用 multiprocessing.Pool 多次打印是因?yàn)槌貙?spawn 5 個(gè)獨(dú)立進(jìn)程.每個(gè)進(jìn)程都會(huì)初始化自己的 Python 解釋器并加載模塊，從而導(dǎo)致頂層 print 再次執(zhí)行.

being printed multiple times with the multiprocessing.Pool is due to the fact that the pool will spawn 5 independent processes. Each process will initialize its own Python interpreter and load the module resulting in the top level print being executed again.

請(qǐng)注意，只有在使用 spawn 進(jìn)程創(chuàng)建方法時(shí)才會(huì)發(fā)生這種情況(僅適用于 Windows 的方法).如果您使用 fork 之一(Unix)，您將看到消息只打印一次，就像線程一樣.

Note that this happens only if the spawn process creation method is used (only method available on Windows). If you use the fork one (Unix), you will see the message printed only once as for the threads.

multiprocessing.pool.ThreadPool 沒(méi)有記錄，因?yàn)樗膶?shí)現(xiàn)從未完成.它缺乏測(cè)試和文檔.你可以在源代碼中看到它的實(shí)現(xiàn).

The multiprocessing.pool.ThreadPool is not documented as its implementation has never been completed. It lacks tests and documentation. You can see its implementation in the source code.

我相信下一個(gè)自然問(wèn)題是:何時(shí)使用基于線程的池以及何時(shí)使用基于進(jìn)程的池?

I believe the next natural question is: when to use a thread based pool and when to use a process based one?

經(jīng)驗(yàn)法則是:

• IO 綁定作業(yè) ->multiprocessing.pool.ThreadPool
• CPU 綁定作業(yè) ->multiprocessing.Pool
• 混合工作 ->取決于工作量，由于進(jìn)程隔離帶來(lái)的優(yōu)勢(shì)，我通常更喜歡 multiprocessing.Pool

在 Python 3 上，您可能需要查看 concurrent.future.Executor 池實(shí)現(xiàn).

On Python 3 you might want to take a look at the concurrent.future.Executor pool implementations.

這篇關(guān)于多處理模塊中的 ThreadPool 與 Pool 有什么區(qū)別?的文章就介紹到這了，希望我們推薦的答案對(duì)大家有所幫助，也希望大家多多支持html5模板網(wǎng)！