問題描述
例如，如果我有一個帶有 2 個處理器的池對象:
If I have a pool object with 2 processors for example:
p=multiprocessing.Pool(2)
我想遍歷目錄中的文件列表并使用 map 函數(shù)
and I want to iterate over a list of files on directory and use the map function
誰能解釋一下這個函數(shù)的塊大小是多少:
could someone explain what is the chunksize of this function:
p.map(func, iterable[, chunksize])
如果我將 chunksize 例如設(shè)置為 10，這是否意味著每 10 個文件都應(yīng)該使用一個處理器進行處理?
If I set the chunksize for example to 10 does that means every 10 files should be processed with one processor?
推薦答案
看Pool.map 的文檔 看來您幾乎是正確的: chunksize 參數(shù)將導(dǎo)致可迭代對象被拆分為大約該大小，并且每件作品都作為單獨的任務(wù)提交.
Looking at the documentation for Pool.map it seems you're almost correct: the chunksize parameter will cause the iterable to be split into pieces of approximately that size, and each piece is submitted as a separate task.
所以在您的示例中，是的，map 將采用前 10 個(大約)，將其作為單個處理器的任務(wù)提交......然后接下來的 10 個將作為另一個任務(wù)提交，等等.請注意，這并不意味著這會使處理器每 10 個文件交替一次，很有可能處理器 #1 最終得到 1-10 和 11-20，而處理器 #2 得到 21-30 和 31-40.
So in your example, yes, map will take the first 10 (approximately), submit it as a task for a single processor... then the next 10 will be submitted as another task, and so on. Note that it doesn't mean that this will make the processors alternate every 10 files, it's quite possible that processor #1 ends up getting 1-10 AND 11-20, and processor #2 gets 21-30 and 31-40.
這篇關(guān)于“塊大小"multiprocessing.Pool.map 中的參數(shù)的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！