問題描述
我想使用 Pool.map() 將關鍵字參數傳遞給我的工作函數.在搜索論壇時，我找不到明確的例子.
I would like to pass keyword arguments to my worker-function with Pool.map(). I can't find a clear example of this when searching forums.
示例代碼:
import multiprocessing as mp
def worker((x,y), **kwargs):
kwarg_test = kwargs.get('kwarg_test', False)
print("kwarg_test = {}".format(kwarg_test))
if kwarg_test:
print("Success")
return x*y
def wrapper_process(**kwargs):
jobs = []
pool=mp.Pool(4)
for i, n in enumerate(range(4)):
jobs.append((n,i))
pool.map(worker, jobs) #works
pool.map(worker, jobs, kwargs) #how to do this?
def main(**kwargs):
worker((1,2),kwarg_test=True) #accepts kwargs
wrapper_process(kwarg_test=True)
if __name__ == "__main__":
main()
輸出:
kwarg_test = True
Success
kwarg_test = False
kwarg_test = False
kwarg_test = False
kwarg_test = False
TypeError: unsupported operand type(s) for //: 'int' and 'dict'
類型錯誤與解析 multiprocessing.Pool 或 Queue 中的參數有關，我嘗試了其他幾種語法，例如制作 kwargs 列表；[kwargs, kwargs, kwargs, kwargs]，以及多次嘗試將 kwarg 包含在工作列表中，但沒有運氣.我從 map 到 map_async 跟蹤 multiprocessing.pool 中的代碼，并達到了task_batches = Pool._get_tasks(func, iterable, chunksize)當我遇到生成器結構時，在 pool.py 中.我很高興將來能進一步了解這方面的信息，但現在我只是想了解一下:
The type error has to do with parsing arguments inside of multiprocessing.Pool or Queue, and I have tried several other syntaxes, like making a list of the kwargs; [kwargs, kwargs, kwargs, kwargs], as well as several attempts to include the kwarg in the jobs list but no luck. I traced the code in multiprocessing.pool from map to map_async and got as far as
task_batches = Pool._get_tasks(func, iterable, chunksize)
in pool.py when I encountered the generator structure. I'm happy to learn more about this in future but for now I am just trying to find out:
是否有允許通過 pool.map 傳遞 kwargs 的簡單語法?
Is there a simple syntax for allowing the passing of kwargs with pool.map?
推薦答案
如果您想遍歷其他參數，請使用@ArcturusB 的答案.
If you want to iterate over the other arguments, use @ArcturusB's answer.
如果您只想傳遞它們，并且每次迭代都具有相同的值，那么您可以這樣做:
If you just want to pass them, having the same value for each iteration, then you can do this:
from functools import partial
pool.map(partial(worker, **kwargs), jobs)
Partial 將參數綁定"到函數.但是，舊版本的 Python 不能序列化部分對象.
Partial 'binds' arguments to a function. Old versions of Python cannot serialize partial objects though.
這篇關于使用 multiprocessing.pool.map 傳遞 kwargs的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！