問題描述

關于使用 Multiprocessing.Pool() 在多個進程之間共享變量的問題.

Have a quick question about a shared variable between multiple processes using Multiprocessing.Pool().

如果我從多個進程中更新全局列表，我會遇到任何問題嗎?IE.如果兩個進程同時嘗試更新列表.

Will I run in to any issues if I am updating a global list from within multiple processes? I.e. if two of the processes were to try to update the list at the same time.

我看過關于在類似事情上使用鎖的文檔，但我想知道是否有必要.

I have seen documentation about using a Lock for similar things but I was wondering if it was necessary.

我共享這個變量的方式是在我的回調函數中使用一個全局變量，'success' 在目標函數完成后，我將所有成功的操作附加到:

The way I am sharing this variable is by using a global variable in my callback function, 'successes' in which i append all of the successful actions to after the target function has completed:

TOTAL_SUCCESSES = []

def func(inputs):
    successes = []

    for input in inputs:
        result = #something with return code
        if result == 0:
            successes.append(input)
    return successes

def callback(successes):
    global TOTAL_SUCCESSES

    for entry in successes:
        TOTAL_SUCCESSES.append(entry)

def main():     
    pool = mp.Pool()
    for entry in myInputs:
         pool.apply_async(func, args=(entry,),callback=callback)         

為任何語法錯誤道歉，很快就寫出來了，但是程序正在運行，只是想知道如果我有問題我是否添加了共享變量.

Apologize for any syntax errors, wrote this up quickly but the program is working just wondering if I add the shared variable if I will have issues.

提前致謝！

推薦答案

使用您當前的代碼，您實際上并沒有在進程之間共享 CURRENT_SUCCESSES.callback 在主進程中的結果處理線程中執行.只有一個結果處理線程，因此每個 callback 將一次運行一個，而不是同時運行.所以你寫的代碼是進程/線程安全的.

With your current code, you're not actually sharing CURRENT_SUCCESSES between processes. callback is executed in the main process, in a result handling thread. There is only one result handling thread, so each callback will be run one at a time, not concurrently. So your code as written is process/thread safe.

但是，您忘記從 func 中返回 success，這是您想要修復的.

However, you are forgetting to return successes from func, which you'll want to fix.

此外，使用 map 可以更簡潔地編寫:

Also, this could be much more succinctly written using map:

def func(inputs):
    successes = []

    for input in inputs:
        result = #something with return code
        if result == 0:
            successes.append(input)
    return successes

def main():     
    pool = mp.Pool()
    total_successes = pool.map(func, myInputs) # Returns a list of lists
    # Flatten the list of lists
    total_successes = [ent for sublist in total_successes for ent in sublist]

這篇關于Python多處理附加列表的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！