問題描述

我已經通過將導入移到頂部聲明解決了我的問題，但這讓我想知道:為什么我不能在函數中使用在 '__main__' 中導入的模塊multiprocessing 的目標?

I've already solved my problem by moving the import to the top declarations, but it left me wondering: Why cant I use a module that was imported in '__main__' in functions that are the targets of multiprocessing?

例如:

import os
import multiprocessing as mp

def run(in_file, out_dir, out_q):
    arcpy.RaterToPolygon_conversion(in_file, out_dir, "NO_SIMPIFY", "Value")
    status = str("Done with "+os.path.basename(in_file))
    out_q.put(status, block=False)

if __name__ == '__main__':
    raw_input("Program may hang, press Enter to import ArcPy...")
    import arcpy

    q = mp.Queue()
    _file = path/to/file
    _dir = path/to/dir
    # There are actually lots of files in a loop to build
    # processes but I just do one for context here
    p = mp.Process(target=run, args=(_file, _dir, q))
    p.start()

# I do stuff with Queue below to status user

當您在 IDLE 中運行它時，它根本不會出錯...只是繼續進行 Queue 檢查(這很好，所以不是問題).問題是，當您在 CMD 終端(操作系統或 Python)中運行它時，會產生 arcpy 未定義的錯誤！

When you run this in IDLE it doesn't error at all...just keeps doing a Queue check (which is good so not the problem). The problem is that when you run this in the CMD terminal (either OS or Python) it produces the error that arcpy is not defined!

只是一個奇怪的話題.

推薦答案

類unix系統和windows的情況不同.在 unixy 系統上，multiprocessing 使用 fork 創建共享父內存空間的寫時復制視圖的子進程.子進程會看到來自父進程的導入，包括父進程在 if __name__ == "__main__": 下導入的任何內容.

The situation is different in unix-like systems and Windows. On the unixy systems, multiprocessing uses fork to create child processes that share a copy-on-write view of the parent memory space. The child sees the imports from the parent, including anything the parent imported under if __name__ == "__main__":.

在 windows 上，沒有 fork，必須執行一個新進程.但是簡單地重新運行父進程是行不通的——它會再次運行整個程序.相反，multiprocessing 運行自己的 python 程序，該程序導入父主腳本，然后腌制/取消腌制父對象空間的視圖，希望這對于子進程來說足夠了.

On windows, there is no fork, a new process has to be executed. But simply rerunning the parent process doesn't work - it would run the whole program again. Instead, multiprocessing runs its own python program that imports the parent main script and then pickles/unpickles a view of the parent object space that is, hopefully, sufficient for the child process.

該程序是子進程的 __main__ 并且父腳本的 __main__ 不運行.主腳本就像任何其他模塊一樣被導入.原因很簡單:運行父 __main__ 只會再次運行完整的父程序，這是 mp 必須避免的.

That program is the __main__ for the child process and the __main__ of the parent script doesn't run. The main script was just imported like any other module. The reason is simple: running the parent __main__ would just run the full parent program again, which mp must avoid.

這是一個測試來顯示發生了什么.一個名為 testmp.py 的主模塊和一個由第一個模塊導入的第二個模塊 test2.py.

Here is a test to show what is going on. A main module called testmp.py and a second module test2.py that is imported by the first.

testmp.py

import os
import multiprocessing as mp

print("importing test2")
import test2

def worker():
    print('worker pid: {}, module name: {}, file name: {}'.format(os.getpid(), 
        __name__, __file__))

if __name__ == "__main__":
    print('main pid: {}, module name: {}, file name: {}'.format(os.getpid(), 
        __name__, __file__))
    print("running process")
    proc = mp.Process(target=worker)
    proc.start()
    proc.join()

test2.py

import os

print('test2 pid: {}, module name: {}, file name: {}'.format(os.getpid(),
        __name__, __file__))

在 Linux 上運行時，test2 被導入一次，worker 運行在主模塊中.

When run on Linux, test2 is imported once and the worker runs in the main module.

importing test2
test2 pid: 17840, module name: test2, file name: /media/td/USB20FD/tmp/test2.py
main pid: 17840, module name: __main__, file name: testmp.py
running process
worker pid: 17841, module name: __main__, file name: testmp.py

在 windows 下，請注意importing test2"打印了兩次 - testmp.py 運行了兩次.但是main pid"只打印了一次——它的 __main__ 沒有運行.那是因為 multiprocessing 在導入期間將模塊名稱更改為 __mp_main__.

Under windows, notice that "importing test2" is printed twice - testmp.py was run two times. But "main pid" was only printed once - its __main__ wasn't run. That's because multiprocessing changed the module name to __mp_main__ during import.

E:	mp>py testmp.py
importing test2
test2 pid: 7536, module name: test2, file name: E:	mp	est2.py
main pid: 7536, module name: __main__, file name: testmp.py
running process
importing test2
test2 pid: 7544, module name: test2, file name: E:	mp	est2.py
worker pid: 7544, module name: __mp_main__, file name: E:	mp	estmp.py

這篇關于為什么在'__main__'中導入模塊不允許multiprocessig使用模塊?的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！