問題描述

我了解使用 subprocess 是調(diào)用外部命令的首選方式.

I understand using subprocess is the preferred way of calling external command.

但是，如果我想并行運(yùn)行多個(gè)命令，但要限制生成的進(jìn)程數(shù)怎么辦?困擾我的是我無法阻止子進(jìn)程.例如，如果我調(diào)用

But what if I want to run several commands in parall, but limit the number of processes being spawned? What bothers me is that I can't block the subprocesses. For example, if I call

subprocess.Popen(cmd, stderr=outputfile, stdout=outputfile)

然后該過程將繼續(xù)，無需等待 cmd 完成.因此，我無法將其包裝在 multiprocessing 庫的工作人員中.

Then the process will continue, without waiting for cmd to finish. Therefore, I can't wrap it up in a worker of multiprocessing library.

例如，如果我這樣做:

def worker(cmd): 
    subprocess.Popen(cmd, stderr=outputfile, stdout=outputfile);

pool = Pool( processes = 10 );
results =[pool.apply_async(worker, [cmd]) for cmd in cmd_list];
ans = [res.get() for res in results];

然后每個(gè)工作人員將在生成子進(jìn)程后完成并返回.所以我不能通過使用Pool 來真正限制subprocess 生成的進(jìn)程數(shù).

then each worker will finish and return after spawning a subprocess. So I can't really limit the number of processes generated by subprocess by using Pool.

限制子進(jìn)程數(shù)量的正確方法是什么?

What's the proper way of limiting the number of subprocesses?

推薦答案

如果要等待命令完成，可以使用subprocess.call.有關(guān)詳細(xì)信息，請參閱 pydoc 子進(jìn)程.

You can use subprocess.call if you want to wait for the command to complete. See pydoc subprocess for more information.

您也可以調(diào)用 Popen.wait 你的工人中的方法:

You could also call the Popen.wait method in your worker:

def worker(cmd): 
    p = subprocess.Popen(cmd, stderr=outputfile, stdout=outputfile);
    p.wait()

這篇關(guān)于在python中控制用于調(diào)用外部命令的子進(jìn)程數(shù)的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！