問(wèn)題描述

我有一個(gè)簡(jiǎn)單的函數(shù)來(lái)遍歷 URL 列表，使用 GET 來(lái)檢索一些信息并相應(yīng)地更新 DB (PostgresSQL).該功能完美運(yùn)行.但是，一次一個(gè)地瀏覽每個(gè) URL 會(huì)占用太多時(shí)間.

I have a simple function that go over a list of URLs, using GET to retrieve some information and update the DB (PostgresSQL) accordingly. The function works perfect. However, going over each URL one at a time talking too much time.

使用 python，我可以執(zhí)行以下操作來(lái)并行執(zhí)行這些任務(wù):

Using python, I'm able to do to following to parallel these tasks:

from multiprocessing import Pool

def updateDB(ip):
     code goes here...

if __name__ == '__main__':
    pool = Pool(processes=4)              # process per core
    pool.map(updateDB, ip)

這工作得很好.但是，我試圖找到如何在 django 項(xiàng)目上做同樣的事情.目前我有一個(gè)函數(shù)(視圖)，可以遍歷每個(gè) URL 以獲取信息并更新數(shù)據(jù)庫(kù).

This is working pretty well. However, I'm trying to find how do the same on django project. Currently I have a function (view) that go over each URL to get the information, and update the DB.

我唯一能找到的就是使用 Celery，但這對(duì)于我想要執(zhí)行的簡(jiǎn)單任務(wù)來(lái)說(shuō)似乎有點(diǎn)過(guò)于強(qiáng)大了.

The only thing I could find is using Celery, but this seems to be a bit overpower for the simple task I want to perform.

有什么簡(jiǎn)單的我可以做或者我必須使用 Celery 嗎?

Is there anything simple that i can do or do I have to use Celery?

推薦答案

目前我有一個(gè)函數(shù)(視圖)可以遍歷每個(gè) URL 以獲取信息，并更新數(shù)據(jù)庫(kù).

Currently I have a function (view) that go over each URL to get the information, and update the DB.

這意味著響應(yīng)時(shí)間對(duì)您來(lái)說(shuō)并不重要，而不是在后臺(tái)(異步)執(zhí)行，如果您的響應(yīng)時(shí)間減少 4(使用 4 個(gè)子進(jìn)程/線程)，您可以在前臺(tái)執(zhí)行.如果是這種情況，您可以簡(jiǎn)單地將示例代碼放在您的視圖中.喜歡

It means response time does not matter for you and instead of doing it in the background (asynchronously), you are OK with doing it in the foreground if your response time is cut by 4 (using 4 sub-processes/threads). If that is the case you can simply put your sample code in your view. Like

from multiprocessing import Pool

def updateDB(ip):
     code goes here...

def my_view(request):
    pool = Pool(processes=4)              # process per core
    pool.map(updateDB, ip)
    return HttpResponse("SUCCESS")

但是，如果您想在后臺(tái)異步執(zhí)行此操作，那么您應(yīng)該使用 Celery 或遵循@BasicWolf 的建議之一.

But, if you want to do it asynchronously in the background then you should use Celery or follow one of @BasicWolf's suggestions.

這篇關(guān)于如何在 django 視圖中使用 python 多處理模塊的文章就介紹到這了，希望我們推薦的答案對(duì)大家有所幫助，也希望大家多多支持html5模板網(wǎng)！