問題描述
以下代碼的時間非常奇怪:
I am getting really weird timings for the following code:
import numpy as np
s = 0
for i in range(10000000):
s += np.float64(1) # replace with np.float32 and built-in float
- 內置浮點:4.9 秒
- float64:10.5 秒
- float32:45.0 秒
- 函數調用
- numpy 與 python float 的轉換
- 對象的創建
為什么 float64 比 float 慢兩倍?為什么 float32 比 float64 慢 5 倍?
Why is float64 twice slower than float? And why is float32 5 times slower than float64?
有什么辦法可以避免使用 np.float64 的懲罰，并讓 numpy 函數返回內置 float 而不是 <代碼>float64?
Is there any way to avoid the penalty of using np.float64, and have numpy functions return built-in float instead of float64?
我發現使用 numpy.float64 比 Python 的 float 慢很多，而 numpy.float32 甚至更慢(即使我在 32 位機器上)).
I found that using numpy.float64 is much slower than Python's float, and numpy.float32 is even slower (even though I'm on a 32-bit machine).
numpy.float32 在我的 32 位機器上.因此，每次我使用各種 numpy 函數(例如 numpy.random.uniform)時，我都會將結果轉換為 float32(以便以 32 位精度執行進一步的操作).
numpy.float32 on my 32-bit machine. Therefore, every time I use various numpy functions such as numpy.random.uniform, I convert the result to float32 (so that further operations would be performed at 32-bit precision).
有沒有辦法在程序或命令行中的某處設置單個變量，并使所有 numpy 函數返回 float32 而不是 float64?
Is there any way to set a single variable somewhere in the program or in the command line, and make all numpy functions return float32 instead of float64?
編輯#1:
numpy.float64 在算術計算中比 float 慢 10 倍.太糟糕了，即使在計算之前轉換為浮點數并返回，程序運行速度也快了 3 倍.為什么?有什么辦法可以解決嗎?
numpy.float64 is 10 times slower than float in arithmetic calculations. It's so bad that even converting to float and back before the calculations makes the program run 3 times faster. Why? Is there anything I can do to fix it?
我想強調，我的時間安排不是由于以下任何原因:
I want to emphasize that my timings are not due to any of the following:
我更新了我的代碼，以更清楚地說明問題所在.使用新代碼，我似乎看到使用 numpy 數據類型會帶來十倍的性能損失:
I updated my code to make it clearer where the problem lies. With the new code, it would seem I see a ten-fold performance hit from using numpy data types:
from datetime import datetime
import numpy as np
START_TIME = datetime.now()
# one of the following lines is uncommented before execution
#s = np.float64(1)
#s = np.float32(1)
#s = 1.0
for i in range(10000000):
s = (s + 8) * s % 2399232
print(s)
print('Runtime:', datetime.now() - START_TIME)
時間是:
- float64:34.56 秒
- float32:35.11 秒
- 浮動:3.53 秒
為了它，我也試過了:
從日期時間導入日期時間將 numpy 導入為 np
from datetime import datetime import numpy as np
START_TIME = datetime.now()
s = np.float64(1)
for i in range(10000000):
s = float(s)
s = (s + 8) * s % 2399232
s = np.float64(s)
print(s)
print('Runtime:', datetime.now() - START_TIME)
執行時間為13.28 s；實際上，將 float64 轉換為 float 并返回比按原樣使用要快 3 倍.盡管如此，轉換還是要付出代價，因此總體而言，與純 Python float 相比，它的速度要慢 3 倍以上.
The execution time is 13.28 s; it's actually 3 times faster to convert the float64 to float and back than to use it as is. Still, the conversion takes its toll, so overall it's more than 3 times slower compared to the pure-python float.
我的機器是:
- 英特爾酷睿 2 雙核 T9300 (2.5GHz)
- WinXP Professional(32 位)
- ActiveState Python 3.1.3.5
- Numpy 1.5.1
編輯 #2:
感謝您的回答，他們幫助我了解如何處理這個問題.
Thank you for the answers, they help me understand how to deal with this problem.
但我仍然想知道為什么下面的代碼使用 float64 比使用 float 慢 10 倍的確切原因(也許基于源代碼).
But I still would like to know the precise reason (based on the source code perhaps) why the code below runs 10 times slow with float64 than with float.
編輯#3:
我在 Windows 7 x64 (Intel Core i7 930 @ 3.8GHz) 下重新運行代碼.
I rerun the code under the Windows 7 x64 (Intel Core i7 930 @ 3.8GHz).
同樣，代碼是:
from datetime import datetime
import numpy as np
START_TIME = datetime.now()
# one of the following lines is uncommented before execution
#s = np.float64(1)
#s = np.float32(1)
#s = 1.0
for i in range(10000000):
s = (s + 8) * s % 2399232
print(s)
print('Runtime:', datetime.now() - START_TIME)
時間是:
- float64:16.1s
- float32:16.1 秒
- 浮動:3.2 秒
現在兩個 np 浮點數(64 或 32)都比內置 float 慢 5 倍.盡管如此，還是有很大的不同.我想弄清楚它是從哪里來的.
Now both np floats (either 64 or 32) are 5 times slower than the built-in float. Still, a significant difference. I'm trying to figure out where it comes from.
編輯結束
推薦答案
總結
如果算術表達式同時包含 numpy 和內置數字，則 Python 算術運行速度較慢.避免這種轉換幾乎可以消除我報告的所有性能下降.
If an arithmetic expression contains both numpy and built-in numbers, Python arithmetics works slower. Avoiding this conversion removes almost all of the performance degradation I reported.
詳情
請注意，在我的原始代碼中:
Note that in my original code:
s = np.float64(1)
for i in range(10000000):
s = (s + 8) * s % 2399232
float 和 numpy.float64 類型混合在一個表達式中.也許 Python 必須將它們全部轉換為一種類型?
the types float and numpy.float64 are mixed up in one expression. Perhaps Python had to convert them all to one type?
s = np.float64(1)
for i in range(10000000):
s = (s + np.float64(8)) * s % np.float64(2399232)
如果運行時沒有改變(而不是增加)，這表明 Python 確實在幕后做了什么，從而解釋了性能拖累.
If the runtime is unchanged (rather than increased), it would suggest that's what Python indeed was doing under the hood, explaining the performance drag.
實際上，運行時間下降了 1.5 倍！這怎么可能?Python 可能要做的最糟糕的事情難道不是這兩次轉換嗎?
Actually, the runtime fell by 1.5 times! How is it possible? Isn't the worst thing that Python could possibly have to do was these two conversions?
我真的不知道.也許 Python 必須動態檢查什么需要轉換成什么，這需要時間，并且被告知要執行哪些精確的轉換可以使其更快.也許，一些完全不同的機制用于算術(根本不涉及轉換)，并且它恰好在不匹配的類型上非常慢.閱讀 numpy 源代碼可能會有所幫助，但這超出了我的技能范圍.
I don't really know. Perhaps Python had to dynamically check what needs to be converted into what, which takes time, and being told what precise conversions to perform makes it faster. Perhaps, some entirely different mechanism is used for arithmetics (which doesn't involve conversions at all), and it happens to be super-slow on mismatched types. Reading numpy source code might help, but it's beyond my skill.
無論如何，現在我們顯然可以通過將轉換移出循環來加快速度:
Anyway, now we can obviously speed things up more by moving the conversions out of the loop:
q = np.float64(8)
r = np.float64(2399232)
for i in range(10000000):
s = (s + q) * s % r
正如預期的那樣，運行時間大幅減少:又減少了 2.3 倍.
As expected, the runtime is reduced substantially: by another 2.3 times.
公平地說，我們現在需要稍微更改 float 版本，將文字常量移出循環.這會導致輕微的 (10%) 減速.
To be fair, we now need to change the float version slightly, by moving the literal constants out of the loop. This results in a tiny (10%) slowdown.
考慮到所有這些變化，代碼的 np.float64 版本現在只比等效的 float 版本慢 30%；可笑的 5 倍性能損失已基本消失.
Accounting for all these changes, the np.float64 version of the code is now only 30% slower than the equivalent float version; the ridiculous 5-fold performance hit is largely gone.
為什么我們仍然看到 30% 的延遲?numpy.float64 數字占用與 float 相同的空間，所以這不是原因.對于用戶定義的類型，算術運算符的解析可能需要更長的時間.當然不是主要問題.
Why do we still see the 30% delay? numpy.float64 numbers take the same amount of space as float, so that won't be the reason. Perhaps the resolution of the arithmetic operators takes longer for user-defined types. Certainly not a major concern.
這篇關于numpy float:比算術運算中內置的慢 10 倍?的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！