How to create a generator/iterator with the Python C API?(如何使用 Python C API 創(chuàng)建生成器/迭代器?)
本文介紹了如何使用 Python C API 創(chuàng)建生成器/迭代器?的處理方法,對(duì)大家解決問題具有一定的參考價(jià)值,需要的朋友們下面隨著小編來一起學(xué)習(xí)吧!
問題描述
如何使用 Python C API 復(fù)制以下 Python 代碼?
How do I replicate the following Python code with the Python C API?
到目前為止,我有這個(gè):
So far, I have this:
/* Define a new object class, Sequence. */
typedef struct {
PyObject_HEAD
size_t max;
} SequenceObject;
/* Instance variables */
static PyMemberDef Sequence_members[] = {
{"max", T_UINT, offsetof(SequenceObject, max), 0, NULL},
{NULL} /* Sentinel */
};
static int Sequence_Init(SequenceObject *self, PyObject *args, PyObject *kwds)
{
if (!PyArg_ParseTuple(args, "k", &(self->max))) {
return -1;
}
return 0;
}
static PyObject *Sequence_data(SequenceObject *self, PyObject *args);
/* Methods */
static PyMethodDef Sequence_methods[] = {
{"data", (PyCFunction)Sequence_data, METH_NOARGS,
"sequence.data() -> iterator object
"
"Returns iterator of range [0, sequence.max)."},
{NULL} /* Sentinel */
};
/* Define new object type */
PyTypeObject Sequence_Type = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"Sequence", /* tp_name */
sizeof(SequenceObject), /* tp_basicsize */
0, /* tp_itemsize */
0, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags*/
"Test generator object", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
Sequence_members, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
(initproc)Sequence_init, /* tp_init */
0, /* tp_alloc */
PyType_GenericNew, /* tp_new */
};
static PyObject *Sequence_data(SequenceObject *self, PyObject *args)
{
/* Now what? */
}
但我不確定下一步該去哪里.誰能提供一些建議?
But I'm not sure where to go next. Could anyone offer some suggestions?
我想我遇到的主要問題是模擬 yield
語句.據(jù)我了解,這是一個(gè)看起來很簡(jiǎn)單,但實(shí)際上很復(fù)雜的語句——它創(chuàng)建了一個(gè)帶有自己的 __iter__()
和 next()
方法的生成器被自動(dòng)調(diào)用.搜索文檔,它似乎與 PyGenObject;但是,如何創(chuàng)建該對(duì)象的新實(shí)例尚不清楚.PyGen_New()
將 PyFrameObject
作為其參數(shù),我能找到的唯一參考是 PyEval_GetFrame()
,這似乎不是我想要的(或者我弄錯(cuò)了嗎?).有誰有這方面的經(jīng)驗(yàn)可以分享嗎?
I suppose the main problem I'm having with this is simulating the yield
statement. As I understand it, it is a pretty simple-looking, but in reality complex, statement —?it creates a generator with its own __iter__()
and next()
methods which are called automatically. Searching through the docs, it seems to be associated with the PyGenObject; however, how to create a new instance of this object is unclear. PyGen_New()
takes as its argument a PyFrameObject
, the only reference to which I can find is PyEval_GetFrame()
, which doesn't seem to be what I want (or am I mistaken?). Does anyone have any experience with this they can share?
當(dāng)我(基本上)擴(kuò)展 Python 在幕后所做的事情時(shí),我發(fā)現(xiàn)這一點(diǎn)更加清晰:
I found this to be clearer when I (essentially) expanded what Python was doing behind the scenes:
從技術(shù)上講,序列相差一個(gè),但你明白了.
Technically the sequence is off by one but you get the idea.
這樣做的唯一問題是,每次需要生成器時(shí)都創(chuàng)建一個(gè)新對(duì)象非常煩人——在 Python 中比在 C 中更是如此,因?yàn)槎x新類型會(huì)帶來巨大的麻煩.C 中不能有 yield
語句,因?yàn)?C 沒有閉包.而我所做的(因?yàn)槲以?Python API 中找不到它——請(qǐng)指出一個(gè)標(biāo)準(zhǔn)對(duì)象,如果它已經(jīng)存在!)是創(chuàng)建一個(gè)簡(jiǎn)單的通用生成器對(duì)象類,它回調(diào)一個(gè)每個(gè) next()
方法調(diào)用的 C 函數(shù).在這里(請(qǐng)注意,我還沒有嘗試編譯它,因?yàn)樗煌暾娤挛?:
The only problem with this is it's very annoying to create a new object every time one needs a generator —?even more so in Python than C because of the required monstrosity that comes with defining a new type. And there can be no yield
statement in C because C has no closures. What I did instead (since I couldn't find it in the Python API —?please point me to a standard object if it already exists!) was create a simple, generic generator object class that called back a C function for every next()
method call. Here it is (note that I have not yet tried compiling this because it is not complete —?see below):
/* A convenient, generic generator object. */
typedef PyObject *(*callback)(PyObject *callee, void *info) PyGeneratorCallback;
typedef struct {
PyObject HEAD
PyGeneratorCallback callback;
PyObject *callee;
void *callbackInfo; /* info to be passed along to callback function. */
bool freeInfo; /* true if |callbackInfo| should be free'()d when object
* dealloc's, false if not. */
} GeneratorObject;
static PyObject *Generator_iter(PyObject *self, PyObject *args)
{
Py_INCREF(self);
return self;
}
static PyObject *Generator_next(PyObject *self, PyObject *args)
{
return self->callback(self->callee, self->callbackInfo);
}
static PyMethodDef Generator_methods[] = {
{"__iter__", (PyCFunction)Generator_iter, METH_NOARGS, NULL},
{"next", (PyCFunction)Generator_next, METH_NOARGS, NULL},
{NULL} /* Sentinel */
};
static void Generator_dealloc(GenericEventObject *self)
{
if (self->freeInfo && self->callbackInfo != NULL) {
free(self->callbackInfo);
}
self->ob_type->tp_free((PyObject *)self);
}
PyTypeObject Generator_Type = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"Generator", /* tp_name */
sizeof(GeneratorObject), /* tp_basicsize */
0, /* tp_itemsize */
Generator_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags*/
0, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
PyType_GenericNew, /* tp_new */
};
/* Returns a new generator object with the given callback function
* and arguments. */
PyObject *Generator_New(PyObject *callee, void *info,
bool freeInfo, PyGeneratorCallback callback)
{
GeneratorObject *generator = (GeneratorObject *)_PyObject_New(&Generator_Type);
if (generator == NULL) return NULL;
generator->callee = callee;
generator->info = info;
generator->callback = callback;
self->freeInfo = freeInfo;
return (PyObject *)generator;
}
/* End of Generator definition. */
/* Define a new object class, Sequence. */
typedef struct {
PyObject_HEAD
size_t max;
} SequenceObject;
/* Instance variables */
static PyMemberDef Sequence_members[] = {
{"max", T_UINT, offsetof(SequenceObject, max), 0, NULL},
{NULL} /* Sentinel */
}
static int Sequence_Init(SequenceObject *self, PyObject *args, PyObject *kwds)
{
if (!PyArg_ParseTuple(args, "k", &self->max)) {
return -1;
}
return 0;
}
static PyObject *Sequence_data(SequenceObject *self, PyObject *args);
/* Methods */
static PyMethodDef Sequence_methods[] = {
{"data", (PyCFunction)Sequence_data, METH_NOARGS,
"sequence.data() -> iterator object
"
"Returns generator of range [0, sequence.max)."},
{NULL} /* Sentinel */
};
/* Define new object type */
PyTypeObject Sequence_Type = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"Sequence", /* tp_name */
sizeof(SequenceObject), /* tp_basicsize */
0, /* tp_itemsize */
0, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags*/
"Test generator object", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
Sequence_members, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
(initproc)Sequence_init, /* tp_init */
0, /* tp_alloc */
PyType_GenericNew, /* tp_new */
};
static PyObject *Sequence_data(SequenceObject *self, PyObject *args)
{
size_t *info = malloc(sizeof(size_t));
if (info == NULL) return NULL;
*info = 0;
/* |info| will be free'()d by the returned generator object. */
GeneratorObject *ret = Generator_New(self, info, true,
&Sequence_data_next_callback);
if (ret == NULL) {
free(info); /* Watch out for memory leaks! */
}
return ret;
}
PyObject *Sequence_data_next_callback(PyObject *self, void *info)
{
size_t i = info;
if (i > self->max) {
return NULL; /* TODO: How do I raise StopIteration here? I can't seem to find
* a standard exception. */
} else {
return Py_BuildValue("k", i++);
}
}
但是,不幸的是,我還沒有完成.我剩下的唯一問題是:如何使用 C API 引發(fā) StopIteration
異常?我似乎無法在 標(biāo)準(zhǔn)例外 中找到它.另外,也許更重要的是,這是解決這個(gè)問題的正確方法嗎?
However, unfortunately, I'm still not finished. The only question I have left is: How do I raise a StopIteration
exception with the C API? I can't seem to find it listed in the Standard Exceptions. Also, perhaps more importantly, is this the correct way to approach this problem?
感謝仍在關(guān)注此內(nèi)容的任何人.
Thanks to anyone that's still following this.
推薦答案
下面是模塊 spam
的簡(jiǎn)單實(shí)現(xiàn),其中一個(gè)函數(shù) myiter(int)
返回迭代器:
Below is a simple implementation of module spam
with one function myiter(int)
returning iterator:
打印從 0 到 9 的數(shù)字.
prints numbers from 0 to 9.
它比您的情況更簡(jiǎn)單,但顯示了要點(diǎn):使用標(biāo)準(zhǔn) __iter__()
和 next()
方法定義對(duì)象,并實(shí)現(xiàn)迭代器行為,包括引發(fā) 適當(dāng)時(shí)停止迭代
.
It is simpler then your case but shows main points: defining object with standard __iter__()
and next()
methods, and implementing iterator behaviour including raising StopIteration
when appropriate.
在您的情況下,迭代器對(duì)象需要持有對(duì)序列的引用(因此您需要將其釋放到 Py_DECREF 它的釋放器方法).序列本身需要實(shí)現(xiàn) __iter()__
并在其中創(chuàng)建一個(gè)迭代器.
In your case iterator object needs to hold reference to Sequence (so you'll need deallocator method for it to Py_DECREF it).
The sequence itself needs to implement __iter()__
and create an iterator inside it.
包含迭代器狀態(tài)的結(jié)構(gòu).(在您的版本而不是 m 中,它將引用序列.)
Structure containing state of iterator.
(In your version instead of m, it would have reference to Sequence.)
迭代器的 __iter__()
方法.它總是簡(jiǎn)單地返回 self
.它允許對(duì)迭代器和集合進(jìn)行相同處理在像 for ... in ...
這樣的結(jié)構(gòu)中.
Iterator's __iter__()
method.
It always simply returns self
.
It allows for both iterator and collection to be treated the same
in constructs like for ... in ...
.
我們迭代的實(shí)現(xiàn):next()
方法.
Implementation of our iteration: next()
method.
我們需要擴(kuò)展版本的 PyTypeObject
結(jié)構(gòu)來為 Python 提供關(guān)于 __iter__()
和 next()
的信息.我們希望它們被有效地調(diào)用,所以在字典中沒有基于名稱的查找.
We need extended version of PyTypeObject
structure to provide Python with
information about __iter__()
and next()
.
We want them to be called efficiently, so no name-based lookup in dictionary.
myiter(int)
函數(shù)創(chuàng)建迭代器.
剩下的就很無聊了……
這篇關(guān)于如何使用 Python C API 創(chuàng)建生成器/迭代器?的文章就介紹到這了,希望我們推薦的答案對(duì)大家有所幫助,也希望大家多多支持html5模板網(wǎng)!
【網(wǎng)站聲明】本站部分內(nèi)容來源于互聯(lián)網(wǎng),旨在幫助大家更快的解決問題,如果有圖片或者內(nèi)容侵犯了您的權(quán)益,請(qǐng)聯(lián)系我們刪除處理,感謝您的支持!