Sunday, 8 September 2013

How to add tasks to a Pool in a task executed by the same Pool?

How to add tasks to a Pool in a task executed by the same Pool?

Suppose I have a task executed by a multiprocessing.Pool. How do I allow
this task to add new tasks to the Pool executing it? For example,
def integers(pool, queue, n1, n2):
print ("integers(%d)" % n)
queue.put(n)
pool.apply_async(multiple_me, (pool, queue, n+1)) # crashes; can't
pickle `pool`
def start():
pool = multiprocessing.Pool()
queue = multiprocessing.Queue()
integers(pool, queue, 1)
while True:
yield queue.get()

No comments:

Post a Comment