async def worker(session):
while True:
uri = await queue.get()
uri, status_code = await delete_url(uri, session)
......
async def delete_url(uri, session):
......
async with session.delete(url, headers=headers) as response:
status_code = response.status
return uri, status_code
async def main():
async with aiohttp.ClientSession() as session:
tasks = []
for _ in range(max_worker):
task = asyncio.create_task(worker(session))
......
早几天看到有个大佬编的那个 [ Python 潮流周刊] 里面推荐一个异步队列 saq
github 地址： https://github.com/tobymao/saq
由于没玩过异步任务队列，就很想试试，结果遇到这样一个问题
async def delete_url(ctx, *, uri, session):
async with session.delete(url, headers=headers) as response:
status_code = response.status
return uri, status_code
settings = {
"functions": [delete_url],
"concurrency": 50
}
async def main():
async with aiohttp.ClientSession() as session:
with open(filename, 'r') as f:
for line in f:
uri = line.strip()
job = await queue.enqueue("delete_url", uri=uri, session=session)
session=session 不能这样传参，因为 saq 的这个 queue.enqueue 只能接收可序列化的作为参数，
而 aiohttp.ClientSession 不是一个可以 JSON 序列化的。这个 session 又不能写全局变量，要 session 共享的话，又不能写在 delete_url 函数里面，想问问大佬，这种情况要咋处理啊？
Session, URI, Async, delete_url
