大佬们,亚马逊爬虫(Amazonbot)是干嘛的?

查看 32|回复 1
作者:茅台   
今天发现亚马逊的爬虫疯狂的爬站,亚马逊不是个购物网站吗,他要爬虫干嘛的?
[14/Oct/2024:21:02:07 +0800] "GET /tag/xuexi.html HTTP/1.1" 200 8191 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)"
[14/Oct/2024:21:01:59 +0800] "GET /go/3_69.html HTTP/1.1" 302 5 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)"
[14/Oct/2024:21:01:58 +0800] "GET /tag/juanzi.html HTTP/1.1" 200 7889 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)"
[14/Oct/2024:21:01:43 +0800] "GET /tag/kexue.html HTTP/1.1" 200 7836 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)"

亚马逊, 爬虫

茅台
OP
  


image.png (26.57 KB, 下载次数: 0)
下载附件
保存到相册
半小时前 上传

卧槽,现在的站长真他妈的难,本来做个站写个内容就没有收录没有流量,完全靠情怀在支撑,他妈的,还到处都是AI的爬虫来获取内容和知识,转眼就把站长给踢了,都是什么玩意。
您需要登录后才可以回帖 登录 | 立即注册

返回顶部