2022版各大搜索引擎蜘蛛关键词汇总及UA名称大全
大家在制定一些防御规则或者屏蔽规则的时候都是需要放行搜索引擎的蜘蛛的,不然的话连蜘蛛都屏蔽了网站自然不会收录,权重也会下降。
尤其是从事seo的朋友来说,需要分析判断来访的爬虫类型,所以整理给大家。
2022版各大搜索引擎蜘蛛UA名称大全
百度User-Agent:
百度蜘蛛关键词:Baiduspider
移动UA :Mozilla/5.0(Linux;u;Android4.2.2;zh-cn;)AppleWebKit/534.46(KHTML,likeGecko)Version/5.1MobileSafari/10600.6.3(compatible;Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
PCUA :Mozilla/5.0(compatible;Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
【新增UA】
移动UA:Mozilla/5.0(iPhone;CPUiPhoneOS9_1likeMacOSX)AppleWebKit/601.1.46(KHTML,likeGecko)Version/9.0Mobile/13B143Safari/601.1(compatible;Baiduspider-render/2.0;+http://www.baidu.com/search/spider.html)
PCUA:Mozilla/5.0(compatible;Baiduspiderrender/2.0;+http://www.baidu.com/search/spider.html)
百度图片:“Baiduspider-image+(+http://www.baidu.com/search/spider.htm)”
神马User-Agent:
神马蜘蛛关键词:YisouSpider
PCUA:YisouSpider
移动UA:
Mozilla/5.0(Linux;U;Android4.3;zh-CN;SCH-N719Build/JSS15J)AppleWebKit/533.1(KHTML,likeGecko)Version/4.0YisouSpider/1.0.0MobileSafari/533.1
360User-Agent:
360蜘蛛关键词:360Spider
PCUA:
User-Agent:Mozilla/5.0(compatible;MSIE9.0;WindowsNT6.1;Trident/5.0);360Spider(compatible;HaosouSpider; http://www.haosou.com/help/help_3_2.html
移动UA:
Mozilla/5.0(Linux;U;Android4.0.2;en-us;GalaxyNexusBuild/ICL53F)AppleWebKit/534.30(KHTML,likeGecko)Version/4.0MobileSafari/534.30;360Spider
Mozilla/5.0(Linux;U;Android4.0.2;en-us;GalaxyNexusBuild/ICL53F)AppleWebKit/534.30(KHTML,likeGecko)Version/4.0MobileSafari/534.30;HaosouSpider
谷歌User-Agent:
谷歌蜘蛛关键词:Googlebot
谷歌UA:“Mozilla/5.0(compatible;Googlebot/2.1;+http://www.google.com/bot.html)”
谷歌图片: “Googlebot-Image/1.0”
谷歌移动:“AdsBot-Google-Mobile(+http://www.google.com/mobile/adsbot.html)Mozilla(iPhone;U;CPUiPhoneOS30likeMacOSX)AppleWebKit(KHTML,likeGecko)MobileSafari”
微软bingUser-Agent:
必应蜘蛛关键词:bingbot
微软UA:“Mozilla/5.0(compatible;bingbot/2.0;+http://www.bing.com/bingbot.htm)”
搜搜User-Agent:
搜索蜘蛛关键词:Sosospider
搜索:“Sosospider+(+http://help.soso.com/webspider.htm)”
搜搜图片:“Sosoimagespider+(+http://help.soso.com/soso-image-spider.htm)”
搜狗User-Agent:
搜狗UA:
Sogouwebspider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Sogouinstspider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
SogouNewsSpider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
SogouPicSpider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
SogouVideoSpider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
头条搜索User-Agent:
头条蜘蛛关键词:Bytespider
头条搜索的爬虫UA为“Bytespider”首写字母为大写,例如:
PC
Mozilla/5.0(compatible;Bytespider; https://zhanzhang.toutiao.com/)AppleWebKit/537.36(KHTML,likeGecko)Chrome/70.0.0.0Safari/537.36
Android
Mozilla/5.0(Linux;Android5.0)AppleWebKit/537.36(KHTML,likeGecko)MobileSafari/537.36(compatible;Bytespider; https://zhanzhang.toutiao.com/)
iOS
Mozilla/5.0(iPhone;CPUiPhoneOS7_1_2likeMacOSX)AppleWebKit/537.36(KHTML,likeGecko)Version/7.0MobileSafari/537.36(compatible;Bytespider; https://zhanzhang.toutiao.com/)