運営事業者
AIクローラーを運営する企業
109 社の事業者が運営するボットを、企業単位でまとめました。各社が公開しているクローラー仕様と robots.txt の扱いを確認できます。
YaDirectFetcher / Yandex / YandexAccessibilityBot / YandexAdditional ほか
3720
AdsBot-Google / AdsBot-Google-Mobile / APIs-Google / Chrome-Lighthouse ほか
3110
baidu / Baiduspider / Baiduspider-ads / Baiduspider-cpro ほか
1000
Amazon Kendra / amazon-QBusiness / Amazonbot / AmazonBuyForMe ほか
701
FacebookBot / facebookexternalhit / Meta-ExternalAds / Meta-ExternalAgent ほか
800
AdIdxBot / AzureAI-SearchBot / bingbot / BingPreview ほか
600
anthropic-ai / Claude-Code / Claude-SearchBot / Claude-User ほか
600
ChatGPT Agent / ChatGPT Operator / ChatGPT-User/2.0 / GPTBot ほか
600
coccocbot-web / Crawl4AI / crawler4j / omgili ほか
006
Bytespider / DoubaoBot / TikTokSpider / Trae
112
AwarioBot / AwarioRssBot / AwarioSmartBot
030
GrokAgent / GrokBot / xAI-SearchBot
300
Kimi-SearchBot / Kimi-User / KimiBot
300
MistralAI-Index / MistralAI-Training / MistralAI-User-1.0
300
SemrushBot / SemrushBot-OCOB / SemrushBot-SWA
030
AhrefsBot / AhrefsSiteAudit
020
AI2Bot / AI2Bot-DeepResearchEval
200
Applebot / Applebot-Extended
200
archive.org_bot / ia_archiver
020
cohere-ai / cohere-training-data-crawler
200
Diffbot / Diffbot-User
011
DotBot / Rogerbot
020
DuckAssistBot / DuckDuckBot
200
ExaBot / ExaSearchBot
200
iAskBot / iaskspider
200
imgproxy / news-please
002
kagi-fetcher / Kagibot
200
Mojeek / MojeekBot
200
PanguBot / PetalBot
200
Perplexity-User-1.0 / PerplexityBot
200
QwenBot / TongyiBot
200
SBIntuitions-SearchBot / SBIntuitionsBot
200
Shap-User / ShapBot
020
Slackbot / Slackbot-LinkExpanding
200
yacy / yacybot
002
AddSearchBot
100
aiHitBot
100
AIWebIndex
100
AliyunSecBot
100
alphalens-bot
010
Andibot
100
Anomura
010
ApifyBot
001
Arquivo-web-crawler
010
atlassian-bot
100
Barkrowler
001
BLEXBot
001
Bravebot
100
Brightbot
001
CCBot
001
ChatGLM-Spider
100
Cloudflare-AutoRAG
100
Cotoyogi
001
Crawlspace
001
Cursor
010
DataForSeoBot
010
DeepSeekBot
100
Devin
100
Echobot Bot
001
EchoboxBot
001
Factset_spyderbot
001
FirecrawlAgent
100
FriendlyCrawler
001
HarkBot
100
IbouBot
100
ICC-Crawler
001
ImagesiftBot
001
img2dataset
100
ISSCyberRiskCrawler
001
Kangaroo Bot
001
KernelSearchBot
100
KlaviyoAIBot
100
LAIONDownloader
100
LinerBot
100
LinkedInBot
100
LinkupBot
100
magpie-crawler
001
Manus-User
100
MauiBot
001
MJ12bot
010
Mozilla-Tabstack
100
MyCentralAIScraperBot
100
netEstate Imprint Crawler
001
NeticleBot
010
Netvibes
010
opencode
010
Panscient
001
PhindBot
100
Pinterestbot
100
Poseidon Research Crawler
001
QualifiedBot
001
QuillBot
100
Quora-Bot
100
Reflectionbot
100
Screaming Frog SEO Spider
010
SeekportBot
001
SeznamBot
100
Sidetrade Indexer Bot
010
Slurp
100
TavilyBot
100
TerraCotta
100
Timpibot
001
TurnitinBot
010
Twitterbot
100
VelenPublicWebCrawler
001
Webzio-Extended
001
WRTNBot
100
YouBot
100
ZipchatBot
010