Free scanTools…

Meta-ExternalFetcher user agent

Operated by Meta · Live answers · Answering you

robots.txt token
Meta-ExternalFetcher

Meta-ExternalFetcher identifies itself with the token Meta-ExternalFetcher. That is the case-insensitive substring to grep for in access logs, and the exact name to put after User-agent: in robots.txt. It fetches a page live, at the moment a user asks something.

Allow or block Meta-ExternalFetcher in robots.txt

Robots.txt applies the most specific matching group, so a group naming Meta-ExternalFetcher overrides your User-agent: * rules for this bot alone.

Allow
User-agent: Meta-ExternalFetcher
Allow: /
Block
User-agent: Meta-ExternalFetcher
Disallow: /

What blocking costs you: Meta AI assistants can't fetch your pages when users ask about them.

To see which AI crawlers your robots.txt allows right now, run your domain through the AI crawler access checker. To check whether AI answers actually cite you, use the AI citation checker.

Meta's other crawlers

Blocking one Meta bot does not block the others — each token is a separate group.

FAQ

What is the Meta-ExternalFetcher user agent?
Meta-ExternalFetcher announces itself with the User-Agent token "Meta-ExternalFetcher". That token is the case-insensitive substring to match in server logs and the exact name to put after "User-agent:" in robots.txt. Meta does not publish one fixed full string, so match on the token rather than the whole line.
Who operates Meta-ExternalFetcher and what does it collect?
Meta-ExternalFetcher is run by Meta and fetches a page live, at the moment a user asks something. In the AI picture that puts it under "Answering you": A live user prompt made an AI fetch your page to answer.
How do I block Meta-ExternalFetcher?
Add a robots.txt group naming the token exactly: User-agent: Meta-ExternalFetcher followed by Disallow: /. Robots.txt matches the most specific group, so a Meta-ExternalFetcher group overrides whatever your User-agent: * group says. Meta AI assistants can't fetch your pages when users ask about them.
Should I block Meta-ExternalFetcher?
That depends on what you lose. Meta AI assistants can't fetch your pages when users ask about them. Sites chasing AI citations usually keep the live-answer and AI-search bots open and make a separate decision about the training crawlers. Run your domain through the AI crawler access checker to see which ones your robots.txt lets in today.
Can Meta-ExternalFetcher be spoofed?
Yes. A User-Agent header is self-reported, so any scraper can claim to be Meta-ExternalFetcher. The token is fine for reporting and for robots.txt, but if you are rate-limiting or firewalling on it, verify the request IP against Meta's published ranges before you trust the name.

Paste a full User-Agent line and identify any crawler with the AI bot user agent list and lookup, or see every token side by side.