Holen Sie sich AI SEO KOSTENLOS.

Erfahren Sie mehr!

What is X-Robots-Tag?

Das X-Robots-Tag ist ein Teil des HTTP-Antwortheaders, der Suchmaschinen mitteilt, wie eine Webseite gecrawlt und indiziert werden soll.

When a crawler bot sends an HTTP request to a webpage, the server responds with an HTTP response containing the X-Robots-Tag. By analyzing the tag, the bot understands whether it can crawl the webpage and how it is allowed to do so. 

For example, a response header containing X-Robots-Tag: noindex instructs a search engine not to index the webpage.

The X-Robots-Tag differs from the meta robots tag. While both tags tell search engines how to crawl the webpage and can be used interchangeably in many cases, the X-Robots-Tag is more versatile. 

For instance, the meta robots tag can only be used on HTML pages. Meanwhile, the X-Robots-Tag can be used on non-HTML properties, including images and videos. 

🇩🇪 Deutsch