By default all robots are
Crawl delay
Sitemap URL. Leave blank for none.
Robots are allowed to crawl your site:
Restricted directories or files. Relative to root, with trailing slash for dirs.
Your Robots.txt
Robots.txt generator |
By default all robots are
Crawl delay
Sitemap URL. Leave blank for none.
Robots are allowed to crawl your site:
Restricted directories or files. Relative to root, with trailing slash for dirs.
Your Robots.txt
Het bestand Robots.txt op een website is help de webmaster die gespecificeerde bestanden of mappen negeert wanneer de spin van de scharreling een site kruipen.
Robots.txt Generator Help u Robost.txt-bestand gemakkelijker te krijgen, WEHN U selecteert de zoekmachine die is toegestaan en kruip de vertragingstijd, voer de sitemap-URL in en selecteer de toegestane Rotots en ontvang uw code. de code zoals hieronder:
User-agent: *
Weigeren: /
User-agent: IA_Archiver
Weigeren:
User-agent: Slurp
Weigeren:
User-agent: Aport
Weigeren:
User-agent: TOMA
Weigeren:
User-agent: Baiduspider
Weigeren:
User-agent: twiceler
Weigeren:
User-agent: Robozilla
Weigeren:
User-agent: Gigabot
Weigeren:
User-agent: GoogleBot
Weigeren:
User-agent: GoogleBot-Image
Weigeren:
User-agent: GoogleBot-Mobile
Weigeren:
User-agent: mail.ru
Weigeren:
User-agent: MSNBOT
Weigeren:
User-agent: PSBOT
Weigeren:
User-agent: NAVERBOT
Weigeren:
User-agent: Yeti
Weigeren:
User-agent: nutch
Weigeren:
User-agent: Stackrambler
Weigeren:
User-agent: Scrubby
Weigeren:
User-agent: Asterias
Weigeren:
User-agent: Yahoo-Slurp
Weigeren:
User-agent: Yandex
Weigeren:
taal selectie:日本語 | 한국어 | Français | Español | ไทย| عربي | русский язык | Português | Deutsch| Italiano | Ελληνικά | Nederlands | Polskie| Tiếng Việt| বাংলা| Indonesia| Pilipino| Türk| فارسی| ລາວ| ဗမာ| български| Català| čeština| Қазақ| Magyar| Română| Україна
Copyright ©2021 - 2031 All Rights Reserved.