Rabu, 06 Oktober 2021
User-agent: Rogerbot
User-agent: Exabot
User-agent: MJ12bot
User-agent: Dotbot
User-agent: Gigabot
User-agent: AhrefsBot
User-agent: SemrushBot
User-agent: SemrushBot-SA
Disallow: /
User-agent: Googlebot
Allow: /
Sitemap: https://www.yourURL.me/sitemap.xml