wah.moe/public/robots.txt

User-agent: *
Disallow: /admin
Disallow: /login
Disallow: /js
Disallow: /css

User-Agent: GPTBot
Disallow: /

User-Agent: ChatGPT-User
Disallow: /

User-Agent: Google-Extended
Disallow: /

User-Agent: CCBot
Disallow: /

User-Agent: PerplexityBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: ClaudeBot
Disallow: /
Initialize laravel app 2023-06-13 20:59:16 +00:00			`User-agent: *`
Add robots.txt 2023-11-29 18:40:14 +00:00			`Disallow: /admin`
			`Disallow: /login`
			`Disallow: /js`
			`Disallow: /css`
Merge MVC rewrite into master (#21) * Just commit it all * Require auth * crap * Update homepage * Block AI scrapers * Update cache update script * Add dummy file * Remove unnecessary lastfm config var * Use withQueryParameters for LastFM API * Fix embeds * Update example env * Smard 2024-06-11 17:02:01 +00:00
			`User-Agent: GPTBot`
			`Disallow: /`

			`User-Agent: ChatGPT-User`
			`Disallow: /`

			`User-Agent: Google-Extended`
			`Disallow: /`

			`User-Agent: CCBot`
			`Disallow: /`

			`User-Agent: PerplexityBot`
			`Disallow: /`

Update robots.txt 2024-12-28 16:37:43 +00:00			`User-agent: anthropic-ai`
			`Disallow: /`

			`User-agent: Claude-Web`
			`Disallow: /`

			`User-agent: ClaudeBot`
			`Disallow: /`