- From: med-ab <notifications@github.com>
- Date: Thu, 20 Jun 2024 09:34:06 -0700
- To: whatwg/fetch <fetch@noreply.github.com>
- Cc: Subscribed <subscribed@noreply.github.com>
Received on Thursday, 20 June 2024 16:34:10 UTC
what about X-Robots-Tag for example where rules specified without a user agent are valid for all ```js console.log( new Headers([ ['x-robots-tag', 'noindex'], ['x-robots-tag', 'unavailable_after: 25 Jun 2099 15:00:00'], ['x-robots-tag', 'somebot: nofollow'], ['x-robots-tag', 'otherbot: noindex, nofollow'], ['x-robots-tag', 'noarchive'], ]).get('x-robots-tag') == 'noindex, unavailable_after: 25 Jun 2099 15:00:00, somebot: nofollow, otherbot: noindex, nofollow, noarchive' ) ``` now is the noarchive rule for all or just for otherbot ?! -- Reply to this email directly or view it on GitHub: https://github.com/whatwg/fetch/issues/973#issuecomment-2181105460 You are receiving this because you are subscribed to this thread. Message ID: <whatwg/fetch/issues/973/2181105460@github.com>
Received on Thursday, 20 June 2024 16:34:10 UTC