mirror of
1
Fork 0
gotosocial/internal/api
alemi.dev d0de3ad492
[bug] respect `X-Robots-Tag` and `robots.txt` on api/v1/instance and nodeinfo (#3756)
* feat: check X-Robots-Tag

when accessing /api/v1/instance or /nodeinfo endpoints respect
X-Robots-Tag

* chore: go fmt ./...

* Check robots.txt as well, add tests

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2025-02-11 13:16:14 +01:00
..
activitypub [feature] Push notifications (#3587) 2025-01-23 16:47:30 -08:00
auth [bugfix] Suggest lowercase username when creating via OIDC (#3780) 2025-02-11 09:42:35 +01:00
client [bug] Add missing `group` property to the Account model (#3746) 2025-02-08 13:04:30 +01:00
fileserver [feature] Push notifications (#3587) 2025-01-23 16:47:30 -08:00
health [feature] Add healthcheck endpoints `/livez` and `/readyz` (#2783) 2024-03-25 17:05:14 +00:00
metrics [chore] update otel libraries (#3740) 2025-02-06 12:14:37 +01:00
model [bug] Add missing `group` property to the Account model (#3746) 2025-02-08 13:04:30 +01:00
nodeinfo [feature] Change `instance-stats-randomize` to `instance-stats-mode` with multiple options; implement nodeinfo 2.1 (#3734) 2025-02-04 16:52:42 +01:00
robots [feature] Use `X-Robots-Tag` headers to instruct scrapers/crawlers (#3737) 2025-02-05 12:47:13 +01:00
util [bug] respect `X-Robots-Tag` and `robots.txt` on api/v1/instance and nodeinfo (#3756) 2025-02-11 13:16:14 +01:00
wellknown [feature] Use `X-Robots-Tag` headers to instruct scrapers/crawlers (#3737) 2025-02-05 12:47:13 +01:00
activitypub.go [feature/performance] Wrap incoming HTTP requests in timeout handler (#2353) 2023-11-13 19:48:51 +01:00
auth.go [feature/performance] Wrap incoming HTTP requests in timeout handler (#2353) 2023-11-13 19:48:51 +01:00
client.go [feature] Push notifications (#3587) 2025-01-23 16:47:30 -08:00
fileserver.go [feature] Ratelimit + serve emoji images on separate router group (#2548) 2024-01-22 15:17:04 +00:00
health.go [feature] Add healthcheck endpoints `/livez` and `/readyz` (#2783) 2024-03-25 17:05:14 +00:00
metrics.go [chore/bugfix] Fix double gzip on prometheus endpoint (#2383) 2023-11-23 19:10:51 +01:00
nodeinfo.go [feature] Use `X-Robots-Tag` headers to instruct scrapers/crawlers (#3737) 2025-02-05 12:47:13 +01:00
robots.go [feature] Use `X-Robots-Tag` headers to instruct scrapers/crawlers (#3737) 2025-02-05 12:47:13 +01:00
wellknown.go [feature/performance] Wrap incoming HTTP requests in timeout handler (#2353) 2023-11-13 19:48:51 +01:00