544cbc6f01
This PR optimizes the SQL query and de-duplicate the labels' ids when generating the query string, on the issue page. <hr/> ### Background Some time ago, BingBot and some other crawlers have been putting my instance on its knees with requests containing a lot of label ids, like this one : ``` [07/Aug/2023:11:28:37 +0200] "GET /Dolibarr/sendrecurringinvoicebymail/issues?q=&type=all&sort=&state=closed&labels=1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c2%2c10%2c2%2c1%2c1%2c10%2c10%2c7%2c6%2c10%2c10%2c3%2c2%2c1%2c5%2c10%2c1%2c6%2c2%2c7%2c3%2c7%2c6%2c10%2c1%2c10%2c1%2c1%2c7%2c7%2c1%2c1%2c1%2c1%2c10%2c10%2c1%2c2%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c2%2c1%2c12%2c6%2c6%2c10&milestone=0&project=-1&poster=0 HTTP/1.1" 499 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/103.0.5060.134 Safari/537.36" ``` Since each of the label ids implies a join, it grows exponentially expensive for the database engine (at least on PostgreSQL but SQLite suffers a little too). Thus, this PR proposes two enhancements: * rewrite the database query to use only one squashed condition, * deduplicate the label ids when generating the URL. ### Performance comparison Here are some timings on Postgresql-backed, Forgejo 7.0.4 instances : ```sh $ time curl -s -o /dev/null "http://localhost:3000/toto/tata/issues?q=&type=all&sort=&labels=19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25&state=open&milestone=0&project=0&assignee=0&poster=0" real 0m10,491s user 0m0,017s sys 0m0,008s ``` ...and with the patch: ```sh $ time curl -s -o /dev/null "http://localhost:3000/toto/tata/issues?q=&type=all&sort=&labels=19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25&state=open&milestone=0&project=0&assignee=0&poster=0" real 0m0,094s user 0m0,012s sys 0m0,013s ``` ### Annex This issue was originally proposed to [Gitea](https://github.com/go-gitea/gitea/pull/26460) but didn't get much attention, and I switched to Forgejo in the meantime :) Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4228 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: Chl <chl@xlii.si> Co-committed-by: Chl <chl@xlii.si> |
||
|---|---|---|
| .devcontainer | ||
| .forgejo | ||
| .gitea | ||
| assets | ||
| build | ||
| cmd | ||
| contrib | ||
| custom/conf | ||
| docker | ||
| models | ||
| modules | ||
| options | ||
| public | ||
| release-notes | ||
| releases/images | ||
| routers | ||
| services | ||
| templates | ||
| tests | ||
| tools | ||
| web_src | ||
| .air.toml | ||
| .deadcode-out | ||
| .dockerignore | ||
| .editorconfig | ||
| .eslintrc.yaml | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| .gitpod.yml | ||
| .golangci.yml | ||
| .ignore | ||
| .markdownlint.yaml | ||
| .npmrc | ||
| .spectral.yaml | ||
| .yamllint.yaml | ||
| BSDmakefile | ||
| CODEOWNERS | ||
| CONTRIBUTING.md | ||
| DCO | ||
| Dockerfile | ||
| Dockerfile.rootless | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| RELEASE-NOTES.md | ||
| build.go | ||
| flake.lock | ||
| flake.nix | ||
| go.mod | ||
| go.sum | ||
| main.go | ||
| package-lock.json | ||
| package.json | ||
| playwright.config.js | ||
| poetry.lock | ||
| poetry.toml | ||
| pyproject.toml | ||
| renovate.json | ||
| stylelint.config.js | ||
| tailwind.config.js | ||
| vitest.config.js | ||
| webpack.config.js | ||
README.md
Welcome to Forgejo
Hi there! Tired of big platforms playing monopoly? Providing Git hosting for your project, friends, company or community? Forgejo (/for'd͡ʒe.jo/ inspired by forĝejo – the Esperanto word for forge) has you covered with its intuitive interface, light and easy hosting and a lot of builtin functionality.
Forgejo was created in 2022 because we think that the project should be owned by an independent community. If you second that, then Forgejo is for you! Our promise: Independent Free/Libre Software forever!
What does Forgejo offer?
If you like any of the following, Forgejo is literally meant for you:
- Lightweight: Forgejo can easily be hosted on nearly every machine. Running on a Raspberry? Small cloud instance? No problem!
- Project management: Besides Git hosting, Forgejo offers issues, pull requests, wikis, kanban boards and much more to coordinate with your team.
- Publishing: Have something to share? Use releases to host your software for download, or use the package registry to publish it for docker, npm and many other package managers.
- Customizable: Want to change your look? Change some settings? There are many config switches to make Forgejo work exactly like you want.
- Powerful: Organizations & team permissions, CI integration, Code Search, LDAP, OAuth and much more. If you have advanced needs, Forgejo has you covered.
- Privacy: From update checker to default settings: Forgejo is built to be privacy first for you and your crew.
- Federation: (WIP) We are actively working to connect software forges with each other through ActivityPub, and create a collaborative network of personal instances.
Learn more
Dive into the documentation, subscribe to releases and blog post on our website, find us on the Fediverse or hop into our Matrix room if you have any questions or want to get involved.
Get involved
If you are interested in making Forgejo better, either by reporting a bug or by changing the governance, please take a look at the contribution guide.