mirror of
1
Fork 0
Commit Graph

91 Commits

Author SHA1 Message Date
merlleu a587d25261
Add auth-required to config.json for Cargo http registry (#26729)
Cargo registry-auth feature requires config.json to have a property
auth-required set to true in order to send token to all registry
requests.
This is ok for git index because you can manually edit the config.json
file to add the auth-required, but when using sparse
(setting index url to
"sparse+https://git.example.com/api/packages/{owner}/cargo/"), the
config.json is dynamically rendered, and does not reflect changes to the
config.json file in the repo.

I see two approaches:
- Serve the real config.json file when fetching the config.json on the
cargo service.
- Automatically detect if the registry requires authorization. (This is
what I implemented in this PR).

What the PR does:
- When a cargo index repository is created, on the config.json, set
auth-required to wether or not the repository is private.
- When the cargo/config.json endpoint is called, set auth-required to
wether or not the request was authorized using an API token.
2023-08-28 07:05:39 +00:00
KN4CK3R ecd51f710b
Fix NuGet search endpoints (#25613)
Fixes #25564
Fixes #23191

- Api v2 search endpoint should return only the latest version matching
the query
- Api v3 search endpoint should return `take` packages not package
versions
2023-08-14 02:50:55 +00:00
KN4CK3R 2d7fe4cc1e
Fix handling of plenty Nuget package versions (#26075)
Fixes #25953

- Do not load full version information (v3)
- Add pagination support (v2)
2023-07-26 19:43:21 +00:00
KN4CK3R 6aa30af724
Fix handling of Debian files with trailing slash (#26087)
Fixes #26022

- Fix handling of files with trailing slash
- Fix handling of duplicate package file errors
- Added test for both
2023-07-24 16:19:44 +02:00
delvh f3d41c61eb
Remove `db.DefaultContext` in `routers/` and `cmd/` (#26076)
Now, the only remaining usages of `models.db.DefaultContext` are in
  - `modules`
  - `models`
  - `services`
2023-07-23 23:47:27 -04:00
wxiaoguang 9b25bfa8f4
Remove redundant "RouteMethods" method (#26024)
The `RouteMethods` is mainly an alias for `Methods` with different
argument order.

Remove it to keep the "route.go" code clear
2023-07-21 00:43:49 +02:00
KN4CK3R bd82d8974e
Add support for different Maven POM encoding (#25873)
Fixes #25853

- Maven POM files aren't always UTF-8 encoded.
- Reject the upload of unparsable POM files
2023-07-14 09:39:15 +00:00
KN4CK3R 115f40e433
Test if container blob is accessible before mounting (#22759)
related #16865

This PR adds an accessibility check before mounting container blobs.

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
2023-07-09 11:24:43 +00:00
silverwind 88f835192d
Replace `interface{}` with `any` (#25686)
Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`.

Basically the same [as golang did](2580d0e08d).
2023-07-04 18:36:08 +00:00
KN4CK3R c890454769
Add direct serving of package content (#25543)
Fixes #24723

Direct serving of content aka HTTP redirect is not mentioned in any of
the package registry specs but lots of official registries do that so it
should be supported by the usual clients.
2023-07-03 15:33:28 +02:00
wxiaoguang 4e2f1ee58d
Refactor web package and context package (#25298)
1. The "web" package shouldn't depends on "modules/context" package,
instead, let each "web context" register themselves to the "web"
package.
2. The old Init/Free doesn't make sense, so simplify it
* The ctx in "Init(ctx)" is never used, and shouldn't be used that way
* The "Free" is never called and shouldn't be called because the SSPI
instance is shared

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-18 09:59:09 +02:00
KN4CK3R cdb088cec2
Add CRAN package registry (#22331)
This PR adds a [CRAN](https://cran.r-project.org/) package registry.

![grafik](https://user-images.githubusercontent.com/1666336/210450039-d6fa6f77-20cd-4741-89a8-1624def267f7.png)
2023-05-22 10:57:49 +08:00
wxiaoguang 6b33152b7d
Decouple the different contexts from each other (#24786)
Replace #16455

Close #21803

Mixing different Gitea contexts together causes some problems:

1. Unable to respond proper content when error occurs, eg: Web should
respond HTML while API should respond JSON
2. Unclear dependency, eg: it's unclear when Context is used in
APIContext, which fields should be initialized, which methods are
necessary.


To make things clear, this PR introduces a Base context, it only
provides basic Req/Resp/Data features.

This PR mainly moves code. There are still many legacy problems and
TODOs in code, leave unrelated changes to future PRs.
2023-05-21 09:50:53 +08:00
KN4CK3R 5968c63a11
Add Go package registry (#24687)
Fixes #7608

This PR adds a Go package registry usable with the Go proxy protocol.

![grafik](https://github.com/go-gitea/gitea/assets/1666336/328feb5c-3df2-4f9d-8eae-fe3126d14c37)
2023-05-14 23:38:40 +08:00
KN4CK3R 9173e079ae
Add Alpine package registry (#23714)
This PR adds an Alpine package registry. You can follow [this
tutorial](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package)
to build a *.apk package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/227779595-b76163aa-eea1-4a79-9583-775c24ad74e8.png)

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-12 17:27:50 +00:00
KN4CK3R 05209f0d1d
Add RPM registry (#23380)
Fixes #20751

This PR adds a RPM package registry. You can follow [this
tutorial](https://opensource.com/article/18/9/how-build-rpm-packages) to
build a *.rpm package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/223806549-d8784fd9-9d79-46a2-9ae2-f038594f636a.png)
2023-05-05 20:33:37 +00:00
KN4CK3R 723598b803
Implement Cargo HTTP index (#24452)
This implements the HTTP index
[RFC](https://rust-lang.github.io/rfcs/2789-sparse-index.html) for Cargo
registries.

Currently this is a preview feature and you need to use the nightly of
`cargo`:

`cargo +nightly -Z sparse-registry update`

See https://github.com/rust-lang/cargo/issues/9069 for more information.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-03 16:58:43 -04:00
KN4CK3R bf999e4069
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry.
You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing.
Source packages are not supported at the moment and I did not find
documentation of the architecture "all" and how these packages should be
treated.


![grafik](https://user-images.githubusercontent.com/1666336/218126879-eb80a866-775c-4c8e-8529-5797203a64e6.png)

Part of #20751.

Revised copy of #22854.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-02 12:31:35 -04:00
Yarden Shoham c0ddec8a2a
Revert "Add Debian package registry" (#24412)
Reverts go-gitea/gitea#22854
2023-04-28 18:06:41 -04:00
KN4CK3R bf77e2163b
Add Debian package registry (#22854)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2023-04-28 17:51:36 -04:00
John Olheiser 5e36024105
Require repo scope for PATs for private repos and basic authentication (#24362)
> The scoped token PR just checked all API routes but in fact, some web
routes like `LFS`, git `HTTP`, container, and attachments supports basic
auth. This PR added scoped token check for them.

---------

Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-04-26 19:24:03 -05:00
wxiaoguang b9a97ccd0e
Refactor web route (#24080)
The old code is unnecessarily complex, and has many misuses.

Old code "wraps" a lot, wrap wrap wrap, it's difficult to understand
which kind of handler is used.

The new code uses a general approach, we do not need to write all kinds
of handlers into the "wrapper", do not need to wrap them again and
again.

New code, there are only 2 concepts:

1. HandlerProvider: `func (h any) (handlerProvider func (next)
http.Handler)`, it can be used as middleware
2. Use HandlerProvider to get the final HandlerFunc, and use it for
`r.Get()`


And we can decouple the route package from context package (see the
TODO).

# FAQ

## Is `reflect` safe?

Yes, all handlers are checked during startup, see the `preCheckHandler`
comment. If any handler is wrong, developers could know it in the first
time.

## Does `reflect` affect performance?

No. https://github.com/go-gitea/gitea/pull/24080#discussion_r1164825901

1. This reflect code only runs for each web handler call, handler is far
more slower: 10ms-50ms
2. The reflect is pretty fast (comparing to other code): 0.000265ms
3. XORM has more reflect operations already
2023-04-20 14:49:06 -04:00
yp05327 bb6c670cff
Add actions support to package auth verification (#23729)
Partly fixes https://github.com/go-gitea/gitea/issues/23642

Error info:

![image](https://user-images.githubusercontent.com/18380374/227827027-4280a368-ec9e-49e0-bb93-6b496ada7cd9.png)
ActionsUser (userID -2) is used to login in to docker in action jobs.

Due to we have no permission policy settings of ActionsUser now,
ActionsUser can only access public registry by this quick fix.
2023-04-10 15:21:03 +08:00
KN4CK3R fbd4eaceed
Display image size for multiarch container images (#23821)
Fixes #23771

Changes the display of different architectures for multiarch images to
show the image size:

![grafik](https://user-images.githubusercontent.com/1666336/228781477-cc76c4d1-4728-434f-8a27-fc008790d924.png)
2023-04-02 17:53:37 +08:00
Mai-Lapyst 5cd1d6c93b
Set repository link based on the url in package.json for npm packages (#20379)
automatically set repository link for package based on the repository
url present inside package.json

closes #20146
2023-03-28 13:55:03 -04:00
Samuel FORESTIER dbdb5ba33e
Sort Python package descriptors by version to mimic PyPI format (#23550)
---

Hi, very naive and **untested** first time ever Go code, feel free to
reject/edit this as needed.
(PyPI actually performs "naive" string comparison too)

---------

Co-authored-by: Lauris BH <lauris@nix.lv>
2023-03-19 15:19:37 +01:00
Zettat123 46addc1f93
Return `repository` in npm package metadata endpoint (#23539)
Close #23444 

Add `Repository` to npm package `Metadata` struct so the `repository` in
`package.json` can be stored and be returned in the endpoint.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-03-17 14:39:19 -04:00
KN4CK3R 8570593d55
Add package registry architecture overview (#23445)
As announced in #22810 I added a readme file to help understanding how
the package registry archictecture works and how the go packages are
related.
2023-03-13 18:15:09 -04:00
KN4CK3R c709fa17a7
Add Swift package registry (#22404)
This PR adds a [Swift](https://www.swift.org/) package registry.


![grafik](https://user-images.githubusercontent.com/1666336/211842523-07521cbd-8fb6-400f-820c-ee8048b05ae8.png)
2023-03-13 15:28:39 -05:00
zeripath 1319ba6742
Use minio/sha256-simd for accelerated SHA256 (#23052)
minio/sha256-simd provides additional acceleration for SHA256 using
AVX512, SHA Extensions for x86 and ARM64 for ARM.

It provides a drop-in replacement for crypto/sha256 and if the
extensions are not available it falls back to standard crypto/sha256.

---------

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2023-02-22 14:21:46 -05:00
KN4CK3R 007d181bb5
Notify on container image create (#22806)
Fixes #22791

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-18 13:36:38 +08:00
KN4CK3R 9057a008a1
Add `/$count` endpoints for NuGet v2 (#22855)
Fixes #22838

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-11 19:30:44 +08:00
KN4CK3R f8c1e14a13
Use import of OCI structs (#22765)
Fixes #22758

Otherwise we would need to rewrite the structs in `oci.go`.
2023-02-06 10:07:09 +00:00
KN4CK3R d987ac6bf1
Add Chef package registry (#22554)
This PR implements a [Chef registry](https://chef.io/) to manage
cookbooks. This package type was a bit complicated because Chef uses RSA
signed requests as authentication with the registry.


![grafik](https://user-images.githubusercontent.com/1666336/213747995-46819fd8-c3d6-45a2-afd4-a4c3c8505a4a.png)


![grafik](https://user-images.githubusercontent.com/1666336/213748145-d01c9e81-d4dd-41e3-a3cc-8241862c3166.png)

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-06 09:49:21 +08:00
KN4CK3R df789d962b
Add Cargo package registry (#21888)
This PR implements a [Cargo registry](https://doc.rust-lang.org/cargo/)
to manage Rust packages. This package type was a little bit more
complicated because Cargo needs an additional Git repository to store
its package index.

Screenshots:

![grafik](https://user-images.githubusercontent.com/1666336/203102004-08d812ac-c066-4969-9bda-2fed818554eb.png)

![grafik](https://user-images.githubusercontent.com/1666336/203102141-d9970f14-dca6-4174-b17a-50ba1bd79087.png)

![grafik](https://user-images.githubusercontent.com/1666336/203102244-dc05743b-78b6-4d97-998e-ef76341a978f.png)

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-05 18:12:31 +08:00
KN4CK3R 6ba9ff7b48
Add Conda package registry (#22262)
This PR adds a [Conda](https://conda.io/) package registry.
2023-02-01 12:30:39 -06:00
KN4CK3R d283a31f03
Check quota limits for container uploads (#22450)
The test coverage has revealed that container packages were not checked
against the quota limits.
2023-01-29 11:34:29 -06:00
KN4CK3R 3510d7e33a
Fix container blob mount (#22226) 2023-01-16 17:35:48 -05:00
KN4CK3R 3fef47b41c
Use ErrInvalidArgument in packages (#22268)
Related to
https://github.com/go-gitea/gitea/pull/22262#discussion_r1059010774

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-31 12:49:37 +01:00
KN4CK3R a35749893b
Move `convert` package to services (#22264)
Addition to #22256

The `convert` package relies heavily on different models which is
[disallowed by our definition of
modules](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#design-guideline).
This helps to prevent possible import cycles.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-29 10:57:15 +08:00
Lunny Xiao ca67c5a8a7
refactor auth interface to return error when verify failure (#22119)
This PR changed the Auth interface signature from 
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) *user_model.User`
to 
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) (*user_model.User, error)`.

There is a new return argument `error` which means the verification
condition matched but verify process failed, we should stop the auth
process.

Before this PR, when return a `nil` user, we don't know the reason why
it returned `nil`. If the match condition is not satisfied or it
verified failure? For these two different results, we should have
different handler. If the match condition is not satisfied, we should
try next auth method and if there is no more auth method, it's an
anonymous user. If the condition matched but verify failed, the auth
process should be stop and return immediately.

This will fix #20563

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Jason Song <i@wolfogre.com>
2022-12-28 13:53:28 +08:00
KN4CK3R 86ace4b5c2
Normalize NuGet package version on upload (#22186)
Fixes #22178

After this change upload versions with different semver metadata are
treated as the same version and trigger a duplicated version error.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-20 22:20:23 -05:00
KN4CK3R 07461e18d3
Fix condition for is_internal (#22095)
depends on #22094

Fixes https://codeberg.org/forgejo/forgejo/issues/77

The old logic did not consider `is_internal`.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-12-14 17:16:01 +08:00
silverwind 0585ac3ac6
Update go dev dependencies (#22064)
`golangci-lint`
[deprecated](https://github.com/golangci/golangci-lint/issues/1841) a
bunch of linters, removed them.
2022-12-08 16:21:37 +08:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
Jason Song 9607750b5e
Replace fmt.Sprintf with hex.EncodeToString (#21960)
`hex.EncodeToString` has better performance than `fmt.Sprintf("%x",
[]byte)`, we should use it as much as possible.

I'm not an extreme fan of performance, so I think there are some
exceptions:

- `fmt.Sprintf("%x", func(...)[N]byte())`
- We can't slice the function return value directly, and it's not worth
adding lines.
    ```diff
    func A()[20]byte { ... }
    - a := fmt.Sprintf("%x", A())
    - a := hex.EncodeToString(A()[:]) // invalid
    + tmp := A()
    + a := hex.EncodeToString(tmp[:])
    ```
- `fmt.Sprintf("%X", []byte)`
- `strings.ToUpper(hex.EncodeToString(bytes))` has even worse
performance.
2022-11-28 11:19:18 +00:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R a1ae83f36e
Workaround for container registry push/pull errors (#21862)
This PR addresses #19586

I added a mutex to the upload version creation which will prevent the
push errors when two requests try to create these database entries. I'm
not sure if this should be the final solution for this problem.

I added a workaround to allow a reupload of missing blobs. Normally a
reupload is skipped because the database knows the blob is already
present. The workaround checks if the blob exists on the file system.
This should not be needed anymore with the above fix so I marked this
code to be removed with Gitea v1.20.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-25 13:47:46 +08:00
KN4CK3R fc7a2d5a95
Add support for HEAD requests in Maven registry (#21834)
Related #18543

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-24 16:25:13 +02:00
Jason Song e4eaa68a2b
Replace yaml.v2 with yaml.v3 (#21832)
I don't see why we have to use two versions of yaml. The difference
between the two versions has nothing to do with our usage.
2022-11-21 16:36:59 +08:00