go panics otherwise with `panic: interface conversion: error is git.ErrNotExist, not *git.ErrNotExist`, thanks to Codeberg/Andi for reporting this.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
It is possible that tag commits could be deleted or missing from repos. This causes
migration 156 to fail and breaks upgrade.
This PR simply logs the failure.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
If an avatar is requested in a particular size ensure that /avatars also gets the size request
Fix#15453
Signed-off-by: Andrew Thornton <art27@cantab.net>
Some postgres users have logging which logs even failed transactions. So
just query the db before trying to insert.
Fix#15451
Signed-off-by: Andrew Thornton art27@cantab.net
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Refactored handleOAuth2SignIn in routers/user/auth.go
The function handleOAuth2SignIn was called twice but some code path could only
be reached by one of the invocations. Moved the unnecessary code path out of
handleOAuth2SignIn.
* Refactored user creation
There was common code to create a user and display the correct error message.
And after the creation the only user should be an admin and if enabled a
confirmation email should be sent. This common code is now abstracted into
two functions and a helper function to call both.
* Added auto-register for OAuth2 users
If enabled new OAuth2 users will be registered with their OAuth2 details.
The UserID, Name and Email fields from the gothUser are used.
Therefore the OpenID Connect provider needs additional scopes to return
the coresponding claims.
* Added error for missing fields in OAuth2 response
* Linking and auto linking on oauth2 registration
* Set default username source to nickname
* Add automatic oauth2 scopes for github and google
* Add hint to change the openid connect scopes if fields are missing
* Extend info about auto linking security risk
Co-authored-by: Viktor Kuzmin <kvaster@gmail.com>
Signed-off-by: Martin Michaelis <code@mgjm.de>
The Session table must have an Expiry field not a created_unix field - somehow
this migration adds the incorrect named field leading to #15445 reports.
Fix#15445
Signed-off-by: Andrew Thornton <art27@cantab.net>
The issue is that the TestPatch will reset the PR MergeBase - and it is possible for TestPatch to update the MergeBase whilst a merge is ongoing. The ensuing merge will then complete but it doesn't re-set the MergeBase it used to merge the PR.
Fixes the intermittent error in git test.
Signed-off-by: Andrew Thornton art27@cantab.net
* _ to unused func options
* rm useless brakets
* rm trifial non used models functions
* rm dead code
* rm dead global vars
* fix routers/api/v1/repo/issue.go
* dont overload import module
* Implemented LFS client.
* Implemented scanning for pointer files.
* Implemented downloading of lfs files.
* Moved model-dependent code into services.
* Removed models dependency. Added TryReadPointerFromBuffer.
* Migrated code from service to module.
* Centralised storage creation.
* Removed dependency from models.
* Moved ContentStore into modules.
* Share structs between server and client.
* Moved method to services.
* Implemented lfs download on clone.
* Implemented LFS sync on clone and mirror update.
* Added form fields.
* Updated templates.
* Fixed condition.
* Use alternate endpoint.
* Added missing methods.
* Fixed typo and make linter happy.
* Detached pointer parser from gogit dependency.
* Fixed TestGetLFSRange test.
* Added context to support cancellation.
* Use ReadFull to probably read more data.
* Removed duplicated code from models.
* Moved scan implementation into pointer_scanner_nogogit.
* Changed method name.
* Added comments.
* Added more/specific log/error messages.
* Embedded lfs.Pointer into models.LFSMetaObject.
* Moved code from models to module.
* Moved code from models to module.
* Moved code from models to module.
* Reduced pointer usage.
* Embedded type.
* Use promoted fields.
* Fixed unexpected eof.
* Added unit tests.
* Implemented migration of local file paths.
* Show an error on invalid LFS endpoints.
* Hide settings if not used.
* Added LFS info to mirror struct.
* Fixed comment.
* Check LFS endpoint.
* Manage LFS settings from mirror page.
* Fixed selector.
* Adjusted selector.
* Added more tests.
* Added local filesystem migration test.
* Fixed typo.
* Reset settings.
* Added special windows path handling.
* Added unit test for HTTPClient.
* Added unit test for BasicTransferAdapter.
* Moved into util package.
* Test if LFS endpoint is allowed.
* Added support for git://
* Just use a static placeholder as the displayed url may be invalid.
* Reverted to original code.
* Added "Advanced Settings".
* Updated wording.
* Added discovery info link.
* Implemented suggestion.
* Fixed missing format parameter.
* Added Pointer.IsValid().
* Always remove model on error.
* Added suggestions.
* Use channel instead of array.
* Update routers/repo/migrate.go
* fmt
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
* Unexport SendUserMail
* Instead of "[]*models.User" or "[]string" lists infent "[]*MailRecipient" for mailer
* adopt
* code format
* TODOs for "i18n"
* clean
* no fallback for lang -> just use english
* lint
* exec testComposeIssueCommentMessage per lang and use only emails
* rm MailRecipient
* Dont reload from users from db if you alredy have in ram
* nits
* minimize diff
Signed-off-by: 6543 <6543@obermui.de>
* localize subjects
* linter ...
* Tr extend
* start tmpl edit ...
* Apply suggestions from code review
* use translation.Locale
* improve mailIssueCommentBatch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add i18n to datas
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a comment
Co-authored-by: Andrew Thornton <art27@cantab.net>
/api/v1/repos/issues/search is a highly inefficient search which is unfortunately
the basis for our dependency searching algorithm. In particular it currently loads
all of the repositories and their owners and their primary coding language all of
which is immediately thrown away.
This PR makes one simple change - just get the IDs.
Related #14560
Related #12827
Signed-off-by: Andrew Thornton <art27@cantab.net>
git gc cron could change the size of the repository therefore we should update the
size of the repo stored in our database.
Also significantly improve the efficiency of counting lfs associated with the
repository
* Create Proper Migration tests
Unfortunately our testing regime has so far meant that migrations do not
get proper testing.
This PR begins the process of creating migration tests for this.
* Add test for v176
* fix mssql drop db
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix Migration 176 yet again
Whilst creating a test for v176 in the migrations_test PR
it has become clear that this was still wrong.
This is now fixed. Genuinely.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* and fix repo transfer
Signed-off-by: Andrew Thornton <art27@cantab.net>
There is a serious issue with the v176 migration where there is a mistaken missing
label_id selection.
*introduced by #14912*
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Never add labels not from this repository or organisation and remove org labels on transfer
Prevent the addition of labels from outside of the repository or
organisation and remove organisation labels on transfer.
Related #14908
Signed-off-by: Andrew Thornton <art27@cantab.net>
* switch to use sql
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove AS
Signed-off-by: Andrew Thornton <art27@cantab.net>
* subquery alias
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Give me some AS?
Signed-off-by: Andrew Thornton <art27@cantab.net>
* double AS
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
* once more around the merry go round
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix api problem
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add outside label consistency check into doctor
This PR adds another consistency check into doctor in order to detect
labels that have been added from outside of repositories and organisations
Fix#14908
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix migration
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for merge
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix postgres ID sequences broken by recreate-table
Unfortunately there is a subtle problem with recreatetable on postgres which
leads to the sequences not being renamed and not being left at 0.
Fix#14725
Signed-off-by: Andrew Thornton <art27@cantab.net>
* let us try information_schema instead
Signed-off-by: Andrew Thornton <art27@cantab.net>
* try again
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
* Ensure validation occurs on clone addresses too
Fix#14984
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix api tests
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* chore: rewrite format.
* chore: update format
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* chore: update format
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* chore: Adjacent parameters with the same type should be grouped together
* chore: update format.
* API: fix set milestone on PR creation
pr creation via API failed with 404, because we searched
for milestoneID 0, due to uninitialized var usage D:
* add tests
* fix expected status codes
* fix tests
Co-authored-by: 6543 <6543@obermui.de>
* Never add labels not from this repository or organisation and remove org labels on transfer
Prevent the addition of labels from outside of the repository or
organisation and remove organisation labels on transfer.
Related #14908
* switch to use sql
* subquery alias
* once more around the merry go round
* fix api problem
Closed milestones and issues should only be marked overdue if they were
closed after their deadline.
Fix: #14536
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make auto check manual merge as a chooseable mod and add manual merge way on ui
as title, Before this pr, we use same way with GH to check manually merge.
It good, but in some special cases, misjudgments can occur. and it's hard
to fix this bug. So I add option to allow repo manager block "auto check manual merge"
function, Then it will have same style like gitlab(allow empty pr). and to compensate for
not being able to detect THE PR merge automatically, I added a manual approach.
Signed-off-by: a1012112796 <1012112796@qq.com>
* make swager
* api support
* ping ci
* fix TestPullCreate_EmptyChangesWithCommits
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* Apply review suggestions and add test
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* fix build
* test error message
* make fmt
* Fix indentation issues identified by @silverwind
Co-authored-by: silverwind <me@silverwind.io>
* Fix tests and make manually merged disabled error on API the same
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a small nit
* fix wrong commit id error
* fix bug
* simple test
* fix test
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Most DBs apart from SQLite will use a default Collation that is not case insensitive.
This means that SearchIssuesByKeyword becomes case sensitive for db indexing - in
contrast to the bleve and elastic indexers.
This PR simply uses UPPER(...) to do the LIKE - and although it may be more efficient
to change collations this would be a non-trivial task.
Fix#13663
Signed-off-by: Andrew Thornton <art27@cantab.net>
* make repo as "pending transfer" if on transfer start doer has no right to create repo in new destination
* if new pending transfer ocured, create UI & Mail notifications
Instead of causing a log.Fatal, we should handle broken OAuth2
providers by disabling them.
Fix#8930
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Create Xorm session provider
This PR creates a Xorm session provider which creates
the appropriate Session table for macaron/session.
Fix#7137
Signed-off-by: Andrew Thornton <art27@cantab.net>
* extraneous l
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use key instead of ID to be compatible with go-macaron/session
Signed-off-by: Andrew Thornton <art27@cantab.net>
* And change the migration too.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update spacing of imports
Co-authored-by: 6543 <6543@obermui.de>
* Update modules/session/xorm.go
Co-authored-by: techknowlogick <matti@mdranta.net>
* add xorm provider to the virtual provider
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prep for master merge
* prep for merge master
* As per @lunny
* move migration out of the way
* Move to call this db session as per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Prevent adding nil label to .AddedLabels or .RemovedLabels
There are possibly a few old databases out there with malmigrated data that can
cause panics with empty labels being migrated.
This PR adds a few tests to prevent nil labels being added.
Fix#14466
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add doctor command to remove the broken label comments
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Fix GPG key deletion when user is deleted
Per #14531, deleting a user account will delete the user's GPG keys
from the `gpg_key` table but not from `gpg_key_import`, which causes
an error when creating an account with the same email and attempting
to re-add the same key. This commit deletes all entries from
`gpg_key_import` that match any GPG key IDs belonging to the user.
* Format added code in models/user.go
* Create a new function for listing GPG keys and apply it
Create a new function `listGPGKeys` and replace a previous use
of `ListGPGKeys`. Thanks to @6543 for the patch.
Co-authored-by: Anton Khimich <anton.khimicha@mail.utoronto.ca>
Co-authored-by: 6543 <6543@obermui.de>
Migrations currently uses the default Xorm mapper which is
not the same as the mapper Gitea actually uses.
This means that there is a difference between the struct
parsing and mapping to database tables in migrations as
compared to normal Sync2.
This was the cause for the catastrophic problem in v168 -
untagged fields are not mapped in the same way in migrations
as compared to outside of migrations.
This is also likely the cause of some weird subtle failures
in other migrations as any untagged field may not be being
mapped exactly the same way.
This PR suggests that we ensure that the mapper is set at
the start of the migrations code - but also enforces a strict
clean mapper between each migration.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix mig 141
* Add Migration to fix it
* update null values to false first
* Alter Table if posible
* use dropTableColumns instead of recreateTable
* MySQL use Alter
* Postgres use Alter
* Update models/migrations/v167.go
* Apply suggestions from code review
* use 2x add col & 2x update & 2x drop col
* let sqlite be the only issue
* use recreate since it just WORKS
Close **Prune hook_task Table (#10741)**
Added a cron job to delete webhook deliveries in the hook_task table. It can be turned on/off and the schedule controlled globally via app.ini. The data can be deleted by either the age of the delivery which is the default or by deleting the all but the most recent deliveries _per webhook_.
Note: I had previously submitted pr #11416 but I closed it when I realized that I had deleted per repository instead of per webhook. Also, I decided allowing the settings to be overridden via the ui was overkill. Also this version allows the deletion by age which is probably what most people would want.
* Add redirect for user
* Add redirect for orgs
* Add user redirect test
* Appease linter
* Add comment to DeleteUserRedirect function
* Fix locale changes
* Fix GetUserByParams
* Fix orgAssignment
* Remove debug logging
* Add redirect prompt
* Dont Export DeleteUserRedirect & only use it within a session
* Unexport newUserRedirect
* cleanup
* Fix & Dedub API code
* Format Template
* Add Migration & rm dublicat
* Refactor: unexport newRepoRedirect() & rm dedub del exec
* if this fails we'll need to re-rename the user directory
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* refactor models.DeleteComment and delete related reactions too
* use deleteComment for UserDeleteWithCommentsMaxDays in DeleteUser
* nits
* Use time.Duration as other time settings have
* docs
* Resolve Fixme & fix potential deadlock
* Disabled by Default
* Update Config Value Description
* switch args
* Update models/issue_comment.go
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
* add notification about running stopwatch to header
* serialize seconds, duration in stopwatches api
* ajax update stopwatch
i should get my testenv working locally...
* new variant: hover dialog
* noscript compatibility
* js: live-update stopwatch time
* js live update robustness
* Add support for ed25519_sk and ecdsa_sk SSH keys
These start with sk-ssh-ed25519@openssh.com and sk-ecdsa-sha2-nistp256@openssh.com.
They are supported in recent versions of go x/crypto/ssh and OpenSSH 8.2
or higher.
* skip ssh-keygen
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Implement ghost comment mitigation
Adds a config option USER_DELETE_WITH_COMMENTS_MAX_DAYS to the [service] section. See https://codeberg.org/Codeberg/Discussion/issues/24 for the underlying issue.
* cleanup
* use setting module correctly
* add to docs
Co-authored-by: Moritz Marquardt <git@momar.de>
* Add review requested filter on pull request overview #13682
fix formatting
* add review_requested filter to /repos/issues/search API endpoint
* only Approve and Reject status should supersede Request status
* add support for team reviews
* refactor: remove duplication of issue filtering conditions
* move SaltGeneration into HashPasswort and rename it to what it does
* Migration: Where Password is Valid with Empty String delete it
* prohibit empty password hash
* let SetPassword("") unset pwd stuff
Fixed#8861
* use ajax on PR review page
* handle review comments
* extract duplicate code
FetchCodeCommentsByLine was initially more or less copied from fetchCodeCommentsByReview. Now they both use a common findCodeComments function instead
* use the Engine that was passed into the method
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix wrong type on hooktask to convert typ from char(16) to varchar(16)
* Fix bugs
* Improve code
* Use different trim function for MSSQL
* Fix bug
* Removed wrong changed line
* Removed wrong changed line
* Fix nullable
* Fix lint
* Ignore sqlite on migration
* Fix mssql modify column failure
* Move modifyColumn to migrations.go so that other migrate function could use it
* Added MirrorInterval to the API
* Remove MirrorInterval from CreateRepository
* Removed Duplicate UpdateMirror Function
* Updated Error Logging
* Update Log Message for is not Mirror
Co-authored-by: 6543 <6543@obermui.de>
* Delete Debug Statement that snuck in
Co-authored-by: zeripath <art27@cantab.net>
* Add Check for If Interval is too small
* Output to API Call
* Add Error Object when time is Less than Min Interval
* Frequency Error Message
Co-authored-by: zeripath <art27@cantab.net>
* Allow Zero Mirror Interval
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Fixes#14187: mention handling extracted from email notification code
Fixes#14013: add notification for mentions in pull request code comments
Fixes#13450: Not receiving any emails with setting "Only Email on Mention"
* Ensure that schema search path is set with every connection on postgres
Unfortunately every connection to postgres requires that the search path is
set appropriately.
This PR shadows the postgres driver to ensure that as soon as a connection
is open, the search_path is set appropriately.
Fix#14088
Signed-off-by: Andrew Thornton <art27@cantab.net>
* no golangci-lint that is not a helpful suggestion
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use Execer if available
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This is "minimal" in the sense that only the Authorization Code Flow
from OpenID Connect Core is implemented. No discovery, no configuration
endpoint, and no user scope management.
OpenID Connect is an extension to the (already implemented) OAuth 2.0
protocol, and essentially an `id_token` JWT is added to the access token
endpoint response when using the Authorization Code Flow. I also added
support for the "nonce" field since it is required to be used in the
id_token if the client decides to include it in its initial request.
In order to enable this extension an OAuth 2.0 scope containing
"openid" is needed. Other OAuth 2.0 requests should not be impacted by
this change.
This minimal implementation is enough to enable single sign-on (SSO)
for other sites, e.g. by using something like `mod_auth_openidc` to
only allow access to a CI server if a user has logged into Gitea.
Fixes: #1310
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
* Change topic name size from 25 to 50
* recreateTable requires full bean definition
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
* Disable SSH key addition and deletion when externally managed
When a user has a login source which has SSH key management
key addition and deletion using the UI should be disabled.
Fix#13983
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make only externally managed keys disabled
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* remove github.com/unknwon/com from models
* dont use "com.ToStr()"
* replace "com.ToStr" with "fmt.Sprint" where its easy to do
* more refactor
* fix test
* just "proxy" Copy func for now
* as per @lunny
* now uses the same permission model as for the activity feed:
only include activities in repos, that the doer has access to.
this might be somewhat slower.
* also improves handling of user.KeepActivityPrivate (still shows
the heatmap to self & admins)
* extend tests
* adjust integration test to new behaviour
* add access to actions for admins
* extend heatmap unit tests
* Show dropdown with all statuses for commit
* Use popups
* Remove unnecessary change
* Style popup
* Use divided list
* As per @silverwind
* Refactor GetLastCommitStatus
* Missing dropdown on repo home and commit page
* Fix tests
* Make status icon be a part of a link on PR list
* Fix missing translation call
* Indent fix
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Trim the branch prefix from action.GetBranch
#13882 has revealed that the refname of an action is actually only a
refname pattern and necessarily a branch. For examplem pushing to
refs/heads/master will result in action with refname refs/heads/master
but pushing to master will result in a refname master.
The simplest solution to providing a fix here is to trim the prefix
therefore this PR proposes this.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update models/action.go
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: a1012112796 <1012112796@qq.com>
The frontpage uses a rather strange method to obtain the commit's avatar
which I've overlooked earlier. I don't exactly understand how it works
but this change fixes the wrong default avatars by using the function
that was in previous use.
Also introduced a few constants for size an size increase factor.
Fixes: https://github.com/go-gitea/gitea/issues/13844
* Direct avatar rendering
This adds new template helpers for avatar rendering which output image
elements with direct links to avatars which makes them cacheable by the
browsers.
This should be a major performance improvment for pages with many avatars.
* fix avatars of other user's profile pages
* fix top border on user avatar name
* uncircle avatars
* remove old incomplete avatar selector
* use title attribute for name and add it back on blame
* minor refactor
* tweak comments
* fix url path join and adjust test to new result
* dedupe functions
* add black list and white list support for migrating repositories
* fix fmt
* fix lint
* fix vendor
* fix modules.txt
* clean diff
* specify log message
* use blocklist/allowlist
* allways use lowercase to match url
* Apply allow/block
* Settings: use existing "migrations" section
* convert domains lower case
* dont store unused value
* Block private addresses for migration by default
* fix lint
* use proposed-upstream func to detect private IP addr
* a nit
* add own error for blocked migration, add tests, imprufe api
* fix test
* fix-if-localhost-is-ipv4
* rename error & error message
* rename setting options
* Apply suggestions from code review
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* ui: show 'owner' tag for real owner
Signed-off-by: a1012112796 <1012112796@qq.com>
* Update custom/conf/app.example.ini
* simplify logic
fix logic
fix a small bug about original author
* remove system manager tag
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Add time filter for issue search
* Add limit option for paggination
* Add Filter for: Created by User, Assigned to User, Mentioning User
* update swagger
* Add Tests for limit, before & since
* Improve error feedback for duplicate deploy keys
Instead of a generic HTTP 500 error page, a flash message is rendered
with the deploy key page template so inform the user that a key with the
intended title already exists.
* API returns 422 error when key with name exists
* Add email validity checking
Add email validity checking for the following routes:
[Web interface]
1. User registration
2. User creation by admin
3. Adding an email through user settings
[API]
1. POST /admin/users
2. PATCH /admin/users/:username
3. POST /user/emails
* Add further tests
* Add signup email tests
* Add email validity check for linking existing account
* Address PR comments
* Remove unneeded DB session
* Move email check to updateUser
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
When migrating repositories with reactions with deleted users, the original
author id may be -1. This means that it is possible to end up attempting
to create multiple reactions with the same [ Type, IssueID, CommentID, UserID,
OriginalAuthorID ] thus breaking the constraints.
On SQLite this appears to cause a deadlock but on other dbs this will
cause the migration to fail.
This PR extends the constraint to include the original author username
in the constraint.
Fix#13271
Signed-off-by: Andrew Thornton <art27@cantab.net>
* When replying to an outdated comment it should not appear on the files page
This happened because the comment took the latest commitID as its base instead of the
reviewID that it was replying to.
There was also no way of creating an already outdated comment - and a
reply to a review on an outdated line should be outdated.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix broken migration
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix mssql
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create temporary table because ... well MSSQL ...
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create temporary table because ... well MSSQL ...
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create temporary table because ... well MSSQL ...
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix mssql
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move session within the batch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* regen the sqlcmd each time round the loop
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Make archival asynchronous
The prime benefit being sought here is for large archives to not
clog up the rendering process and cause unsightly proxy timeouts.
As a secondary benefit, archive-in-progress is moved out of the
way into a /tmp file so that new archival requests for the same
commit will not get fulfilled based on an archive that isn't yet
finished.
This asynchronous system is fairly primitive; request comes in, we'll
spawn off a new goroutine to handle it, then we'll mark it as done.
Status requests will see if the file exists in the final location,
and report the archival as done when it exists.
Fixes#11265
* Archive links: drop initial delay to three-quarters of a second
Some, or perhaps even most, archives will not take all that long to archive.
The archive process starts as soon as the download button is initially
clicked, so in theory they could be done quite quickly. Drop the initial
delay down to three-quarters of a second to make it more responsive in the
common case of the archive being quickly created.
* archiver: restructure a little bit to facilitate testing
This introduces two sync.Cond pointers to the archiver package. If they're
non-nil when we go to process a request, we'll wait until signalled (at all)
to proceed. The tests will then create the sync.Cond so that it can signal
at-will and sanity-check the state of the queue at different phases.
The author believes that nil-checking these two sync.Cond pointers on every
archive processing will introduce minimal overhead with no impact on
maintainability.
* gofmt nit: no space around binary + operator
* services: archiver: appease golangci-lint, lock queueMutex
Locking/unlocking the queueMutex is allowed, but not required, for
Cond.Signal() and Cond.Broadcast(). The magic at play here is just a little
too much for golangci-lint, as we take the address of queueMutex and this is
mostly used in archiver.go; the variable still gets flagged as unused.
* archiver: tests: fix several timing nits
Once we've signaled a cond var, it may take some small amount of time for
the goroutines released to hit the spot we're wanting them to be at. Give
them an appropriate amount of time.
* archiver: tests: no underscore in var name, ungh
* archiver: tests: Test* is run in a separate context than TestMain
We must setup the mutex/cond variables at the beginning of any test that's
going to use it, or else these will be nil when the test is actually ran.
* archiver: tests: hopefully final tweak
Things got shuffled around such that we carefully build up and release
requests from the queue, so we can validate the state of the queue at each
step. Fix some assertions that no longer hold true as fallout.
* repo: Download: restore some semblance of previous behavior
When archival was made async, the GET endpoint was only useful if a previous
POST had initiated the download. This commit restores the previous behavior,
to an extent; we'll now submit the archive request there and return a
"202 Accepted" to indicate that it's processing if we didn't manage to
complete the request within ~2 seconds of submission.
This lets a client directly GET the archive, and gives them some indication
that they may attempt to GET it again at a later time.
* archiver: tests: simplify a bit further
We don't need to risk failure and use time.ParseDuration to get 2 *
time.Second.
else if isn't really necessary if the conditions are simple enough and lead
to the same result.
* archiver: tests: resolve potential source of flakiness
Increase all timeouts to 10 seconds; these aren't hard-coded sleeps, so
there's no guarantee we'll actually take that long. If we need longer to
not have a false-positive, then so be it.
While here, various assert.{Not,}Equal arguments are flipped around so that
the wording in error output reflects reality, where the expected argument is
second and actual third.
* archiver: setup infrastructure for notifying consumers of completion
This API will *not* allow consumers to subscribe to specific requests being
completed, just *any* request being completed. The caller is responsible for
determining if their request is satisfied and waiting again if needed.
* repo: archive: make GET endpoint synchronous again
If the request isn't complete, this endpoint will now submit the request and
wait for completion using the new API. This may still be susceptible to
timeouts for larger repos, but other endpoints now exist that the web
interface will use to negotiate its way through larger archive processes.
* archiver: tests: amend test to include WaitForCompletion()
This is a trivial one, so go ahead and include it.
* archiver: tests: fix test by calling NewContext()
The mutex is otherwise uninitialized, so we need to ensure that we're
actually initializing it if we plan to test it.
* archiver: tests: integrate new WaitForCompletion a little better
We can use this to wait for archives to come in, rather than spinning and
hoping with a timeout.
* archiver: tests: combine numQueued declaration with next-instruction assignment
* routers: repo: reap unused archiving flag from DownloadStatus()
This had some planned usage before, indicating whether this request
initiated the archival process or not. After several rounds of refactoring,
this use was deemed not necessary for much of anything and got boiled down
to !complete in all cases.
* services: archiver: restructure to use a channel
We now offer two forms of waiting for a request:
- WaitForCompletion: wait for completion with no timeout
- TimedWaitForCompletion: wait for completion with timeout
In both cases, we wait for the given request's cchan to close; in the latter
case, we do so with the caller-provided timeout. This completely removes the
need for busy-wait loops in Download/InitiateDownload, as it's fairly clean
to wait on a channel with timeout.
* services: archiver: use defer to unlock now that we can
This previously carried the lock into the goroutine, but an intermediate
step just added the request to archiveInProgress outside of the new
goroutine and removed the need for the goroutine to start out with it.
* Revert "archiver: tests: combine numQueued declaration with next-instruction assignment"
This reverts commit bcc5214023.
Revert "archiver: tests: integrate new WaitForCompletion a little better"
This reverts commit 9fc8bedb56.
Revert "archiver: tests: fix test by calling NewContext()"
This reverts commit 709c35685e.
Revert "archiver: tests: amend test to include WaitForCompletion()"
This reverts commit 75261f56bc.
* archiver: tests: first attempt at WaitForCompletion() tests
* archiver: tests: slight improvement, less busy-loop
Just wait for the requests to complete in order, instead of busy-waiting
with a timeout. This is slightly less fragile.
While here, reverse the arguments of a nearby assert.Equal() so that
expected/actual are correct in any test output.
* archiver: address lint nits
* services: archiver: only close the channel once
* services: archiver: use a struct{} for the wait channel
This makes it obvious that the channel is only being used as a signal,
rather than anything useful being piped through it.
* archiver: tests: fix expectations
Move the close of the channel into doArchive() itself; notably, before these
goroutines move on to waiting on the Release cond.
The tests are adjusted to reflect that we can't WaitForCompletion() after
they've already completed, as WaitForCompletion() doesn't indicate that
they've been released from the queue yet.
* archiver: tests: set cchan to nil for comparison
* archiver: move ctx.Error's back into the route handlers
We shouldn't be setting this in a service, we should just be validating the
request that we were handed.
* services: archiver: use regex to match a hash
This makes sure we don't try and use refName as a hash when it's clearly not
one, e.g. heads/pull/foo.
* routers: repo: remove the weird /archive/status endpoint
We don't need to do this anymore, we can just continue POSTing to the
archive/* endpoint until we're told the download's complete. This avoids a
potential naming conflict, where a ref could start with "status/"
* archiver: tests: bump reasonable timeout to 15s
* archiver: tests: actually release timedReq
* archiver: tests: run through inFlight instead of manually checking
While we're here, add a test for manually re-processing an archive that's
already been complete. Re-open the channel and mark it incomplete, so that
doArchive can just mark it complete again.
* initArchiveLinks: prevent default behavior from clicking
* archiver: alias gitea's context, golang context import pending
* archiver: simplify logic, just reconstruct slices
While the previous logic was perhaps slightly more efficient, the
new variant's readability is much improved.
* archiver: don't block shutdown on waiting for archive
The technique established launches a goroutine to do the wait,
which will close a wait channel upon termination. For the timeout
case, we also send back a value indicating whether the timeout was
hit or not.
The timeouts are expected to be relatively small, but still a multi-
second delay to shutdown due to this could be unfortunate.
* archiver: simplify shutdown logic
We can just grab the shutdown channel from the graceful manager instead of
constructing a channel to halt the caller and/or pass a result back.
* Style issues
* Fix mis-merge
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
* When replying to an outdated comment it should not appear on the files page
This happened because the comment took the latest commitID as its base instead of the
reviewID that it was replying to.
There was also no way of creating an already outdated comment - and a
reply to a review on an outdated line should be outdated.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Refactor Logger to make a logger interface and make it possible to
wrap loggers for specific purposes.
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Create function to group label comments
* Combine multiple label additions into one
* Group removed and added labels in the same comment
* Fix indentation on comments.tmpl
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
* added mastodon provider to oauth code
* changed go.mod and vendor/modules.txt to add updated goth
* vendored mastodon and new goth
* committing result of go mod tidy && go mod vendor
* added pic and mastodon to oauth models
* handled instance url
* applied lafriks suggestion
* Update web_src/js/index.js
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <matti@mdranta.net>
* Ensure topics added using the API are added to the repository
Fix#12426
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Store task errors following migrations and display them
When migrate tasks fail store the error in the task table
and ensure that they show on the status page.
Fix#13242
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update web_src/js/index.js
* Hide the failed first
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Keep database transactions not too big
* Fix#13255
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Only cache the last repo
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
techknowlogick