mirror of
1
Fork 0
Commit Graph

214 Commits

Author SHA1 Message Date
zeripath 62eb1b0f25 Graceful Queues: Issue Indexing and Tasks (#9363)
* Queue: Add generic graceful queues with settings

* Queue & Setting: Add worker pool implementation

* Queue: Add worker settings

* Queue: Make resizing worker pools

* Queue: Add name variable to queues

* Queue: Add monitoring

* Queue: Improve logging

* Issues: Gracefulise the issues indexer

Remove the old now unused specific queues

* Task: Move to generic queue and gracefulise

* Issues: Standardise the issues indexer queue settings

* Fix test

* Queue: Allow Redis to connect to unix

* Prevent deadlock during early shutdown of issue indexer

* Add MaxWorker settings to queues

* Merge branch 'master' into graceful-queues

* Update modules/indexer/issues/indexer.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/indexer/issues/indexer.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/queue/queue_channel.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/queue/queue_disk.go

* Update modules/queue/queue_disk_channel.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Rename queue.Description to queue.ManagedQueue as per @guillep2k

* Cancel pool workers when removed

* Remove dependency on queue from setting

* Update modules/queue/queue_redis.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* As per @guillep2k add mutex locks on shutdown/terminate

* move unlocking out of setInternal

* Add warning if number of workers < 0

* Small changes as per @guillep2k

* No redis host specified not found

* Clean up documentation for queues

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

* Update modules/indexer/issues/indexer_test.go

* Ensure that persistable channel queue is added to manager

* Rename QUEUE_NAME REDIS_QUEUE_NAME

* Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME"

This reverts commit 1f83b4fc9b.

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-01-07 12:23:09 +01:00
zeripath e3c3b33ea7
Graceful: Xorm, RepoIndexer, Cron and Others (#9282)
* Change graceful to use a singleton obtained through GetManager instead of a global.
* Graceful: Make TestPullRequests shutdownable
* Graceful: Make the cron tasks graceful
* Graceful: AddTestPullRequest run in graceful ctx
* Graceful: SyncMirrors shutdown
* Graceful: SetDefaultContext for Xorm to be HammerContext
* Avoid starting graceful for migrate commands and checkout
* Graceful: DeliverHooks now can be shutdown
* Fix multiple syncing errors in modules/sync/UniqueQueue &  Make UniqueQueue closable
* Begin the process of making the repo indexer shutdown gracefully
2019-12-15 09:51:28 +00:00
zeripath 60c5339042 Graceful: Cancel Process on monitor pages & HammerTime (#9213)
* Graceful: Create callbacks to with contexts

* Graceful: Say when Gitea is completely finished

* Graceful: Git and Process within HammerTime

Force all git commands to terminate at HammerTime
Force all process commands to terminate at HammerTime
Move almost all git processes to run as git Commands

* Graceful: Always Hammer after Shutdown

* ProcessManager: Add cancel functionality

* Fix tests

* Make sure that process.Manager.Kill() cancels

* Make threadsafe access to Processes and remove own unused Kill

* Remove cmd from the process manager as it is no longer used

* the default context is the correct context

* get rid of double till
2019-11-30 16:40:22 +02:00
zeripath 54dab5aed9 Properly fix displaying virtual session provider in admin panel (#9137)
* Properly fix #7127

Although #7300 properly shadows the password from the virtual session
provider, the template displaying the provider config still presumed
that the config was JSON.

This PR updates the template and properly hides the Virtual Session
provider.

Fixes #7127

* update per @silverwind's suggestion
2019-11-28 12:35:41 -05:00
QuaSoft 7b4d2f7a2a Add single sign-on support via SSPI on Windows (#8463)
* Add single sign-on support via SSPI on Windows

* Ensure plugins implement interface

* Ensure plugins implement interface

* Move functions used only by the SSPI auth method to sspi_windows.go

* Field SSPISeparatorReplacement of AuthenticationForm should not be required via binding, as binding will insist the field is non-empty even if another login type is selected

* Fix breaking of oauth authentication on download links. Do not create new session with SSPI authentication on download links.

* Update documentation for the new 'SPNEGO with SSPI' login source

* Mention in documentation that ROOT_URL should contain the FQDN of the server

* Make sure that Contexter is not checking for active login sources when the ORM engine is not initialized (eg. when installing)

* Always initialize and free SSO methods, even if they are not enabled, as a method can be activated while the app is running (from Authentication sources)

* Add option in SSPIConfig for removing of domains from logon names

* Update helper text for StripDomainNames option

* Make sure handleSignIn() is called after a new user object is created by SSPI auth method

* Remove default value from text of form field helper

Co-Authored-By: Lauris BH <lauris@nix.lv>

* Remove default value from text of form field helper

Co-Authored-By: Lauris BH <lauris@nix.lv>

* Remove default value from text of form field helper

Co-Authored-By: Lauris BH <lauris@nix.lv>

* Only make a query to the DB to check if SSPI is enabled on handlers that need that information for templates

* Remove code duplication

* Log errors in ActiveLoginSources

Co-Authored-By: Lauris BH <lauris@nix.lv>

* Revert suffix of randomly generated E-mails for Reverse proxy authentication

Co-Authored-By: Lauris BH <lauris@nix.lv>

* Revert unneeded white-space change in template

Co-Authored-By: Lauris BH <lauris@nix.lv>

* Add copyright comments at the top of new files

* Use loopback name for randomly generated emails

* Add locale tag for the SSPISeparatorReplacement field with proper casing

* Revert casing of SSPISeparatorReplacement field in locale file, moving it up, next to other form fields

* Update docs/content/doc/features/authentication.en-us.md

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Remove Priority() method and define the order in which SSO auth methods should be executed in one place

* Log authenticated username only if it's not empty

* Rephrase helper text for automatic creation of users

* Return error if more than one active SSPI auth source is found

* Change newUser() function to return error, letting caller log/handle the error

* Move isPublicResource, isPublicPage and handleSignIn functions outside SSPI auth method to allow other SSO methods to reuse them if needed

* Refactor initialization of the list containing SSO auth methods

* Validate SSPI settings on POST

* Change SSPI to only perform authentication on its own login page, API paths and download links. Leave Toggle middleware to redirect non authenticated users to login page

* Make 'Default language' in SSPI config empty, unless changed by admin

* Show error if admin tries to add a second authentication source of type SSPI

* Simplify declaration of global variable

* Rebuild gitgraph.js on Linux

* Make sure config values containing only whitespace are not accepted
2019-11-23 01:33:31 +02:00
guillep2k e4ec32de2e Fix password checks on admin create/edit user (#9076)
* Fix password checks on admin create/edit user

* Remove incorrect trimspace
2019-11-20 00:07:51 +00:00
guillep2k c57edb6c7b Add password requirement info on error (#9074)
* Add password requirement info on error

* Move BuildComplexityError to the password pkg

* Unexport complexity type

* Fix extra line

* Update modules/password/password.go

Co-Authored-By: Lauris BH <lauris@nix.lv>
2019-11-19 22:44:58 +00:00
zeripath ae36ed7ecb
Shadow password correctly for session config (#8984)
Fix #8718

This PR shadows passwords in session config correctly by detecting
the VirtualProvider, unmarshalling the original config and then
shadowing config within that.
2019-11-14 20:06:02 +00:00
Lauris BH 86a44f6b48
Fix new user form for non-local users (#8826) 2019-11-04 21:10:37 +02:00
Lunny Xiao 9e85358777
Move some repositories' operations to a standalone service package (#8557)
* Move some repositories' operations to a standalone service package

* improve code

* remove unused codes

* add rollback when fork failed

* add repo when return
2019-10-26 14:54:11 +08:00
guillep2k 31655aabfc Fix password complexity regex for special characters (on master) (#8525)
* Fix extra space

* Fix regular expression

* Fix error template name

* Simplify check code, fix default values, add test

* Fix router tests

* Fix fmt

* Fix setting and lint

* Move cleaning up code to test, improve comments

* Tidy up variable declaration
2019-10-16 11:09:58 +08:00
Maxim Tkachenko db657192d0 Password Complexity Checks (#6230)
Add password complexity checks. The default settings require a lowercase, uppercase, number and a special character within passwords.

Co-Authored-By: T-M-A <maxim.tkachenko@gmail.com>
Co-Authored-By: Lanre Adelowo <adelowomailbox@gmail.com>
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-Authored-By: Lauris BH <lauris@nix.lv>
2019-10-14 16:24:26 +01:00
Lunny Xiao 5a438ee3c0
Move all mail related codes from models to services/mailer (#7200)
* move all mail related codes from models to modules/mailer

* fix lint

* use DBContext instead Engine

* use WithContext not WithEngine

* Use DBContext instead of Engine

* don't use defer when sess.Close()

* move DBContext to context.go and add some methods

* move mailer from modules/ to services

* fix lint

* fix tests

* fix fmt

* add gitea copyright

* fix tests

* don't expose db functions

* make code clear

* add DefaultDBContext

* fix build

* fix bug
2019-09-24 13:02:49 +08:00
Lunny Xiao f83db078f0 Move database settings from models to setting (#7806)
* move database settings from models to setting

* update docs

* fix checkout pr

* fix tests

* fix lint

* remove unsupported tidb options

* correct wrong variable name

* remove tidb totally
2019-08-24 11:24:45 +02:00
Tamal Saha 171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
Lunny Xiao 85202d4784
Display ui time with customize time location (#7792)
* display ui time with customize time location

* fix lint

* rename UILocation to DefaultUILocation

* move time related functions to modules/timeutil

* fix tests

* fix tests

* fix build

* fix swagger
2019-08-15 22:46:21 +08:00
Lunny Xiao 161e12e157
Shadow the password on cache and session config on admin panel (#7300)
* shadow the password on cache and session config on admin panel

* add shadow password of mysql/postgres/couchbase

* fix log import
2019-06-27 00:12:38 +08:00
Lunny Xiao aa7c34cf86 Fix error log when loading issues caused by a xorm bug (#7271)
* fix error log when loading issues caused by a xorm bug

* upgrade packages

* fix fmt

* fix Consistency

* fix tests
2019-06-23 18:22:43 +03:00
Lunny Xiao 1e8a616426 show lfs config on admin panel (#7220) 2019-06-16 19:24:49 +01:00
kolaente f9ec2f89f2 Add golangci (#6418) 2019-06-12 15:41:28 -04:00
Mario Lubenka 8eba27c792 Repository avatar fallback configuration (#7087)
* Only show repository avatar in list when one was selected

Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>

* Adds fallback configuration option for repository avatar

Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>

* Implements repository avatar fallback

Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>

* Adds admin task for deleting generated repository avatars

Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>

* Solve linting issues

Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>

* Save avatar before updating database

* Linting

* Update models/repo.go

Co-Authored-By: zeripath <art27@cantab.net>
2019-06-02 14:40:12 +08:00
Lunny Xiao 710245e81e
Refactor models.NewRepoContext to extract git related codes to modules/git (#6941)
* refactor models.NewRepoContext to extract git related codes to modules/git

* fix imports

* refactor
2019-05-15 09:57:00 +08:00
Lunny Xiao 01ebd52a1f fix config ui error about cache ttl (#6861) 2019-05-06 17:35:11 +03:00
Mario Lubenka fcbac38d6f Unifies pagination template usage (#6531) (#6533) 2019-04-20 00:15:19 -04:00
zeripath 704da08fdc
Better logging (#6038) (#6095)
* Panic don't fatal on create new logger

Fixes #5854

Signed-off-by: Andrew Thornton <art27@cantab.net>

* partial broken

* Update the logging infrastrcture

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reset the skip levels for Fatal and Error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* broken ncsa

* More log.Error fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove nal

* set log-levels to lowercase

* Make console_test test all levels

* switch to lowercased levels

* OK now working

* Fix vetting issues

* Fix lint

* Fix tests

* change default logging to match current gitea

* Improve log testing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* reset error skip levels to 0

* Update documentation and access logger configuration

* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE

* Fix broken level caching

* Refactor the router log

* Add Router logger

* Add colorizing options

* Adjust router colors

* Only create logger if they will be used

* update app.ini.sample

* rename Attribute ColorAttribute

* Change from white to green for function

* Set fatal/error levels

* Restore initial trace logger

* Fix Trace arguments in modules/auth/auth.go

* Properly handle XORMLogger

* Improve admin/config page

* fix fmt

* Add auto-compression of old logs

* Update error log levels

* Remove the unnecessary skip argument from Error, Fatal and Critical

* Add stacktrace support

* Fix tests

* Remove x/sync from vendors?

* Add stderr option to console logger

* Use filepath.ToSlash to protect against Windows in tests

* Remove prefixed underscores from names in colors.go

* Remove not implemented database logger

This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.

* Ensure that log paths are relative to ROOT_PATH

* use path.Join

* rename jsonConfig to logConfig

* Rename "config" to "jsonConfig" to make it clearer

* Requested changes

* Requested changes: XormLogger

* Try to color the windows terminal

If successful default to colorizing the console logs

* fixup

* Colorize initially too

* update vendor

* Colorize logs on default and remove if this is not a colorizing logger

* Fix documentation

* fix test

* Use go-isatty to detect if on windows we are on msys or cygwin

* Fix spelling mistake

* Add missing vendors

* More changes

* Rationalise the ANSI writer protection

* Adjust colors on advice from @0x5c

* Make Flags a comma separated list

* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING

* Ensure matching is done on the non-colored message - to simpify EXPRESSION
2019-04-02 08:48:31 +01:00
mrsdizzie f125330fcf Include more variables on admin/config page (#6378)
Include the current CustomPath location in the admin section and also
display GITEA_WORK_DIR and/or GITEA_CUSTOM env var if they are set.

Right now there is no easy way to see this information, and if you try
and help most users they won't be able to tell you anything about these
values -- just that their custom template isn't working, files aren't in
the right place, etc... Now you can see all paths and if they were set
by ENV or not.
2019-03-19 18:40:13 -04:00
Russell Aunger b34996a629 Implement Default Webhooks (#4299)
Partially implement #770.
Add "Default Webhooks" page in site admin UI.
Persist to the existing webhooks table, but store with RepoID=0 and OrgID=0.
Upon repo creation, copy the set of default webhooks into the new repo.
2019-03-18 22:33:20 -04:00
Zsombor 538a26d56f Show private organization for admin, fix #6111 (#6112) 2019-02-19 09:16:06 +08:00
Lanre Adelowo d0cb5aa9f9 fix #3200 (#6033) 2019-02-11 03:27:24 +02:00
zeripath 2a660a1de1 Support reverse proxy providing email (#5554)
This PR implements #2347
2018-12-18 12:05:48 -05:00
Lanre Adelowo 2a6d3ba058 Allow admin toggle forcing a password change for newly created users (#4563) 2018-09-13 12:34:36 -04:00
Lanre Adelowo 126ba796dc Force user to change password (#4489)
* redirect to login page after successfully activating account

* force users to change password if account was created by an admin

* force users to change password if account was created by an admin

* fixed build

* fixed build

* fix pending issues with translation and wrong routes

* make sure path check is safe

* remove unneccessary newline

* make sure users that don't have to view the form get redirected

* move route to use /settings prefix so as to make sure unauthenticated users can't view the page

* update as per @lafriks review

* add necessary comment

* remove unrelated changes

* support redirecting to location the user actually want to go to before being forced to change his/her password

* run make fmt

* added tests

* improve assertions

* add assertion

* fix copyright year

Signed-off-by: Lanre Adelowo <yo@lanre.wtf>
2018-09-13 15:04:25 +03:00
Magnus Lindvall cdb9478774 LDAP Public SSH Keys synchronization (#1844)
* Add LDAP Key Synchronization feature

Signed-off-by: Magnus Lindvall <magnus@dnmgns.com>

* Add migration: add login source id column for public_key table

* Only update keys if needed

* Add function to only list pubkey synchronized from ldap

* Only list pub ssh keys synchronized from ldap. Do not sort strings as ExistsInSlice does it.

* Only get keys belonging to current login source id

* Set default login source id to 0

* Some minor cleanup. Add integration tests (updete dep testify)
2018-05-24 07:59:02 +03:00
Lauris BH 501fb228e6
Add option to use paged LDAP search when synchronizing users (#3895) 2018-05-05 17:30:47 +03:00
Allen Wild 8606d9f5bc Add admin dashboard option to run health checks (#3606)
There's one for git gc, why not git fsck too?

Also add a couple more trace logs to GitFsck to see progress
2018-03-02 11:09:43 +02:00
Kim "BKC" Carlbäcker e5b8b4b5ec Cleanup models.User.HashPassword (#3334) 2018-01-12 00:19:38 +02:00
Morgan Bazalgette 65861900cd Handle refactor (#3339)
* Replace all ctx.Handle with ctx.ServerError or ctx.NotFound

* Change Handle(403) to NotFound, avoid using macaron's NotFound
2018-01-10 23:34:17 +02:00
Morgan Bazalgette 674cfb7cac Change EncodePasswd to HashPassword (#3329)
* Change EncodePasswd to HashPassword

* Create test+benchmark for HashPassword
2018-01-08 23:28:18 +01:00
Ethan Koenig 716ac1214f Enable admin to search by email (#2888) 2017-11-26 16:40:38 +08:00
Lauris BH eca05b09aa Add commit count caching (#2774)
* Add commit count caching

* Small refactoring

* Add different key prefix for refs and commits

* Add configuratuion option to allow to change caching time or disable it
2017-10-26 09:37:33 +08:00
Lunny Xiao 6eeadb2082 Hide unactive on explore users and some refactors (#2741)
* hide unactive on explore users and some refactors

* fix test for removed Organizations

* fix test for removed Organizations

* fix imports

* fix logic bug

* refactor the toConds

* Rename TestOrganizations to TestSearchUsers and add tests for users

* fix other tests

* fix other tests

* fix watchers tests

* fix comments and remove unused code
2017-10-24 20:36:19 +03:00
Morlinest dff26e25ca Remove redudant functions and code (#2652)
* Remove redudant functions and code
2017-10-10 22:37:18 +02:00
Ethan Koenig b689bb6180 Webhooks for repo creation/deletion (#1663)
* Webhooks for repo creation/deletion

* add createHookTask

* Add handles for GetSlackPayload and GetDiscordPayload
2017-09-03 11:20:24 +03:00
Lauris BH 32fc44aa83 Make time diff translatable (#2057) 2017-06-28 13:43:28 +08:00
Lunny Xiao 75b5be236a Display config file path on admin panel (#2030) 2017-06-22 08:21:56 -05:00
Lauris BH 524885dd65 LDAP user synchronization (#1478) 2017-05-10 15:10:18 +02:00
Willem van Dreumel 950f2e2074 Additional OAuth2 providers (#1010)
* add google+

* sort signin oauth2 providers based on the name so order is always the same

* update auth tip for google+

* add gitlab provider

* add bitbucket provider (and some go fmt)

* add twitter provider

* add facebook provider

* add dropbox provider

* add openid connect provider incl. new format of tips section in "Add New Source"

* lower the amount of disk storage for each session to prevent issues while building cross platform (and disk overflow)

* imports according to goimport and code style

* make it possible to set custom urls to gitlab and github provider (only these could have a different host)

* split up oauth2 into multiple files

* small typo in comment

* fix indention

* fix indentation

* fix new line before external import

* fix layout of signin part

* update "broken" dependency
2017-05-01 21:26:53 +08:00
Bo-Yi Wu 95574a3640 fix: Admin can see all private repositories on Explore page. (#1026)
* fix: Admin can see all private repositories on Explore page.

* refactor: fix session
2017-02-26 13:59:31 +08:00
Lunny Xiao 0e6b9ea786 Take back control of hooks (#1006)
* git: delegate all server-side Git hooks (#1623)

* create hooks directories

* take control hooks back

* fix lint

* bug fixed and minor changes

* fix imports style

* fix migration scripts
2017-02-23 11:40:44 +08:00
Willem van Dreumel 01d957677f Oauth2 consumer (#679)
* initial stuff for oauth2 login, fails on:
* login button on the signIn page to start the OAuth2 flow and a callback for each provider
Only GitHub is implemented for now
* show login button only when the OAuth2 consumer is configured (and activated)
* create macaron group for oauth2 urls
* prevent net/http in modules (other then oauth2)
* use a new data sessions oauth2 folder for storing the oauth2 session data
* add missing 2FA when this is enabled on the user
* add password option for OAuth2 user , for use with git over http and login to the GUI
* add tip for registering a GitHub OAuth application
* at startup of Gitea register all configured providers and also on adding/deleting of new providers
* custom handling of errors in oauth2 request init + show better tip
* add ExternalLoginUser model and migration script to add it to database
* link a external account to an existing account (still need to handle wrong login and signup) and remove if user is removed
* remove the linked external account from the user his settings
* if user is unknown we allow him to register a new account or link it to some existing account
* sign up with button on signin page (als change OAuth2Provider structure so we can store basic stuff about providers)

* from gorilla/sessions docs:
"Important Note: If you aren't using gorilla/mux, you need to wrap your handlers with context.ClearHandler as or else you will leak memory!"
(we're using gorilla/sessions for storing oauth2 sessions)

* use updated goth lib that now supports getting the OAuth2 user if the AccessToken is still valid instead of re-authenticating (prevent flooding the OAuth2 provider)
2017-02-22 08:14:37 +01:00