mirror of
1
Fork 0
Commit Graph

16 Commits

Author SHA1 Message Date
tobi 941893a774
[chore] The Big Middleware and API Refactor (tm) (#1250)
* interim commit: start refactoring middlewares into package under router

* another interim commit, this is becoming a big job

* another fucking massive interim commit

* refactor bookmarks to new style

* ambassador, wiz zeze commits you are spoiling uz

* she compiles, we're getting there

* we're just normal men; we're just innocent men

* apiutil

* whoopsie

* i'm glad noone reads commit msgs haha :blob_sweat:

* use that weirdo go-bytesize library for maxMultipartMemory

* fix media module paths
2023-01-02 12:10:50 +00:00
kim fcb9c0bb8b
[chore] cleanup storage implementation, no need for multiple interface types (#1131)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-24 09:35:46 +01:00
kim a156188b3e
[chore] update dependencies, bump to Go 1.19.1 (#826)
* update dependencies, bump Go version to 1.19

* bump test image Go version

* update golangci-lint

* update gotosocial-drone-build

* sign

* linting, go fmt

* update swagger docs

* update swagger docs

* whitespace

* update contributing.md

* fuckin whoopsie doopsie

* linterino, linteroni

* fix followrequest test not starting processor

* fix other api/client tests not starting processor

* fix remaining tests where processor not started

* bump go-runners version

* don't check last-webfingered-at, processor may have updated this

* update swagger command

* update bun to latest version

* fix embed to work the same as before with new bun

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-09-28 18:30:40 +01:00
kim 6934ae378a
[chore] improved router logging, recovery and error handling (#705)
* move panic recovery to logging middleware, improve logging + panic recovery logic

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* remove skip paths code

Signed-off-by: kim <grufwub@gmail.com>

* re-enable log quoting

Signed-off-by: kim <grufwub@gmail.com>

* use human-friendly bytesize in logging body size

Signed-off-by: kim <grufwub@gmail.com>

* only disable quoting in debug builds

Signed-off-by: kim <grufwub@gmail.com>

* use logrus level instead of debug.DEBUG() to enable/disable quoting

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* fix instance tests

Signed-off-by: kim <grufwub@gmail.com>

* fix gin test contexts created with missing engine HTML renderer

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding not logging query parameters

Signed-off-by: kim <grufwub@gmail.com>

* better explain 'DisableQuoting' logic

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>
2022-07-12 08:32:20 +01:00
Dominik Süß 9d0df426da
[feature] S3 support (#674)
* feat: vendor minio client

* feat: introduce storage package with s3 support

* feat: serve s3 files directly

this saves a lot of bandwith as the files are fetched from the object
store directly

* fix: use explicit local storage in tests

* feat: integrate s3 storage with the main server

* fix: add s3 config to cli tests

* docs: explicitly set values in example config

also adds license header to the storage package

* fix: use better http status code on s3 redirect

HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL

307/TemporaryRedirect would mean that this resource is usually located
here, not in this case

303/SeeOther indicates that the redirection does not link to the
requested resource but to another page

* refactor: use context in storage driver interface
2022-07-03 12:08:30 +02:00
tobi dfdc473cef
[chore] Webfinger rework (#627)
* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test
2022-06-11 11:01:34 +02:00
kim 43ac0cdb9c
[chore] Global server configuration overhaul (#575)
* move config flag names and usage to config package, rewrite config package to use global Configuration{} struct

Signed-off-by: kim <grufwub@gmail.com>

* improved code comment

Signed-off-by: kim <grufwub@gmail.com>

* linter

Signed-off-by: kim <grufwub@gmail.com>

* fix unmarshaling

Signed-off-by: kim <grufwub@gmail.com>

* remove kim's custom go compiler changes

Signed-off-by: kim <grufwub@gmail.com>

* generate setter and flag-name functions, implement these in codebase

Signed-off-by: kim <grufwub@gmail.com>

* update deps

Signed-off-by: kim <grufwub@gmail.com>

* small change

Signed-off-by: kim <grufwub@gmail.com>

* appease the linter...

Signed-off-by: kim <grufwub@gmail.com>

* move configuration into ConfigState structure, ensure reloading to/from viper settings to keep in sync

Signed-off-by: kim <grufwub@gmail.com>

* lint

Signed-off-by: kim <grufwub@gmail.com>

* update code comments

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* improved version string (removes time + go version)

Signed-off-by: kim <grufwub@gmail.com>

* fix version string build to pass test script + consolidate logic in func

Signed-off-by: kim <grufwub@gmail.com>

* add license text, update config.Defaults comment

Signed-off-by: kim <grufwub@gmail.com>

* add license text to generated config helpers file

Signed-off-by: kim <grufwub@gmail.com>

* defer unlock on config.Set___(), to ensure unlocked on panic

Signed-off-by: kim <grufwub@gmail.com>

* make it more obvious which cmd flags are being attached

Signed-off-by: kim <grufwub@gmail.com>
2022-05-30 14:41:24 +02:00
kim 223025fc27
[security] transport.Controller{} and transport.Transport{} security and performance improvements (#564)
* cache transports in controller by privkey-generated pubkey, add retry logic to transport requests

Signed-off-by: kim <grufwub@gmail.com>

* update code comments, defer mutex unlocks

Signed-off-by: kim <grufwub@gmail.com>

* add count to 'performing request' log message

Signed-off-by: kim <grufwub@gmail.com>

* reduce repeated conversions of same url.URL object

Signed-off-by: kim <grufwub@gmail.com>

* move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue

Signed-off-by: kim <grufwub@gmail.com>

* fix security advisories regarding max outgoing conns, max rsp body size

- implemented by a new httpclient.Client{} that wraps an underlying
  client with a queue to limit connections, and limit reader wrapping
  a response body with a configured maximum size
- update pub.HttpClient args passed around to be this new httpclient.Client{}

Signed-off-by: kim <grufwub@gmail.com>

* add httpclient tests, move ip validation to separate package + change mechanism

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* use singular mutex in transport rather than separate signer mus

Signed-off-by: kim <grufwub@gmail.com>

* improved useragent string

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding missing test

Signed-off-by: kim <grufwub@gmail.com>

* remove useragent field from transport (instead store in controller)

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* reset other signing headers on each loop iteration

Signed-off-by: kim <grufwub@gmail.com>

* respect request ctx during retry-backoff sleep period

Signed-off-by: kim <grufwub@gmail.com>

* use external pkg with docs explaining performance "hack"

Signed-off-by: kim <grufwub@gmail.com>

* use http package constants instead of string method literals

Signed-off-by: kim <grufwub@gmail.com>

* add license file headers

Signed-off-by: kim <grufwub@gmail.com>

* update code comment to match new func names

Signed-off-by: kim <grufwub@gmail.com>

* updates to user-agent string

Signed-off-by: kim <grufwub@gmail.com>

* update signed testrig models to fit with new transport logic (instead uses separate signer now)

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter

Signed-off-by: kim <grufwub@gmail.com>
2022-05-15 11:16:43 +02:00
kim 420e2fb22b
replace async client API / federator msg processing with worker pools (#497)
* replace async client API / federator msg processing with worker pools
* appease our lord-and-saviour, the linter
2022-04-28 13:23:11 +01:00
tsmethurst e0f9323b9a test the media manager a bit, add shutdown logic 2022-01-10 18:36:09 +01:00
tobi ef5a9256a8
Extend license notices to 2022 (#354) 2021-12-20 18:42:19 +01:00
tobi e2daf0f012
Add `Accept` header negotiation to relevant API endpoints (#337)
* start centralizing negotiation logic for API

* swagger document nodeinfo endpoint

* go fmt

* document negotiate function

* use content negotiation

* tidy up negotiation logic

* negotiate content throughout client api

* swagger

* remove attachment on Content

* add accept header to test requests
2021-12-11 17:50:00 +01:00
tobi 0884f89431
Implement Cobra CLI tooling, Viper config tooling (#336)
* start pulling out + replacing urfave and config

* replace many many instances of config

* move more stuff => viper

* properly remove urfave

* move some flags to root command

* add testrig commands to root

* alias config file keys

* start adding cli parsing tests

* reorder viper init

* remove config path alias

* fmt

* change config file keys to non-nested

* we're more or less in business now

* tidy up the common func

* go fmt

* get tests passing again

* add note about the cliparsing tests

* reorganize

* update docs with changes

* structure cmd dir better

* rename + move some files around

* fix dangling comma
2021-12-07 13:31:39 +01:00
tobi 829a934d23
update dependencies (#296) 2021-11-13 12:29:08 +01:00
tobi 2aaec82732
smtp + email confirmation (#285)
* add smtp configuration

* add email confirm + reset templates

* add email sender to testrig

* flesh out the email sender interface

* go fmt

* golint

* update from field with more clarity

* tidy up the email formatting

* fix tests

* add email sender to processor

* tidy client api processing a bit

* further tidying in fromClientAPI

* pin new account to user

* send msg to processor on new account creation

* generate confirm email uri

* remove emailer from account processor again

* add processCreateAccountFromClientAPI

* move emailer accountprocessor => userprocessor

* add email sender to user processor

* SendConfirmEmail function

* add noop email sender

* use noop email sender in tests

* only assemble message if callback is not nil

* use noop email sender if no smtp host is defined

* minify email html before sending

* fix wrong email address

* email confirm test

* fmt

* serve web hndler

* add email confirm handler

* init test log properly on testrig

* log emails that *would* have been sent

* go fmt ./...

* unexport confirm email handler

* updatedAt

* test confirm email function

* don't allow tokens older than 7 days

* change error message a bit

* add basic smtp docs

* add a few more snippets

* typo

* add email sender to outbox tests

* don't use dutch wikipedia link

* don't minify email html
2021-10-31 15:46:23 +01:00
tobi 15621f5324
Follow request improvements (#282)
* tiny doc update

* add rejectfollowrequest to db

* add follow request reject to processor

* add reject handler

* tidy up follow request api

* tidy up federation call

* regenerate swagger docs

* api endpoint tests

* processor test

* add reject federatingdb handler

* start writing reject tests

* test reject follow request

* go fmt

* increase sleep for slow test setups

* more relaxed time.sleep
2021-10-16 13:27:43 +02:00