mirror of
1
Fork 0
Commit Graph

217 Commits

Author SHA1 Message Date
Daenney 236604bd6b
[chore] Log less output on failed test (#2804)
* [chore] Log less output on failed test

This changes the testrig log level to be error by default instead of
info. This makes test failures a lot easier to read, as we don't have
the parade of info logs for each failure to scroll through.

It speeds up the test suite by a couple of seconds since we need
to buffer and flush a lot less messages. On a clean run, so no test
failures, it's about a 3s difference on my machine. Depending on the
amount of test failures, total time saved can vary.

This also introduces a GTS_TESTRIG_LOG_LEVEL environment variable that
we explicitly check for, making it easy to override the log level should
we have a need for it. This would be primarily for running locally, and
not so much as part of go test.

Lastly, it updates the syslog tests to use log.Error because if the log
level is set to error but we call log.Info no message is emitted and we
hang indefinitely on the channel read.

* [chore] Rename the testrig log level env var
2024-04-04 16:42:42 +01:00
kim d61d5c8a6a
[bugfix] httpclient not signing subsequent redirect requests (#2798)
* move http request signing to transport

* actually hook up the http roundtripper ...

* add code comments for the new gtscontext functions
2024-04-02 13:12:26 +02:00
tobi f05874be30
[feature] Option to hide followers/following (#2788) 2024-04-02 10:42:24 +01:00
tobi 7f4a0a1aeb
[chore] Move local account settings to separate db table (#2770)
* [chore] Move local account settings to separate database model

* don't use separate settings_id
2024-03-22 14:03:46 +01:00
Vyr Cossont 61a2b91f45
[feature] Filters v1 (#2594)
* Implement client-side v1 filters

* Exclude linter false positives

* Update test/envparsing.sh

* Fix minor Swagger, style, and Bun usage issues

* Regenerate Swagger

* De-generify filter keywords

* Remove updating filter statuses

This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered.

* Add a test for filter statuses specifically

* De-generify filter statuses

* Inline FilterEntry

* Use vertical style for Bun operations consistently

* Add comment on Filter DB interface

* Remove GoLand linter control comments

Our existing linters should catch these, or they don't matter very much

* Reduce memory ratio for filters
2024-03-06 11:15:58 +01:00
kim d85727e184
[bugfix] check remote status permissibility (#2703)
* add more stringent checks for remote status permissibility

* add check for inreplyto of a remote status being a boost

* do not permit inReplyTo boost wrapper statuses

* change comment wording

* fix calls to NewFederator()

* add code comments for NotPermitted() and SetNotPermitted()

* improve comment

* check that existing != nil before attempting delete

* ensure replying account isn't suspended

* use a debug log instead of info. check for boost using ID

* shorten log string length. make info level

* add note that replying to boost wrapper status shouldn't be able to happen anyways

* update to use onFail() function
2024-03-04 12:30:12 +00:00
Daenney 40ba93654e
[docs] Use human sizes for size limitations (#2706)
This switches the documentation to use human sizes, so 50MiB instead of
an integer number of bytes. This makes it much easier to understand what
values are set, and less likely to accidentally set the wrong value.
2024-03-01 14:47:03 +00:00
9p4 9bf448be7a
[feature/oidc] Add support for very basic RBAC (#2642)
* Add support for very basic RBAC

* Add some small tests for allowedGroup and adminGroup

* Switch to table-driven tests
2024-02-27 16:07:29 +01:00
tobi 9cadc764b3
[feature] Add experimental `instance-federation-spam-filter` option (#2685)
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
tobi 4b0eefbcc9
[chore] Increase default max image description to 1500 chars, collapse cw char count into status (#2682)
* [chore] Make default max image description 1500 chars, collapse cw char count into status

* oops

* tests
2024-02-23 18:28:09 +00:00
kim 2bafd7daf5
[bugfix] add stricter checks during all stages of dereferencing remote AS objects (#2639)
* add stricter checks during all stages of dereferencing remote AS objects

* a comment
2024-02-14 12:13:38 +01:00
tobi aa396c78d3
[feature] serdes for moved/also_known_as (#2600)
* [feature] serdes for moved/also_known_as

* document `alsoKnownAs` and `movedTo` properties

* only implicitly populate AKA uris from DB for local accounts

* don't let remotes store more than 20 AKA uris to avoid shenanigans
2024-02-06 09:45:46 +00:00
kim 7ec1e1332e
[performance] overhaul struct (+ result) caching library for simplicity, performance and multiple-result lookups (#2535)
* rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial

* use actual go-structr release version (not just commit hash)

* revert go toolchain changes (damn you go for auto changing this)

* fix go mod woes

* ensure %w is used in calls to errs.Appendf()

* fix error checking

* fix possible panic

* remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop

* fix copy-paste artifact... 😇

* fix all comment copy-paste artifacts

* remove dropID() function, now we can just use slices.DeleteFunc()

* use util.Deduplicate() instead of collate(), move collate to util

* move orderByIDs() to util package and "generify"

* add a util.DeleteIf() function, use this to delete entries on failed population

* use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol)

* add note about how collate differs from deduplicate
2024-01-19 12:57:29 +00:00
tobi c36f9ac37b
[feature] Account alias / move API + db models (#2518)
* [feature] Account alias / move API + db models

* go fmt

* fix little cherry-pick issues

* update error checking, formatting

* add and use new util functions to simplify alias logic
2024-01-16 16:22:44 +00:00
tobi d5e3996a18
[feature] Parse instance descriptors as markdown, show T&C on /about (#2481)
* [feature] Parse instance descriptors as markdown, show T&C on /about

* lint

* remove unnecessary nullzero tags
2024-01-05 13:39:31 +01:00
tobi 0ff52b71f2
[chore] Refactor HTML templates and CSS (#2480)
* [chore] Refactor HTML templates and CSS

* eslint

* ignore "Local"

* rss tests

* fiddle with OG just a tiny bit

* dick around with polls a bit more so SR stops saying "clickable"

* remove break

* oh lord

* don't lazy load avatar

* fix ogmeta tests

* clean up some cruft

* catch remaining calls to c.HTML

* fix error rendering + stack overflow in tag

* allow templating attributes

* fix indent

* set aria-hidden on status complementary content, since it's already present in the label anyway

* tidy up templating calls a little

* try to make styling a bit more consistent + readable

* fix up some remaining CSS issues

* fix up reports
2023-12-27 11:23:52 +01:00
tobi d60edf7ec6
[bugfix] Ensure `pre` renders as expected, fix orderedCollectionPage (#2434) 2023-12-10 12:36:00 +01:00
tobi cc91ea057d
[bugfix] Fix web media not showing as sensitive (#2433)
* [bugfix] Fix web media not showing as sensitive

* test

* go fmt
2023-12-09 16:54:38 +01:00
tobi 8c2d94c168
[feature] Poll web view (#2377)
* [feature] Render polls nicely on the web view

* use figure for poll, other small tweaks

* reverse share + count (lines up better)

* poll options list entries

* fix up some remaining things
2023-11-22 12:17:42 +01:00
Tsuribori 1ba3e14b36
[feature] Initial Prometheus metrics implementation (#2334)
* feat: Initial OTEL metrics

* docs: add metrics documentation

* fix: metrics endpoint conditional check

* feat: metrics endpoint basic auth

* fix: make metrics-auth-enabled default false

* fix: go fmt helpers.gen.go

* fix: add metric-related env vars to envparsing.sh

* fix: metrics docs

* fix: metrics related stuff in envparsing.sh

* fix: metrics docs

* chore: metrics docs wording

* fix: metrics stuff in envparsing?

* bump otel versions

---------

Co-authored-by: Tsuribori <user@acertaindebian>
Co-authored-by: Tsuribori <none@example.org>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-11-20 16:43:55 +01:00
kim 16275853eb
[bugfix] self-referencing collection pages for status replies (#2364) 2023-11-20 12:22:28 +00:00
tobi fc02d3c6f7
[feature] Set/show instance language(s); show post language on frontend (#2362)
* update go text, include text/display

* [feature] Set instance langs, show post lang on frontend

* go fmt

* WebGet

* set language for whole article, don't use FA icon

* mention instance languages + other optional config vars

* little tweak

* put languages in config properly

* warn log language parse

* change some naming around

* tidy up validate a bit

* lint

* rename LanguageTmpl in template
2023-11-17 11:35:28 +01:00
tobi 4ee436e98a
[bugfix] process account delete side effects in serial, not in parallel (#2360)
* [bugfix] process account delete side effects in serial, not in parallel

* StartWorkers / StartNoopWorkers for tests

* undo testrig trace logging

* log errors instead of immediately returning
2023-11-14 15:57:25 +01:00
tobi 8d0c017cf2
[feature/performance] Wrap incoming HTTP requests in timeout handler (#2353)
* deinterface router, start messing about with deadlines

* weeeee

* thanks linter (thinter)

* write Connection: close when timing out requests

* update wording

* don't replace req

* don't bother with fancy Cause functions (I'll use them one day...)
2023-11-13 19:48:51 +01:00
kim deaea100c3
[bugfix] support endless polls, and misskey's' method of inferring expiry in closed polls (#2349) 2023-11-11 10:15:04 +00:00
tobi ba9d6b467a
[feature] Media attachment placeholders (#2331)
* [feature] Use placeholders for unknown media types

* fix read of underreported small files

* switch to reduce nesting

* simplify cleanup
2023-11-10 19:29:26 +01:00
tobi c7ecab9e6f
[chore/bugfix/horror] Allow `expires_in` and poll choices to be parsed from strings (#2346) 2023-11-10 16:42:48 +00:00
kim e9e5dc5a40
[feature] add support for polls + receiving federated status edits (#2330) 2023-11-08 14:32:17 +00:00
kim 41435a6c4e
[feature] support canceling scheduled tasks, some federation API performance improvements (#2329) 2023-11-04 20:21:20 +00:00
tobi 4dc0547dc0
[feature] Customizable media cleaner schedule (#2304) 2023-10-30 17:35:11 +00:00
tobi c7b6cd7770
[feature] Status thread mute/unmute functionality (#2278)
* add db models + functions for keeping track of threads

* give em the old linty testy

* create, remove, check mutes

* swagger

* testerino

* test mute/unmute via api

* add info log about new index creation

* thread + allow muting of any remote statuses that mention a local account

* IsStatusThreadMutedBy -> IsThreadMutedByAccount

* use common processing functions in status processor

* set = NULL

* favee!

* get rekt darlings, darlings get rekt

* testrig please, have mercy muy liege
2023-10-25 15:04:53 +01:00
kim 69ba9a79a1
[chore] de-interface{} the federator and dereferencer structs (#2285)
* de-interface{} the federator and dereferencer structs

* fix broken type signatures
2023-10-23 10:58:13 +01:00
tobi 9114c5ca1b
[bugfix/frontend] fix typo and other oddness in patchRemoteEmojis (#2281)
* fix emoji test model

* found the bug!

* remove unused 'current' import

* comment useChecklistReducer

* wah

* lint

* fix cleaner tests
2023-10-21 17:23:05 +02:00
kim 8f67dd583d
[chore] deinterface the typeutils.Converter and update to use state structure (#2217)
* update typeconverter to use state structure

* deinterface the typeutils.TypeConverter -> typeutils.Converter

* finish copying over old type converter code comments

* fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-09-23 18:44:11 +02:00
tobi 183eaa5b29
[feature] Implement explicit domain allows + allowlist federation mode (#2200)
* love like winter! wohoah, wohoah

* domain allow side effects

* tests! logging! unallow!

* document federation modes

* linty linterson

* test

* further adventures in documentation

* finish up domain block documentation (i think)

* change wording a wee little bit

* docs, example

* consolidate shared domainPermission code

* call mode once

* fetch federation mode within domain blocked func

* read domain perm import in streaming manner

* don't use pointer to slice for domain perms

* don't bother copying blocks + allows before deleting

* admonish!

* change wording just a scooch

* update docs
2023-09-21 12:12:04 +02:00
kim 7293d6029b
[feature] add paging to account follows, followers and follow requests endpoints (#2186) 2023-09-12 14:00:35 +01:00
tobi 4b594516ec
[feature] Allow admins to expire remote public keys; refetch expired keys on demand (#2183) 2023-09-12 10:43:12 +01:00
tobi 3ed1ca68e5
[feature] Store admin actions in the db, prevent conflicting actions (#2167) 2023-09-04 14:55:17 +01:00
Daenney 4ae16bce8c
[feature] Make log format configurable (#2130)
* [feature] Don't emit timestamp in log lines

When running gotosocial with a service manager like systemd, or a
container runtime, the associated log driver usually emits timestamps
itself. In those cases, having the extra timestamp from our own log
lines ends up being a bit noisy and when centrally ingesting logs is
duplicate information.

This introduces a configuration flag that allows disabling emitting the
timestamp. It's only wired up for "daemonised" processes, meaning server
and testrig.

* [chore] Add docs for log-timestamp

* [feature] Simplify timestamp handling

Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>

* [chore] Less escaped double-quotes

* [chore] Fix help string

---------

Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-08-21 19:07:55 +01:00
f0x52 92de8fb396
[feature] Instance rules (#2125)
* init instance rules database model, admin api

* expose instance rules in public instance api

* public /api/v1/instance/rules route

* GET ruleById

* createRule route

* createRule auth check

* updateRule

* deleteRule

* list rules on about page

* ruleGet auth

* add about page ids for anchors

* process and store adding violated rules to reports

* admin api models for instance rules

* instance rule edit frontend

* change rule inputs to textareas

* database fixes after rebase (#2124)

* remove unused imports

* fix db migration column name

* fix tests

* fix more tests

* fix postgres error with wrongly used Ident

* add some tests, fiddle with rule model a bit, fix postgres migration

* swagger docs

---------

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-19 14:33:15 +02:00
tobi 9770d54237
[feature] List replies policy, refactor async workers (#2087)
* Add/update some DB functions.

* move async workers into subprocessor

* rename FromFederator -> FromFediAPI

* update home timeline check to include check for current status first before moving to parent status

* change streamMap to pointer to mollify linter

* update followtoas func signature

* fix merge

* remove errant debug log

* don't use separate errs.Combine() check to wrap errs

* wrap parts of workers functionality in sub-structs

* populate report using new db funcs

* embed federator (tiny bit tidier)

* flesh out error msg, add continue(!)

* fix other error messages to be more specific

* better, nicer

* give parseURI util function a bit more util

* missing headers

* use pointers for subprocessors
2023-08-09 19:14:33 +02:00
kim 3920bc87d1
[bugfix] don't accept unrelated statuses (#2078)
Co-authored-by: Daenney <daenney@users.noreply.github.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-08 12:26:34 +01:00
Daenney be3718f6e4
[chore] Use generic pointer function (#2080)
This replaces the different $TypePtr functions with a generic
implementation.
2023-08-07 18:38:11 +01:00
tobi 2796a2e82f
[feature] Hashtag federation (in/out), hashtag client API endpoints (#2032)
* update go-fed

* do the things

* remove unused columns from tags

* update to latest lingo from main

* further tag shenanigans

* serve stub page at tag endpoint

* we did it lads

* tests, oh tests, ohhh tests, oh tests (doo doo doo doo)

* swagger docs

* document hashtag usage + federation

* instanceGet

* don't bother parsing tag href

* rename whereStartsWith -> whereStartsLike

* remove GetOrCreateTag

* dont cache status tag timelineability
2023-07-31 15:47:35 +02:00
Vyr Cossont b874e9251e
[feature] Implement markers API (#1989)
* Implement markers API

Fixes #1856

* Correct import grouping in markers files

* Regenerate Swagger for markers API

* Shorten names for readability

* Cache markers for 6 hours

* Update DB ref

* Update envparsing.sh
2023-07-29 12:49:14 +02:00
Vyr Cossont 22ac4607a1
[feature] Support setting private notes on accounts (#1982)
* Support setting private notes on accounts

* Reformat comment whitespace

* Add missing license headers

* Use apiutil.ParseID

* Rename Note model and cache to AccountNote

* Update golden cache config in test/envparsing.sh

* Rename gtsmodel/note.go to gtsmodel/accountnote.go

* Update AccountNote uniqueness constraint name

Now has same prefix as other indexes on this table.

---------

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2023-07-27 10:30:39 +02:00
kim 9eff0d46e4
[feature/performance] support uncaching remote emoji + scheduled cleanup functions (#1987) 2023-07-24 13:14:13 +01:00
Daenney da8282081c
[feature] Lower remote media cache config duration (#2007)
The old default of 30d can lead to a lot of media getting cached and
significant disk usage, even on small or single person instances. A lot
of deployments decrease this value, to 15 or even less. This is less of
an issue when using object storage, but for local storage which is the
more popular deployment option running out of disk space is unpleasant.

With GoToSocial's aim to fit in small places, this changes the default
to a much more conservative 7 days. In all likelihood people aren't
scrolling that far back in their timeline so this change shouldn't
result in any issue. Existing deployments will only be affected by
this change if the admin hasn't already configured this value, or didn't
bootstrap from the example configuration.
2023-07-21 16:22:59 +02:00
kim 9a22102fa8
[bugfix/chore] oauth entropy fix + media cleanup tasks rewrite (#1853) 2023-06-22 20:46:36 +01:00
kim 5faeb4de20
[chore] tidy up media manager, add calling func to errors, build-script improvements (#1835)
* media manager tidy-up: de-interface and remove unused PostDataFunc

Signed-off-by: kim <grufwub@gmail.com>

* remove last traces of media.Manager being an interface

Signed-off-by: kim <grufwub@gmail.com>

* update error to provide caller, allow tuneable via build tags

Signed-off-by: kim <grufwub@gmail.com>

* remove kim-specific build script changes

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* update build-script to support externally setting build variables

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-05-28 14:08:35 +02:00