* Implement client-side v1 filters
* Exclude linter false positives
* Update test/envparsing.sh
* Fix minor Swagger, style, and Bun usage issues
* Regenerate Swagger
* De-generify filter keywords
* Remove updating filter statuses
This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered.
* Add a test for filter statuses specifically
* De-generify filter statuses
* Inline FilterEntry
* Use vertical style for Bun operations consistently
* Add comment on Filter DB interface
* Remove GoLand linter control comments
Our existing linters should catch these, or they don't matter very much
* Reduce memory ratio for filters
* tidy up account, status, webfingering logic a wee bit
* go fmt
* invert published check
* alter resp initialization
* get Published from account in typeutils
* don't instantiate error for no darn good reason
* shadow err
* don't repeat error codes in wrapped errors
* don't wrap error unnecessarily
* [feature] Account alias / move API + db models
* go fmt
* fix little cherry-pick issues
* update error checking, formatting
* add and use new util functions to simplify alias logic
* [bugfix] Replace named unique constraint on header filter header with generic unique directive
* add migration retry
* the old fixie uppie
* fix constraint name
* my goodness
* [feature] Federate status language in + out
* go fmt
* tests, little fix
* improve comments
* unnest a bit
* avoid unnecessary nil check
* use more descriptive variable for contentMap
* prefer instance languages when selecting from contentMap
* update docs to reflect lang selection
* rename rdfLangString -> rdfLangs
* update comments to mention Pollable
* iter through slice instead of map
* add db models + functions for keeping track of threads
* give em the old linty testy
* create, remove, check mutes
* swagger
* testerino
* test mute/unmute via api
* add info log about new index creation
* thread + allow muting of any remote statuses that mention a local account
* IsStatusThreadMutedBy -> IsThreadMutedByAccount
* use common processing functions in status processor
* set = NULL
* favee!
* get rekt darlings, darlings get rekt
* testrig please, have mercy muy liege
* love like winter! wohoah, wohoah
* domain allow side effects
* tests! logging! unallow!
* document federation modes
* linty linterson
* test
* further adventures in documentation
* finish up domain block documentation (i think)
* change wording a wee little bit
* docs, example
* consolidate shared domainPermission code
* call mode once
* fetch federation mode within domain blocked func
* read domain perm import in streaming manner
* don't use pointer to slice for domain perms
* don't bother copying blocks + allows before deleting
* admonish!
* change wording just a scooch
* update docs
* init instance rules database model, admin api
* expose instance rules in public instance api
* public /api/v1/instance/rules route
* GET ruleById
* createRule route
* createRule auth check
* updateRule
* deleteRule
* list rules on about page
* ruleGet auth
* add about page ids for anchors
* process and store adding violated rules to reports
* admin api models for instance rules
* instance rule edit frontend
* change rule inputs to textareas
* database fixes after rebase (#2124)
* remove unused imports
* fix db migration column name
* fix tests
* fix more tests
* fix postgres error with wrongly used Ident
* add some tests, fiddle with rule model a bit, fix postgres migration
* swagger docs
---------
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* [chore] Remove go-playground/validator
It turns out we're not actually using the validator code. This is a
remnant from when we intended to use it, but the presence of it and its
struct tags creates the illusion we're validating a lot of things we're
not. It resulted in some confusion when we were trying to figure out
language valdiation.
Remove all this code, so that only the validation functions from the
validate package we actually use remain. I'm not touching the struct
tags in the migrations in order to avoid things potentially thinking
migrations need to be re-run.
* [chore] Bring back a struct tag on api
The validate on internal/api is Gin doing form validation, not the
validator from go-playground/validator.
* update go-fed
* do the things
* remove unused columns from tags
* update to latest lingo from main
* further tag shenanigans
* serve stub page at tag endpoint
* we did it lads
* tests, oh tests, ohhh tests, oh tests (doo doo doo doo)
* swagger docs
* document hashtag usage + federation
* instanceGet
* don't bother parsing tag href
* rename whereStartsWith -> whereStartsLike
* remove GetOrCreateTag
* dont cache status tag timelineability
* Support setting private notes on accounts
* Reformat comment whitespace
* Add missing license headers
* Use apiutil.ParseID
* Rename Note model and cache to AccountNote
* Update golden cache config in test/envparsing.sh
* Rename gtsmodel/note.go to gtsmodel/accountnote.go
* Update AccountNote uniqueness constraint name
Now has same prefix as other indexes on this table.
---------
Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
* start working on lists
* further list work
* test list db functions nicely
* more work on lists
* peepoopeepoo
* poke
* start list timeline func
* we're getting there lads
* couldn't be me working on stuff... could it?
* hook up handlers
* fiddling
* weeee
* woah
* screaming, pissing
* fix streaming being a whiny baby
* lint, small test fix, swagger
* tidying up, testing
* fucked! by the linter
* move timelines to state like a boss
* add timeline start to tests using state
* invalidate lists
* revamp http client to not limit requests, instead use sender worker
Signed-off-by: kim <grufwub@gmail.com>
* remove separate sender worker pool, spawn 2*GOMAXPROCS batch senders each time, no need for transport cache sweeping
Signed-off-by: kim <grufwub@gmail.com>
* improve batch senders to keep popping recipients until remote URL found
Signed-off-by: kim <grufwub@gmail.com>
* fix recipient looping issue
Signed-off-by: kim <grufwub@gmail.com>
* move request id ctx key to gtscontext, finish filling out more code comments, add basic support for not logging client IP
Signed-off-by: kim <grufwub@gmail.com>
* first draft of status refetching logic
Signed-off-by: kim <grufwub@gmail.com>
* fix testrig to use new federation alloc func signature
Signed-off-by: kim <grufwub@gmail.com>
* fix log format directive
Signed-off-by: kim <grufwub@gmail.com>
* add status fetched_at migration
Signed-off-by: kim <grufwub@gmail.com>
* remove unused / unchecked for error types
Signed-off-by: kim <grufwub@gmail.com>
* add back the used type...
Signed-off-by: kim <grufwub@gmail.com>
* add separate internal getStatus() function for derefThread() that doesn't recurse
Signed-off-by: kim <grufwub@gmail.com>
* improved mention and media attachment error handling
Signed-off-by: kim <grufwub@gmail.com>
* fix log and error format directives
Signed-off-by: kim <grufwub@gmail.com>
* update account deref to match status deref changes
Signed-off-by: kim <grufwub@gmail.com>
* very small code formatting change to make things clearer
Signed-off-by: kim <grufwub@gmail.com>
* add more code comments
Signed-off-by: kim <grufwub@gmail.com>
* improved code commenting
Signed-off-by: kim <grufwub@gmail.com>
* only check for required further derefs if needed
Signed-off-by: kim <grufwub@gmail.com>
* improved cache invalidation
Signed-off-by: kim <grufwub@gmail.com>
* tweak cache restarting to use a (very small) backoff
Signed-off-by: kim <grufwub@gmail.com>
* small readability changes and fixes
Signed-off-by: kim <grufwub@gmail.com>
* fix account sync issues
Signed-off-by: kim <grufwub@gmail.com>
* fix merge conflicts + update account enrichment to accept already-passed accountable
Signed-off-by: kim <grufwub@gmail.com>
* remove secondary function declaration
Signed-off-by: kim <grufwub@gmail.com>
* normalise dereferencer get status / account behaviour, fix remaining tests
Signed-off-by: kim <grufwub@gmail.com>
* fix remaining rebase conflicts, finish commenting code
Signed-off-by: kim <grufwub@gmail.com>
* appease the linter
Signed-off-by: kim <grufwub@gmail.com>
* add source file header
Signed-off-by: kim <grufwub@gmail.com>
* update to use TIMESTAMPTZ column type instead of just TIMESTAMP
Signed-off-by: kim <grufwub@gmail.com>
* don't pass in 'updated_at' to UpdateEmoji()
Signed-off-by: kim <grufwub@gmail.com>
* use new ap.Resolve{Account,Status}able() functions
Signed-off-by: kim <grufwub@gmail.com>
* remove the somewhat confusing rescoping of the same variable names
Signed-off-by: kim <grufwub@gmail.com>
* update migration file name, improved database delete error returns
Signed-off-by: kim <grufwub@gmail.com>
* formatting
Signed-off-by: kim <grufwub@gmail.com>
* improved multi-delete database functions to minimise DB calls
Signed-off-by: kim <grufwub@gmail.com>
* remove unused type
Signed-off-by: kim <grufwub@gmail.com>
* fix delete statements
Signed-off-by: kim <grufwub@gmail.com>
---------
Signed-off-by: kim <grufwub@gmail.com>
* refactor visibility checking, add caching for visibility
* invalidate visibility cache items on account / status deletes
* fix requester ID passed to visibility cache nil ptr
* de-interface caches, fix home / public timeline caching + visibility
* finish adding code comments for visibility filter
* fix angry goconst linter warnings
* actually finish adding filter visibility code comments for timeline functions
* move home timeline status author check to after visibility
* remove now-unused code
* add more code comments
* add TODO code comment, update printed cache start names
* update printed cache names on stop
* start adding separate follow(request) delete db functions, add specific visibility cache tests
* add relationship type caching
* fix getting local account follows / followed-bys, other small codebase improvements
* simplify invalidation using cache hooks, add more GetAccountBy___() functions
* fix boosting to return 404 if not boostable but no error (to not leak status ID)
* remove dead code
* improved placement of cache invalidation
* update license headers
* add example follow, follow-request config entries
* add example visibility cache configuration to config file
* use specific PutFollowRequest() instead of just Put()
* add tests for all GetAccountBy()
* add GetBlockBy() tests
* update block to check primitive fields
* update and finish adding Get{Account,Block,Follow,FollowRequest}By() tests
* fix copy-pasted code
* update envparsing test
* whitespace
* fix bun struct tag
* add license header to gtscontext
* fix old license header
* improved error creation to not use fmt.Errorf() when not needed
* fix various rebase conflicts, fix account test
* remove commented-out code, fix-up mention caching
* fix mention select bun statement
* ensure mention target account populated, pass in context to customrenderer logging
* remove more uncommented code, fix typeutil test
* add statusfave database model caching
* add status fave cache configuration
* add status fave cache example config
* woops, catch missed error. nice catch linter!
* add back testrig panic on nil db
* update example configuration to match defaults, slight tweak to cache configuration defaults
* update envparsing test with new defaults
* fetch followingget to use the follow target account
* use accounnt.IsLocal() instead of empty domain check
* use constants for the cache visibility type check
* use bun.In() for notification type restriction in db query
* include replies when fetching PublicTimeline() (to account for single-author threads in Visibility{}.StatusPublicTimelineable())
* use bun query building for nested select statements to ensure working with postgres
* update public timeline future status checks to match visibility filter
* same as previous, for home timeline
* update public timeline tests to dynamically check for appropriate statuses
* migrate accounts to allow unique constraint on public_key
* provide minimal account with publicKey
---------
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* [chore] Remove years from all license headers
Years or year ranges aren't required in license headers. Many projects
have removed them in recent years and it avoids a bit of yearly toil.
In many cases our copyright claim was also a bit dodgy since we added
the 2021-2023 header to files created after 2021 but you can't claim
copyright into the past that way.
* [chore] Add license header check
This ensures a license header is always added to any new file. This
avoids maintainers/reviewers needing to remember to check for and ask
for it in case a contribution doesn't include it.
* [chore] Add missing license headers
* [chore] Further updates to license header
* Use the more common // indentend comment format
* Remove the hack we had for the linter now that we use the // format
* Add SPDX license identifier
* Advertise rich text formats, support content_type field
* Update JSON in instance patch tests
* Replace format with content_type everywhere
* update migration to work with both pg and sqlite
* regenerate swagger docs
* update instance serialization + tests
* fix up
* learn to code tobi please, i'm begging you
---------
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* implement status pin client api + web handler
* make test names + comments more descriptive
* don't use separate table for status pins
* remove unused add + remove checking
* tidy up + add some more tests
* start fiddling with media + account queries a little
* initialize state when pruning
* allow for unsetting remote media
make sure to wait til media loaded
fix silly tiny bug
* move comment a bit for readability
* slight reformat of fetchRemoteAccount{Avatar,Header}
* fix issue after rebase
* slightly neaten up logic of avatar/header media handling
* remove log prefix (callername log field handles this)
---------
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
* reformat GetAccount() functionality, and add UpdateAccount() function.
* use fetched_at instead of last_webfingered_at
* catch local "not found" errors. small formatting / error string changes
* remove now unused error type
* return nil when wrapping nil error
* update expected error messages
* return correct url for foss satan webfinger
* add AP model for Some_User
* normalize local domain
* return notretrievable where appropriate
* expose NewErrNotRetrievable
* ensure webfinger for new accounts searched by uri
* update local account short circuit
* allow enrich to fail for already-known accounts
* remove unused LastWebfingeredAt
* expose test maps on mock http client
* update Update test
* reformat GetAccount() functionality, and add UpdateAccount() function.
* use fetched_at instead of last_webfingered_at
* catch local "not found" errors. small formatting / error string changes
* remove nil error checks (we shouldn't be passing nil errors to newError() initializers)
* remove mutex unlock on transport init fail (it hasn't yet been locked!)
* woops add back the error wrapping to use ErrNotRetrievable
* caches were never being started... 🙈
---------
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* start messing about with different mp4 metadata extraction
* heyyooo it works
* add test cow
* move useful multierror to gtserror package
* error out if video doesn't seem to be a real mp4
* test parsing mkv in disguise as mp4
* tidy up error handling
* remove extraneous line
* update framerate formatting
* use float32 for aspect
* fixy mctesterson
* [feature] overhaul the oidc system
this allows for more flexible username handling and prevents account
takeover using old email addresses
* [feature] add migration path for old OIDC users
* [feature] nicer error reporting for users
* [docs] document the new OIDC flow
* [fix] return early on oidc error
* [docs]: add comments on the finalization logic
* [feature] Add emoji categories GET
Serialize emojis in appropriate categories; make it possible to get categories via the admin API
* [feature] Create (or use existing) category for new emoji uploads
* fix lint issue
* update misleading line in swagger docs
tobi