* media manager tidy-up: de-interface and remove unused PostDataFunc
Signed-off-by: kim <grufwub@gmail.com>
* remove last traces of media.Manager being an interface
Signed-off-by: kim <grufwub@gmail.com>
* update error to provide caller, allow tuneable via build tags
Signed-off-by: kim <grufwub@gmail.com>
* remove kim-specific build script changes
Signed-off-by: kim <grufwub@gmail.com>
* fix merge conflicts
Signed-off-by: kim <grufwub@gmail.com>
* update build-script to support externally setting build variables
Signed-off-by: kim <grufwub@gmail.com>
---------
Signed-off-by: kim <grufwub@gmail.com>
* [chore] Remove years from all license headers
Years or year ranges aren't required in license headers. Many projects
have removed them in recent years and it avoids a bit of yearly toil.
In many cases our copyright claim was also a bit dodgy since we added
the 2021-2023 header to files created after 2021 but you can't claim
copyright into the past that way.
* [chore] Add license header check
This ensures a license header is always added to any new file. This
avoids maintainers/reviewers needing to remember to check for and ask
for it in case a contribution doesn't include it.
* [chore] Add missing license headers
* [chore] Further updates to license header
* Use the more common // indentend comment format
* Remove the hack we had for the linter now that we use the // format
* Add SPDX license identifier
* [chore] Move ShortcodeDomain to its own little util func
* [feature] Add RefetchEmojis function to media manager
* [feature] Expose admin media refresh via admin API
* update following review feedback
- change/fix log levels
- make sure not to try to refetch local emojis
- small style refactoring + comments
* log on emoji refetch start
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
* select emoji using image_static_url
* use updated on AP emojis
* allow refetch of updated emojis
* cheeky workaround for test
* clean up old files for refreshed emoji
* check error for originalPostData
* shorten GetEmojiByStaticImageURL
* delete kirby (sorry nintendo)
* feat: vendor minio client
* feat: introduce storage package with s3 support
* feat: serve s3 files directly
this saves a lot of bandwith as the files are fetched from the object
store directly
* fix: use explicit local storage in tests
* feat: integrate s3 storage with the main server
* fix: add s3 config to cli tests
* docs: explicitly set values in example config
also adds license header to the storage package
* fix: use better http status code on s3 redirect
HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL
307/TemporaryRedirect would mean that this resource is usually located
here, not in this case
303/SeeOther indicates that the redirection does not link to the
requested resource but to another page
* refactor: use context in storage driver interface
* rename + tidy up remote pruning
* fix media attachment account join
see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships
* update logging to new function name
* add get avatars and headers to bun
* add pruneallmeta function
* don't set uncached since we're deleting anyway
* fix totalPruned being in wrong place
* test pruning meta
* go fmt ./...
* rename mediaprune
* add meta pruning to routine mediaprune
* tidy up cleanup job scheduling
* rename adminmediaremoteprune
* update mediacleanup to use renamed prune func
* update swagger docs a little bit
* reuse cancel + context
* Add whereNotEmptyAndNotNull
* Add GetRemoteOlderThanDays
* Add GetRemoteOlderThanDays
* Add PruneRemote to Manager interface
* Start implementing PruneRemote
* add new attachment + status to tests
* fix up and test GetRemoteOlderThan
* fix bad import
* PruneRemote: return number pruned
* add Cached column to mediaattachment
* update + test pruneRemote
* update mediaTest
* use Cached column
* upstep bun to latest version
* embed structs in mediaAttachment
* migrate mediaAttachment to new format
* don't default cached to true
* select only remote media
* update db dependencies
* step bun back to last working version
* update pruneRemote to use Cached field
* fix storage path of test attachments
* add recache logic to manager
* fix trimmed aspect ratio
* test prune and recache
* return errwithcode
* tidy up different paths for emoji vs attachment
* fix incorrect thumbnail type being stored
* expose TransportController to media processor
* implement tee-ing recached content
* add thoughts of dog to test fedi attachments
* test get remote files
* add comment on PruneRemote
* add postData cleanup to recache
* test thumbnail fetching
* add incredible diagram
* go mod tidy
* buffer pipes for recache streaming
* test for client stops reading after 1kb
* add media-remote-cache-days to config
* add cron package
* wrap logrus so it's available to cron
* start and stop cron jobs gracefully
Remote media is now dereferenced and attached properly to incoming federated statuses.
Mentions are now dereferenced and attached properly to incoming federated statuses.
Small fixes to status visibility.
Allow URL params for filtering statuses:
// ExcludeRepliesKey is for specifying whether to exclude replies in a list of returned statuses by an account.
// PinnedKey is for specifying whether to include pinned statuses in a list of returned statuses by an account.
// MaxIDKey is for specifying the maximum ID of the status to retrieve.
// MediaOnlyKey is for specifying that only statuses with media should be returned in a list of returned statuses by an account.
Add endpoint for fetching an account's statuses.
Inbox POST from federated servers now working for statuses and follow requests.
Follow request client API added.
Start work on federating outgoing messages.
Other fixes and changes/tidying up.
This PR adds:
Statuses
New status creation.
View existing status
Delete a status
Fave a status
Unfave a status
See who's faved a status
Media
Upload media attachment and store/retrieve it
Upload custom emoji and store/retrieve it
Fileserver
Serve files from storage
Testing
Test models, testrig -- run a GTS test instance and play around with it.
* start work on accounts module
* plodding away on the accounts endpoint
* groundwork for other account routes
* add password validator
* validation utils
* require account approval flags
* comments
* comments
* go fmt
* comments
* add distributor stub
* rename api to federator
* tidy a bit
* validate new account requests
* rename r router
* comments
* add domain blocks
* add some more shortcuts
* add some more shortcuts
* check email + username availability
* email block checking for signups
* chunking away at it
* tick off a few more things
* some fiddling with tests
* add mock package
* relocate repo
* move mocks around
* set app id on new signups
* initialize oauth server properly
* rename oauth server
* proper mocking tests
* go fmt ./...
* add required fields
* change name of func
* move validation to account.go
* more tests!
* add some file utility tools
* add mediaconfig
* new shortcut
* add some more fields
* add followrequest model
* add notify
* update mastotypes
* mock out storage interface
* start building media interface
* start on update credentials
* mess about with media a bit more
* test image manipulation
* media more or less working
* account update nearly working
* rearranging my package ;) ;) ;)
* phew big stuff!!!!
* fix type checking
* *fiddles*
* Add CreateTables func
* account registration flow working
* tidy
* script to step through auth flow
* add a lil helper for generating user uris
* fiddling with federation a bit
* update progress
* Tidying and linting
kim