* [docs] Add a certificates guide in Advanced
This adds some documentation about the process of getting certificates
through ACME in general. It also provides a number of links to
alternative clients and certbot deployment guides that are up to date.
Slightly restructure the NGINX and Apache reverse proxy documentation
and insert mentions to the Provisioning TLS certificates advanced
documentation in them.
* [docs] Add firewall section in Advanced
* [docs] Add new guides to section indexes
* [docs] Fix spelling issue
* [docs] Fix a few typos
* [docs] Made Advanced its own section
This splits the Advanced page off from the Getting Started guide and
makes it its own thing. It now has some additional sub-sections for
bigger topics like caching and enhanced security. This also moves
tracing from Getting Started to Advanced as that feels like a more
appropriate location for it.
The enhanced security looks a little silly with a single section, but I
have guides pending for firewall configurations and I'd also like to
consolidate our how to provision TLS certificates in there as we repeat
this information multiple times.
* [docs] Fix all my spelling errors
* [docs] Inline the links in sandboxing
This tries to revamp/restructure the installation guide. It's renamed to
"Getting Started" as it now contains a few more things than just the
installation, especially the deployment considerations which we didn't
use to spell out as much ahead of time.
Installation is now a section with the guides on their own. I've removed
a bit of redundancy like the user creation. I also removed the rogue
reverse proxy section in the Docker guide and lifted that into the
reverse proxy section.
This adds an example on how to get Grafana Tempo up to receive spans as
well as Grafana itself to view them.
I've added this as a separate Tracing doc in the installation guide as
the Advanced one was starting to get rather full.
Fixes: #1791
Our default configuration places the SQLite DB in /gotosocial/, but the
AppArmor profile doesn't allow us to write there. Instead of making the
whole directory writable, add a writable area in /gotosocial/db/ instead
and advise in the docs to move the DB there.
Instead of a manually curated list of distribution packages, this adds
the Repology[1] widget instead.
This also separates out the distribution packages from the
self-hosting/deployment options into their own sections.
[1]: https://repology.org/project/gotosocial/versions
* start working on lists
* further list work
* test list db functions nicely
* more work on lists
* peepoopeepoo
* poke
* start list timeline func
* we're getting there lads
* couldn't be me working on stuff... could it?
* hook up handlers
* fiddling
* weeee
* woah
* screaming, pissing
* fix streaming being a whiny baby
* lint, small test fix, swagger
* tidying up, testing
* fucked! by the linter
* move timelines to state like a boss
* add timeline start to tests using state
* invalidate lists
This adds the preferences endpoint to our Mastodon Client API
implementation. It's a read-only endpoint that returns a number of
user preferences. Applications can query these settings when logging in
a user (for the first time) to configure themselves.
* Explain that markdown is enabled on the user settings page.
The current wording seems to imply that the input formatting can be
set on a per-post basis, which no client currently supports. It's not
clear that you have to go to the user settings page to change it.
* Update FAQ; some of the missing features are not missing any more!
* start working on notifs for new posts
* tidy up a bit
* update swagger
* carry over show reblogs + notify from follow req
* test notify on status post
* update column slice
* dedupe update logic + add tests
* fix own boosts not being timelined
* avoid type check, passing unnecessary accounts
* remove unnecessary 'inReplyToID' check
* add a couple todo's for future db functions
* start poking timelines
* OK yes we're refactoring, but it's nothing like the last time so don't worry
* more fiddling
* update tests, simplify Get
* thanks linter, you're the best, mwah mwah kisses
* do a bit more tidying up
* start buggering about with the prepare function
* fix little oopsie
* start merging lists into 1
* ik heb een heel zwaar leven
nee nee echt waar
* hey it works we did it reddit
* regenerate swagger docs
* tidy up a wee bit
* adjust paging
* fix little error, remove unused functions
* start fiddling about with email sending to allow multiple recipients
* do some fiddling
* notifs working
* notify on closed report
* finishing up
* envparsing
* use strings.ContainsAny
This adds a section to the docs instructing how to ensure apps will be
able to successfully login to an instance when host- and account-domain
differ.
Resolves#1609
* [feature] Allow admins to send test emails
* implement unwrap on new error type
* add + use gtserror types
* GoToSocial Email Test -> GoToSocial Test Email
* add + use getInstance db call
* removed unused "unknown" error type
* [chore] Remove years from all license headers
Years or year ranges aren't required in license headers. Many projects
have removed them in recent years and it avoids a bit of yearly toil.
In many cases our copyright claim was also a bit dodgy since we added
the 2021-2023 header to files created after 2021 but you can't claim
copyright into the past that way.
* [chore] Add license header check
This ensures a license header is always added to any new file. This
avoids maintainers/reviewers needing to remember to check for and ask
for it in case a contribution doesn't include it.
* [chore] Add missing license headers
* [chore] Further updates to license header
* Use the more common // indentend comment format
* Remove the hack we had for the linter now that we use the // format
* Add SPDX license identifier
* [feature] Provide .well-known/host-meta endpoint
This adds the host-meta endpoint as Mastodon clients use this to
discover the API domain to use when the host and account domains aren't
the same.
* Address review comments
Currently, GtS only supports using the built-in LE client directly for
TLS. However, admins may still want to use GtS directly (so without a
reverse proxy) but with certificates provided through some other
mechanism. They may have some centralised way of provisioning these
things themselves, or simply prefer to use LE but with a different
challenge like DNS-01 which is not supported by autocert.
This adds support for loading a public/private keypair from disk instead
of using LE and reconfigures the server to use a TLS listener if we
succeed in doing so.
Additionally, being able to load TLS keypair from disk opens up the path
to using a custom CA for testing purposes avoinding the need for a
constellation of containers and something like Pebble or Step CA to
provide LE APIs.
* Advertise rich text formats, support content_type field
* Update JSON in instance patch tests
* Replace format with content_type everywhere
* update migration to work with both pg and sqlite
* regenerate swagger docs
* update instance serialization + tests
* fix up
* learn to code tobi please, i'm begging you
---------
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* start fiddling
* the ol' fiddle + update
* start working on fetching statuses
* poopy doopy doo where r u uwu
* further adventures in featuring statuses
* finishing up
* fmt
* simply status unpin loop
* move empty featured check back to caller function
* remove unnecessary log.WithContext calls
* remove unnecessary IsIRI() checks
* add explanatory comment about status URIs
* change log level to error
* better test names
This removes the current default of checking for membership of the admin
or admins group and makes it required to explicitly configure which
groups should grant admin access, if any.
Relying on the implicit default of admin or admins is potentially
dangerous as that group may contain a different subset of people that we
may wish to grant admin access to GtS. This is probably not an issue for
a single-person instance, but for a community instance different admin
groups may exist in an OIDC provider for different applications.
I'm explicitly opting for not defaulting the value of oidc-admin-groups
to admin,admins because I think it's better for those things to be
explicitly configured.
* implement status pin client api + web handler
* make test names + comments more descriptive
* don't use separate table for status pins
* remove unused add + remove checking
* tidy up + add some more tests
* [feature] Use Material theme for docs
This changes the theme to use Material with the slate (dark) theme and
the accent colour set to orange. It also replaces the swagger plugin
with one that works correctly with the Material theming.
* Add theme by @f0x52
Co-authored-by: f0x52 <f0x@cthu.lu>
* Enable social cards
The dependencies for cairosvg and pillow can be satisfied on RTD, so
this adds them to our requirements.txt and enables the social plugin.
This generates opengraph and twitter card tags as well as fancy preview
images in GTS colour style for platforms that show that as part of link
previews.
* Update Conda environment.yml
* update theme for swagger, basic light theme option
* dark mode mobile header bg color
---------
Co-authored-by: f0x52 <f0x@cthu.lu>
This adds a lightweight form of tracing to GTS. Each incoming request is
assigned a Request ID which we then pass on and log in all our log
lines. Any function that gets called downstream from an HTTP handler
should now emit a requestID=value pair whenever it logs something.
Co-authored-by: kim <grufwub@gmail.com>
* [bug] Fix nginx fileserver caching example
This updates the example to ensure the nginx proxies the request on to
GTS if the file is not found on disk. This can happen due to media
pruning.
* [chore] Set cache-control in nginx to private
This makes the header match with the backend. For things from the
fileserver it may not be appropriate for anything other than a private
cache (i.e the client) to cache things.
* serve publickey separately from AP, don't throttle it
* update nginx cache documentation, cache main-key too
* throttle public key, but separately from other endpoints
since 2.4.47 (released April 22nd 2021), Apache httpd can ProxyPass to
websockets on the same URL, without mod_rewrite (and, without
mod_proxy_wstunnel).
* [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite
* reduce cache size default to 8MiB, reduce connections to 2 * cpu
* introduce max open conns multiplier, tune sqlite and pg separately
* go fmt
* start adding report client api
* route + test reports get
* start report create endpoint
* you can create reports now babyy
* stub account report processor
* add single reportGet endpoint
* fix test
* add more filtering params to /api/v1/reports GET
* update swagger
* use marshalIndent in tests
* add + test missing Link info
* Add local user and post count to nodeinfo responses
This fixes#1307 (at least partially). The nodeinfo endpoint should now
return the total users on an instance, along with their post count.
* Update NodeInfoUsers docstring and swagger yaml file
* AWS S3 config details added
It was interesting to note that since presigned urls are used buckets dont need to be exposed publically. this was an interesting change compared to other mastodon specific s3 bucket guides hence documented here for correct directions.
* Update storage.md
1. Added AWS identified to make it clear its aws specific.
2. Adjusted text around data migration
* updation as requested
Refining the doc as per request.
* [feature] Add throttling middleware to AP endpoints
* refactor a lil bit
* use config setting, start updating docs
* doc updates
* use relative links in faq doc
* small docs fixes
* return code 503 instead of 429 when throttled
* throttle other endpoints too
* simplify token channel prefills
* interim commit: start refactoring middlewares into package under router
* another interim commit, this is becoming a big job
* another fucking massive interim commit
* refactor bookmarks to new style
* ambassador, wiz zeze commits you are spoiling uz
* she compiles, we're getting there
* we're just normal men; we're just innocent men
* apiutil
* whoopsie
* i'm glad noone reads commit msgs haha :blob_sweat:
* use that weirdo go-bytesize library for maxMultipartMemory
* fix media module paths
Lots of these were appearing:
```
*459 connect() failed (111: Connection refused) while connecting to upstream
```
This change resolves it, see https://stackoverflow.com/a/52550758
* [docs] Serve static assets with nginx
This explains how to use nginx to serve static assets and offload GTS
from that responsibility. It also shows how to have nginx add caching
headers to indicate to clients how long they may cache an asset.
* [docs] Move additional nginx config to advanced
This moves a bunch of additional nginx configuration into the Advanced
page instead. It declutters the nginx configuration page.
This explains how nginx can be used to cache webfinger responses and
potentially serve stale responses in case GTS is down. This can be
useful to do in order to ensure webfinger keeps working even if you're
doing some maintenance.
* [chore] Move ShortcodeDomain to its own little util func
* [feature] Add RefetchEmojis function to media manager
* [feature] Expose admin media refresh via admin API
* update following review feedback
- change/fix log levels
- make sure not to try to refetch local emojis
- small style refactoring + comments
* log on emoji refetch start
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
* [docs] Update contributing.md
- Add Pull Request process and guidelines.
- Add feature/bug issue process.
- Rearrange some sections for clarity.
- Add overview of package structure.
* [docs] Add build from source links
* [chore] add pull request templates
These link to the new CONTRIBUTING.md document, and include a checklist to validate that contributors have read the guidelines.
* [docs] Put existing stub CoC in separate doc
* update web related stuff in CONTRIBUTING.md
Co-authored-by: f0x <f0x@cthu.lu>
* Implement Bookmarks
* Update based on review comments
* Update swagger doc
* Fix argument passing to status.Bookmark
* Update changed test
* Updates based on latest PR review
* [feature] overhaul the oidc system
this allows for more flexible username handling and prevents account
takeover using old email addresses
* [feature] add migration path for old OIDC users
* [feature] nicer error reporting for users
* [docs] document the new OIDC flow
* [fix] return early on oidc error
* [docs]: add comments on the finalization logic
* remove filesystem logging directives from example systemd unit config
* [docs] Update docs to reflect new systemd config
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* Enable the 'admonitions' Markdown extension for Mkdocs.
The admonitions extension to Python-Markdown allows you to include
rST-style "admonitions" to Markdown documents, for instance,
!!! note
Here's an important note to keep in mind!
In general, the current documentation uses bold text to try to achieve
the same effect, which is a bit harder to notice and makes it difficult
to differentiate between "here's something useful to know" versus "here
there be dragons".
* Add AppArmor profile and documentation for LSM-related sandboxing
This commit adds an AppArmor profile for gotosocial in
examples/apparmor/gotosocial. This will (hopefully) serve as a helpful
security mitigation for people are planning on deploying GTS on a
Debian-family Linux distribution.
I've also updates the documentation to include some information about
deploying GTS with either AppArmor or SELinux (moving the documentation
for the former out of the "binary installation guide" docs).
Since the documentation site only shows the latest version of the
docs, we need the docs to explain how to use the latest stable
release, not just the latest git version.
* add FilePath regex
* add `admin media prune orphaned` command
* add prune orphaned function to media manager
* don't mark flag as required
* document admin media prune orphaned cmd
* oh envparsing.sh you coy minx
The NewSignup method was already being called with
requireApproval=false, but it had emailVerified=false as well, which
meant that it was required to use the `admin account confirm` command
to verify the email before the newly-created user could log in.
I think that was probably an oversight; effectively it did require
approval anyway. Changing emailVerified to true allows you to just
create the account and log in immediately, reducing the opportunity
for manual error to sneak in.
Also updated the docs to remove the mention of needing to confirm new
accounts. However, I've left the confirmation command alone because I
think once we have web signups, it will be needed in that context.
* start adding admin emoji PATCH stuff
* updating works OK, now how about copying
* allow emojis to be copied
* update swagger docs
* update admin processer to use non-interface storage driver
* remove shortcode updating for local emojis
* go fmt
Co-authored-by: f0x52 <f0x@cthu.lu>
* Clarify that gotosocial doesn't use recommendation algorithms
It uses lots of algorithms.
~/src/gotosocial $ rgrep algorithm . | wc
856 8971 143395
* Add first draft of the FAQ.
* Suggested FAQ changes.
Mention backfill of posts as not yet being implemented; link to FAQ
from readme/index.
Mention that GtS does serve up some UI; just not most interaction.
Daenney