Throat

https://phuks.co/

A phoxy link and discussion aggregator with snek (python3)

Dependencies:

• A database server, MySQL, MariaDB and Postgres have been tested. Sqlite should work for messing locally
• Redis
• Python >= 3.9
• A recent node/npm
• libmagic
• libpq-dev

Setup:

We recommend using a virtualenv or Pyenv and setting it up to use Python 3.9 for best compatibility (higher versions might fail to build dependencies). See for e.g. pyenv to learn how to install multiple Python versions on your workstation. Refer to your virtual environment manager for documentation on selecting specific Python versions.

1. Install Python dependencies with poetry install
2. Install Node dependencies with npm install
3. Build the bundles with npm run build
4. Copy example.config.yaml to config.yaml and edit it
5. Set up the database by executing poetry run ./throat.py migration apply
6. Compile the translation files with poetry run ./throat.py translations compile

And you're done! You can run a test server by executing poetry run ./throat.py. For production instances we recommend setting up gunicorn

Production deployments

Please read doc/deploy.md for instructions to deploy on gunicorn or using docker.

Develop on Docker

If you prefer to develop on docker - The provided Docker resources only support Postgres - You still must copy example.config.yaml to config.yaml and make any changes you want - In addition, configs are overridden by environment variables set in docker-compose.yml which reference the redis and postgres services created by docker-compose.

make up will bring the containerized site up and mount the app/html and app/template directories inside the container for dev. It also runs the migrations on start-up. make down will spin down the containerized services.

To add an admin user to a running docker-compose application: docker exec throat_throat_1 ./throat.py admin add {{username}}

If Wheezy templates are not automatically reloading in docker between changes, try docker restart throat_throat_1.

Database Configuration

The default hot sort function is simple for speed, but it does not prioritize new posts over old ones as much as some people prefer. If you define a function named hot in SQL in your database, you can use that instead of the default by setting custom_hot_sort to True in your config.yaml. The function needs to take two arguments, a post's current score and the date it was posted. To allow the database to cache the results, the function should only depend on the values of its arguments and should be marked immutable.

In addition to defining the function, you should also create an index on it to speed up the hot sort query. Once that is done, custom functions will be faster than the default hot sort. To implement Reddit's version of hot sort in Postgres, add the following SQL statements to your database using psql:

```sql create or replace function hot(score integer, date double precision) returns numeric as $$ select round(cast(log(greatest(abs($1), 1)) * sign($1) + ($2 - 1134028003) / 45000.0 as numeric), 7) $$ language sql immutable;

create index on sub_post (hot(score, (EXTRACT(EPOCH FROM sub_post.posted)))); ```

Other databases may require variations in the handling of the date. Custom hot sorts are not supported for Sqlite.

Docker Deployments

Gunicorn

CMD [ "gunicorn", \ "-w", "4", \ "-k", "geventwebsocket.gunicorn.workers.GeventWebSocketWorker", \ "-b", "0.0.0.0:5000", \ "throat:app" ]

Authenticating with a Keycloak server

Optionally, user authentication can be done using a Keycloak server. You will need to create a realm for the users on the server, as well as Keycloak clients with appropriate permissions. See doc/keycloak.org for instructions.

Deploying to AWS

You can check out the CDK Definition of Infrastructure maintained by Ovarit

Management commands

• ./throat.py admin to list, add or remove administrators.
• ./throat.py default to list, add or remove default subs.

Tests

Python tests

1. Python, redis, and libmagic are required, but node and postgres are not.

2. Install dependencies with pip install -r requirements.txt

3. Run the tests with python -m pytest

4. The tests are not affected by your configuration in config.yaml. If you wish to run the tests against production database or authentication servers (instead of the defaults, which are sqlite and local authentication), you may put configuration settings in test_config.yaml and run the tests with TEST_CONFIG=test_config.yaml python -m pytest. The tests will erase the database supplied in the test configuration. You can also supply a logging configuration in test_config.yaml and then pytest will show you the logs from failing tests.

Note: The tests currently work with Postgres and Sqlite. Testing with MySQL is not yet supported.

Testing under Docker

You can run pytest in a Docker container via docker-compose with make test.

To pass arguments to pytest, invoke make like so: make test ARGS="-x -k my_test"

Chat

If you have any questions, you can reach us on #throat:phuks.co on Matrix

Issues

GitHub CodeQL code scanning alerts for Throat

opened on 2023-03-15 13:28:36 by globalistas

I believe these particular alerts are not enabled on this repo, but on my fork I have them enabled and they have produced the following report:

https://github.com/globalistas/ceknito/security/code-scanning

Might be worth it to enable them here as well and then fix these security issues, unless they are false positives of some sort.

Issues with the Email Forwarded Notifications PR

opened on 2023-03-14 08:44:51 by globalistas

          I have tested this PR and found several issues:

1) PR is missing an updated example.config.yaml with the new config setting email_forwarded_notifications = False in it

2) When email_forwarded_notifications = True is specified in config.yaml, running poetry run ./throat.py migration apply gets stuck and then crashes with the following error:

INFO:migration:: :Starting migrations DEBUG:app.sql_timing:: :(CREATE TABLE IF NOT EXISTS "migratehistory" ("id" SERIAL NOT NULL PRIMARY KEY, "name" VARCHAR(255) NOT NULL, "migrated_at" TIMESTAMP NOT NULL), []) (executed in 3 ms) DEBUG:app.sql_timing:: :(SELECT "t1"."id", "t1"."name", "t1"."migrated_at" FROM "migratehistory" AS "t1" ORDER BY "t1"."id", []) (executed in 1 ms) INFO:migration:: :There is nothing to migrate Process Process-1: Traceback (most recent call last): File "/home/app/.pyenv/versions/3.9.16/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/app/.pyenv/versions/3.9.16/lib/python3.9/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/app/ceknito/app/email_manager.py", line 98, in _produce Notification.select( File "/home/app/.pyenv/versions/3.9.16/envs/app/lib/python3.9/site-packages/peewee.py", line 1960, in inner raise InterfaceError('Query must be bound to a database in order ' peewee.InterfaceError: Query must be bound to a database in order to call "execute". Process Process-3: Traceback (most recent call last): File "/home/app/.pyenv/versions/3.9.16/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/app/.pyenv/versions/3.9.16/lib/python3.9/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/app/ceknito/app/email_manager.py", line 98, in _produce Notification.select( File "/home/app/.pyenv/versions/3.9.16/envs/app/lib/python3.9/site-packages/peewee.py", line 1960, in inner raise InterfaceError('Query must be bound to a database in order ' peewee.InterfaceError: Query must be bound to a database in order to call "execute".

This error does not occur when False is specified.

3) When email_forwarded_notifications = True is specified in config.yaml, site user sees the following error when trying to open user preferences html:

File "user/settings/preferences.html", line 39, in main AttributeError: 'EditUserForm' object has no attribute 'email_forwarded_notifications'

Originally posted by @globalistas in https://github.com/Phuks-co/throat/issues/520#issuecomment-1467647931

Optimized CI with: jobs + caching; updated actions used in steps.

opened on 2023-03-13 22:06:58 by why-not-try-calmer None

Possible issue with the 'session' cookie missing a proper "samesite" attribute value

opened on 2023-03-08 09:12:00 by globalistas

Is this an actual issue currently or going forward?

Background task seems to be failing to upload an mp4

opened on 2022-10-28 14:10:02 by globalistas

Trying to upload this file, a Background task seems to be failing.

https://user-images.githubusercontent.com/64347618/198634789-1a016041-d99b-4309-9f39-b28ad4077a6a.mp4

Easy installtion manual

opened on 2022-09-08 17:28:16 by jayabie

Hello Team,

I'm a beginner and i would like to install Thraot on a VPS. is there any easy manual i can follow will allow me to make a full setup?