90 Commits

Author SHA1 Message Date
Mia Herkt
0cd289d981 Move uploaded file handling to class
So uploading files via remote URLs was completely broken and
apparently nobody noticed. This commit fixes that, too.
Wouldn’t it be nice if there were a test suite!
2025-03-01 10:08:27 +01:00
Mia Herkt
a2b322f868 Avoid holding in-memory copies of file content
Werkzeug uses tempfile.SpooledTemporaryFile, so we can make use of
file-like object properties. This may result in more disk writes,
but that’s probably better than eating up RAM.

I hope this fixes #84.
2024-09-27 20:45:42 +02:00
Mia Herkt
f65bccc2aa Remove blacklist reference from default index template 2024-09-27 19:15:57 +02:00
Mia Herkt
de19212a71 PEP8 compliance 2024-09-27 18:30:33 +02:00
Mia Herkt
a2147cc964 Remove broken tests
Will be readded after some major refactoring and modernization.
2024-09-27 18:30:31 +02:00
Mia Herkt
45a414c5ee Implement request filters
This moves preexisting blacklists to the database, and adds the
following filter types:

    * IP address
    * IP network
    * MIME type
    * User agent

In addition, IP address handling is now done with the ipaddress
module.
2024-09-27 18:30:28 +02:00
Mia Herkt
6393538333 Replace NSFW detector implementation 2024-09-27 06:34:14 +02:00
Mia Herkt
3330a85c2c ModUI: Update for Textual 0.54.0 2024-03-30 18:23:37 +01:00
polina4096
8a912e8744 Fix remote URL content length check off-by-one
Fixes #85
2023-06-04 06:35:11 +02:00
Mia Herkt
c2b5e95903 ModUI: Handle opening filter panel with NULL user agent 2023-03-29 07:49:56 +02:00
Mia Herkt
c189c47306 ModUI: Allow LIKE matching for address filtering 2023-03-29 07:38:36 +02:00
Mia Herkt
3d1facaec3 Store user agent with files
Needed for moderation.
2023-03-29 07:36:49 +02:00
Mia Herkt
e00866f5e4 URL: Explicitly set upper-case table name
Looks like recent SQLAlchemy/Alembic chose to lower-case it by
default. Try not to break existing schemas.
2023-03-29 07:19:47 +02:00
jonas-w
3950f6e8eb fix 500 error when file extension could not be guessed
when a file without an extension was uploaded
and the mimetypes.guess_extension returned None
because there is no official file extension
for that mimetype a NoneType was subscripted
which yielded a 500 http error
2023-01-15 20:36:39 +01:00
Mia Herkt
e1e99957b6 ModUI: Fix crash when encountering null NSFW score
Fixes #78
2022-12-29 19:51:04 +01:00
Mia Herkt
647e3a54f1 ModUI: Add application/xml to text handler 2022-12-22 09:55:41 +01:00
Mia Herkt
0e4f0206ab ModUI: Fix jinja2 func call in ban action 2022-12-22 09:44:32 +01:00
Mia Herkt
53249df28d README: Kitty support was merged in mpv 2022-12-21 19:47:49 +01:00
Mia Herkt
556cd8aeae README: Add ModUI screenshot 2022-12-20 16:57:07 +01:00
Mia Herkt
8b04e08fd6 ModUI: Add application/json to text handler 2022-12-20 16:23:35 +01:00
Mia Herkt
455863c138 Update requirements.txt 2022-12-20 16:19:49 +01:00
Mia Herkt
eebd5d8c6d Add moderation TUI
This ended up way fancier than I imagined.
2022-12-20 16:19:49 +01:00
Mia Herkt
dcea8bffe1 migrations: Fix file expirations on SQLite
Well that was what we feared. I love arbitrary hardcoded limits.
2022-12-20 14:23:14 +01:00
Mia Herkt
f76dbef82f Fix NSFW detection 2022-12-17 02:32:51 +01:00
Mia Herkt
57c4b6853f Prevent unreasonably long MIME types 2022-12-13 23:41:12 +01:00
Mia Herkt
77801efd21 Fix URL test issue 2022-12-13 23:18:40 +01:00
Mia Herkt
d5763a9854 File: Fix 404 case with secret URLs 2022-12-13 23:17:56 +01:00
Mia Herkt
aaf0e4492a Record file sizes in db
Moderation interface is going to use this.
2022-12-13 23:04:48 +01:00
Mia Herkt
6055a50948 File: Add is_nsfw property 2022-12-13 21:51:39 +01:00
Mia Herkt
b1ed63c401 README: Add note about StreamMaxLength in clamd.conf 2022-12-12 07:40:38 +01:00
Mia Herkt
a904922cbd Add support for ClamAV 2022-12-12 07:35:05 +01:00
Mia Herkt
da30c8f8ff index.html: Document appending file names 2022-12-01 03:28:25 +01:00
Mia Herkt
0b80a62f80 Add support for “secret” file URLs
Closes #47
2022-12-01 02:49:28 +01:00
Mia Herkt
ed84d3752c Fix 500 on invalid paths 2022-12-01 01:26:32 +01:00
Mia Herkt
7661216bc0 Fix handling double file name extensions
Long names would get truncated at the end, causing problems
including unresolvable file URLs. Example with default settings:
    .package.lst → .package.

Fixes #61
2022-12-01 01:19:05 +01:00
Mia Herkt
9214bb4832 Add X-Expires to file response headers
Tells clients when files will expire, in milliseconds since Unix epoch.

Closes #50.
2022-11-30 02:30:52 +01:00
Mia Herkt
e168534258 Allow changing expiration date 2022-11-30 02:19:29 +01:00
Mia Herkt
afe2329bf5 templates/index: Remove unnecessary escaping 2022-11-30 02:19:29 +01:00
Mia Herkt
a182b6199b Allow management operations like deleting files
This introduces the X-Token header field in the response of newly
uploaded files as a simple way for users to manage their own files.

It does not need to be particularly secure.
2022-11-30 02:19:29 +01:00
Mia Herkt
eb0b1d2f69 nsfw_detect: Use PyAV instead of ffmpegthumbnailer 2022-11-29 21:54:43 +01:00
Mia Herkt
14cfe3da58 nsfw_detect: Use pathlib, fix deprecation warning
Also fix glog suppression
2022-11-29 21:44:07 +01:00
Mia Herkt
aa443178e1 README: Also run db upgrade after git pull! 2022-11-29 17:23:56 +01:00
Mia Herkt
db9a20c94d Add example systemd unit files for prune job 2022-11-29 17:23:30 +01:00
Mia Herkt
f25619b7e3 nsfw_detect: Tolerate score computation failure 2022-11-29 13:31:35 +01:00
Emi Simpson
af4b3b06c0 Add support for expiring files
SUPPLEMENTALLY:
- Add an `expiration` field to the `file` table of the database
- Produce a migration for the above change
- Overhaul the cleanup script, and integrate into fhost.py
  (now run using FLASK_APP=fhost flask prune)
- Replace the old cleanup script with a deprecation notice
- Add information about how to expire files to the index
- Update the README with information about the new script

Squashed commits:

Add a note explaining that expired files aren't immediately removed

Show correct times on the index page graph

Improve the migration script, removing the need for --legacy

Use automap in place of an explicit file map in migration

Remove vestigial `touch()`

Don't crash when upgrading a fresh database

Remove vestigial warning about legacy files

More efficiently filter to unexpired files when migrating

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-224

Coalesce updates to the database during migration

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-226

Remove vestigial database model

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-261

prune:  Stream expired files from the database

(as opposed to collecting them all first)

config.example.py:  Add min & max expiration + description
2022-11-29 13:09:26 +01:00
Mia Herkt
00dba0e189 config.example.py: Clarify MIME ext mapping 2022-11-28 22:30:53 +01:00
Ember Hearth
be796b9b5b Add example configuration file
See #73.
2022-11-28 22:30:43 +01:00
Mia Herkt
b5f0cfdf6f README: Clarify why serving file requests from the app is bad 2022-11-20 16:54:46 +01:00
Mia Herkt
4ba18146e1 README: Clarify how to change configuration 2022-11-20 13:05:17 +01:00
Mia Herkt
afb5811879 README: Warn users about URL fetch network security implications 2022-08-19 22:27:29 +02:00