I’ve had a Lemmy instance running on a VPS with 100 GB of storage for a few months and it has filled up. I’ve been searching for ways to reduce the amount of storage used but so far I am coming up empty. Can anyone point me in the right direction?
Firstly move pict-rs to object storage. My instance’s pict-rs uses 150GB alone. I pay less than $2/mo to put it on Cloudflare R2. Backblaze B2 might be even cheaper. Instructions: https://crates.io/crates/pict-rs#filesystem-to-object-storage-migration
If that doesn’t help enough and you’re comfortable with SQL, you can purge the unnecessary entries in
received-activities.Command:
delete from received_activity where published & NOW() - INTERVAL '3 days';(Lemmy has problems with ampersands so you’ll have to edit it)Then do a
vacuum full received_activity;to reclaim the space.This deleted 98 million entries for me and reduced my database size from 49GB to 20GB a week ago when I started running out of space. No other effect as far as I can tell. Thanks @illecors@lemmy.cafe
what is stored in
received_activity? anything important? I mean obviously it’s something the instance has received from other instances but is this then stored somewhere else (like comments then stored elsewhere for eg)As far as I’ve been told it’s basically just a log of all received activities. Nothing references it.
Nothing seems to have gone wrong in the past week on thelemmy.club since I removed it. I do have backups though.
Nice, thank you.
in v0.19.0 Lemmy will automatically delete entries over 7 days old
https://github.com/LemmyNet/lemmy/issues/4113
https://github.com/LemmyNet/lemmy/commit/cb01427dcff14b3d88e30220695fc97978786a9a
currently it waits 3 months before deleting
Time to update then!
it’s in alpha currently, but you could still run it
I think you might have to use the
:devtag to get this update, a bit risky to stay on that tag though, maybe wait for the next docker image of an alpha releaseTa, good to know
Well now you can use
:0.19.0-rc.5:)epic :)
Instead of using the hard drive for pictrs, I suggest using it’s S3 capabilities and migrating to bucket based storage. You’ll save way more money and keep the expensive VPS hard drive just doing text and DB things. I think I spend maybe a dollar a month in S3 storage.
For anyone doing this, set up your spending and budget alerts and actions. It’s possible to accidentally fuck something up and end up with an aws bill that’ll suck, but this will give you some measure of protection from that in case you accidentally misconfigure something.
Is there a guide for switching over?
I ran into the same problem and ended up switching to an S3 with Vultr. It’s been a while since I did it but here are the links that I used to figure it out. I’m deployed using Lemmy-Easy-Deploy.
I used a combination of:
https://lemmy.world/post/538280
https://github.com/ubergeek77/Lemmy-Easy-Deploy/blob/main/ADVANCED_CONFIGURATION.md
https://git.asonix.dog/asonix/pict-rs/#user-content-filesystem-to-object-storage-migration
Good luck!
pict-rs has the option to compress images. Ours is set to WEBP with 1280 pixels either side max.
I love how OP doesn’t say it, but everyone immediately goes “it’s the pictures”
Honestly, what else would it be? Text takes ridiculously little storage compared to a single picture of a decent resolution.
in v0.19.0 you could try disabling pictrs caching https://github.com/LemmyNet/lemmy/commit/1d23df37d86cc5cb6d7d9efaaf4360ecc9a9796f
cache_external_link_previews: falseI don’t think that will clear the existing cache though
