It’s still the same issue, RAID or Ceph. If a physical drive can only write 100 MB/s, a 36TB drive will take 360,000 seconds (6000 minutes or 100 hours) to write. During the 100-hour window, you’ll be down a drive, and be vulnerable to a second failure. Both RAID and Ceph can be configured for more redundancy at the cost of less storage capacity, but even Ceph fails (down to read only mode, or data loss) if too many physical drives fail.

all the quadratic communication and caching growth it requires.

I have trouble visualizing and understanding how the Internet works at scale, but can generally grasp how page-by-page or resource-by-resource requests work. I struggle to understand how one could efficiently parse the firehose of activity coming from every user on every instance that your own users follow, at least in user-focused services like Mastodon (or Twitter or Bluesky). With Lemmy, there will be many more people following the biggest communities with the most activity, so caching naturally scales. But with Twitter-like follows of individual accounts, there are going to be a lot of accounts on the long tail, with lots of different accounts being followed only by a few people. The most efficient method is to just ignore the small accounts, but obviously that ends up affecting a large number of accounts. But on the other hand, keeping up with the many small accounts will end up occupying all the resources on stuff very few people want to see.

A centralized service has to struggle with this as well, but might have better control over caching and other on-demand retrieval of content in lower demand, without inadvertently DDoSing someone else’s server.