Re: Quesion about querying distributed databases

On Wed, Mar 5, 2025 at 9:44 PM me nefcanto <sn(dot)1361(at)gmail(dot)com> wrote:

> I once worked with a monolithic SQL Server database with more than 10
> billion records and about 8 Terabytes of data. A single backup took us more
> than 21 days. It was a nightmare.
>

25 years ago (meaning *much* slower hardware), I managed a 1TB database.
Backups took about 4 hours. Could have gotten it down to two hours if I'd
wanted to use more tape drives.

Right now, I manage a 5TB database. Backups take 110 minutes, and that's
when using one channel for all IO, writing to not the fastest NAS, and
other 3+TB databases backing up to it at the same time.

> Almost everybody knows that scaling up has a ceiling
>

And that ceiling is much, much higher than you think it is.

> , but scaling out has no boundaries.
>

Except for complexity and fragility. I bet I could get good scaled up
performance out of the amount of hardware you're using to scale out.

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!