Re: How to store "blobs" efficiently for small and large sizes, with random access

På torsdag 20. oktober 2022 kl. 10:32:44, skrev Dominique Devienne <
ddevienne(at)gmail(dot)com <mailto:ddevienne(at)gmail(dot)com>>:
On Wed, Oct 19, 2022 at 5:05 PM Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at> wrote:
> On Wed, 2022-10-19 at 12:48 +0200, Dominique Devienne wrote:
> > On Wed, Oct 19, 2022 at 12:17 PM Andreas Joseph Krogh <andreas(at)visena(dot)com>
wrote:
> > > First advice, don't do it. We started off storing blobs in DB for “TX
safety”
> > Not really an option, I'm afraid.
> You should reconsider. Ruling out that option now might get you into trouble
> later. Large Objects mean trouble.

Andreas, Ericson, Laurenz, thanks for the advice.
I'll be sure to discuss these concerns with the team.
[…]
But before I finish this thread for now, I'd like to add that I
consider unfortunate a state of affairs where
NOT putting the data in the DB is the mostly agreed upon advice. It
IMHO points to a weak point of
PostgreSQL, which does not invest in those use-cases with large data,
perhaps with more file-system
like techniques. Probably because most of the large users of
PostgreSQL are more on the "business"
side (numerous data, but on the smaller sizes) than the "scientific"
side, which (too often) uses files and
files-in-a-file formats like HDF5.
[…]

Note that my views were not PG-specific and applies to all
applications/architectures involving RDBMS.
From my point of view having all data in RDBMS is (maybe) theoretically sound,
but given that IO is not instant I consider it a design-flaw, for some reasons
which I've already pointed out.

--
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
andreas(at)visena(dot)com <mailto:andreas(at)visena(dot)com>
www.visena.com <https://www.visena.com>
<https://www.visena.com>