From: | Melanie Plageman <melanieplageman(at)gmail(dot)com> |
---|---|
To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
Cc: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Avoiding hash join batch explosions with extreme skew and weird stats |
Date: | 2019-07-31 03:07:21 |
Message-ID: | CAAKRu_a6nUv3GXhxG_a=Tr2+Y9-H1VTtqtuiSLCm9cgizzhcEg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, Jul 30, 2019 at 4:36 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Jul 30, 2019 at 2:47 PM Melanie Plageman
> <melanieplageman(at)gmail(dot)com> wrote:
> > I did the "needlessly dumb implementation" Robert mentioned, though,
> > I thought about it and couldn't come up with a much smarter way to
> > write match bits to a file. I think there might be an optimization
> > opportunity in not writing the current_byte to the file each time that
> > the outer tuple matches and only doing this once we have advanced to a
> > tuple number that wouldn't have its match bit in the current_byte. I
> > didn't do that to keep it simple, and, I suspect there might be a bit
> > of gymnastics needed to make sure that that byte is actually written
> > to the file in case we exit from some other state before we encounter
> > the tuple represented in the last bit in that byte.
>
> I mean, I was assuming we'd write in like 8kB blocks or something.
> Doing it a byte at a time seems like it'd produce way too many
> syscals.
>
>
For the actual write to disk, I'm pretty sure I get that for free from
the BufFile API, no?
I was more thinking about optimizing when I call BufFileWrite at all.
--
Melanie Plageman
From | Date | Subject | |
---|---|---|---|
Next Message | Peter Geoghegan | 2019-07-31 03:11:47 | Re: Avoiding hash join batch explosions with extreme skew and weird stats |
Previous Message | Amit Langote | 2019-07-31 02:29:56 | Re: Runtime pruning problem |