Re: Batching in executor

Hi,

I just discovered this very interesting thread on batch execution. It's
very relevant to some of the work we've done on columnar engines and table
access methods in TimescaleDB. I am happy to share some of our learnings. I
have not digested the full history of the thread yet, so I apologize in
advance if I repeat things you've already discussed. I intend to study the
current patch set in detail later. I'll just describe at a high level what
we did and the approaches we took. I am not sure yet how much this overlaps
with the ongoing batch executor work, but hopefully it still provides
valuable insights---if nothing else, perhaps only to validate your current
approach.

Obviously, for TimescaleDB, we approached batch processing from the
point-of-view of an extension where we could not change the existing
interfaces. That led us to initially build a columnar engine using custom
executor nodes to implement columnar processing on top of the heap engine.
This approach wasn't very principled, making our code less compatible with
both existing and future PostgreSQL APIs, which is not ideal. One example
is the table access method API (TAM), which didn't exist when we built the
first version of our columnar engine. However, as TAM became available,
along with other new APIs, we attempted to be more principled by building a
TAM over our existing columnar engine. Perhaps that work can inform and
help improve this work on batch execution?

I think the initial instinct when looking at this problem is to extend the
TAM API with new batch functions. However, since that wasn't an option, we
decided to build it mostly as a custom TupleTableSlot implementation using
its extensible interface. On reflection, I think that's actually the more
principled approach because it keeps the TAM and underlying storage very
transparent to "upper" executor nodes and APIs. So, for this purpose we
built a custom TupleTableSlot for batching that we called
ArrowTupleTableSlot because we use the Arrow format internally for
processing columnar data. The implementation of the slot can be found here:
https://github.com/timescale/timescaledb/blob/2.21.x/tsl/src/hypercore/arrow_tts.h

So, the way this functions is as follows:

If the TAM supports columnar format or batching (which is transparent to
nodes from the TAM), it produces an ArrowTupleTableSlot which internally
holds a batch of data and an index into that batch. Initially, when the
TAM's `getnextslot()` produces the slot, the batch is not even
materialized; it's only a reference to the relevant data in storage (which
could be compressed). When a processing node (let's say SeqScan for
simplicity) eventually calls getsomeattrs() on the slot, the batch is
materialized (e.g., decompressed) and the slot's index points to the first
entry in the batch array. This "late materialization" is critical for good
performance because it delays decompression until after filtering, or it
decompresses only the columns needed for the filtering. Internally, the
slot's batch might hold relevant metadata, like min/max values so an
executor node can filter the entire batch slot before materialization. Even
without min/max values, a vectorized filter can quickly scan a column in
the batch and determine that no "rows" pass the filter. Skipping the entire
batch is done by setting the slot's batch index to "end".

When TAM's getnextslot() is called for the second time, one of two things
happens:

1. The TAM increments the index in the existing slot's batch and sees that
it is not yet done, just returning the existing slot again without reading
new data.
2. The TAM notices that the slot's batch index is at "end", and then reads
the next batch into the slot, setting the index to 0.

I am skipping many other details for now, but hopefully this is enough to
get the gist of it.

So, what's the advantage of this approach?

1. It is entirely transparent to existing processing nodes and the TAM
interface. In fact, SeqScan and any other similar nodes can work on top of
the TAM and the slot without knowing it produces batches internally.
However, a SeqScan doesn't take full advantage of all the benefits either.
2. Custom "batch-aware" nodes (e.g., a "BatchSeqScan") can detect that
they read an ArrowTupleTableSlot from the TAM based on the slot type, and
therefore initiate special batch processing. E.g., filtering entire batches
or running vectorized expressions over them.
3. The TAM and slot interfaces remain largely the same. However, to really
take advantage of columnar processing, some API extensions are needed.
4. The batch slot can carry the batch data up to a vector aggregation node,
which can efficiently compute functions like avg() over the entire batch in
one go.

Here's a non-exhaustive discussion of some API changes that might be
necessary with the above approach:

TupleTableSlot:

- getsomeattrs(): This API will materialize all columns up to the given
index. This does not work for columnar data because each column needs
separate decompression. Instead, it should be possible to materialize only
the columns used by filters or projections. In fact, if your query projects
columns 2,1 (in that order) you might initially decompress only column 4
for vectorized filtering, end then _after_ filtering, columns 2,1 for
projection on the batches that survive. We worked around this by keeping
filter and projection column masks in the ArrowTupleTableSlot so that we
only materialized the necessary columns whenever getsomeattrs was called,
using the separate masks for filtering and projection.

- For batch processing, the TupleTableSlot API probably needs some
additional API functions for batch-aware processing nodes, e.g.,
getbatchsize(), increment_batch_index(), etc.

- One issue we had was with projections: When passing the slot up to an Agg
node, PostgreSQL typically wants to do projection in order to convert the
slot to a VirtualTupleTableSlot with only the projected columns and data
and maybe change the column order. However, that removes the batch data and
prevents vectorized aggregation. Instead, there must be a way to project
columns while preserving the batch and/or columnar format. One way to do
this is to introduce indirection: keep the batch data in its original
column order and, instead, apply projection to an array of column indexes
in the slot to reorder them, rather than actually reordering and copying
the data into a VirtualTupleTableSlot.

TAM:

For basic batch processing, the TAM API didn't actually need any changes
with the approach we took. However, you probably want a way to push down
certain expressions, for example, skipping entire RowGroups (in Parquet
terminology). Someone already proposed doing this in beginscan(), which
might work. We used ScanKeys to push down some basic min/max filters. But
maybe the API can be improved? One thing that is worth keeping in mind is
that the semantics of columnar filtering might differ from existing
filtering in PostgreSQL. Normally, when providing a ScanKey the contract is
that only data passing the filter is passed upward and no additioanl quals
filtering is needed. However, with batches and "sparse" metadata, like
min/max you can only exclude batches that cannot possibly contain any
relevant values. For example, if min/max "temp" of a batch is 5 and 10, and
the filter is "temp" > 7, the entire batch must be passed "upward" even
though some values within the batch should be excluded. Therefore, you must
still run the same qual filters on the batch even though you passed down
ScanKeys. Otherwise the entire filtering and expression engines must move
down into the TAM, which I don't think is the right approach. That said,
some TAMs might provide a full expression and filtering engine internally,
and should probably be allowed to do that. Other columnar/batch TAMs might
want to offload at least some filtering to PostgreSQL's filtering engine
instead. For example, they could perform basic min/max batch filtering
internally but offload advanced expression filtering to higher-level
processing nodes in PostgreSQL.

FWIW, SeqScan currently doesn't provide ScanKeys to TAMs even if the API
exists, so we implemented a custom SeqScan node to push down the relevant
keys. Still, executor nodes can perform similar filtering using metadata
within the TupleTableSlot, as I described. If no materialization has
occurred, there is no performance loss.

For columnar data, the TID format was a challenge. I know there have been
proposals to change it already, so won't add much to that discussion for
now, except mentioning that we hacked around it by re-encoding the TID to
hold <block, offset, batch-index>. This was an imperfect approach, but it
worked with some limitations. This approach allowed the standard index AM
API to work on top of our columnar TAM, enabling support for cursors and
other features.

We also dealt with many other issues, such as cost models for planning when
producing columnar data, multi-stage filtering (batch skipping ->
vectorized filters -> regular quals over each value), batch deletions and
inserts, index support via IAM (e.g., should min/max filters be an IAM or
internal to TAM, or both).

I will stop here for now, as this is already a lengthy addition to this
thread. Happy to continue the discussion and answer questions if you have
any.

Best regards,

Erik

On Tue, Jul 21, 2026 at 8:03 AM Amit Langote <amitlangote09(at)gmail(dot)com>
wrote:

> Hi,
>
> I still haven't gotten a chance to review Denis's proposal and the
> patches closely, but I wanted to second what Antonin wrote below.
>
> On Fri, Jul 17, 2026 at 9:12 PM Antonin Houska <ah(at)cybertec(dot)at> wrote:
> > Denis Smirnov <darthunix(at)gmail(dot)com> wrote:
> >
> > > I am not sure adding a new table AM callback for this is the right
> > > direction, at least for this patch.
> > >
> > > My concern is that scan_getnextbatch still looks like a row-oriented
> > > interface. For a Parquet-like AM, with columnar storage and block-level
> > > filters such as bloom/fuse filters, the useful API would need to pass
> > > down things like the required columns, pushed-down predicates, and
> maybe
> > > a limit. Just asking the AM for the next batch of rows does not give
> the
> > > storage layer enough information to avoid unnecessary work.
> >
> > Is there a reason not to include this information in the scan descriptor
> of
> > particular AM?
>
> Yeah, I think that information is better passed to beginscan() than to
> getnextslot() or getnextbatch(). The scan descriptor is scan-lifetime
> state, so telling the AM which columns and predicates it can use
> belongs at scan setup; what the getnext* functions return per call is
> a separate question. That work could be undertaken independently of
> allowing them to return batches.
>
> I'll try to reply properly on the rest of Denis's points later this week.
>
> --
> Thanks, Amit Langote
>
>
>