Passing values to a dynamic background worker

From: Keith Fiske <keith(at)omniti(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Passing values to a dynamic background worker
Date: 2017-04-17 20:19:13
Message-ID: CAG1_KcAFJ60pac_QnnZX0qeO12NENiPOcohuoQvs297WaT_ObQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

So after reading a recent thread on the steep learning curve for PG
internals [1], I figured I'd share where I've gotten stuck with this in a
new thread vs hijacking that one.

One of the goals I had with pg_partman was to see if I could get the
partitioning python scripts redone as C functions using a dynamic
background worker to be able to commit in batches with a single call. My
thinking was to have a user-function that can accept arguments for things
like the interval value, batch size, and other arguments to the python
script, then start/stop a dynamic bgw up for each batch so it can commit
after each one. The dymanic bgw would essentially just have to call the
already existing partition_data() plpgsql function, but I have to be able
to pass the argument values that the user gave down into the dynamic bgw.

I've reached a roadblock in that bgw_main_arg can only accept a single
argument that must be passed by value for a dynamic bgw. I already worked
around this for passing the database name to the my existing use of a bgw
with doing partition maintenance (pass a simple integer to use as an index
array value). But I'm not sure how to do this for passing multiple values
in. I'm assuming this would be the place where I'd see about storing values
in shared memory to be able to re-use later? I'm not even sure if that's
the right approach, and if it is, where to even start to understand how to
do that. Let alone in the context of how that would interact with the
background worker system. If you look at my existing C code, you can see
it's very simple and doesn't do much more than the worker_spi example. I've
yet to have to interact with any memory contexts or such things, and as the
referenced thread below mentions, doing so is quite a steep learning curve.

Any guidance for a newer internals dev here would be great.

1.
https://www.postgresql.org/message-id/CAH%3Dt1kqwCBF7J1bP0RjgsTcp-SaJaHrF4Yhb1iiQZMe3W-FX2w%40mail.gmail.com

--
Keith Fiske
Database Administrator
OmniTI Computer Consulting, Inc.
http://www.keithf4.com

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2017-04-17 20:27:30 Re: Self-signed certificate instructions
Previous Message Bruce Momjian 2017-04-17 20:09:28 Re: Self-signed certificate instructions