Skip site navigation (1) Skip section navigation (2)

[proposal] protocol extension to support loadable stream filters

From: Brent Verner <brent(at)rcfile(dot)org>
To: pgsql-hackers(at)postgresql(dot)org
Subject: [proposal] protocol extension to support loadable stream filters
Date: 2005-04-25 17:32:30
Message-ID: 20050425173230.GA4347@rcfile.org (view raw or flat)
Thread:
Lists: pgsql-hackers
Hackers,

  I'd like to introduce the concept of (dynamically loaded) stream
filters that would be used to wrap calls to send/recv by the FE/BE 
protocol.  The initial "StreamFilter" will be a zlib compression 
filter.  Yeah, I know it could just be added along-side (in the
same way as) the SSL code, in fact, having done just that is 
precisely why I think a generic filter API is a good idea.  The 
SSL code could then be moved out into a loadable filter.  At 
first blush, this looks like it will entail the following:


  1) Define a StreamFilter struct (see below) and a base implementation
     that simply wraps send/recv calls.  This base StreamFilter source
     file will also contain a handful of utility functions to deal
     with the loading/managing subsequent filters.

  2) Add StreamFilter member to Port and PGconn, initialized with the
     base StreamFilter.

  3) Add support for a new ProtocolVersion in ProcessStartupPacket
       NEGOTIATE_FILTER_CODE PG_PROTOCOL(1234,5680)
     In addition, the client would send the name of the filter
     for which it is requesting support along with some filter-
     specific options...maybe a string like this,
       "zlib:required=1;level=7"
     where everything up to the ':' is the filter name, and
     the remainder would be optional config options.

     Q: Would this /require/ a bump of the protocol version?

     Q: Is there some chunk of existing code that I could easily
        use to parse the options bit of the filter request
        string above?  I looked at PQconninfoOption, but I'm
        not sure that's what I want.  All I want is for that
        options string to be parsed into a simple structure
        from which I can request values by name.

  4) Add support to PQconnectPoll (and throughout client...) for 
     requesting and initializing a StreamFilter.

  5) Create a pg_dlsym_ptr() function that returns a void*, instead
     of a PGFunction like pg_dlsym.

  6) The server will (attempt to) load the filter by name using 
       pg_dlopen( expand_dynamic_library_name( "sf_" + filterName ) )
     or something along those lines. That library's create()
     function will return a StreamFilter...

  7) add support for StreamFilter(s) to pqsecure_read/write and 
     secure_read/write.  If SSL were factored out as a streamFilter,
     fe-secure.c and be-secure.c could really be factored away, and
     the calls to [pq]secure_read/write would be replaced by direct
     calls to StreamFilter->read/write.

  Ok, I think that is a fair high-level description of what I'd
like to do.  Comments?  Questions?  I don't find much time to work
on fun stuff during the week (or even to keep up with fun stuff...),
but I do expect to have time this weekend to wrap up a working 
prototype of this.  Hopefully, I'll have time knock out the jdbc
zlib client as well.

Thanks.
	Brent


typedef struct _StreamFilter {
  struct _StreamFilter* next; /* next StreamFilter.  NULL iff last filter. */
  Port* port;
  PGconn conn;
  int sock;
  int version;
  char* name;
  void* filterOptions;  /* type other than void* ??? */
  /* filter-specific data...ptr to some-struct-or-another */
  void* filterStateData;

  // ...
  PostgresPollingStatusType (*connectPoll)(struct _StreamFilter*);
  struct _StreamFilter* (*create)(void);
  int (*initialize)(struct _StreamFilter*);
  int (*destroy)(struct _StreamFilter*);
  int (*openServer)(struct _StreamFilter* filter, Port* port);
  int (*openClient)(struct _StreamFilter* filter, PGconn* conn);
  ssize_t (*read)(struct _StreamFilter* filter, void* ptr, size_t len);
  ssize_t (*write)(struct _StreamFilter* filter, void* ptr, size_t len);
  int (*close)(struct _StreamFilter* filter);
  const char* (*getName)(struct _StreamFilter* filter);
  char* (*getInfo)(struct _StreamFilter* filter);

} StreamFilter;


Responses

pgsql-hackers by date

Next:From: Michael PaesoldDate: 2005-04-25 17:51:31
Subject: Re: [PATCHES] Continue transactions after errors in psql
Previous:From: Michael PaesoldDate: 2005-04-25 17:30:24
Subject: Re: Continue transactions after errors in psql

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group