Re: number of loaded/unloaded COPY rows

Volkan YAZICI wrote:
> I prepared a patch for "Have COPY return the number of rows
> loaded/unloaded?" TODO. (Sorry for disturbing list with such a simple
> topic, but per warning from Bruce Momjian, I send my proposal to -hackers
> first.)
>
> I used the "appending related information to commandTag" method which is
> used for INSERT/UPDATE/DELETE/FETCH commands too. Furthermore, I edited
> libpq to make PQcmdTuples() interpret affected rows from cmdStatus value
> for COPY command. (Changes don't cause any compatibility problems for API
> and seems like work with triggers too.)
>
> One of the problems related with the used concept is trying to encapsulate
> processed number of rows within an uint32 variable. This causes an internal
> limit for counting COPY when we think it can process billions of rows. I
> couldn't find a solution for this. (Maybe, two uint32 can be used to store
> row count.) But other processed row counters (like INSERT/UPDATE) uses
> uint32 too.
>
> What's your suggestions and comments?

Good question. The libpq interface returns the count as a character
string using PQcmdTuples(), meaning libpq doesn't have any size
limitation. The problem is only the counter you are using, and I think
an int64 is the proper solution. If int64 isn't really 64-bits, the
code should still work though.

In fact we have this TODO, which is related:

* Change LIMIT/OFFSET and FETCH/MOVE to use int8

This requires the same type of change.

I have added this TODO:

* Allow the count returned by SELECT, etc to be to represent as an int64
to allow a higher range of values

This requires a change to es_processed, I think.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073