Re: Conflict handling for COPY FROM

From: Karen Huddleston <khuddleston(at)pivotal(dot)io>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Surafel Temesgen <surafel3000(at)gmail(dot)com>
Subject: Re: Conflict handling for COPY FROM
Date: 2018-08-18 05:17:20
Message-ID: 153456944015.1410.12495599902221553748.pgcf@coridan.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Surafel,

Andrew and I began reviewing your patch. It applied cleanly and seems to mostly have the functionality you describe. We did have some comments/questions.

1. It sounded like you added the copy_max_error_limit GUC as part of this patch to allow users to specify how many errors they want to swallow with this new functionality. The GUC didn't seem to be defined and we saw no mention of it in the code. My guess is this might be good to address one of the concerns mentioned in the initial email thread of this generating too many transaction IDs so it would probably be good to have it.
2. I was curious why you only have support for skipping errors on UNIQUE and EXCLUSION constraints and not other kinds of constraints? I'm not sure how difficult it would be to add support for those, but it seems they could also be useful.
3. We think the wording "ON CONFLICT IGNORE" may not be the clearest description of what this is doing since it is writing the failed rows to a file for a user to process later, but they are not being ignored. We considered things like STASH or LOG as alternatives to IGNORE. Andrew may have some other suggestions for wording.
4. We also noticed this has no tests and thought it would be good to add some to ensure this functionality works how you intend it and continues to work. We started running some SQL to validate this, but haven't gotten the chance to put it into a clean test yet. We can send you what we have so far, or we are also willing to put a little time in to turn it into tests ourselves that we could contribute to this patch.

Thanks for writing this patch!
Karen

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2018-08-18 08:08:18 Re: docs: note ownership requirement for refreshing materialized views
Previous Message Michael Paquier 2018-08-18 02:07:08 Re: Fix help option of contrib/oid2name