RE: Complete data erasure

From: "asaba(dot)takanori(at)fujitsu(dot)com" <asaba(dot)takanori(at)fujitsu(dot)com>
To: 'Tom Lane' <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, 'Masahiko Sawada' <masahiko(dot)sawada(at)2ndquadrant(dot)com>, "imai(dot)yoshikazu(at)fujitsu(dot)com" <imai(dot)yoshikazu(at)fujitsu(dot)com>, "'pgsql-hackers(at)postgresql(dot)org'" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Complete data erasure
Date: 2020-04-10 08:23:32
Message-ID: OSBPR01MB4728FC5244A9823507AB572A8CDE0@OSBPR01MB4728.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

I was off the point.
I want to organize the discussion and suggest feature design.

There are two opinions.
1. COMMIT should not take a long time because errors are more likely to occur.
2. The data area should be released when COMMIT is completed because COMMIT has to be an atomic action.

These opinions are correct.
But it is difficult to satisfy them at the same time.
So I suggest that users have the option to choose.
DROP TABLE works as following two patterns:

1. Rename data file to "...del" instead of ftruncate(fd,0).
After that, bgworker scan the directory and run erase_command.
(erase_command is command set by user like archive_command.
For example, shred on Linux.)

2. Run erase_command for data file immediately before ftruncate(fd,0).
Wait until it completes, then reply COMMIT to the client.
After that, it is the same as normal processing.

If error of erase_command occurs, it issues WARNING and don't request unlink to CheckPointer.
It’s not a security failure because I think that there is a risk when data area is returned to OS.

I will implement from pattern 2 because it's more similar to user experience than pattern 1.
This method has been pointed out as follows.

From Stephen
> We certainly can't run external commands during transaction COMMIT, so
> this can't be part of a regular DROP TABLE.

I think it means that error of external commands can't be handled.
If so, it's no problem because I determined behavior after error.
Are there any other problems?

Regards,

--
Takanori Asaba

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeremy Morton 2020-04-10 08:34:51 Support for DATETIMEOFFSET
Previous Message Michael Paquier 2020-04-10 08:09:10 pg_basebackup, manifests and backends older than ~12