Re: WIP/PoC for parallel backup

rebased and updated to current master (d025cf88ba). v12 is attahced.

Also, changed the grammar for LIST_WAL_FILES and SEND_FILE to:

- LIST_WAL_FILES 'startptr' 'endptr'
- SEND_FILE 'FILE' [NOVERIFY_CHECKSUMS]

On Wed, Apr 8, 2020 at 10:48 AM Rajkumar Raghuwanshi <
rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com> wrote:

> Hi Asif,
>
> Thanks for new patches.
>
> Patches need to be rebased on head. Getting a failure while applying the
> 0003 patch.
> edb(at)localhost postgresql]$ git apply
> v11/0003-Parallel-Backup-Backend-Replication-commands.patch
> error: patch failed: src/backend/storage/ipc/ipci.c:147
> error: src/backend/storage/ipc/ipci.c: patch does not apply
>
> I have applied v11 patches on commit -
> 23ba3b5ee278847e4fad913b80950edb2838fd35 to test further.
>
> pg_basebackup has a new option "--no-estimate-size", pg_basebackup
> crashes when using this option.
>
> [edb(at)localhost bin]$ ./pg_basebackup -D /tmp/bkp --no-estimate-size
> --jobs=2
> Segmentation fault (core dumped)
>
> --stacktrace
> [edb(at)localhost bin]$ gdb -q -c core.80438 pg_basebackup
> Loaded symbols for /lib64/libselinux.so.1
> Core was generated by `./pg_basebackup -D /tmp/bkp --no-estimate-size
> --jobs=2'.
> Program terminated with signal 11, Segmentation fault.
> #0 ____strtol_l_internal (nptr=0x0, endptr=0x0, base=10, group=<value
> optimized out>, loc=0x392158ee40) at ../stdlib/strtol_l.c:298
> 298 while (ISSPACE (*s))
> Missing separate debuginfos, use: debuginfo-install
> keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-65.el6.x86_64
> libcom_err-1.41.12-24.el6.x86_64 libselinux-2.0.94-7.el6.x86_64
> openssl-1.0.1e-58.el6_10.x86_64 zlib-1.2.3-29.el6.x86_64
> (gdb) bt
> #0 ____strtol_l_internal (nptr=0x0, endptr=0x0, base=10, group=<value
> optimized out>, loc=0x392158ee40) at ../stdlib/strtol_l.c:298
> #1 0x0000003921233b30 in atoi (nptr=<value optimized out>) at atoi.c:28
> #2 0x000000000040841e in main (argc=5, argv=0x7ffeaa6fb968) at
> pg_basebackup.c:2526
>
> Thanks & Regards,
> Rajkumar Raghuwanshi
>
>
> On Tue, Apr 7, 2020 at 11:07 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
>> On Tue, Apr 7, 2020 at 1:25 PM Asif Rehman <asifr(dot)rehman(at)gmail(dot)com>
>> wrote:
>> > I will, however parallel backup is already quite a large patch. So I
>> think we should first
>> > agree on the current work before adding a backup manifest and
>> progress-reporting support.
>>
>> It's going to be needed for commit, but it may make sense for us to do
>> more review of what you've got here before we worry about it.
>>
>> I'm gonna try to find some time for that as soon as I can.
>>
>> --
>> Robert Haas
>> EnterpriseDB: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>

--
--
Asif Rehman
Highgo Software (Canada/China/Pakistan)
URL : www.highgo.ca