Re: Synchronous replication, reading WAL for sending

Fujii Masao wrote:
>> - WALSender reads from WAL buffers and/or WAL files and sends the
>> buffers to WALReceiver. In phase one, we may assume that WALSender can
>> only read from WAL buffers and WAL files in pg_xlog directory. Later
>> on, this can be improved so that WALSender can temporarily restore
>> archived files and read from that too.
>>
> You mean that only walsender performs xlog streaming and copying
> from pg_xlog serially? I think that this would degrade the performance.
> And, I'm worried about the situation that the speed to generate xlog
> on the primary is higher than that to copy them to the standby. We
> might not be able to start xlog streaming forever.
>

I've seen a few references to this. Somebody else mentioned how a single
TCP/IP stream might not have the bandwidth to match changes to the database.

TCP/IP streams do have a window size that adjusts with the load, and
unless one gets into aggressive networking such as bittorrent which
arguably reduce performance of the entire network, why shouldn't one
TCP/IP stream be enough? And if one TCP/IP stream isn't enough, doesn't
this point to much larger problems, that won't be solved by streaming it
some other way over the network? As in, it doesn't matter what you do -
your network pipe isn't big enough?

Over the Internet from my house to a co-located box, I can reliably get
1.1+ Mbyte/s using a single TCP/IP connection. The network connection
at the co-lo is 10Mbit/s and my Internet connection to my house is also
10Mbit/s. One TCP/IP connection seems pretty capable to stream data to
the full potential of the network...

Also, I assume that most database loads have peaks and lows. Especially
for very larger updates, perhaps end of day processing, I see it as a
guarantee that all of the stand bys will fall "more behind" for a period
(a few seconds to a minute?), but they will catch up shortly after the
peak is over.

Cheers,
mark

--
Mark Mielke <mark(at)mielke(dot)cc>