| From: | Mladen Gogala <gogala(dot)mladen(at)gmail(dot)com> |
|---|---|
| To: | pgsql-admin(at)lists(dot)postgresql(dot)org |
| Subject: | Re: Improve "select count(*)" query - takes more than 30 mins for some large tables |
| Date: | 2022-07-12 12:51:42 |
| Message-ID: | 1eb233ff-90f0-529b-f0cc-fa982f668a8f@gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-admin |
On 7/11/22 03:23, Florents Tselai wrote:
> psql “select id from my_table" | sort -u | wc -l
That will be a lot slower than just "select count(*) from my_table".
You are delivering data to the user program (psql) and then shipping
them to pipe and then processing the output with "wc". Depending on the
version, PostgreSQL has very reliable parallelism and can do counting
rather quickly. The speed of "select count(*) from my_table" depends on
the speed of I/O. Since the table is big, it cannot be cached in the
file system cache, so all that you have at your disposal is the raw disk
speed. For the smaller machines, NVME is the king. For larger rigs, you
should consider something like Pure, XTremIO or NetApp SolidFire. People
frequently expect database to do miracles with under par hardware.
--
Mladen Gogala
Database Consultant
Tel: (347) 321-1217
https://dbwhisperer.wordpress.com
| From | Date | Subject | |
|---|---|---|---|
| Next Message | MichaelDBA Vitale | 2022-07-12 14:31:17 | Re: Improve "select count(*)" query - takes more than 30 mins for some large tables |
| Previous Message | Aleksey M Boltenkov | 2022-07-12 06:02:42 | Re: Storing large large JSON objects in JSONB |