| From: | "Tristan Partin" <tristan(at)partin(dot)io> |
|---|---|
| To: | "Andres Freund" <andres(at)anarazel(dot)de>, "Nazir Bilal Yavuz" <byavuz81(at)gmail(dot)com>, "Jacob Champion" <jacob(dot)champion(at)enterprisedb(dot)com>, "Thomas Munro" <thomas(dot)munro(at)gmail(dot)com> |
| Cc: | "pgsql-hackers" <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: ci: CCache churns through available space too quickly |
| Date: | 2026-06-17 22:02:09 |
| Message-ID: | DJBNW7DKLMV1.2WHERLY9NSEJX@partin.io |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri Jun 5, 2026 at 8:09 PM UTC, Andres Freund wrote:
> Hi,
>
> I noticed that a handfull of CI runs already lead to exceeding the available
> cache space. One can pay for more cache space, but I think the problem is
> more that what we currently do doesn't work well.
>
> With cirrus-ci all branches shared one cache, but that's not the case with
> github actions. Except for being able to read caches from the default branch
> (master in our case), other branches have completely separate cache
> namespaces. That's probably the right call, safety wise, but makes our ccache
> approach .. not great.
>
> We should only upload a new cache when the ccache cache hit ratio of the
> existing cache entry has gotten low.
I had started reviewing this patch the day it was originally sent, but
due to circumstances I couldn't finish the review before it was
committed. I had some thoughts with regard to improving the Python
script itself. Attached are some improvements that make the code
a little more pythonic as well as more easily usable locally for testing
purposes. Some of the patches may be more valuable than others.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Use-long-options-for-ccache-commands.patch | text/x-patch | 1.6 KB |
| v1-0002-Use-json-to-parse-ccache-statistics.patch | text/x-patch | 1.9 KB |
| v1-0003-Remove-the-shutil.which-call.patch | text/x-patch | 1.4 KB |
| v1-0004-Use-a-context-manager-to-write-grouped-CI-output.patch | text/x-patch | 2.9 KB |
| v1-0005-Provide-ccache-target-rate-as-an-input-to-the-scr.patch | text/x-patch | 4.1 KB |
| v1-0006-Check-that-GITHUB_OUTPUT-exists-before-trying-to-.patch | text/x-patch | 1.4 KB |
| v1-0007-Make-gha_ccache_decide.py-executable.patch | text/x-patch | 675 bytes |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jacob Champion | 2026-06-17 22:04:34 | Re: Require SSL connection to postgres for oauth |
| Previous Message | Noah Misch | 2026-06-17 21:59:52 | Re: Non-text mode for pg_dumpall |