| From: | Josh Andrews <josh(dot)andrews(at)dorianinsights(dot)com> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | Measuring Fix Pace and Disclosed AI Involvement Across PG 12-18 |
| Date: | 2026-09-28 19:56:15 |
| Message-ID: | CACzdQB4Ojc0dC7-USnr5Ga8BA8WjZRswz=8cKAqCUPmrdzutWQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hello All -
I've been trying to measure, rather than guess, how much LLMs are changing
software development in general, and, in particular, what impact they are
having on discovering bugs and vulnerabilities. I use Postgres and saw a
significant volume increase in the August minor patch release, so I decided
to use the Postgres project as a candidate for this evaluation. I analyzed
only public data: the git repo (commits, tags and the release-notes SGML),
the pgsql-bugs and pgsql-hackers mbox archives, and the CVE list on
postgresql.org.
I published an article with my findings here:
https://dorianinsights.com/insights/llms_and_postgres_bug_fixing/. More
details on the code and data supporting the analysis follow.
What the data shows:
- The August 2026 minor releases carry 142 release-notes items, versus
63-78 for every scheduled release in 2025 and early 2026.
- CVEs went from 2-3 per release (0-6% of items) to 14 in May and 29 in
August 2026 (19% and 20% of items).
- 26 stable-branch fix commits disclose AI involvement in the commit
message, all since April 2026; 22 of them credit an AI tool with finding
the bug (OpenAI Codex Security, Claude via Calif.io, Xint Code).
- New threads on -hackers and -bugs that disclose AI involvement in the
first message went from under 1% before June 2026 to about 8% in August (29
threads) and September so far.
- Total list traffic and total bug-report volume show no significant
changes from historical volume.
- Lines churned per quarter on stable branches (one representative commit
per backpatched fix, so the sum deduplicates accounting for the same
patches pushed back to multiple supported branches) is at 212% of the
quarterly mean in Q3 2026, with the quarter still open.
The code and data are open source (MIT / CC BY 4.0):
https://github.com/dorianinsights/postgres_bug_analysis. There are many
more findings and charts; I am sharing only a sample of the most
interesting ones here.
The figures above are as of 2026-09-18 and cover version 12.1 onward.
I'd love to hear feedback and thoughts from others. Issues and pull
requests on the repo are welcome too. I have a lot of respect for the
Postgres project and those who work hard to maintain it, so thanks to
everyone for all of their contributions.
Regards,
Josh Andrews
Josh Andrews · Founder & Principal Consultant
DORIAN INSIGHTS, L.L.C. · Data & Analytics · Business
Strategy · Technology Architecture
josh(dot)andrews(at)dorianinsights(dot)com · (310) 948-4641 · dorianinsights.com
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Mats Kindahl | 2026-09-28 19:57:05 | Re: pg_rewind does not rewind diverging timelines |
| Previous Message | Masahiko Sawada | 2026-09-28 19:48:59 | Re: Fix "unexpected logical decoding status change" error; from concurrent logical decoding activation |