| From: | Magnus Hagander <magnus(at)hagander(dot)net> |
|---|---|
| To: | Christoph Berg <myon(at)debian(dot)org> |
| Cc: | Dave Page <dpage(at)pgadmin(dot)org>, Daniel Gustafsson <daniel(at)yesql(dot)se>, "Jonathan S(dot) Katz" <jkatz(at)postgresql(dot)org>, pgsql-www(at)lists(dot)postgresql(dot)org, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, PostgreSQL Contributors <contributors(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Introduce contributor badges - v14 |
| Date: | 2026-11-04 19:38:05 |
| Message-ID: | CABUevExExU=XWHgLBujLQXrAmbtqYo9h4AqbQ6_cZmDU5=hQpw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-www |
On Fri, 16 Oct 2026 at 19:21, Christoph Berg <myon(at)debian(dot)org> wrote:
> The v14 patch series addresses Dave's review, Daniels 150x150 comment
> and includes Magnus feedback:
>
I've lost track of much of this, but none of these fixes is actually
applied yet, right?
And they're just adressed in a random order here rather than from 1 and
onwards? :)
I'll start with this patch series as it's already been done, and then
review the underlying stuff later.
Re: Dave Page
> > 1. **Removing a holder purges nothing for that person.** `BadgeForm.save`
> > calls `badge.holders.remove()` (`pgweb/contributors/forms.py`). That
> > deletes the through-table rows without any `post_save`, and pgweb only
> > hooks `post_save` (`pgweb/util/signals.py:169`). The badge page still
> gets
> > purged, because `simple_form` saves the badge afterwards. The removed
> > person's `/community/people/<username>/` page keeps showing the badge.
>
> This is v14-0007-Purge-badge-pages-and-holder-profiles-through-xk.patch
>
I wonder if we shouldn't add a `badgelist_<n>` xkey and use that to purge
the pages when a contributor is edited, rather than do it url based. Url
based, particularly with looping, can generate an overhead (since
technically they are not purges even though they are called that -- they to
regexp matching agianst all incoming requests for banning).
Comments are about how the xkey is duplicated between the responses, but
not the etag? (but the comment seems to go into a level of detail we're
unusued to anyway, so we can maybe just drop that part)
> 2. **Deletes never purge.** This applies to deleting a `Badge`,
> > `Badgeholder` or `Contributor` in the admin. The rest of pgweb has the
> same
> > gap, but it matters more here: a deleted contributor's public profile
> page,
> > which can include their email, stays cached. If someone asks for their
> > profile to be removed, it isn't really gone.
>
> v14-0008-Purge-on-delete-as-well-as-on-save.patch
>
This commit message looks so painfully LLM generated that I can't read
through it all. But it also looks like a long explanation of a discussion
rather than the actual patch.
And it seems to introduce a new type of logging that's not used elsewhere,
just trapping and logging *all* exceptions as warnings completely
regardless of what they are. Doesn't seem like something that belongs in
such a patch.
And since it affects everything, not just the badges, it should probably
not just be patch 8 in a long series, it deserves looking at separtely.
> ## Database and app server
> >
> > 5. **The `has_image` annotation reads every image out of TOAST**,
> > PostgreSQL's out-of-line storage for large values
> > (`BadgeManager.with_has_image`). The comment says the value is "never
> > transferred", which is true for the network, but `imagedata > ''`
> compares
> > the full bytea, so PostgreSQL has to fetch and reassemble the whole
> value.
> > A test of 200 rows of about 96 KB each on the local database showed the
> > comparison reading about 3,000 buffers in 31 ms, whilst
> `octet_length(img)
> > > 0` read 2 buffers in 0.04 ms, because it only reads the TOAST header.
> > Switching to `Length('imagedata') > 0` is a one-line fix. It runs on
> every
> > cache miss of the people page (up to 200 badges) and on every account
> > profile page load, which is never cached.
>
> v14-0002-Determine-has-image-without-full-detoasting.patch
>
>
Looks fine, but is it really needed? Do we even (want to) support badges
without images? Whta's the point of a badge that has no image?
> > 6. **Oversized images are decoded in full (a "decompression bomb").**
> > `ImageBinaryField.to_python` decodes the entire image with Pillow before
> > rescaling. A PNG under 1 MB can still be about 170 megapixels before
> Pillow
> > refuses it, which is several hundred MB of RAM plus a bicubic resize on
> the
> > app server. `ModelForm` full-clean also re-parses the stored image on
> every
> > badge edit, even when the image hasn't changed. Only managers of an
> > approved organisation can upload, so the risk is moderate, but checking
> > `img.size` before decoding would be cheap.
>
> v14-0003-Reject-images-with-wrong-size-before-loading-the.patch
>
Why are we doing this whole exif orientation thing in the first place?
Surely we can expect people to upload badges that are not upside down?
> v14-0004-Don-t-re-parse-the-stored-badge-image-on-every-e.patch
See above comments about commit messages.
> > 7. **Minor, and only on cache misses:**
> > - `badge.html` uses `holders|length` and then `holders.all`, which
> runs
> > the holder query and its two prefetches a second time.
>
> v14-0005-Don-t-run-the-badge-holder-query-twice.patch
>
Looks fine, but the commit message is... Yeha, see above.
> - `select_related` would do better than `prefetch_related` for the
> > foreign-key lookups in `peoplelist` and `badge_view`.
>
> v14-0006-Use-select_related-for-the-badge-holder-and-peop.patch
>
Dito.
> - `badge_image` fetches the image and computes its MD5 before it checks
> > `If-None-Match`; storing the hash would avoid both.
>
> v14-0009-Store-the-badge-image-md5-in-the-database.patch
>
> Since 0009 is touching the database, we can now pull up the staged
> extended badge title length:
> v14-0010-Extend-badge-title-to-255-chars.patch
>
I agree with Daniel we should do this extension.
Both together share a migration file that is part of
> v14-0011-Migrate-imagehash-and-serve-badge-images-from-it.patch
>
> If we decide we don't want the MD5 change, we can postpone these
> three.
>
This should probably be just one patch, so the model changes go along with
the migration.
0011 looks good.
> - Every account profile save that has a contributor record purges
> > `/community/contributors/`, `/community/people/` and all of that user's
> > badge pages, even when nothing changed.
>
> v14-0012-Don-t-re-purge-contributor-pages-on-a-no-op-acco.patch
>
Looks good.
Re: Daniel Gustafsson
> > +1 on restricting the upload to 150x150 and make the user responsible
> for the
> > badge image looking good rather than rescaling.
>
> That's v14-0001-Require-user-supplied-badges-to-be-150x150.patch
>
We could perhaps just remove the autoscaling code now that we've turned it
off?
Why do we want to support both PNG and JPG? Seems a badge should be only
PNG?
//Magnus
| From | Date | Subject | |
|---|---|---|---|
| Next Message | A Henry | 2026-11-05 07:47:03 | Wiki request: add HiDataLinker to PostgreSQL_Clients (Open Source) |
| Previous Message | David Wheeler | 2026-11-03 20:19:21 | PGDay Boston CFfP Link? |