Re: Bugs in new announcement system

On Mon, Nov 9, 2020 at 4:16 AM David Fetter <david(at)fetter(dot)org> wrote:
>
> On Sun, Nov 08, 2020 at 06:25:17PM +0100, Magnus Hagander wrote:
> > On Mon, Nov 2, 2020 at 1:10 AM David Fetter <david(at)fetter(dot)org> wrote:
> > Yeah, that seems a lot more useful.
>
> > > Please fix this either by making something that highlights the
> > > offending section(s) so people have some idea what to fix, or renders
> > > them harmless automatically, whichever seems easier. I went to the
> >
> > Do you have any suggestions for how to actually accomplish such highlighting?
>
> I'd imagine that the thing that can tell there's HTML in there can
> also tell where it is and hand back a line number at a minimum.

Oh, that's the easy part -- even getting a regexp to do that is pretty
easy. But how do you get that feedback into a standard HTML input
box, what amount of black magic is needed there?

> > There are also some further issues around the preview code for that,
> > since it uses a different markdown engine, but that one already has
> > some issues so we should probably try to figure that part out at the
> > same time.
> >
> >
> > > trouble of tracking this down because I have a lot of readers each
> > > week who expect me to get it there, but I doubt anyone else who ran
> > > into this bothered.
> >
> > Well, nobody else has reported any problems, but my guess is nobody
> > else has tried pasting HTML before :)
>
> I did not try pasting HTML in there. There was no HTML anywhere in the
> newsletter before. What there was was a false positive that I had the
> pleasure of tracking down.

Oh, gotcha. Would you care to actually share *what* the problematic
match was? If nothing else, that would be good to test against with a
new implementation.

> What is it precisely that you don't want in HTML? I'm asking because
> if you can come up with a list of things you want blocked, a gizmo
> that removes same from that AST (er, DOM) seems like the thing that
> would actually work and not burden people.

We don't want anything in HTML in general, other than what's generated
out of the markdown. So it's really a question of what we *want*,
which is just the basic formatting tags + links.

Looking some more at the bleach thing it does seem to work with this
kind of whitelist model, so that is indeed probably a good way
forward. It will require some bigger hackings around the pgweb code
though, but that will likely pay off.

> You're inferring that no complaints means no one had problems other
> than me. I think a much more likely explanation is survivorship bias,
> i.e. lots of people noticed it was buggy and unhelpful, and silently
> gave up.

This is certainly possible. But given the number of other people who
have contacted us with questions around *different* things in that
system after the change, I'm willing to guess that the number are
fairly low. And we've generally seen about the same number of posts /
week as we had before, so there has certainly not been a big drop.
Whereas the actual delivery rate has gone up *massively*.

--
Magnus Hagander
Me: https://www.hagander.net/
Work: https://www.redpill-linpro.com/