Earth Notes: On Website Technicals (2025-10)

To do... When inlining small SVG images in HTML with data URLs, URL encode rather than base64, since the former will typically compress much better. Preferably have the %xx hex digits be lower case, since the majority of the surrounding text will be.

No, using jq to URL encode was not on my bingo card either... Note that jq outputs upper case hex digits, so is not ideal.

% cat img/LIVE-20w.svg | wc -c
     208
% cat img/LIVE-20w.svg | gzip -9 | wc -c
     174
% cat img/LIVE-20w.svg | brotli | wc -c
     142
% cat img/LIVE-20w.svg | base64 | wc -c
     281
% cat img/LIVE-20w.svg | jq -sRr @uri | wc -c
     349
% cat img/LIVE-20w.svg | base64 | gzip -9 | wc -c
     247
% cat img/LIVE-20w.svg | jq -sRr @uri | gzip -9 | wc -c
     210
% cat img/LIVE-20w.svg | base64 | brotli | wc -c
     235
% cat img/LIVE-20w.svg | jq -sRr @uri | brotli | wc -c
     182

This became obvious when looking at the EOU homepage compressed in the flateview ZIP visualiser.

This also revealed that I have been using an uppercase letter in the ⭐ Unicode expansion of the 'star' marker that I inject for popular pages. Now adjusted to ⭐ for better compression.

Also I now only put a "Count" after "References" when > 1, removing that paragraph from ~40 desktop pages.

2025-10-22: hand-rolled URL encoder

I have now written a script script/urlencode.sh to URL encode stdin to stdout with lowercase escape hex.

% cat img/LIVE-20w.svg | sh script/urlencode.sh | wc -c
     348
% cat img/LIVE-20w.svg | sh script/urlencode.sh | gzip -9 | wc -c
     202
% cat img/LIVE-20w.svg | sh script/urlencode.sh | brotli | wc -c
     176

This avoids adding a trailing newline. A few bytes are saved in compression stand-alone with this encoding style: 8 with gzip -9 and 6 with brotli.

I then let svgo have a go at the SVG source, and it managed to strip out 3 bytes (though may have changed behaviour), with the result still working in my browser!

% ls -alS img/LIVE-20w.*
205 img/LIVE-20w.svg
169 img/LIVE-20w.svggz
136 img/LIVE-20w.svgbr
% cat img/LIVE-20w.svg | sh script/urlencode.sh | brotli | wc -c
170

But maybe I need URL-encode only the # character?

2025-10-23: first pass SVG inline URL encode

A first pass change has been made to inline SVG URL-encoded. Strict URL encoding is being used at this point. Much of the purple has already gone for the flateview of the inlined image.

That this micro-optimisation currently only affects about six (desktop) pages, all for the same 'LIVE' SVG beacon image!

A second pass script script/urlencode-dataURL.sh allows :/@ characters to remain unescaped, and was tested against Lynx, Firefox, Chrome, Safari, and the W3C HTML validator.

% cat img/LIVE-20w.svg | sh script/urlencode.sh | wc -c
     343
% cat img/LIVE-20w.svg | sh script/urlencode-dataURL.sh .sh | wc -c
     319
% cat img/LIVE-20w.svg | sh script/urlencode.sh | brotli | wc -c
     170
% cat img/LIVE-20w.svg | sh script/urlencode-dataURL.sh .sh | brotli | wc -c
     166

This relaxed encoding saves 24 bytes (~7%) uncompressed, and 4 bytes (~2%) after brotli compression.

A few more bytes are saved by dropping the redundant class=raw attribute from such images, since I do not style raw on EOU. That affects ~7 (desktop) pages.

I fixed svgo's breakage of my animation, adding back a few bytes! I also expanded the set of unencoded characters to /:@,.

% cat img/LIVE-20w.svg | wc -c
     208
% cat img/LIVE-20w.svg | sh script/urlencode-dataURL.sh | wc -c
     322
% cat img/LIVE-20w.svg | sh script/urlencode-dataURL.sh | brotli | wc -c
     169

The current data URI for the img src attribute is:

data:image/svg+xml,%3csvg%20xmlns%3d%22http://www.w3.org/2000/svg%22%20width%3d%2220%22%20height%3d%2220%22%3e%3cstyle%3e@keyframes%20c%7b0%25,to%7bfill:red%7d50%25%7bfill:%23fcc%7d%7d%3c/style%3e%3ccircle%20cx%3d%2210%22%20cy%3d%2210%22%20r%3d%2210%22%20style%3d%22animation:c%2010s%20linear%200s%2020%22%20fill%3d%22%23fcc%22/%3e%3c/svg%3e

To save a few more bytes for inlined images, since height and width are omitted for them, decoding=async is no longer inserted (nor loading=lazy) to try to provide the browser access to the image dimensions ASAP and thus reduce the risk of content movement.

To find any cached inlined headers with decoding=async, to remove them and force rebuilds:

% find img/a/h -type f -exec egrep -l 'data:image' {} +

AI code assist

I asked for AI (Google's search AI Mode) help with the encoder script, because it is hard to do with sed and awk which are my usual tools, because of gaps in their functionality, at least portably including the Mac. The AI solution hallucinated some gawk functionality, for the critical hard-to-do part. I have now retained a part that the AI got tidily right, and solved the hard bit with normal Olde Worlde search results.

I have taken a few bytes out of the ORCID byline link on 'research' articles. I shortened the image path (with an alias image, ORCID.png). I also shortened the alt text to just "ORCID", thus no longer requiring quotes either. I changed the link URL to "ORCID.org" to help compression, with all mentions now being in the same case.

# Greedy podcast feed pullers: keys are MD5 hashed User-Agent.
# A non-empty txt map lookup of the %{md5:%{HTTP:User-Agent}} means bad!
# Built: 2025-10-17T11:36+00:00Z
# MAXHITSPERUAPERDAY: 7
# MAXUAS: 25
#----------------
# request-count User-Agent
# MD5hash approx-hits-per-day
#----------------
# 475 iTMS
97f76eb7e02c5ff923e1198ff1c288cd 475
# 304 Podbean/FeedUpdat
54e0e9df937b06cc83fab29f44c02b7f 304
# 203 Spotify/1.0
4582d9bdbcef42af27d89da91c6eb804 203
# 138 Google-Podcast
8dea568b39db0451edd6b30f29238eaf 138
# 98 Gofeed/1.0
4a9d728c458902d6ff716779ff72841d 98
# 73 -
d41d8cd98f00b204e9800998ecf8427e 73
# 34 axios/1.11.0
5c8ae194a6f98a725c992886f3da6e04 34
# 32 MuckRackFeedParse
62b46fff1cf5f8af7b4b37a2f783b57a 32
# 26 Amazon Music Podc
d69be2563c9f1929edf2906d41809aea 26
# 21 itms
2e7f714a929b3f52f3c094710819a99a 21
# 15 deezer/curl-3.0
b99188f8b12adffe0f92ae9c03f03c7c 15
# 13 PocketCasts/1.0 (
5caee5a0a53fcbcae25244d8770516ed 13
# 12 axios/1.6.8
b534882134248c9a5957e0c011a37037 12
# 11 TPA/1.0.0
86e71fcf5ba78d28f18270f7f83256bb 11
# 11 Mozilla/5.0 (Wind
6b9a00393fb1607b0ada13520f814ab5 11
# 11 FeedMaster/4.1
dca30fcc6f45af71c2825280bebba04d 11
# 10 Mozilla/5.0 (Maci
209eaa299125147cabf09b35d947b794 10
# 8 Overcast/1.0 Podc
c8bf931c39e0b216181afc441001e58b 8
# 8 AntennaPod/3.9.0
3975d4c899c67ddd6e9fa312444f71a8 8
# 7 PodchaserParser/2
599ce17a1a1a11800a9905e39fa49f10 7

I will put the threshold up to ~10, to avoid trapping relatively well-behaved clients, such as AntennaPod, with multiple distinct users.

My small static dashboard HTML page, which uses meta http-equiv=refresh content=XXX to auto-update, is small, but not fully minified, because it is maintained by hand.

% ls -alS _dashboard.html*
3295 Oct 26  2024 _dashboard.html
1175 Oct 26  2024 _dashboard.htmlgz
 889 Oct 26  2024 _dashboard.htmlbr

It occurs to me that that I could at least run it through the 'gentle' minimisation level to scrape a few more bytes. (Though this is a really tiny amount of network traffic anyway!) This changes a couple of lines of the source. (I also updated the copyright to 2025.) I then have to force regeneration of the pre-compressed versions.

3254 Oct 14 11:17 _dashboard.html
1155 Oct 14 11:17 _dashboard.htmlgz
 890 Oct 14 11:17 _dashboard.htmlbr

Oh look, the br is actually 1 byte larger!

So I tweaked some metadata, including removing some Twitter-only stuff:

3117 Oct 14 11:27 _dashboard.html
1122 Oct 14 11:27 _dashboard.htmlgz
 867 Oct 14 11:27 _dashboard.htmlbr

I finally ran the updated page through the W3C Markup Validation Service to make sure that I had not broken anything!

I have added automatic linking from ISSNs in bibliography entries via The ISSN Portal.

Something similar could be done with ISBNs.

Ads have generally not been allowed on the m-dot / mobile / lite site since those users' bandwidth may be most precious.

However, I want to probe if there is significant potential revenue there, for I have added a 'soft' configuration parameter to allow me to turn this stuff on and off more easily.

As ever, this took a couple of rounds to get right. Having created a soft parameter to allow ad injection on 'lite' pages, I then also had to enable computation of page popularity rank for 'lite' pages since unpopular pages generally do not get to show ads anyway!

Some (podcast) RSS users want all the text of all the episodes in the RSS feed. I do not want to do that since:

• Detailed HTML styling and other control (eg for efficiency) does not seem possible.
• It is a terrible redundant waste of bandwidth for a full archive feed, with every reader having to pay over and over in full for items of no interest or already read.

However, what I have now done, where safe (no lurking HTML tags nor entities), I append the pgintro paragraph to the RSS description for new-ish episodes (up to ~90 days old).

Feed bloat is capped, but readers get to see a little more to reel them in...

Google keeps claiming that I do not publish the ads.txt for my hd.org zone/domain, though it never moved or went away. So today rather than yet again manually convince Google to actually look at the URL unchanged for years, I removed all ads for the zone and dropped authorisation in AdSense. Google's (tiny) loss I think, if it cannot get some basics right.

I gave Sitebulb 8.10 a whirl and found/fixed a few minor errors such as subtle broken internal links.

But Sitebulb claimed that the majority of my HTML pages are so badly broken that browsers may not be even able to display them. Those pages pass validation with the W3C validator and others, and work fine in Firefox, Chrome, Safari and Lynx... (I suspect I know exactly why, because of a valid minification that I use.)

So that suggests to me that the tool will not be much use to me for now.