Earth Notes: On Website Technicals (2025-08)

The HTML5 minifier that I have been using (html-minifier) has not been maintained for a long time.

I would like switch to to the html-minifier-next fork.

I would like to enable the new no-newlines-before-tag-close option.

But the server's JavsScript is too old:

% html-minifier --help
Usage: html-minifier [options] [files...]
...

% html-minifier-next --help
/usr/lib/node_modules/html-minifier-next/cli.js:28
import fs from 'fs';
^^^^^^

SyntaxError: Unexpected token import
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:616:28)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:188:16)
    at bootstrap_node.js:609:3

I sent polite emails a few days ago to two of the pollers in the 'greedy' list, asking them to follow the good practice in the Feed Reader Behavior Project. Both poll far too often, though one manages to do conditional polls and the other allows compression.

One has not reacted at all and the other seems to have stopped polling.

I have lowered the maximum polls per day threshold to 7, which should be plenty if I am sending the right signals and any attention is being paid to even a subset of them. The default Cache-Control max-age is a little over 4h. though Retry-After can be shorter when attempting to steer clients toward the preferred noon-ish poll time.

# Greedy podcast feed pullers: keys are MD5 hashed User-Agent.
# A non-empty txt map lookup of the %{md5:%{HTTP:User-Agent}} means bad!
# Built: 2025-08-29T12:47+00:00Z
# MAXHITSPERUAPERDAY: 7
# MAXUAS: 25
#----------------
# request-count User-Agent
# MD5hash approx-hits-per-day
#----------------
# 257 Podbean/FeedUpdat
54e0e9df937b06cc83fab29f44c02b7f 257
# 203 Spotify/1.0
4582d9bdbcef42af27d89da91c6eb804 203
# 138 Google-Podcast
8dea568b39db0451edd6b30f29238eaf 138
# 98 Gofeed/1.0
4a9d728c458902d6ff716779ff72841d 98
# 63 Amazon Music Podc
d69be2563c9f1929edf2906d41809aea 63
# 63 -
d41d8cd98f00b204e9800998ecf8427e 63
# 60 iTMS
97f76eb7e02c5ff923e1198ff1c288cd 60
# 35 MuckRackFeedParse
62b46fff1cf5f8af7b4b37a2f783b57a 35
# 21 PocketCasts/1.0 (
5caee5a0a53fcbcae25244d8770516ed 21
# 19 itms
2e7f714a929b3f52f3c094710819a99a 19
# 16 Mozilla/5.0 (Wind
6b9a00393fb1607b0ada13520f814ab5 16
# 16 fyyd-poll-1/0.5
0222c1d79b7a96a5bc998a674717f750 16
# 13 axios/1.6.8
b534882134248c9a5957e0c011a37037 13
# 12 Podcasts/1555.2.1
1155c008fa426b4f89c4a1e832eacaee 12
# 12 Overcast/1.0 Podc
c8bf931c39e0b216181afc441001e58b 12
# 12 Mozilla/5.0 (Maci
9bb586dfc3329d2b522978a294f5c138 12
# 11 deezer/curl-3.0
b99188f8b12adffe0f92ae9c03f03c7c 11
# 10 axios/1.11.0
5c8ae194a6f98a725c992886f3da6e04 10
# 9 FeedBurner/1.0 (h
7678d16f662d98189ceb769c5dd002bc 9
# 8 TPA/1.0.0
86e71fcf5ba78d28f18270f7f83256bb 8
# 7 PodchaserParser/2
599ce17a1a1a11800a9905e39fa49f10 7
# 7 iVoox Global Podc
5651428ffa1960ddd9c4e5ff5ec4c270 7

Ignore Sec-CH-UA-Mobile

I was forcing clients with a Sec-CH-UA-Mobile: ?1 HTTP header to receive the 'lite' EOU page versions. That can be confusing since it takes away any choice to see the full page on a capable device without expensive data for example.

I do still force Save-Data: on clients, and those (likely badly-written bots) that will clearly not accept compression, to 'lite' mode, to conserve bandwidth and to keep a little rich data from bad actors.

Sample from the updated Apache EOU configuration:

  # No precompression usable, or client does not support gzip or br compression.
  RewriteCond %{HTTP:Accept-Encoding} !((gzip)|(br)) [OR]
  #RewriteCond %{HTTP:Sec-CH-UA-Mobile} "[?]1" [OR]
  RewriteCond %{HTTP:Save-Data} on [NC]
  RewriteCond %{DOCUMENT_ROOT}/m%{REQUEST_FILENAME} -s
  RewriteRule ^/([^_][^/]+)\.(html)$ /m/$1.$2 [L]

Because of the data sampling period, and maybe other factors, this does not fully represent any change in response to the 429 change.

# Greedy podcast feed pullers: keys are MD5 hashed User-Agent.
# A non-empty txt map lookup of the %{md5:%{HTTP:User-Agent}} means bad!
# Built: 2025-08-25T10:05+00:00Z
# MAXHITSPERUAPERDAY: 12
# MAXUAS: 25
#----------------
# request-count User-Agent
# MD5hash approx-hits-per-day
#----------------
# 256 Podbean/FeedUpdat
54e0e9df937b06cc83fab29f44c02b7f 256
# 202 Spotify/1.0
4582d9bdbcef42af27d89da91c6eb804 202
# 137 Google-Podcast
8dea568b39db0451edd6b30f29238eaf 137
# 97 Gofeed/1.0
4a9d728c458902d6ff716779ff72841d 97
# 62 Amazon Music Podc
d69be2563c9f1929edf2906d41809aea 62
# 59 iTMS
97f76eb7e02c5ff923e1198ff1c288cd 59
# 59 -
d41d8cd98f00b204e9800998ecf8427e 59
# 35 MuckRackFeedParse
62b46fff1cf5f8af7b4b37a2f783b57a 35
# 22 PocketCasts/1.0 (
5caee5a0a53fcbcae25244d8770516ed 22
# 20 fyyd-poll-1/0.5
0222c1d79b7a96a5bc998a674717f750 20
# 19 itms
2e7f714a929b3f52f3c094710819a99a 19
# 15 Mozilla/5.0 (Wind
6b9a00393fb1607b0ada13520f814ab5 15
# 13 axios/1.6.8
b534882134248c9a5957e0c011a37037 13
# 12 Podcasts/1555.2.1
1155c008fa426b4f89c4a1e832eacaee 12