Earth Notes: On Website Technicals (2025-06)

Riku Mattila, also @rmattila74@energydon.fi, is a Nuke engineer retired due to ALS. Defined by the former, not by the latter.

Under-voltage

Sampling the under-voltages that I have captured, yields:

% egrep "voltage low" data/eddi/log/live/*.stop.log | awk '{print $NF}' | sort | uniq -c
      1 2312
      1 2313
      1 2314
      1 2316
      1 2317
      1 2318
      1 2319
      4 2322
      1 2323
      3 2324
      2 2325
      2 2326
      1 2327
      3 2328
      2 2329
      2 2332
      3 2333
      1 2335
      1 2336
      5 2337
      2 2338
      3 2339
      3 2340
      5 2341
      4 2342
      5 2343
      3 2344
      4 2345
      7 2346
      4 2347
      4 2348
      9 2349
      7 2350
      6 2351
     11 2352
     19 2353
     14 2354
     12 2355
     20 2356
     15 2357
      9 2358
     21 2359

There was a huge block yesterday, to the extend that I thought there may have been some deliberate demand reduction taking place:

% egrep -c "voltage low" data/eddi/log/live/*.stop.log
data/eddi/log/live/20250530.stop.log:0
data/eddi/log/live/20250531.stop.log:2
data/eddi/log/live/20250601.stop.log:0
data/eddi/log/live/20250602.stop.log:1
data/eddi/log/live/20250603.stop.log:7
data/eddi/log/live/20250604.stop.log:0
data/eddi/log/live/20250605.stop.log:1
data/eddi/log/live/20250606.stop.log:1
data/eddi/log/live/20250607.stop.log:0
data/eddi/log/live/20250608.stop.log:1
data/eddi/log/live/20250609.stop.log:2
data/eddi/log/live/20250610.stop.log:0
data/eddi/log/live/20250611.stop.log:0
data/eddi/log/live/20250612.stop.log:9
data/eddi/log/live/20250613.stop.log:0
data/eddi/log/live/20250614.stop.log:3
data/eddi/log/live/20250615.stop.log:2
data/eddi/log/live/20250616.stop.log:0
data/eddi/log/live/20250617.stop.log:1
data/eddi/log/live/20250618.stop.log:0
data/eddi/log/live/20250619.stop.log:34
data/eddi/log/live/20250620.stop.log:16
data/eddi/log/live/20250621.stop.log:141
data/eddi/log/live/20250622.stop.log:45

(The final value was updated the following day...)

A more focussed look at yesterday:

% egrep "voltage low" data/eddi/log/live/20250621.stop.log | awk '{print $NF}' | sort | uniq -c
      1 2325
      1 2328
      1 2329
      1 2332
      3 2337
      1 2338
      2 2339
      2 2340
      5 2341
      4 2342
      4 2343
      3 2344
      4 2345
      6 2346
      4 2347
      3 2348
      6 2349
      6 2350
      4 2351
     10 2352
     16 2353
     11 2354
      5 2355
     14 2356
      7 2357
      6 2358
     11 2359

Also evident from the logs are runs of many consecutive minutes of low voltage response, which means that is has a chance to be helping rather than just adding noise.

Oh, alright, just one more then...

Wilde Life is a supernatural adventure/horror series set in a small town in rural Oklahoma.

Yet more? Here is one that is getting a bit intermittent but just updated a few days ago: Madam & Eve! A slice-of-life from South Africa, where I have relatives, and it has kept me in touch with some of the politics there over decades!

Madam & Eve takes a humorous look at the daily lives of two people from very different backgrounds as they experience life in postapartheid South Africa. The strip's main characters have become icons of a changing South Africa, providing light relief through the days of transition to democracy.

Still more? OK, here is Mr Willis' DoA: a webcomic about college freshmen in the girls wing within a co-ed dorm at Indiana University, learning everything about life and themselves usually in the most difficult ways.

MOAR! you roar again. OK, so here is Poorly Drawn Lines by Reza Farazmand, absurdist and satirical and always slightly weird!

MOAR COMIX! you roar. A fine upstanding (and longstanding) member of the community is XKCD by Randall Munroe: A webcomic of romance, sarcasm, math, and language.

xkcd.com is best viewed with Netscape Navigator 4.0 or below on a Pentium 3±1 emulated in Javascript on an Apple IIGS at a screen resolution of 1024x1. Please enable your ad blockers, disable high-heat drying, and remove your device from Airplane Mode and set it to Boat Mode. For security reasons, please leave caps lock on while browsing.

O Westin is a prolific creator of very, very short science fiction and fantasy stories on various social media platforms including Twitter and then the Fediverse. Flavour for my feeds!

Am I just copying Mr Rigby today, or does Neil Brown simply deserve to win the Internets (for fifteen minutes off-peak, batteries not included)?

From his legal business' diversity section: We have no religion or belief, but Neil prefers vim over emacs.

Another of my webcomic favourites is Questionable Content which I have been reading since before it was famous... This embodies a positive vision of AI that I can get behind.

Diversion delay until after noon

I successfully implemented a scheme to delay remaining diversion to the heat battery once nearly full until after solar noon. It was implemented with 67%-full and 12:00Z thresholds, and as shown in the chart most (~2.5kWh) diversion is delayed until after solar noon (cf ~17kWh exports) on 2025-06-14. The following day again diversion is delayed until after solar noon.

The aim is to reduce peak spill to grid at around solar noon when many other PV systems will be exporting at their maxima. This scheme should be able to bite ~2kWh out of each day's solar peak. This is export 'shaping'.

2025-06-16: noon fade as planned

I have now (at EOD) moved the time threshold to a few minutes before UTC noon to avoid a sudden power transition close to an hour boundary, and to eat a little of the near-peak just before noon.

2025-06-18: noon notch

I have had many science-based chuckles from Saturday Morning Breakfast Cereal over the years! I commend it and Zach W to you!

One of my supervisors works on, and contributes occasional blog posts to, the Modern Energy Cooking Services UK International Development supported effort to accelerate a transition from biomass to genuinely ‘clean’ cooking.

I learnt just today that while an induction hob (like 16WW's) uses only ~80% of the energy of a plain old resistance hotplate, an EPC (Electric Pressure Cooker in this case) is nearer half!

Andrea is a UCL chemistry prof with lots of 'rizz'. He has been very kind to me, and to my daughter who he took on a 1.5h private tour of the chemistry facilities while she was deciding what to do at university.

He writes well and is an excellent science communicator, and is active in the Fediverse.

Andrea has just returned his 2014 Royal Society Faraday Prize since it will not eject Elon Musk for his rampage across US science.

Andrea has a heat pump also!

Emil strongly advocates for nuclear electricity generation and fairly hard-left politics, which is an unusual combination. I agree that we need nukes in the generation mix, but find him a bit harsh on the main intermittent renewables (solar and wind). Emil is very active on the Fediverse, which is how I follow him.

He says, in response, trimmed a little:

As for my harsh takes on solar and wind: that is functional.

• I simply see too much wavy takes on energy, stemming from a lack of understanding of how much energy we as a society actually use.
• I mirror typical attacks on nuclear (waste, cost, etc etc) trying to let people consider a fuller picture.
• Solar and wind on itself introduce all kinds of issues that feel-good vibe posters ignore (or don't know about)

So, it is a narrative of push back that I try to establish. I'm not against solar and wind and I disagree strongly with pro-nuclear advocates that say we shouldn't care about solar and wind at all (for ironically very similar reasons only-renewable people use against nuclear). We need to build it all. It is just that on fediverse I rarely have to deal with the second group, heh.

Michael's blog is full of physics and renewable heating and determination to fix the climate. He used to measure things at the NPL, so he is accurate and only tends to make sweeping generalisations about politicians not moving fast enough!

Default closed captions for video and audio!

With a steer from Mr Cridland (see below) I have added the default attribute to my captions track for both AUDIO and VIDEO. Now my videos (such as on smart TRVs) will show closed captions by default if I have a .vtt file and I am displaying this under an https URL. Hurrah!

James' solution for a HTML5 audio player with VTT captions actually does work for me (again, hurrah!), but produced protests from my current page generation pipeline (and poorly optimised CSS) mainly because of the little bit of inlined JavaScript to tie it together.

So I exported the JS for the case required for the first audio tag on the page, ie the main one for a podcast, and made it async, and now it does work. Tested on Firefox, Chrome and Safari.

There is some ugly cross-origin stuff to sort out on the m-dot pages ... later.

I am giving into peer pressure and quietly tagging along with the cool kids on Junited 2025. (I thought that I would be linking to relentlessly tech 'blogs' or adjacent-ish, but the gamut has run rather wider than that while hunting for 30 entries!)

I back-filled to the start of the month: Ministry of Truth stylee...

I enjoy reading James' blogs on audio, but discovered him though Podnews, from my work on RSS Podcast Feed Inefficiency.

A couple of years ago Dan was in the audience at Kinsgton Efficient Homes Show. This year he presented his own short talk about all the work he has done to move his home towards net zero!

Tim's blog cannot help being quite technical because he is, and has form! But it strays into real life often.

Ken Shirriff's blog on retro tech stuff including IC reverse engineering, is fun. I have ridden to the first page of Hacker News at least once I think, posting Interesting BiCMOS circuits in the Pentium, reverse-engineered.

I would like to bring to your attention a little known blogger, the UK Secretary of State for Energy Security and Net Zero, and his RSS feed of written answers to parliamentary questions.

While a lot of the questions are political bluster (and a few are indeed quite unpleasant in their antisocial implications), their general flavour and responses are useful for tracking the government's views and actions (and inactions and bluster).

Chris often has his "slice of sysadmin life" featured on Hacker News and the like, so needs no exposure from me, but here is a recent musing of several, where he states:

... For me, blocking web scrapers here on Wandering Thoughts is partly an editorial decision of whether I want any of my resources or my writing to be fed into whatever they're doing. I will certainly block scrapers for doing what I consider an abusive level of crawling, and in practice most of the scrapers that I block come to my attention due to their volume, but I will block low-volume scrapers because I simply don't like what they're doing it for.

Are you a 'brand intelligence' firm that scrapes the web and sells your services to brands and advertisers? Blocked. ...

I blocked Semrush many years ago because its business model seemed to be from my point of view making my content easier for lowlife to steal to detour search engine users from what they were actually looking for. Several parts bad.

Plus Semrush did all sorts of such technically bad things that I had to repeatedly raise the stakes legally (eg explicitly denying it any implied rights of access under the Computer Misuse Acts) before it started adhere to even basic etiquette.

Thus Semrush earned permanent spots in several of my robots.txt files such as:

# Asked Semrush 2017/12/09 privately to stop touching my servers entirely.
User-agent: SemrushBot
Disallow: /
User-agent: SemrushBot-BA
Disallow: /
User-agent: SemrushBot-SA
Disallow: /
User-agent: SemrushBot-SI
Disallow: /

OpenAI GPTBot Misbehaviour

One of the reasons that I mark up my HTML semantically, such as with schema.org microdata, is to help search engines and others (even AI engines) absorb the information that I wish to share.

But like many others my systems have been compromised for real human users by the blundering, greedy and sometimes deceitful behaviour of the AI 'bots'.

I make available archives of the data and main HTML pages of the site, eg to facilitate off-line reading of the site. But those archives are big, and some of the more idiotic crawlers (not just AI so far as I can tell) were downloading them over and over. So after a couple of intermediate steps to try not to annoy actual humans that might benefit, I moved the big archive in a directory that has a robots.txt "Keep Out" sign on it:

# DHD20250209: avoid huge consolidated archives being redundantly spidered.
User-agent: *
Disallow: /out/monthly/archive/

www.earth.org.uk:443 20.171.X.X - - [07/May/2025:20:27:30 +0000] "GET /out/monthly/archive/public-data-files.tar.xz HTTP/2.0" 200 437823930 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"

and:

www.earth.org.uk:443 20.171.X.X - [26/May/2025:18:57:40 +0000] "GET /out/monthly/archive/public-data-files.tar.xz HTTP/2.0" 200 437823930 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"
www.earth.org.uk:80 20.171.X.X - [26/May/2025:23:05:52 +0000] "GET /out/monthly/archive/public-data-files.tar.xz HTTP/1.1" 302 547 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"
www.earth.org.uk:443 20.171.X.X - [26/May/2025:23:05:52 +0000] "GET /out/monthly/archive/public-data-files.tar.xz HTTP/2.0" 200 437823930 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"
www.earth.org.uk:443 85.93.X.X - [29/May/2025:17:18:20 +0000] "GET /out/monthly/archive/public-data-files.tar.xz HTTP/2.0" 200 437830560 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"

The 20.171.X.X IP accesses are from OpenAI / GPTBot's official IP address range.

What gives? This is obviously unjustifiable behaviour. Someone should start an 'unauthorised access' criminal action under the Computer Misuse Acts or similar, whatever one separately thinks of the ethics of AI scraping, etc. This is clearly "the law is for little people" and some of the same "we do not care about you" attitude that took years to resolve with Semrush, see below...

I noticed that Mr Rigby was doing Junited 2025, not necessarily on the 1st, nor did he necessarily start on the 1st, but in the spirit of rewriting history, let me go with this!