Earth Notes: On Website Technicals (2025-04)

In the energy series dataset display for each datum there is a meter which in its body has a variable length bar of + symbols as a fall-back for browsers that do not implement meter such as lynx. But that case applies to very few visitors. Currently the bar is 0 to 10 +s long, but many many equivalents these days, eg star ratings, are up to 5.

So in the script that generates the tables I have trimmed this to be 0 to 5. This will save a tiny bit of bandwidth, most people will not see the change, and for those that do the result may be more countable and salient.

% ls -alS energy-series-dataset.html{,gz,br}
-r--r--r--  1 dhd  staff  548189 15 Apr 07:58 energy-series-dataset.html
-r--r--r--  1 dhd  staff   42149 15 Apr 15:58 energy-series-dataset.htmlgz
-r--r--r--  1 dhd  staff   29744 15 Apr 15:58 energy-series-dataset.htmlbr
% make energy-series-dataset.html{,gz,br}
% ls -alS energy-series-dataset.html{,gz,br}
-r--r--r--  1 dhd  staff  541786 15 Apr 16:07 energy-series-dataset.html
-r--r--r--  1 dhd  staff   41728 15 Apr 16:07 energy-series-dataset.htmlgz
-r--r--r--  1 dhd  staff   29235 15 Apr 16:07 energy-series-dataset.htmlbr

Yes, I am procrastinating: how could you tell?

It seems like many of the Enphase monthly net energy report files have been full of errors. I noticed this when seeing suspect battery monthly discharge summaries. Note the missing final columns.

% gzip -d < data/16WWHiRes/Enphase/adhoc/net_energy_202501.csv.gz | head
Date/Time,Energy Produced (Wh),Energy Consumed (Wh),Exported to Grid (Wh),Imported from Grid (Wh)
2025-01-01 00:00:00 +0000,0,14,0,18
2025-01-01 00:15:00 +0000,0,30,0,34
2025-01-01 00:30:00 +0000,0,28,0,32
2025-01-01 00:45:00 +0000,0,62,0,66
2025-01-01 01:00:00 +0000,0,45,0,49
2025-01-01 01:15:00 +0000,0,28,0,32
2025-01-01 01:30:00 +0000,0,32,0,36
2025-01-01 01:45:00 +0000,0,31,0,35
2025-01-01 02:00:00 +0000,0,30,0,34
% head ~/Downloads/14XXXXX_monthly_energy_report.csv
Date/Time,Energy Produced (Wh),Energy Consumed (Wh),Exported to Grid (Wh),Imported from Grid (Wh),Stored in batteries (Wh),Discharged from batteries (Wh)
2025-01-01 00:00:00 +0000,0,14,0,18,4,0
2025-01-01 00:15:00 +0000,0,30,0,34,4,0
2025-01-01 00:30:00 +0000,0,28,0,32,4,0
2025-01-01 00:45:00 +0000,0,62,0,66,4,0
2025-01-01 01:00:00 +0000,0,45,0,49,4,0
2025-01-01 01:15:00 +0000,0,28,0,32,4,0
2025-01-01 01:30:00 +0000,0,32,0,36,4,0
2025-01-01 01:45:00 +0000,0,31,0,35,4,0
2025-01-01 02:00:00 +0000,0,30,0,34,4,0

Inspection of the captured files shows that all such captured reports from August last year to January this year were/are missing the final two columns. I shall have to eyeball for this in future, or maybe set an automated check.

Re-fetching January's data returned an apparently complete set. I have manually re-fetched and replaced the other five damaged months' files. I also rebuilt the net_energy_2024.csv.xz full-year file. The charge/discharge monthly and yearly graphs look much more plausible now (not just zeroes)!

One method to try to minimise the size of compressed HTML on the wire is to sort the attributes in each element in a consistent (eg lexical) order.

When I cared about posting articles to Twitter I could not safely sort, because Twitter would fail to read important meta tags in the head unless the attributes were ordered a certain way.

I still use some Twitter tags in the head to help (eg) Mastodon provide previews, but I should no longer need to work round buggy attribute ordering issues for other users of those tags.

So prompted by a Fediverse thread I reinstated attribute ordering in the HTML5 minification script.

I expect this in general to save ~1% of gzip- and br- compressed HTML, though sometimes it might undo careful manual ordering and cause some expansion:

% ls -lS 4{04,29}.html{,gz,br}
   881 429.html
   834 404.html
   505 429.htmlgz
   483 404.htmlgz
   392 429.htmlbr
   337 404.htmlbr
% make 4{04,29}.html{gz,br}
% ls -lS 4{04,29}.html{,gz,br}
   881 429.html
   834 404.html
   503 429.htmlgz
   482 404.htmlgz
   353 429.htmlbr
   334 404.htmlbr

% ls -lS note-on-site-technicals-94.html{,gz,br}
 21808 note-on-site-technicals-94.html
  7081 note-on-site-technicals-94.htmlgz
  5968 note-on-site-technicals-94.htmlbr
% make note-on-site-technicals-94.html{,gz,br}
% ls -lS note-on-site-technicals-94.html{,gz,br}
 21919 note-on-site-technicals-94.html
  7083 note-on-site-technicals-94.htmlgz
  5988 note-on-site-technicals-94.htmlbr

% ls -lS energy-series-dataset.html{,gz,br}
547826 energy-series-dataset.html
 42273 energy-series-dataset.htmlgz
 29381 energy-series-dataset.htmlbr
% make energy-series-dataset.html{,gz,br}
% ls -lS energy-series-dataset.html{,gz,br}
547831 energy-series-dataset.html
 42186 energy-series-dataset.htmlgz
 29531 energy-series-dataset.htmlbr

% ls -lS bibliography.html{,gz,br}
775365 bibliography.html
 98856 bibliography.htmlgz
 74714 bibliography.htmlbr
% make bibliography.html{,gz,br}
% ls -lS bibliography.html{,gz,br}
775424 bibliography.html
 98867 bibliography.htmlgz
 74762 bibliography.htmlbr