Earth Notes: On Website Technicals (2020-08)
Updated 2024-12-18.https is available...2020-08-31: FAQ Off The Top Table
There was a huge drop-off in FAQ rich results impressions for EOU between the and (~1000 to ~200 for those days). The peak seen on this snapshot is ~1400 impressions on , dropping to 15 on and down as low as 7 on the !
It appears that Google is pruning the rich and FAQ / How-To results, maybe to reduce search results clutter.
2020-08-22: Brotli Precompression
I have now stopped using the RPi2 (which does not support Brotli) to serve any of the EOU site in any variant.
I am tentatively adding support for Brotli-precompressed files (for https). I have hand-compressed and checked in the Share42 JavaScript for desktop and lite for a nominal first-page-load 280 byte (~20%) saving over gzip:
2787 share42-3.js 1180 share42-3.jsgz 900 share42-3.jsbr
I can see in the browser developer tools that the Brotli version is used over https.
A part of the configuration looks like this (preferring br over gz):
# Serve pre-compressed content (eg HTML) if possible.
# If client accepts brotli (br) compressed files...
RewriteCond %{HTTP:Accept-Encoding} br
# ... and if the pre-compressed file exists...
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME}br -s
# ... then send .XXXbr content instead of .XXX compressed on the fly.
RewriteRule ^/(.+)\.(html|css|js|xml)$ /$1.$2br [E=no-gzip:1,L]
# If client accepts gzip compressed files...
RewriteCond %{HTTP:Accept-Encoding} gzip
# ... and if the pre-compressed file exists...
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME}gz -s
# ... then send .XXXgz content instead of .XXX compressed on the fly.
RewriteRule ^/(.+)\.(html|css|js|xml)$ /$1.$2gz [E=no-gzip:1,L]
Though it seems to happen automagically, there is no harm in explicitly adding a Vary: Accept-Encoding header when doing the above rewrites, eg with config of the form:
<IfModule mod_headers.c>
<FilesMatch ".(html|css|js|xml)$">
Header append Vary: Accept-Encoding
</FilesMatch>
</IfModule>
2020-08-21: MODBUS Moved
Today MODBUS stuff has been shifted over to the RPi3B, and there are some other odds and ends to do, and log files to stitch together carefully.
HTTPS for www.EOU
To extend the EOU certificate to allow https://www.earth.org.uk (and https://earth.org.uk):
sudo certbot -a webroot -i apache --cert-name amp.earth.org.uk -d amp.earth.org.uk -d m.earth.org.uk -d static.earth.org.uk -d www.earth.org.uk -d earth.org.uk
All versions of EOU (desktop/lite/AMP) are now available over http and https.
Now that the stand-alone static.earth.org.uk is no longer required, eg as a host for https image, audio and video targets, I am redirecting it to www again. (Such redirects for unfashionable aliases go to the https version, even though the canonical remains http for now.)
2020-08-20: Reviews Fixed
All reviews are now converted to schema.org/Review nested within Product / Event / Book as appropriate to stop GSC complaining.
This has given rise to a curious inflation of somewhat spurious metadata in that what was one review has become (as reported in GSC) two Review 'fragments' and a Product/Event/Book, ie three items in GSC!
There is some extra metadata information in there, mainly brand and global identifier such as barcode where available. But I think that in trying to fix an index SPAMming issue, Google has created a new noise monster...
Anyway, no errors currently reported in GSC for EOU, hurrah!
In other news: a 5V supply has now been wired from the RPi3B to the dump load relay, so the RPi3B is controlling the load from ~19:30Z.
2020-08-29: after a bit of a delay, a page that I had forgotten popped out of the woodwork with an error in GSC, using hreview! And indeed it looks as if there are about another three pages to convert too...
Here is part of one of the hreview items to be retired:
<div class="hreview">
<ul>
<li class="item"><span class="fn"><a href="https://www.google.com/drive/">Google Drive</a> - <q>Cloud Storage < File Backup for Photos, Docs < More</q></span></li>
<li>Reviewed by: <span class="reviewer">Damon Hart-Davis</span> on
<span class="dtreviewed">2017-08-05</span>.</li>
<li class="summary">Generally reliable cloud backup and collaboration tool.</li>
<li class="description">
An application that integrates with my Mac desktop, and Firefox
and Chrome browsers, making cloud backup easy and cheap with
a bit of command-line stuff from me. On top of that Google Docs
is an excellent collaboration tool, and anything in one of Google's
formats doesn't count towards the storage limit.
...
</li>
<li>Rating: <span class="rating">4.5</span> out of 5</li>
</ul>
</div>
2020-08-15: Moar Moves to RPI3B
I have now moved across the remaining hd.org sites, just as http (not https): www.hd.org, d.hd.org, aj.hd.org.
With Ansible and some templating, the process was a matter of minutes.
I also checked that the SunnyBeam works plugged into the RPi3B, ie that I could read a live power generation value from my grid-tied system. I waited until after dark so that I did not lose a reading on the live server.
On the 18th I integrated the new GPIO driver code into powermng and moved the control wiring across, but need to provide +5V to the driver/relay also.
2020-08-11: AutoAds and Floats Don't Play Nicely
Google's AutoAds seems to have a problem where a float above it causes AutoAds not to push down the space created for an ad, but does push down the ad itself, obscuring following content.
I do not understand why such a basic thing should go wrong. I report the issue by closing/reporting the ad as "covering content". I do not know if that is the best way.
2020-08-16: I have started adding contain:content to the wrapper for the non-AMP explicit AdSense inserts, in preparation for turning AutoAds off again. This should be OK because I disable ads that expand outside their borders for example. The few pages I have rebuilt like this seem OK.
Also in preparation for turning off, or at least watering-down the effects of, AutoAds I have boosted the number of explicit ad slots on the most popular pages. There is less cause for AutoAds to insert anything at all.
2020-08-24: I am seeing clipping of the sides of ads in narrow (portrait) mobile view, since the ads try to bust out of the container to the edges of the viewport.
So I have weakened containment to contain:layout to avoid this.
2020-08-07: CSS for Large Pages
I have added the facility to insert extra CSS support to improve browser performance of pages that are large (source over ~16kB for now) or that contain an insert, since and insert may be large and/or complex. I could modify that to be source+insert size over ~16kB for example.
This size/complexity threshold means that small pages which are fast anyway do not incur a size penalty for something that they do not need.
The first part of this simply attempts to make footers render a little more lazily on large pages where they will not be seen for a while, with footer { contain: content; }.
This constrains footers to be simple beasts with neither overflowing paint nor floats, basically, which is as they should be. They will not let things float into them on big pages, for example. But that will rarely actually make a difference nor be noticeable when there is a difference.
This initial implementation will only retain this CSS tweak if the body/insert text contains at least one footer, such as a Sources section, else the extra CSS will be stripped. The page wrapper text is not seen by the CSS minifier.
This large page support provides separate classes for layout and content containment. Parts of (for example) the home page can use content containment. All SECTIONs can use layout containment when not in desktop mode, but not content because full-width stuff busts out of the container's margins. The non-desktop part of this is implemented, and that is where it may make the most difference.
2020-08-08: the SECTION larding up with contain:layout now only happens (deep) below the fold. If in or near the viewport at the top of the page it would be unlikely to be able to help at all anyway. The containment is used in all page forms now, ie including desktop.
2020-08-05: CSS contain
I have experimentally applied contain:layout to the big table in the LED lighting page to try to speed up initial rendering. For that particular table contain:content (ie including paint) will not work because (on desktop) we bust out of its container's margins on purpose.
div, so it is now arriving via the full-width table style.) Further, because of similar margin-busting elsewhere, contain:layout is likely the most that can be applied blindly in most cases, and a clear:both before may be good to avoid astonishing float behaviour...
SECTION may be a good place to apply contain:layout routinely, maybe only below the fold, with some tweaks.
I am also testing contain:content on the home page for a couple of early simple blocks, and in a benign footer on all pages.
Argh: The property 'contain' in attribute 'style' in tag 'footer' is disallowed. (see https://amp.dev/documentation/guides-and-tutorials/develop/style_and_layout/style_pages/)
Nor does AMP like contain on div. The issue seems to be the style attribute itself: CSS contain can be applied in other ways, eg footer { contain: content; }.
I have also applied contain:layout to all EOU's 'full width' CSS styles, both to potentially speed up layout, and to make such 'fullw' objects less likely to interfere with surrounding page elements. (AMP does not seem to be objecting to these.)
2020-08-04: Reviewed
Over the last few days I have made a big change in how (schema.org) Review snippets are implemented, because Google stopped allowing them as children of Article and similar, to reduce SPAMming of its SERPs. GSC (Google Search Console) switched to reporting all such Review snippets (and older hReview versions) as in error.
I now have such Reviews nested inside Product (or Event or Book), with the LED lighting page getting a major face-lift in passing.