Earth Notes: On Website Technicals (2018-12)

Doing a Google search from mobile for one of my key terms, my "Why XXX?" heading followed by the start of the following para, showed up as a featured snippet. The same (m.) page shows up as a normal SERP entry a little further down with rich text (for a review) and the same snippet.

Hurrah!

Note that there is no special (eg schema.org) markup around the snippet, just a clear short question in the (h2) heading, and a simple short and sweet para immediately underneath answering it.

Tentatively, I have added sharing with amp-social-share.

In all the recent upheaval, AMP support included, a number of things seem to have been silently broken such as dropping the og:description meta tag.

I have added unit tests for the test page to cover variants of some of the issues found today.

The amp.earth.org.uk site went live!

(Incidentally that home page can be accessed also via the Google AMP Cache.)

There are few pages that are not supported for AMP. Most of them may never be, eg because they provide on-page JavaScript-based demos. Some just need a little further housekeeping to join the AMP universe.

In such a case, no AMP page is created nor linked to, and any attempt to access it will be redirected to the vanilla mobile/lite 'm.' page.

An annoying issue: the AMP cache knows that it is loading from an http: source, and rewrites relative links to absolute (http:) links. But it does not rewrite protocol-relative (scheme-relative?) links (eg //WWW.earth.org.uk) to absolute, so they will fail trying to reach a currently non-existent https: server. So I am now introducing pseudo-protocol-relative //WWW.earth.org.uk links to match the existing //STATIC.earth.org.uk links, that get re-written to the primary absolute form, though remain syntactically valid as raw HTML.

Now all but 15 (out of ~207) pages have AMP versions. I manually fixed up nearly 600 links to deal with the protocol-relative issue also...

I had somehow convinced myself that AMP did not allow inline CSS style in the page body. So I did a lot of work to eliminate common inline styling, partly because doing so can also make the page smaller. But I was sure that it was going to be a big problem for many remaining pages.

(Maybe this would indeed have been an issue until recently, see eg You can't use inline 'style="..."' tags, but AMP: Supported CSS says that inline styling is allowed when reading it today.)

However, inline styling seems not to be a significant problem since only ~40 of the ~200 main pages are failing to validate in AMP form. And many of those have simple/known img and script issues...

The validator uses the latest published set of rules to apply across the network, which means that details and summary are already passing validation (hurrah!). But it also means that attempting to build and validate AMP pages fully off-line, as I can with desktop and vanilla mobile, will result in:

ERROR: validation: Unable to fetch https://cdn.ampproject.org/v0/validator.js - getaddrinfo ENOTFOUND cdn.ampproject.org cdn.ampproject.org:443

I am not sure that I want to be prodding the CDN for every single page rebuild, for a number of reasons.

Maybe amphtml-validator-rules would be part of a mechanism to help me work more locally.

I have reduced the size of the custom 404 error page to 894 bytes of body when pre-GZIP-compressed, plus ~340 bytes of desktop HTTP/1.1 headers. Thus the HTTP response for it should be able to fit in a single TCP frame to most clients. The mobile/lite page is even smaller.

All informational footers, and social-media header support such as og:image and twitter:card, are omitted for such an error page. This saves ~200 bytes from the GZIPed size.

There is a little more that could be stripped out, eg a little residue of social media button support (~80 bytes uncompressed) that could go, and would benefit all desktop pages not needing such support.

Though it is unlikely to be used any time soon (ie probably only for US-originated Google News searches for now), I am starting to fold in some support for the 'pending' Schema.org 'speakable'.

I was partly spurred on by the relevant parts of the discussion in What to Expect in 2019 with Google's John Mueller.

This may in future help screen readers and voice searches. Being another site providing this data may in a tiny way speed its adoption.

Google's documentation has this firmly marked as BETA for now. (Also see "Add vocabulary to indicate which sections of a document are particularly 'speakable'".)

Note that Google's docs say not to use both xpath and cssSelector, but I am picking out title and description with the former and an optional intro para with the latter. I have split the structured microdata for the latter into its own itemscope and Google's Structured Data Testing Tool seems OK with that, showing the implied value for each item correctly.

All the new meta/structured data is at the very end of the document, so not in the CRP (Critical Rendering Path). Hurrah!

Note that HTML minification for the m-dot version rearranges (sorts) tag attributes to try to improve compression. However this seems to silently break extraction by Twitter of some header meta data such as description under some circumstances. So I have stopped doing that particular sorting.

The m-dot minifier also omits the inferrable head tag, and this minified HTML apparently defeated parsing when in full precise form:

/html/head/meta[@property='og:title']/@content

All is happy again when I slightly generalise the xpath, with minimal risk of picking up stray tags later!

//meta[@property='og:title']/@content

Schema.org SpeakableSpecification Example

Grabbed from the EOU home page (with some wrapping for readability):

<span itemprop=speakable itemscope itemtype=http://schema.org/SpeakableSpecification>
  <meta itemprop=xpath content="//meta[@property='og:title']/@content">
  <meta itemprop=xpath content="//meta[@property='og:description']/@content">
</span>
<span itemprop=speakable itemscope itemtype=http://schema.org/SpeakableSpecification>
  <meta itemprop=cssSelector content=.pgintro>
</span>

Note that this is largely fixed because it refers to existing pieces of text. The .pgintro part is left out if the page does not have a pgintro chunk of text. The title and description are always present, however.

At times there may be more than one make running to try to rebuild EOU. For example, while the battery charge is high a make -k all may be run every hour.

In particular, the rebuild of each page has a lock around it. Two or more make processes may end up continually trying to make the same page next, with one of them being excluded by the lock after a timeout, and moving on. The multiple processes tend to stay in lockstep, and all that lock contention wastes time and reduces parallelism.

In general make tries to be reasonably dependable and consistent, and shaking that up is hard. A reasonable solution for my gmake on *nx is, given my list of main pages in a "simply expanded variable" PAGES:

PAGES := pageA.html pageB.html ... another.html

... and given that each page's build is independent of the others, then adding this line afterwards mixes things up:

PAGES := $(shell echo $(PAGES) | xargs -n1 | sort -R | xargs)

This works fine with independent make runs with or without -j to add parallelism.

The cost is a the execution of the shell command once per make invocation.

It's a day off for me today, so of course what I do before breakfast is add a couple of tricky unit test cases for IMG!

Let joy be unconfined!

I am modifying IMG to be able to accept as src a (standard) thumbnail URL in my 'CMS' gallery.hd.org, at least for body images in the first instance.

This means that I need not copy, minify and check-in to the VCS (Version Control System, eg SVN) body images for EOU if they are already being hosted in the Gallery.

(This also means that the IMG tag remains valid HTML if not translated, even if it will be non-optimal in a number of ways.)

The appropriately scaled images will still be served from directly under /img/a/ rather than by the Gallery, and clicking the image will take the visitor through to the Gallery catalogue page.

Which means that I can drop almost any still raster image in, on whim!

The IMG tag is helping me to spruce up existing pages, eg in adding new images to them. Even if I never take AMP pages live, the mechanism is useful. It helps to only need src and class attributes. It is proving helpful to be able to manually set alt sometimes.

So, for example, On Greening Christmas had its rather poor lone image improved and a new one added, and that second image was also added to Low Carbon Family Holidays. I automatically get smaller lower-weight versions for the mobile pages, along with really-low-weight Save-Data versions, and a link back to the source image if too large to be used directly, so possibly containing other information of interest to the visitor. What's not to like?

I freed up a little space on the CRP (Critical Rendering Path) for the home pages (desktop and mobile) and so inserted a header link to the basic Atom site feed. My Firefox "Brief" plugin picks that up and shows a feed button in the URL bar, and I am hoping that other browsers give similar signals.

Soon I shall drop both the RSS/Atom and G+ 'social media' buttons for each page, and so keeping the feed on the home page in this way may be useful.