Creating this site yet again: gem Jekyll Safe Mode issue


For this blog site, the category and tags index files are not generated by GitHub Actions as the Jekyll build runs in safe mode there. Solution is to generate them separately locally and have them committed to the blog site repository.

Discussion

Previous posts covered moving this Jekyll Blog Site from Azure Devops to GitHub Pages. Part of the structure of this site is that each page has one category and multiple tags which can be used for searching. An index page is generated for each category and for each tag, along with an index of categories and an index of tags which point to these tag pages.

Previously the site was built locally and committed to the Azure Devops as a repository where it deployed to Azure Blob Storage so as to be made available as a web site.

GitHub Pages simplifies this (and can be cheaper) in that the un-built site is committed to the repository where Actions build the Jekyll site and make it available as web site.

A problem has arisen in that it was noticed that new tags on posts where not resulting in new tag index pages. A similar issue probably existed for new categories. The reason for this is that “with github-pages bundle, Jekyll runs in safe mode and does not load the local plugins that generate those index pages.”

A powershell script for each has been added so as to build those index pages and index thereof locally so that they can be committed to GitHub and thus used when the Jekyll site is built there by GitHub Actions. These:

  • Make sure the cats and tags folders exist in the root of the site content
  • Clears any files in those folders
  • For each category, generates a cats index file of all posts of that category and an index of all categories pointing to the category index pages
  • For each tag, generate a tag index file of all all posts with that tag and an index page of all tags pointing to those tag index pages.

Tag and category pages

Source files versus build output

Location Purpose Commit it?
tags/*.md Source pages for /tags/<tag>/; tags/index.md provides /tags/. Yes
cats/*.md Source pages for /cats/<category>/. Yes
categories.md Source page for the /cats/ index. Yes
_site/tags/ and _site/cats/ HTML generated by a local Jekyll build. No; _site/ is ignored

Deleting _site/tags/ or _site/cats/ is different from deleting the root tags/ or cats/ source directories. Jekyll can rebuild HTML from committed source pages, but it cannot infer and restore missing source pages in the current safe-mode build.

Why the old plugins appeared to generate pages

_plugins/tags.rb and _plugins/cats.rb register :site, :post_read hooks. When local plugins are enabled, the hooks inspect site.tags and site.categories and write Markdown files into the root tags/ and cats/ directories. They do not merely create HTML in _site/. Because post_read runs after Jekyll discovers source pages, a newly written file may require another build to appear in the output.

The site’s Gemfile includes github-pages. In the tested bundle exec jekyll build configuration, that gem enables Jekyll safe mode and prevents the repository’s local _plugins/*.rb files from loading. Consequently, the old hooks do not recreate missing source pages in that build. Safe mode is a Jekyll/GitHub Pages build setting, not an inherent property of GitHub Actions. No .github/workflows/ files are tracked in this repository; check the repository’s Pages settings if you need to know which deployment method GitHub currently uses. An external workflow would only regenerate source pages if it explicitly ran a generator and used its results in the build.

Current regeneration workflow

From the repository root, after changing post tags or _data/sections.yml:

.\scripts\createtags.ps1 -WhatIf
.\scripts\createcats.ps1 -WhatIf
.\scripts\createtags.ps1
.\scripts\createcats.ps1
bundle exec jekyll build
git status --short

The scripts validate their inputs before replacing the corresponding root directory. createtags.ps1 gets unique tag names from Jekyll’s post index, recreates tags/*.md, and writes tags/index.md. createcats.ps1 gets category abbreviations from _data/sections.yml and recreates cats/*.md. The /cats/ index comes from categories.md, so the category script does not write a second index. Keep the YAML abbreviations in sync with post categories so every category link has a source page.

These commands are destructive to any custom files inside the root tags/ and cats/ directories: they clear each directory before writing the generated pages. Review git status and the resulting changes before committing. scripts/jekyll-clean.ps1 also clears both root directories; run the two generators afterward if you use that cleanup script.

Commit the source pages along with the scripts and YAML changes, for example:

git add tags cats _data/sections.yml scripts/createtags.ps1 scripts/createcats.ps1

Conclusion GitHub Pages can then build the committed Markdown into published HTML, even when local Ruby plugins are disabled. Neither Git nor the current safe-mode Jekyll build runs the PowerShell generators automatically. Run them and commit their output again when new tags or categories are introduced.

Code

createcats.ps1

[CmdletBinding(SupportsShouldProcess)]
param(
    [string]$SiteRoot = (Join-Path $PSScriptRoot '..')
)

$siteRoot = (Resolve-Path -LiteralPath $SiteRoot).Path
$sectionsPath = Join-Path $siteRoot '_data\sections.yml'
$catsPath = Join-Path $siteRoot 'cats'
if (-not (Test-Path -LiteralPath $sectionsPath -PathType Leaf)) {
    throw "Missing category data: $sectionsPath"
}

$categoryJson = & ruby -rjson -ryaml -e "sections = YAML.safe_load(File.read(ARGV.fetch(0))); abort 'Expected abbreviation and label pairs in sections.yml' unless sections.is_a?(Array) && sections.all? { |entry| entry.is_a?(Array) && entry.length == 2 && entry.all? { |value| value.is_a?(String) } }; STDOUT.write(JSON.generate(sections.map(&:first)))" $sectionsPath
if ($LASTEXITCODE -ne 0) {
    throw 'Could not read categories from sections.yml.'
}

$categories = @($categoryJson | ConvertFrom-Json) | Sort-Object -CaseSensitive -Unique
if ($categories.Count -eq 0) {
    throw 'No categories found in sections.yml.'
}
$pages = @{}
foreach ($category in $categories) {
    if (-not $category -or $category -match '[\\/:*?"<>|]' -or $category -match '(^\.{1,2}$|[. ]$|^(con|prn|aux|nul|com[1-9]|lpt[1-9])(\.|$))' -or $category -eq 'index') {
        throw "Category cannot be used as a page filename: $category"
    }
    if ($pages.ContainsKey($category)) {
        throw "Categories share the same page filename: $category and $($pages[$category])"
    }
    $pages[$category] = $category
}

if (-not $PSCmdlet.ShouldProcess($catsPath, "Clear and create $($categories.Count) category pages")) {
    return
}

if (-not (Test-Path -LiteralPath $catsPath)) {
    New-Item -ItemType Directory -Path $catsPath | Out-Null
}
Get-ChildItem -LiteralPath $catsPath -Force | Remove-Item -Recurse -Force

$utf8 = New-Object System.Text.UTF8Encoding($false)
foreach ($category in $categories) {
    $content = @(
        '---'
        'layout: catpage'
        "cat: $($category | ConvertTo-Json -Compress)"
        "permalink: $("/cats/$category/" | ConvertTo-Json -Compress)"
        '---'
        ''
    ) -join "`n"
    [System.IO.File]::WriteAllText((Join-Path $catsPath "$category.md"), $content, $utf8)
}
Write-Output "Created $($categories.Count) category pages in $catsPath"

createtags.ps1

[CmdletBinding(SupportsShouldProcess)]
param(
    [string]$SiteRoot = (Join-Path $PSScriptRoot '..')
)

$siteRoot = (Resolve-Path -LiteralPath $SiteRoot).Path
$postsPath = Join-Path $siteRoot '_posts'
$tagsPath = Join-Path $siteRoot 'tags'
if (-not (Test-Path -LiteralPath $postsPath -PathType Container) -or
    -not (Test-Path -LiteralPath (Join-Path $siteRoot 'Gemfile') -PathType Leaf)) {
    throw "Not a Jekyll site: $siteRoot"
}

Push-Location $siteRoot
try {
    $tagJson = & bundle exec ruby -e "require 'json'; require 'jekyll'; site = Jekyll::Site.new(Jekyll.configuration({'source' => ARGV.fetch(0), 'safe' => true, 'plugins_dir' => 'disabled-local-plugins', 'quiet' => true})); site.read; STDOUT.write(JSON.generate(site.tags.keys))" $siteRoot
    if ($LASTEXITCODE -ne 0) {
        throw 'Could not read tags from Jekyll posts.'
    }
} finally {
    Pop-Location
}

$tags = @($tagJson | ConvertFrom-Json) | Sort-Object -CaseSensitive -Unique
$pages = @{}
foreach ($tag in $tags) {
    $slug = $tag.TrimStart('/')
    if (-not $slug -or $slug -match '[\\/:*?"<>|]' -or $slug -match '(^\.{1,2}$|[. ]$|^(con|prn|aux|nul|com[1-9]|lpt[1-9])(\.|$))' -or $slug -eq 'index') {
        throw "Tag cannot be used as a page filename: $tag"
    }
    if ($pages.ContainsKey($slug)) {
        throw "Tags share the same page filename: $tag and $($pages[$slug])"
    }
    $pages[$slug] = $tag
}

if (-not $PSCmdlet.ShouldProcess($tagsPath, "Clear and create $($tags.Count) tag pages and an index")) {
    return
}

if (-not (Test-Path -LiteralPath $tagsPath)) {
    New-Item -ItemType Directory -Path $tagsPath | Out-Null
}
Get-ChildItem -LiteralPath $tagsPath -Force | Remove-Item -Recurse -Force

$utf8 = New-Object System.Text.UTF8Encoding($false)
foreach ($slug in $pages.Keys) {
    $tag = $pages[$slug]
    $content = @(
        '---'
        'layout: tagpage'
        "tag: $($tag | ConvertTo-Json -Compress)"
        "permalink: $("/tags/$slug/" | ConvertTo-Json -Compress)"
        '---'
        ''
    ) -join "`n"
    [System.IO.File]::WriteAllText((Join-Path $tagsPath "$slug.md"), $content, $utf8)
}

$index = @(
    '---'
    'layout: page'
    'title: Tags'
    'permalink: /tags/'
    '---'
    ''
    '<h1>Tags</h1>'
    '<ul>'
      '=== See image below ==='
    '</ul>'
    ''
) -join "`n"
[System.IO.File]::WriteAllText((Join-Path $tagsPath 'index.md'), $index, $utf8)
Write-Output "Created $($tags.Count) tag pages and an index in $tagsPath"

Code at ‘See image below’ as above

 TopicSubtopic
<  Prev:   Blazor
   
 This Category Links 
Category:Web Sites Index:Web Sites