I first heard about the idea of a
/colophon page from Slash Pages. I was not
familiar with this concept, but apparently that is the term used to
define the section of a printed work that describes the publisher and/or
publishing process. We can of course adopt this to web pages, describing
the development process and technologies involved.
I have had many iterations of this site over the years, but most of them have been based on GitHub Pages. This is an offering by GitHub where you get a free personal static website at a URL matching your username. It is incredibly easy to use and is a perfect fit for a simple purpose like a personal blog.
I originally started with artisanal, hand-written HTML. This was about as painful to maintain as you might guess. After a few years of that I migrated over to Hugo (I was going through a bit of a phase with Go). This served me reasonably well, but I was always a little uneasy with the seemingly perpetual v0 nature of the project. This did not directly impact me at first though, so I put those concerns to the side. But after a while I got a new laptop and when I was setting up my development environment I realized I was about 100 minor versions behind. When I finally upgraded Hugo it broke my site in a dozen tiny ways, and I decided it was time to revisit this project. I had been playing with the idea of my own static site generator for some time and this was the moment that pushed me over the edge.
I had heard of Pandoc before and was interested in trying it out but never had a compelling use-case. I thought this was an interesting idea: use Pandoc to hand-roll a static site generator. I always felt like Hugo had just a few too many bells and whistles for my needs. To complete this migration though I needed to nail down my core requirements:
- Build and deployment of the site must all be contained in 1-2 simple scripts that will be easy to maintain. I will often take a break for several months (or years) between iterations on this site, and it might be transparent how exactly the entire process functions.
- Several of my blog posts include TeX embeddings for properly displaying mathematical equations, and any replacement should retain that feature.
- I must be able to enrich individual posts with embedded HTML and JavaScript.
- I also need support for embedded Mermaid diagrams.
- Pages must be marked as draft or ready to publish. I will often write out the skeleton of a post idea and then reject it or simply not finish it. I need to be able to store those posts within the repo without automatically publishing.
- There should be some form of tagging system. Honestly I do not use tags as heavily as I could, primarily because the volume of posts on this site is pretty low. But tags are incredibly powerful as a search/filter tool and they should be supported within this new model.
- The output format for all pages should prioritize simple semantic HTML with minimal CSS.
- (Optional) RSS feed generation. I doubt anyone actually subscribes to my current RSS feed, but I think it is good practice to build in that support early.
After a bit of testing I landed on 3 scripts:
build.ps1: Pulls together the .md files in the source repo and converts them to .html via Pandoc.serve.ps1: A simple helper script to run a local web server to test your changes as you go.deploy.ps1: Run a build and then copy over the latest .html and asset files from/public/to a sibling GitHub Pages repo, then push that to master.
build.ps1
$root = $PSScriptRoot
$out = "$root/public"
$staticDir = "$root/static"
$siteUrl = "https://nobleator.github.io"
$svgPath = "$root/assets/img/Hexaflake_Logo.svg"
$logoSvg = Get-Content -Raw -Path $svgPath
$navFile = "$root/nav.html"
$navHtml = @"
<header>
<a href='/' id='logo' alt='nobleator logo, links to homepage'$logoSvg</a>
<nav>
<a href='/'>Home</a>
<a href='/blog.html'>Blog</a>
<a href='/colophon.html'>Colophon</a>
<a href='/blogroll.html'>Blogroll</a>
</nav>
<button id='theme-toggle-button' onclick="toggleTheme()">Theme</button>
</header>
"@
$navHtml | Out-File $navFile -Encoding utf8
function Get-PostMeta($mdFile) {
$json = pandoc $mdFile.FullName --template="$root/templates/metadata.json" -t html 2> $null # TeX rendering warnings will always emit here so ignoring
$meta = $json | ConvertFrom-Json
$meta | Add-Member -NotePropertyName BaseName -NotePropertyValue $mdFile.BaseName
return $meta
}
function Convert-File($mdFile, $outFile) {
pandoc $mdFile `
--standalone `
--template="$root/templates/page.html" `
--include-before-body="$navFile" `
--mathjax `
-o $outFile
}
function Get-TagSlug($tag) {
return ($tag.ToLower() -replace '[^a-z0-9]+', '-').Trim('-')
}
function Format-PostList($postMetas) {
$links = $postMetas | ForEach-Object {
$formattedDate = ([datetime]$_.date).ToString('yyyy-MM-dd')
$tagLinks = (@($_.tags) | ForEach-Object {
"<a href='/tags/$(Get-TagSlug $_).html' class='tag'>$_</a>"
}) -join ' '
"<li><span><i><time datetime='$formattedDate'>$formattedDate</time></i></span><a href='/posts/$($_.BaseName).html'>$($_.title)</a><span class='tags'>$tagLinks</span></li>"
}
return $links -join ''
}
function Format-RssItem($m) {
$link = "$siteUrl/posts/$($m.BaseName).html"
$postDate = [datetime]::SpecifyKind([datetime]$m.date, [DateTimeKind]::Utc)
$pubDate = $postDate.ToString('R')
$title = [System.Security.SecurityElement]::Escape($m.title)
$desc = if ($m.description) { [System.Security.SecurityElement]::Escape($m.description) } else { $title }
return @"
<item>
<title>$title</title>
<link>$link</link>
<guid isPermaLink='true'>$link</guid>
<pubDate>$pubDate</pubDate>
<description>$desc</description>
</item>
"@
}
function Get-GitLastModified($filePath) {
$gitDate = git -C $root log -1 --format=%cd --date=short -- $filePath 2>$null
if ([string]::IsNullOrWhiteSpace($gitDate)) {
return (Get-Item $filePath).LastWriteTime.ToString('yyyy-MM-dd')
}
return $gitDate
}
if (Test-Path $out) {
Get-ChildItem $out -Recurse | Remove-Item -Recurse -Force
} else {
New-Item $out -ItemType Directory | Out-Null
}
Copy-Item "$root/assets" "$out/assets" -Recurse
$sitemapEntries = [System.Collections.Generic.List[string]]::new()
$today = (Get-Date).ToString('yyyy-MM-dd')
$sitemapEntries.Add("<url><loc>$siteUrl/</loc><lastmod>$today</lastmod></url>")
$pages = Get-ChildItem "$root/content/pages/*.md"
foreach ($p in $pages) {
Convert-File $p.FullName "$out/$($p.BaseName).html"
$lastmod = Get-GitLastModified $p.FullName
$sitemapEntries.Add("<url><loc>$siteUrl/$($p.BaseName).html</loc><lastmod>$lastmod</lastmod></url>")
}
$posts = Get-ChildItem "$root/content/posts/*.md"
$metas = $posts | ForEach-Object { Get-PostMeta $_ }
$metas = $metas | Where-Object { $_.draft -ne "true" } | Sort-Object { [datetime]$_.date } -Descending
$postDates = @{}
foreach ($m in $metas) {
$srcFile = Join-Path "$root/content/posts" "$($m.BaseName).md"
Convert-File $srcFile "$out/posts/$($m.BaseName).html"
$postDates[$m.BaseName] = Get-GitLastModified $srcFile
$sitemapEntries.Add("<url><loc>$siteUrl/posts/$($m.BaseName).html</loc><lastmod>$($postDates[$m.BaseName])</lastmod></url>")
}
$allTags = $metas | ForEach-Object { @($_.tags) } | Where-Object { $_ } | Sort-Object -Unique
$tagCloud = ($allTags | ForEach-Object {
"<a href='/tags/$(Get-TagSlug $_).html' class='tag'>$_</a>"
}) -join ' '
$blogMd = "# Posts`n`n<div class='tag-cloud'>$tagCloud</div>`n`n<ul class='blog-posts'>$(Format-PostList $metas)</ul>"
$blogMd | Out-File "$root/_blog_tmp.md" -Encoding utf8
Convert-File "$root/_blog_tmp.md" "$out/blog.html"
Remove-Item "$root/_blog_tmp.md"
$latestPostDate = if ($postDates.Count -gt 0) {
($postDates.Values | ForEach-Object { [datetime]$_ } | Sort-Object -Descending | Select-Object -First 1).ToString('yyyy-MM-dd')
} else {
$today
}
$sitemapEntries.Add("<url><loc>$siteUrl/blog.html</loc><lastmod>$latestPostDate</lastmod></url>")
if (!(Test-Path "$out/tags")) {
New-Item "$out/tags" -ItemType Directory | Out-Null
}
foreach ($tag in $allTags) {
$slug = Get-TagSlug $tag
$taggedMetas = $metas | Where-Object { @($_.tags) -contains $tag }
$tagMd = "# Posts tagged '$tag'`n`n<ul class='blog-posts'>$(Format-PostList $taggedMetas)</ul>"
$tagMd | Out-File "$root/_tag_tmp.md" -Encoding utf8
Convert-File "$root/_tag_tmp.md" "$out/tags/$slug.html"
Remove-Item "$root/_tag_tmp.md"
$tagLatest = ($taggedMetas | ForEach-Object { [datetime]$postDates[$_.BaseName] } | Sort-Object -Descending | Select-Object -First 1).ToString('yyyy-MM-dd')
$sitemapEntries.Add("<url><loc>$siteUrl/tags/$slug.html</loc><lastmod>$tagLatest</lastmod></url>")
}
Remove-Item $navFile
$sitemapXml = @"
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
$($sitemapEntries -join "`n")
</urlset>
"@
$sitemapXml | Out-File "$out/sitemap.xml" -Encoding utf8
$rssItems = ($metas | ForEach-Object { Format-RssItem $_ }) -join "`n"
$feedXml = @"
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>nobleator</title>
<link>$siteUrl</link>
<description>nobleator's personal blog</description>
<lastBuildDate>$((Get-Date).ToUniversalTime().ToString('R'))</lastBuildDate>
<atom:link xmlns:atom="http://www.w3.org/2005/Atom" href="$siteUrl/feed.xml" rel="self" type="application/rss+xml" />
$rssItems
</channel>
</rss>
"@
$feedXml | Out-File "$out/feed.xml" -Encoding utf8
foreach ($f in @("robots.txt", "security.txt")) {
$srcPath = Join-Path $staticDir $f
if (Test-Path $srcPath) {
Copy-Item $srcPath (Join-Path $out $f) -Force
}
}
if (Test-Path "$out/security.txt") {
New-Item "$out/.well-known" -ItemType Directory -Force | Out-Null
Copy-Item "$out/security.txt" "$out/.well-known/security.txt" -Force
}
serve.ps1
$root = $PSScriptRoot
Write-Host "Running fresh build..."
& "$root/build.ps1"
# start server in a background job, rooted in public/
$serverJob = Start-Job -ScriptBlock {
param($dir)
Set-Location $dir
# python -m http.server 1313
# You may need the following on macOS:
python3 -m http.server 1313
} -ArgumentList "$root/public"
Write-Host "Server running at http://localhost:1313 (job id $($serverJob.Id))"
try {
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "$root/content"
$watcher.IncludeSubdirectories = $true
$watcher.Filter = "*.md"
Write-Host "Watching for changes... Ctrl+C to stop."
while ($true) {
$result = $watcher.WaitForChanged([System.IO.WatcherChangeTypes]::All, 1000)
if ($result.TimedOut) { continue }
Write-Host "Change detected: $($result.Name) - rebuilding..."
& "$root/build.ps1"
}
}
finally {
Stop-Job $serverJob
Remove-Job $serverJob
}
deploy.ps1
param([string]$additionalMessage)
"Deploying updates for GitHub pages site"
$currentDirectory = Get-Location
$today = Get-Date
$message = "Rebuilding site and publishing on $today"
if ($additionalMessage) {
$message = $message + ": " + $additionalMessage
}
$root = $PSScriptRoot
Write-Host "Running fresh build..."
& "$root/build.ps1"
"Copying files from .\public into ..\nobleator.github.io"
if (-Not (Test-Path ..\nobleator.github.io)) {
throw "..\nobleator.github.io must exist relative to this script!"
}
Copy-Item -Path .\public\* -Destination ..\nobleator.github.io -Recurse -Force
"Navigating to ..\nobleator.github.io"
Set-Location ..\nobleator.github.io
"Adding changes"
git add .
"Committing changes"
git commit -m "$message"
"Pushing changes to master"
git push origin master
"Navigating to ${currentDirectory}"
Set-Location $currentDirectory
"Deployment complete"
And there you have it, a very simple static site built with Pandoc!