One issue with Hugo is that the way I’m deploying it is via Github actions, which means every time I want to update, the site has to be totally rebuilt. Now the primary flaw with that process is that when Hugo builds a lot of images, it takes a lot of time. About 8 minutes.
The reason Hugo takes this long is that every time it runs its builds, it regenerates all the images and resizes them. This is not a bad thing, since Hugo smartly caches everything in the /resources/_gen/ folder, which is not sync’d to Github, and when you run builds locally it doesn’t take half as long.
Now, this speed is about the same whether the images are locally (as in, stored in the repository) or remote (which is where mine are located – assets.example.com), because regardless it has to build the resized images. This only runs on a build, since it’s only needed for a build. Once the content is on the server, it’s unnecessary.
The obvious solution to solve my speed issues would be to include the folder in Github, only I don’t want to store any images on Github if I can help it (legal reasons, if there’s a DMCA its easier to nuke them from my own storage). The less obvious solution is how we got here.
The Basic Solution
Here’s your overview:
- Checkout the repo
- Install Hugo
- Run the repo installer (all the dependancies etc)
- Copy the files from ‘wherever’ to the Git resource
- Run the build (which will use what’s in the resource folder to speed it up)
- Copy the
resourcesfolder content back down to the server
This would allow me to have a ‘source of truth’ and update it as I push code.
The Setup
To start with, I had to decide where to upload the content. The folder is (right now) about 500 megs, and that’s only going to get bigger. Thankfully I have a big VPS and I was previous hosting around 30 gigs there, so I’m pretty sure this will be okay.
But the ‘where’ specifics needed a little more than that. I went with a subdomain like secretplace.example.com and in there is a folder called /resources/_gen/
Next, how do I want to upload to for starters? I went with only uploading the static CSS files because my plan involves pushing things back down after I re-run the build.
Then comes the downloading. Did you know that there’s nearly no documentation about how to rsync from a remote source to your Github Action instance? It doesn’t help that the words are all pretty generic, and search engines think “Oh you want to know about rsync and a Github Action? You must want to sync from your action to your server!” No, thank you, I wanted the opposite.
While there’s a nifty wrapper for syncing over SSH for Github, it only works one way. In order to do it the other way, you have to understand the actual issue that action is solving. The SSH-sync isn’t solving rsync at all, that’s baked in to the action image (assuming you’re using ubuntu…). No, what the action solves is the mishegas of adding in your SSH details (the key, the known hosts, etc).
I could use that action to copy back down to the server, but if you’re going to have to solve the issue once, you may as well use it all the time. Once that’s solved, the easy part begins.
Your Actions
Once we’ve understood where we’re going, we can start to get there.
I’ve set this up in my ci.yml, which runs on everything except production, and it’s a requirement for a PR to pass it before it can be merged into production. I could skip it (as admin) but I try very hard not to, so I can always confirm my code will actually push and not error when I run it.
name: 'Preflight Checks'
on:
push:
branches:
- '!production' # excludes production.
concurrency:
group: ${{ github.ref }}-ci
cancel-in-progress: true
jobs:
preflight-checks:
runs-on: ubuntu-latest
steps:
- name: Do a git checkout including submodules
uses: actions/checkout@v4
with:
submodules: true
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2
with:
key: ${{ secrets.SERVER_SSH_KEY }}
known_hosts: unnecessary
- name: Adding Known Hosts
run: ssh-keyscan -H ${{ secrets.REMOTE_HOST }} >> ~/.ssh/known_hosts
- name: Setup Hugo
uses: peaceiris/actions-hugo@v3
with:
hugo-version: 'latest'
extended: true
- name: Setup Node and Install
uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
cache: 'npm'
- name: Install Dependencies
run: npm install && npm run mod:update
- name: Lint
run: npm run lint
- name: Make Resources Folder locally
run: mkdir resources
- name: Download resources from server
run: rsync -rlgoDzvc -i ${{ secrets.REMOTE_USER }}@${{ secrets.REMOTE_HOST }}:/home/${{ secrets.REMOTE_USER }}/${{ secrets.HUGO_RESOURCES_URL }}/ resources/
- name: Test site
run: npm run tests
- name: Copy back down all the regenerated resources
run: rsync -rlgoDzvc -i --delete resources/ ${{ secrets.REMOTE_USER }}@${{ secrets.REMOTE_HOST }}:/home/${{ secrets.REMOTE_USER }}/${{ secrets.HUGO_RESOURCES_URL }}/
Obviously this is geared towards Hugo. My command npm run tests is a home-grown command that runs a build and then some tests on said build. It’s separate from the linting, which comes with my theme. Because it’s running a build, this is where I can make use of my pre-built resources.
You may notice I set known_hosts to ‘unnecessary’ — this is a lie. They’re totally needed but I had a devil of a time making it work at all, so I followed the advice from Zell, who had a similar headache, and put in the ssh-keyscan command.
When I run my deploy action, it only runs the build (no tests), but it also copies down the resources folder to speed it up. I only copy it back up on testing for the teeny speed boost.
Results
Before all this, my builds took 8 to 9 minutes.
After they took 1 to 2, which is way better. Originally I only had it down to 4 minutes, but I was using wget to test things (and that’s generally not a great idea — it’s slow). Once I switched to rsync, it’s incredibly fast. The build of Hugo is still the slowest part, but it’s around 90 seconds.
