This website is a static site hosted by nearlyfreespeech.net (NFS). I normally deploy it by writing raw HTML on my own machine, committing my changes to GitHub, SSHing into my NFS machine, pulling my changes, and copying the HTML files to a publicly-served folder. I wanted to automate this process. GitHub Actions seemed like an intuitive way to do this, so I started to look into it.
On the NFS machine, I have a script that looks something like this:
rm -r /home/public/* cp -r /home/private/repo/src/* /home/public/ mkdir /home/public/feed ATOM_OUTPUT_PATH=/home/public/feed/atom.xml \ RSS_OUTPUT_PATH=/home/public/feed/rss.xml \ python repo/feed/generate.py
From this, you can see that what we need to do is run a Python script to generate the feed files, then upload the website and feed to the NFS server. Once the files are replaced, the Nginx server running my website will start serving the new files.
The workflow
Here is the full workflow file I ended up with. I'll pull snippets from the file into this page and explain them.
on:
workflow_dispatch:
push:
branches:
- master
This part is pretty self-explanatory. I wanted to be able to trigger deploys whenever I wanted, so that's why I included the workflow_dispatch key.
- name: Check out repository code
uses: actions/checkout@v4
- name: Set up SSH hosts and SSH identity
env:
VICTORZHOUDEV_SSH_KEY_PASSPHRASE: ${{ secrets.VICTORZHOUDEV_SSH_KEY_PASSPHRASE }}
run: |
mkdir ~/.ssh
ssh-keyscan "${{ secrets.VICTORZHOUDEV_SSH_HOST }}" >> ~/.ssh/known_hosts
SSH_KEY=`mktemp -p "${GITHUB_WORKSPACE}" id_rsaXXXXXX`
echo "${{ secrets.VICTORZHOUDEV_SSH_KEY }}" >> "${SSH_KEY}"
eval "$(ssh-agent)"
DISPLAY=1 SSH_ASKPASS="${GITHUB_WORKSPACE}/.github/util/print_ssh_passphrase.sh" ssh-add "${SSH_KEY}"
echo "SSH_AUTH_SOCK=${SSH_AUTH_SOCK}" >> $GITHUB_ENV
echo "SSH_AGENT_PID=${SSH_AGENT_PID}" >> $GITHUB_ENV
The first step checks out the repository into the GitHub actions runner.
The second step sets up an SSH agent so that GitHub Actions can SSH to my NFS server. In order, this step:
- Adds the NFS host as an SSH Known Host.
- Creates a file to hold an SSH private key and populates it from GitHub Actions Secrets.
- Starts an SSH agent.
- Adds the SSH key to the agent using the passphrase, which is also stored in GitHub Actions Secrets.
- Saves some SSH agent environment variables so that future steps can use the SSH agent.
I did not come up with these steps myself; I mostly followed a post from Max Schmitt. The first main difference is that I did not hardcode an agent socket path, but instead allowed the SSH agent to generate one automatically. This meant that I had to use GitHub to propogate the environment variables for the SSH agent between job steps, as environment variables aren't maintained between job steps automatically.
The second main difference is that I decided to require a passphrase on my private key. This meant that I had to get the passphrase from GitHub Actions Secrets into the SSH Agent. I found a StackExchange response that allows passphrases to come from environment variables, so I just had to pass the passphrase as an environment variable, which is straightforward to do in GitHub Actions.
- name: Deploy main website
env:
SSH_AUTH_SOCK: ${{ env.SSH_AUTH_SOCK }}
SSH_AGENT_PID: ${{ env.SSH_AGENT_PID }}
run: |
ssh "${{ secrets.VICTORZHOUDEV_SSH_ADDRESS }}" "rm -r /home/public/*"
scp -r "${GITHUB_WORKSPACE}/src"/* "${{ secrets.VICTORZHOUDEV_SSH_ADDRESS }}:/home/public/"
This step uses the SSH environment variables we set up in the previous step to remove the previous website, and uploads the newest website using SCP.
- name: Install Python environment
uses: actions/setup-python@v4
with:
python-version: "3.x"
- name: Install Python dependencies
run: pip install -r "${GITHUB_WORKSPACE}/feed/requirements.txt"
victorzhou.dev serves two feed XML files for those are interested in that. These steps installs the Python environment to generate the feed files.
- name: Generate and upload RSS feed files
env:
SSH_AUTH_SOCK: ${{ env.SSH_AUTH_SOCK }}
SSH_AGENT_PID: ${{ env.SSH_AGENT_PID }}
run: |
ATOM_OUTPUT_PATH="${GITHUB_WORKSPACE}/feed/atom.xml" RSS_OUTPUT_PATH="${GITHUB_WORKSPACE}/feed/rss.xml" python "${GITHUB_WORKSPACE}/feed/generate.py"
ssh "${{ secrets.VICTORZHOUDEV_SSH_ADDRESS }}" "mkdir /home/public/feed"
scp "${GITHUB_WORKSPACE}/feed/atom.xml" "${{ secrets.VICTORZHOUDEV_SSH_ADDRESS }}:/home/public/feed/"
scp "${GITHUB_WORKSPACE}/feed/rss.xml" "${{ secrets.VICTORZHOUDEV_SSH_ADDRESS }}:/home/public/feed/"
Finally, in the last step, we generate the feed XML files. The SCP upload uses again the SSH agent we set up earlier.
Final thoughts
One lesson I learned while working on this was that environment variables do not propogate automatically between GitHub Action workflow steps. I was having trouble getting the SSH and SCP commands to authenticate correctly and it turns out that it was because the SSH agent environment variables weren't present in those steps, even though they were set earlier. I had to use ssh -vvv flags to figure out that SSH wasn't even trying to use my temporary SSH key, and confirmed that the environment variable wasn't present by printing out the relevant environment variable in testing.
I also accidentally deleted my website once. Thankfully, I'm not really worried about the uptime of my personal site, and it was easy to recover because I still have my old git-based deployment working. I might have been better off my testing this deployment to a staging path.
Finally, I'm not sure that having a passphrase on my SSH key does anything to protect it, since both of these secrets are stored in the same system, and are retrieved using the same system. Not having a passphrase might have simplified the workflow a little for no increased risk.
In the end, I'm happy that I can just write posts, commit them to GitHub, and they'll just appear on my website without further intervention.