../

Migrating VMs

After finishing my exams, I started to look at the miscellaneous tasks which were on my todo list. While messing around on my VM, I thought that it was probably a good time to upgrade to Ubuntu 20.04.

I first checked do-release-upgrade. For some reason, it listed eoan instead of focal as the release to upgrade to. Because I didn’t have many services running on the VM (only Gitea and ZNC), I thought that it wouldn’t be a bad idea to start fresh on a fresh VM.

After stopping all the running services, I moved all the files which weren’t on the attached block storage volume to the volume (misc files in /home like /home/ubuntu/.znc, configs from /etc/nginx, letsencrypt certificates from /etc/letsencrypt etc).

Then, I swapped this volume from the old machine to the new one, and started installing docker and docker-compose. This went quite smoothly now that the focal source respository for docker is present. After that, I disabled the snap and Oracle related services which were on the machine.

Because my domains pointed to the IP address of the original VM, I reassigned the reserved IP of the old VM and to the new one. This turned out to be a good decision, as it became possible to ProxyJump from the new machine to the old machine using its internal IP address, and I used this to double check some configuration files (e.g. iptables).

ssh -J vm [email protected]

Then, it was just a matter of reconfiguring ZNC, gitea and the other relevant pieces (e.g. nginx, iptables, sshd_config, fail2ban).

Containerising ZNC

I use znc as an IRC bouncer for freenode. Because I configured the old instance in a rush, I didn’t bother investigating whether it had a docker image or not and just let it run on the machine. This time round, I thought that it would be a good idea to run it as a container, if only to make the migration process easier the next time.

I referred to the reference for the official docker image and the compose file from linuxserver and adapted it into the following docker-compose file. Note that 6667 is the port which I originally chose.

---
version: "2.1"
services:
  znc:
    image: znc:latest
    container_name: znc
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Australia/Melbourne
    volumes:
      - ./data:/znc-data
    ports:
      - 6667:6667
    restart: unless-stopped

I renamed the original .znc folder to data and ran docker-compose start, immediately running into the following error:

Unable to locate pem file: /home/ubuntu/.znc/znc.pem

After playing around with the configuration files, I eventually realised that znc.conf contains a reference to the original location of the pem file. Therefore, I changed the corresponding line to reflect the new location, and all was well.

Gitea with SSH passthrough

I use gitea for hosting some of my personal git repositories. Because it is desirable to use SSH to perform operations on remote repositories, it becomes necessary to use SSH passthrough to pass SSH from host to container.

Though the guide on Gitea provides a step by step process, it isn’t trivial to set it up correctly.

Received disconnect from <IP> port 22:2: Too many authentication failures
Disconnected from <IP> port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Here are some strategies for debugging such issues:

  • Get the SSH client to output some logs when performing operations on remote repositories. e.g.
GIT_SSH_COMMAND="ssh -v" git clone <url>

When having many SSH keys, use a ~/.ssh/config file to specify the keys that should be used.

Host git.domain
    HostName git.domain
    IdentityFile ~/.ssh/<private_key>
    User git
  • Check the SSH logs (e.g. /var/log/auth.log). Often, you’ll find something intuitive, e.g.
Jun 20 09:58:31 sshd[13579]: message repeated 5 times: [ Authentication refused: bad ownership or modes for directory /volume/gitea/gitea]
  • Change to the git user (e.g. sudo su git) and see if the authorized_keys file is accessible.
← Previous Post