Author: Ryan Sawyer
Saturday, 6 August 2022
Upgrading Urbit on your Linux server has finally become ridiculously easy, with two quick caveats.
An upgrade for the Urbit binary, version 1.10, was released last week.
Version 1.9 was the most recent release before this upgrade, and it’s the one which we documented previously. I wish it were called 1.09, but I haven’t come out as Prince of the Earth yet and for reasons out of our immediate scope won’t be permitted to do so until Charles of Wales is crowned King.
Version 1.9 was in part unique for its incorporation of a simplified upgrade process thanks to a new, built-in upgrade mechanism, so this version marks the first time we get to try it out.
Update Instructions
Quick, clear, and simple instructions for upgrading via the new method have been very nicely documented by the Galactic Tribune. Have a look; it’s an extremely simple, three-step process.
Two one-time caveats to these instructions, however – specific to the update from 1.9 to 1.10 – merit additional documentation.
Pace
The new upgrade mechanism supports release channels (like “Stable” vs “Dev”) called paces, and the version we previously installed happens to have been released on the “wrong” pace.
Before running the commands from the instructions linked immediately above, simply open the file called “pace” inside the hidden subdirectory “.bin” inside your pier folder (the folder designating your planet name on your Linux system). In the pace file, replace the word “once” with “live”.
Alternatively worded, you can execute this in the Linux command line as follows:
echo "live" > /path/to/your-planet/.bin/pace
Binary Location
The new upgrade process prepares your pier for the update and installs the new binary. It does not necessarily, however, replace the specific Linux binary file that you’re used to running. Once you figure this out, you shouldn’t have to deal with it again; the new upgrade process will be almost ridiculously simple and you shouldn’t need anything other than what the Galactic Tribune has provided.
If you followed verbatim the official documentation for your original installation as I had, you are likely used to running a binary file named “urbit” located in the same directory as your pier folder (i.e. parallel to the pier folder, not inside it).
If your upgrade was successful, that file is now outdated and can be deleted or backed up. This will be the same regardless as to whether you ran the “next” command on the original “.run” binary as advised or ran it on your original “urbit” binary. In either case, the latter file is untouched.
As demonstrated by the Galactic Tribune, the “.run” file inside your pier folder is now your new, upgraded binary.
Moving forward, you can simply run Urbit directly from this binary file. In our case, however, we wanted a quick way to retain functionality of our existing environmental scripts. We opted to create a symbolic link named “urbit” in the location of the original binary, pointing to updated one.
If you want to do the same, assuming your original urbit binary and pier folder were located next to each other inside an “urbit” folder in your home directory, you would execute the following:
ln -s ~/urbit/pier-name/.run ~/urbit/urbit
Don’t forget that Linux interprets relative paths literally in symbolic links, so be sure to use absolute paths as I have above.
Conclusion
The new upgrade feature is impressively simple and efficient. If for any reason you need to use the old method, those instructions are included as well.
The pace problem should not come up again for most users. And if you created a symbolic link as we did, you will not need to create it again when the .run binary is upgraded in the future.
Live long and prosper.
Author: Ryan Sawyer
Friday, 8 July 2022
The official guide for installing Urbit on a Linux server is accurate enough and up to date. At the same time, it’s easy for documentation on Urbit at this stage in its development to get deprecated substantially, and running any project like this will involve variation by some combination of preference and environmental necessity.
I recommend following the official guide, to which I defaulted throughout the process, from which I deviated slightly as follows. This is not intended as a replacement for or even supplement to the official guide so much as a place of refuge in case someone runs into trouble.
Identity
Your unique identity within the Urbit namespace is basically a phonetic IP address and is called a planet (with stars and galaxies up the network hierarchy). If you don’t have any connections in the Urbit community, you can start by booting up a comet and then asking around for a planet, or you can just buy one. I’ve skimmed at least a dozen websites that sell them either in USD or ETH, and after recent cost reductions in the way these are generated it’s not uncommon to see them sold for $10 – $30. This is a great way to support someone you like – or you can pay (currently) much less by buying a planet on Azimuth for 0.002 ETH which for me 5 days ago amounted to less than $3 with gas fees.
Server Basics
I’ve had good luck with another VPS provider over the years, but for this project I opted for a vanilla Ubuntu 20.04 LTS server through AWS Lightsail for $5/month – mainly because the first 3 months are free.
Urbit these days is accessed both as a command-line shell called dojo and broadcast through a web interface called landscape. The official documentation by default advises you to run sudo setcap ‘cap_net_bind_service=+ep’ ~/urbit/urbit so that Urbit is prioritized access to the HTTP port 80 as the default web server though I ended up electing to reverse this via sudo setcap -r ~/urbit/urbit so that it runs on 8080 instead. I then ran the former command to the path of my nginx binary instead, and configured Nginx as a proxy server.
Note that Urbit guides which point you in this direction tend to be older, and it’s likely that future versions of the urbit binary will likely require flags like –http-port for such configurations to work consistently.
I’m a hacker, and I learn things my own way; despite the fact that I’ve run various headless web servers, I’ve never really had to think about port access before and I got a better understanding of the basics while trying to diagnose what I initially suspected were improperly configured SSL certificates.
I eventually discovered AWS has their own proprietary network firewall, accessed through the web console and independent of the firewalls running inside the actual Linux system like ufw/iptables. HTTPS (port 443) is disabled by default.
I suppose this might not have been the case had I opted for a VM that had a webserver pre-installed, but the choice was between installing things myself on Ubuntu or loading the RHEL-based “Amazon Linux,” about which I had at least a couple reservations.
For setting up HTTPS with LetsEncrypt, Amazon recommends you use the snap version on their servers rather than python3-certbot-nginx, that you do it while the web server’s systemd service is stopped, and that you then go in and incorporate the public keys into your server configuration manually. Note that Amazon’s exact instructions vary depending on which type of distribution package you have.
I don’t think this is necessary, but in my case I used their method to validate my certificates and copied over pieces from a configuration I had generated with python3-certbot-nginx previously.
Media Management
To facilitate uploading media, Landscape is designed to integrate with a file service Amazon created called S3. The official guide walks you through the process of setting up a S3 “bucket” wherein they recommend purchasing a package from DigitalOcean. Amazon of course offers the service as well, but their documentation suggests their API no longer validates V2 Signatures, which Urbit requires.
Fortunately, the Urbit team accurately outlines how to install and configure MinIO for a self-hosted solution. One thing they don’t spell out for you (unless I missed it) is that you’re creating a docker container for MinIO that is saved and can be run after future reboots via docker start minio-urbit
I deviated from their instructions only in that I used Nginx instead of using Caddy. I had made this decision only while misdiagnosing the problems I mentioned above, so I do not think it should be necessary.
Nginx Configuration
This site was very helpful in translating the recommended setup for Caddy into an identically functional Nginx configuration. The MinIO configuration involves the creation of two proxy servers, into which I integrated a third for Landscape.
One advantage to doing this with Caddy is that it would have automated the validation and installation of the SSL certificates. In the case of other web servers, you just have to set everything up for HTTP and use a version of Certbot that will update the config for you. In the configuration template below, Urbit runs on 8080 (HTTP), MinIO runs on 9000 and 9001, and Nginx receives all three and sends them out on 80 (HTTP) and 443 (HTTPS).
/etc/nginx/sites-enabled/main
server {
server_name urbit.yourdomain.org;
location / {
proxy_set_header Host $host;
proxy_set_header Connection '';
proxy_http_version 1.1;
proxy_pass http://127.0.0.1:8080;
chunked_transfer_encoding off;
proxy_buffering off;
proxy_cache off;
proxy_redirect default;
proxy_set_header Forwarded for=$remote_addr;
}
listen 80; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/yourpath/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/yourpath/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
# Redirect non-https traffic to https
if ($scheme != "https") {
return 301 https://$host$request_uri;
} # managed by Certbot
}
/etc/nginx/sites-enabled/bucket
server {
# MinIO console
server_name console.s3.yourdomain.org;
ignore_invalid_headers off;
client_max_body_size 0;
proxy_buffering off;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://localhost:9001;
}
listen 80;
listen 443 ssl;
ssl_certificate /etc/letsencrypt/yourpath/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/yourpath/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
server {
# MinIO server
server_name s3.yourdomain.org;
ignore_invalid_headers off;
client_max_body_size 0;
proxy_buffering off;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://localhost:9000;
}
listen 80;
listen 443 ssl;
ssl_certificate /etc/letsencrypt/yourpath/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/yourpath/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
server {
# MinIO server
server_name media.s3.yourdomain.org;
ignore_invalid_headers off;
client_max_body_size 0;
proxy_buffering off;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://localhost:9000;
}
listen 80;
listen 443 ssl;
ssl_certificate /etc/letsencrypt/yourpath/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/yourpath/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
Shell Scripts
In order to run Urbit as a background process but also be able to access its shell in the foreground as needed, I downloaded and compiled abduco which is very easy and should have no awkward dependencies. This serves the same purpose as tmux or screen as Urbit recommends except that these latter two are terminal multiplexers and abduco is only a session manager.
I then incorporated abduco into 3 shell scripts in my user’s ~/bin directory (I can’t recall if this is part of $PATH by default) and in which its function should be clarified below. In the words of Dr. Emmett L. Brown, “please excuse the crudity of this model, I didn’t have time to build it to scale.”
~/bin/urbit-boot
#!/bin/sh
# If urbit isn’t running, launch an abduco session named “urbit,” in which the command urbit
# is run, and for which Ctrl+Q is bound to hide the session.
pidof urbit && {
printf '\n%s\n\n' "An instance of urbit is already running."; exit 1;
}
printf '\n%s\n%s\n\n' \
"Launching new detachable Urbit instance via abduco." \
"Press Ctrl+Q to return to shell. Restore with urbit-restore."
abduco -e ^q -c urbit $HOME/urbit/urbit -p 34543 $HOME/urbit/sampel-palnet
~/bin/urbit-restore
#!/bin/sh
# Find the abduco process with the session name urbit and
# re-attach it to the current command-line. If urbit isn’t running, return an error.
printf '\n%s\n%s\n\n' \
"Restoring existing Urbit instance via abduco." \
"Press Ctrl+Q to return to shell. Restore \
with urbit-restore."
abduco -e ^q -a urbit || {
pidof urbit && { printf '\n%s\n\n' "Urbit is already running without an attachable session." ; exit 0; }
#must return successful exit code, 0; error means urbit needs to be run!
printf '\n%s\n\n' "Urbit is not running."
exit 1
}
Finally, this third script really could just be a function:
~/bin/urbit
#!/bin/sh
# If urbit-restore returns an error, make sure urbit really isn’t
# running, and then run urbit-boot.
urbit-restore || pidof urbit || urbit-boot
Creating a Systemd Service
Finally, a stable web server should be able to survive reboots without manual intervention. In other words we need to automate the launching of the MinIO docker container as well as the urbit server. There are different ways of doing this, but here’s what I did.
/opt/root-launches-users-urbit
#!/bin/sh
# Root automatically logs user in, runs usual start script as the
# user, then starts docker
#launch urbit for user as reattachable abduco session
runuser -l myuser -c "/home/myuser/bin/urbit-boot" >/dev/null 2>&1 &
#launch minio for connected media bucket
docker start minio-urbit
/etc/systemd/system/urbit-daemon.service
[Unit]
Description=root starts urbit and media bucket for user at system boot
[Service]
ExecStart=/opt/root-launches-users-urbit
[Install]
WantedBy=multi-user.target