I like having my own lightweight “datacenter” at home using ARM servers. It was (and still is) highly effective way of learning how to manage multiple servers using Linux: how to setup networking, security, use config management (I use Chef but today I would’ve chosen Ansible I guess), remote access patterns, installing and maintaining multiple applications, etc. But, when I got my Dell laptop from my previous job as a departure artifact I got into a small predicament: how to safely, securely and efficiently utilize it in my setup which until then only had cheap small ARM servers, not a 1K€ laptop which can be used for many more things (like running Jupyter notebooks, DroneCI docker containers etc).
To be clear, in the context of this blog and my requirements, safe means fully disk encrypted laptop (if a thief takes it, data can’t be extracted), efficient means turning it off and on remotely, securely means that only I can do it, using my Yubikey, no matter where I am in the world.
For quite some time I did it in a simple way: it was always ON (with battery out as some people suggested it on the Internetz for fire safety reasons) but the feeling of doubt was omnipresent: this wastes electricity 99% of the time in this way. I wanted the damn thing on only when I wanted it! It’s the cloud age when things should run only when needed, a small laptop in my basement should be trivial!
I did notice that people are able to do a very cool thing: use something called initramfs/dropbear to access fully encrypted laptop. I also figured out that Intel AMT can also turn on or off a modern PC so I googled, I’ve written some Go code and I’ve decided to write this follow up blog to connect all I’ve figured out for the next one wanting the same thing to do it more simply.
Manual setup stage
Here we need to understand what we are dealing with in my setup to know how applicable it is for you.
FDE
My laptop has full disk encryption. I have Debian Stretch (9) on it during the installation (but derivates — like Ubuntu — should also support FDE). You can’t turn it on after installation AFAIK, at least not the default ecryptfs. You should set it up as you please and at least one decryption method should be passphrase-driven so that you can send it as cleartext over SSH as a parameter to the decryption command. I will not explain how to do it, please google it. A hit I got is for example: https://xo.tc/setting-up-full-disk-encryption-on-debian-9-stretch.html
Intel AMT
I have Intel AMT which I had to activate and set an “admin” password. Depending on your PC vendor AMT can be activated with an empty password or pre-set with a fixed one etc. You should be able to set it up and allow for power management usage commands. This effectively means that ports 16992 and/or 16993 are active on a certain IP that you set up manually. On Linux, you can verify using amttool that AMT is correctly set up using another machine. You should be able to do a powerup and powerdown commands. You can even do it from any linux machine. For example: article 1 and article 2
Dropbear SSH
Dropbear is another very interesting piece of software: it is minimalistic collection of some basic utilities for Linux related to SSH, all of which having minimal dependencies and (important!) being able to run in initramfs which is activated after initial boot sequence in Linux is complete and before disk is decrypted.
Dropbear allows SSH connection to be established to it before your Linux distro is fully up! Don’t forget that no full real drive mount points are available in dropbear
In latest Debian/Ubuntu releases it is not that complicated to set it all up. There are many not up-to-date (and thus — complex) Dropbear initramfs SSH setup blogs to be found on Internet but I found them deprecated. A correct and simple one is for example here: https://hamy.io/post/0005/remote-unlocking-of-luks-encrypted-root-in-ubuntu-debian/
Effectively you need to:
- Install dropbear package for your distro
- Add your public key (your home is not accessible, so another path on the system is used for dropbear authorized keys)
- (Optionally, but recommended) Change the port for SSH
You might need this to know to which SSH service you are talking to: the real one (part of your Linux distro) or dropbear one, running in initramfs; both can’t be running at the same time
Bastion host
I don’t let my servers appear on Internet except one, so called “dmz” or “bastion” node. This is the node that is used primarily for things like SSH tunneling and nginx proxying to other servers. I recommend having one since it allows easier setup (you can use it to setup vpn also for example either using OpenVPN or more simply using ngrok as I have explained in another article).
SSH agent
I stopped using private keys in a form of files. I bought myself a Yubikey in the last Amazon Prime day and I’m loving it. I went through the magnificent manual to setup the GPG and SSH using Yubikey and since then I’ve removed my previous flow and replaced it with a yubikey-driven GPG agent (that can and does correctly work in SSH agent mode also).
I hope you are using some sort of SSH agent, right? Even if you use private key files, you don’t type in your password every time you need that private key, right? You would of course also never dream of keeping unencrypted private key laying around? Right?
Sanity check
Now is the time to verify all is set up correctly:
- Make sure you can connect to AMT via amttool package from another host;
- Make sure your passphrase is correct and that you can type it in and unlock the laptop after boot is complete;
- Make sure after booting and before you unlock the drive you can connect to dropbear ssh port and that you can run decryption command crypt-unlock;
- Now rinse & repeat with an SSH tunnel through your bastion host.
Automation
Finally, you might consider that all of this is fine and nice but it takes a lot to turn on: you need to start the machine, connect remotely to dropbear ssh, issue unlock procedure etc. That’s correct, it’s tedious. That’s why I wrote https://github.com/milanaleksic/laptop-booter/blob/master/README.md to do all the steps above. I recommend setting up aliases to boot up certain servers.
Many things can be improved here! You might for example wish for:
- only partial support for my flow: for example avoid bastion ssh tunnel or use local file for ssh private key etc;
- env variables or config files instead of CLI args;
- omit some warning messages or introduce logging levels.
I can only say: PRs are welcome, requirements in the form of GitHub issues also (although depending on priority it might take some time for me to do sth about them).
