Categories
DevOps

How to install LXD on Raspberry Pi

Install LXD a next generation system container and virtual machine manager on Raspberry Pi using DietPi a highly optimized minimal Debian OS.

Preparations

Update package index.

$ sudo apt update

Upgrade packages.

$ sudo apt upgrade

Install Logical Volume Manager utilities.

$ sudo apt install lvm2 util-linux

Determine if a disk needs to be wiped up.

$ sudo wipefs --no-act --all /dev/sda
/dev/sda: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 8 bytes were erased at offset 0xe8decffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sda: calling ioctl to re-read partition table: Success

Wipe signatures from a device if needed.

$ sudo wipefs --all /dev/sda
/dev/sda: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 8 bytes were erased at offset 0xe8decffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/sda: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sda: calling ioctl to re-read partition table: Success

Initialize LVM physical volume.

$ sudo pvcreate /dev/sda
  Physical volume "/dev/sda" successfully created.

Create LVM volume group.

$ sudo vgcreate lxd_storage /dev/sda
  Volume group "lxd_storage" successfully created

Install and configure firewall

Install a dynamically managed firewall with support for network zones.

$ sudo apt install firewalld

Add an existing interfaces to the public zone.

$ sudo firewall-cmd --add-interface wlan0 --zone public
success
$ sudo firewall-cmd --add-interface eth0 --zone public
success

Add a bridge interface that will be managed by the LXD daemon to the trusted zone.

$ sudo firewall-cmd --add-interface lxdbr0 --zone trusted
success

Display and verify used zones.

$ firewall-cmd --get-active-zones 
public
  interfaces: eth0 wlan0
trusted
  interfaces: lxdbr0

Display and verify allowed services.

$ sudo firewall-cmd --list-services --zone=public 
dhcpv6-client ssh

Ensure that changes are permanent.

$ sudo firewall-cmd --runtime-to-permanent
success

Configure cgroup

Inspect kernel boot parameters.

$ cat /boot/cmdline.txt 
root=PARTUUID=8b0ac6f5-02 rootfstype=ext4 rootwait fsck.repair=yes net.ifnames=0 logo.nologo quiet console=tty1

Enable memory accounting and use cgroup v1.

$ sudo sed -i -e "1 s/$/ cgroup_enable=memory systemd.unified_cgroup_hierarchy=0/" /boot/cmdline.txt 
$ cat /boot/cmdline.txt 
root=PARTUUID=8b0ac6f5-02 rootfstype=ext4 rootwait fsck.repair=yes net.ifnames=0 logo.nologo quiet console=tty1 cgroup_enable=memory systemd.unified_cgroup_hierarchy=0

Reboot operating system.

$ sudo reboot

Install LXD

Install snap package manager and bridge utils.

$ sudo apt install snapd bridge-utils

Inspect LXD snap package.

$ snap info lxd                                                  
name:      lxd                                                                                                
summary:   LXD - container and VM manager                                                                     
publisher: Canonical✓                                                                                         
store-url: https://snapcraft.io/lxd
contact:   https://github.com/lxc/lxd/issues                                                                  
license:   Apache-2.0                                                                                         
description: |                                         
  LXD is a system container and virtual machine manager.
                                                                                                              
  It offers a simple CLI and REST API to manage local or remote instances,
  uses an image based workflow and support for a variety of advanced features.
                                                                                                              
  Images are available for all Ubuntu releases and architectures as well
  as for a wide number of other Linux distributions. Existing
  integrations with many deployment and operation tools, makes it work
  just like a public cloud, except everything is under your control.
                                                                                                              
  LXD containers are lightweight, secure by default and a great
  alternative to virtual machines when running Linux on Linux.
                                                                                                              
  LXD virtual machines are modern and secure, using UEFI and secure-boot
  by default and a great choice when a different kernel or operating
  system is needed.                                                                                           
                                                                                                              
  With clustering, up to 50 LXD servers can be easily joined and managed
  together with the same tools and APIs and without needing any external
  dependencies.                                                                                               
                                                                                                              
                                                                                                              
  Supported configuration options for the snap (snap set lxd [=...]):

    - ceph.builtin: Use snap-specific Ceph configuration [default=false]
    - ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
    - criu.enable: Enable experimental live-migration support [default=false]
    - daemon.debug: Increase logging to debug level [default=false]
    - daemon.group: Set group of users that can interact with LXD [default=lxd]
    - daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
    - daemon.syslog: Send LXD log events to syslog [default=false]
    - lvm.external: Use the system's LVM tools [default=false]
    - lxcfs.pidfd: Start per-container process tracking [default=false]
    - lxcfs.loadavg: Start tracking per-container load average [default=false]
    - lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
    - openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
    - shiftfs.enable: Enable shiftfs support [default=auto]
   
  For system-wide configuration of the CLI, place your configuration in
  /var/snap/lxd/common/global-conf/ (config.yml and servercerts)
snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
channels:
  latest/stable:    4.20        2021-11-10 (21863) 69MB -
  latest/candidate: 4.20        2021-11-05 (21863) 69MB -
  latest/beta:      ↑                                    
  latest/edge:      git-fa40c17 2021-11-12 (21887) 69MB -
  4.20/stable:      4.20        2021-11-10 (21863) 69MB -
  4.20/candidate:   4.20        2021-11-05 (21863) 69MB -
  4.20/beta:        ↑                                    
  4.20/edge:        ↑                                    
  4.19/stable:      4.19        2021-10-28 (21783) 68MB -
  4.19/candidate:   4.19        2021-11-04 (21844) 68MB -
  4.19/beta:        ↑                                    
  4.19/edge:        ↑                                    
  4.0/stable:       4.0.8       2021-11-06 (21843) 63MB -
  4.0/candidate:    4.0.8       2021-11-04 (21843) 63MB -
  4.0/beta:         ↑                                    
  4.0/edge:         git-79ea78f 2021-11-04 (21849) 63MB -
  3.0/stable:       3.0.4       2019-10-10 (11376) 49MB -
  3.0/candidate:    3.0.4       2019-10-10 (11376) 49MB -
  3.0/beta:         ↑                                    
  3.0/edge:         git-81b81b9 2019-10-10 (11378) 49MB -
  2.0/stable:       2.0.12      2020-08-18 (16884) 34MB -
  2.0/candidate:    2.0.12      2021-03-22 (19874) 34MB -
  2.0/beta:         ↑                                    
  2.0/edge:         git-82c7d62 2021-03-22 (19869) 35MB -

Install LXD snap package.

$ sudo snap install lxd --channel=latest/stable
2021-11-13T21:45:04+01:00 INFO Waiting for automatic snapd restart...
lxd 4.20 from Canonical✓ installed

Create initial LXD configuration.

$ cat << EOF | tee lxd_configuration.yml
networks:
- config:
    ipv4.address: 10.114.53.1/28
    ipv4.nat: "true"
    ipv6.address: none
  description: ""
  name: lxdbr0
  type: bridge
  project: default
storage_pools:
- config:
    source: lxd_storage
  description: ""
  name: default
  driver: lvm
profiles:
- config: {}
  description: Default LXD profile
  devices:
    eth0:
      name: eth0
      network: lxdbr0
      type: nic
    root:
      path: /
      pool: default
      type: disk
  name: default
projects:
- config:
    features.images: "true"
    features.networks: "true"
    features.profiles: "true"
    features.storage.volumes: "true"
  description: Default LXD project
  name: default
EOF

Initialize LXD using created configuration.

$ cat lxd_configuration.yml | sudo -i lxd init --preseed

LVS thin pool will be created automatically.

$ sudo lvs
  LV          VG          Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  LXDThinPool lxd_storage twi-a-tz-- 929.48g             0.00   1.93   

Inspect warnings as these are expected.

$ sudo -i lxc warning list                                                          
+--------------------------------------+---------------------------------------------+--------+----------+-------+---------+------------------------------+                                            
|                 UUID                 |                    TYPE                     | STATUS | SEVERITY | COUNT | PROJECT |          LAST SEEN           |                                            
+--------------------------------------+---------------------------------------------+--------+----------+-------+---------+------------------------------+                                            
| 4cd6ebfe-5e9b-44d2-b9f2-571cab5c1017 | AppArmor support has been disabled          | NEW    | LOW      | 1     |         | Nov 13, 2021 at 9:28pm (UTC) |                                            
+--------------------------------------+---------------------------------------------+--------+----------+-------+---------+------------------------------+                                            
| 8c35852b-5bc2-4d48-a92d-99367edb7f16 | Couldn't find the CGroup hugetlb controller | NEW    | LOW      | 1     |         | Nov 13, 2021 at 9:28pm (UTC) |                                            
+--------------------------------------+---------------------------------------------+--------+----------+-------+---------+------------------------------+                                            
| 754c45bf-9109-4298-98c3-ed4884648d92 | Couldn't find the CGroup blkio.weight       | NEW    | LOW      | 1     |         | Nov 13, 2021 at 9:28pm (UTC) |                                            
+--------------------------------------+---------------------------------------------+--------+----------+-------+---------+------------------------------+ 

Acknowledge these warnings.

$ sudo -i lxc warning acknowledge local:4cd6ebfe-5e9b-44d2-b9f2-571cab5c1017
$ sudo -i lxc warning acknowledge local:8c35852b-5bc2-4d48-a92d-99367edb7f16
$ sudo -i lxc warning acknowledge local:754c45bf-9109-4298-98c3-ed4884648d92
$ sudo -i lxc warning list
+------+------+--------+----------+-------+---------+-----------+
| UUID | TYPE | STATUS | SEVERITY | COUNT | PROJECT | LAST SEEN |
+------+------+--------+----------+-------+---------+-----------+

Basic usage

Search for Debian Bullseye images.

$ sudo -i lxc image list images: architecture=armhf os=Debian release=bullseye
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|             ALIAS              | FINGERPRINT  | PUBLIC |              DESCRIPTION               | ARCHITECTURE |   TYPE    |  SIZE   |          UPLOAD DATE          |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
| debian/11/armhf (3 more)       | feee44c6e671 | yes    | Debian bullseye armhf (20211113_05:45) | armv7l       | CONTAINER | 76.13MB | Nov 13, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
| debian/11/cloud/armhf (1 more) | e223fe0d38d2 | yes    | Debian bullseye armhf (20211113_05:38) | armv7l       | CONTAINER | 93.33MB | Nov 13, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 01cabc9c7479 | yes    | Debian bullseye armhf (20211111_05:39) | armv7l       | CONTAINER | 68.20MB | Nov 11, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 4f0fe70ee58c | yes    | Debian bullseye armhf (20211113_05:45) | armv7l       | CONTAINER | 67.01MB | Nov 13, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 6b22688e95a4 | yes    | Debian bullseye armhf (20211112_05:38) | armv7l       | CONTAINER | 93.33MB | Nov 12, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 9c202c8d2d6c | yes    | Debian bullseye armhf (20211111_05:39) | armv7l       | CONTAINER | 76.13MB | Nov 11, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 09b709e4b991 | yes    | Debian bullseye armhf (20211111_05:41) | armv7l       | CONTAINER | 93.33MB | Nov 11, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 878c19633be3 | yes    | Debian bullseye armhf (20211112_05:38) | armv7l       | CONTAINER | 84.34MB | Nov 12, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 9171e38c4ffb | yes    | Debian bullseye armhf (20211111_05:41) | armv7l       | CONTAINER | 82.96MB | Nov 11, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | 1873486d24b4 | yes    | Debian bullseye armhf (20211113_05:38) | armv7l       | CONTAINER | 84.58MB | Nov 13, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | b3c977d51b5d | yes    | Debian bullseye armhf (20211112_05:38) | armv7l       | CONTAINER | 76.13MB | Nov 12, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+
|                                | eb6f8bf12e8f | yes    | Debian bullseye armhf (20211112_05:38) | armv7l       | CONTAINER | 68.27MB | Nov 12, 2021 at 12:00am (UTC) |
+--------------------------------+--------------+--------+----------------------------------------+--------------+-----------+---------+-------------------------------+

Inspect specific image.

$ sudo -i lxc image  info images:debian/11/armhf
Size: 76.13MB
Architecture: armv7l
Type: container
Public: yes
Timestamps:
    Created: 2021/11/13 00:00 UTC
    Uploaded: 2021/11/13 00:00 UTC
    Expires: never
    Last used: never
Properties:
    serial: 20211113_05:45
    variant: default
    type: squashfs
    os: Debian
    architecture: armhf
    release: bullseye
    description: Debian bullseye armhf (20211113_05:45)
Aliases:
    - debian/bullseye/default/armhf
    - debian/11/default/armhf
    - debian/bullseye/armhf
    - debian/11/armhf
Cached: no
Auto update: disabled
Profiles: []

Copy image to local storage.

$ sudo -i lxc image copy images:debian/11/armhf local:
Image copied successfully!  

Display local images.

$ sudo -i lxc image list local:
+-------+--------------+--------+----------------------------------------+--------------+-----------+---------+------------------------------+
| ALIAS | FINGERPRINT  | PUBLIC |              DESCRIPTION               | ARCHITECTURE |   TYPE    |  SIZE   |         UPLOAD DATE          |
+-------+--------------+--------+----------------------------------------+--------------+-----------+---------+------------------------------+
|       | feee44c6e671 | no     | Debian bullseye armhf (20211113_05:45) | armv7l       | CONTAINER | 76.13MB | Nov 13, 2021 at 9:57pm (UTC) |
+-------+--------------+--------+----------------------------------------+--------------+-----------+---------+------------------------------+

Beware, these images will be stored using local filesystem.

$ sudo ls /var/snap/lxd/common/lxd/images/
feee44c6e6715b24593a2f0c85a577a5356816504d3c230a7200ad9525215569  feee44c6e6715b24593a2f0c85a577a5356816504d3c230a7200ad9525215569.rootf

Create an alias.

$ sudo -i lxc image alias create local:debian/11 feee44c6e671

Display local images, again.

$ sudo -i lxc image list local:
+-----------+--------------+--------+----------------------------------------+--------------+-----------+---------+------------------------------+
|   ALIAS   | FINGERPRINT  | PUBLIC |              DESCRIPTION               | ARCHITECTURE |   TYPE    |  SIZE   |         UPLOAD DATE          |
+-----------+--------------+--------+----------------------------------------+--------------+-----------+---------+------------------------------+
| debian/11 | feee44c6e671 | no     | Debian bullseye armhf (20211113_05:45) | armv7l       | CONTAINER | 76.13MB | Nov 13, 2021 at 9:57pm (UTC) |
+-----------+--------------+--------+----------------------------------------+--------------+-----------+---------+------------------------------+

Create an instance with random name.

$ sudo -i lxc launch local:debian/11
Creating the instance
Instance name is: sharing-arachnid
Starting sharing-arachnid

List instances.

$ sudo -i lxc list
+------------------+---------+---------------------+------+-----------+-----------+
|       NAME       |  STATE  |        IPV4         | IPV6 |   TYPE    | SNAPSHOTS |
+------------------+---------+---------------------+------+-----------+-----------+
| sharing-arachnid | RUNNING | 10.114.53.14 (eth0) |      | CONTAINER | 0         |
+------------------+---------+---------------------+------+-----------+-----------+

Update fstab inside created instance to use a virtual memory filesystem in the same way as DietPi does.

$ cat << EOF | sudo -i lxc exec sharing-arachnid tee /etc/fstab
tmpfs /tmp tmpfs size=64M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=64M,noatime,lazytime,nodev,nosuid,mode=1777
EOF

Restart instance.

$ sudo -i lxc restart sharing-arachnid

Create override for root disk size.

$ sudo -i lxc config device override sharing-arachnid root size=20GB
Device root overridden for sharing-arachnid

Alter root disk size override.

$ sudo -i lxc config device set sharing-arachnid root size=30GB

Define CPU limit.

$ sudo -i lxc config set sharing-arachnid limits.cpu 2

Define memory limit.

$ sudo -i lxc config set sharing-arachnid limits.memory 256MB

Execute commands inside a running container.

$ sudo -i lxc exec sharing-arachnid -- df -h /
Filesystem                                     Size  Used Avail Use% Mounted on
/dev/lxd_storage/containers_sharing--arachnid   28G  340M   26G   2% /
$ sudo -i lxc exec sharing-arachnid -- free -h
               total        used        free      shared  buff/cache   available
Mem:           244Mi       6.0Mi       210Mi       7.0Mi        26Mi       237Mi
Swap:             0B          0B          0B
$ sudo -i lxc exec sharing-arachnid -- cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 1
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

Display container details.

$ sudo -i lxc info sharing-arachnid
Name: sharing-arachnid
Status: RUNNING
Type: container
Architecture: armv7l
PID: 5566
Created: 2021/11/13 23:09 CET
Last Used: 2021/11/13 23:15 CET

Resources:
  Processes: 9
  Disk usage:
    root: 331.59MiB
  CPU usage:
    CPU usage (in seconds): 1
  Memory usage:
    Memory (current): 32.72MiB
    Memory (peak): 33.71MiB
  Network usage:
    eth0:
      Type: broadcast
      State: UP
      Host interface: veth92ba84c3
      MAC address: 00:16:3e:22:62:53
      MTU: 1500
      Bytes received: 756B
      Bytes sent: 1.68kB
      Packets received: 3
      Packets sent: 15
      IP addresses:
        inet:  10.114.53.14/28 (global)
        inet6: fe80::216:3eff:fe22:6253/64 (link)
    lo:
      Type: loopback
      State: UP
      MTU: 65536
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0
      IP addresses:
        inet:  127.0.0.1/8 (local)
        inet6: ::1/128 (local)

Install dnsutils to locally use DNS names.

$ sudo apt install dnsutils
$ dig sharing-arachnid.lxd +short @10.114.53.1 
10.114.53.14

Delete instance.

$ sudo -i lxc delete --force  sharing-arachnid