How to define multiple disks inside Vagrant using VirtualBox provider

Define multiple disks inside Vagrant using VirtualBox provider. It is not immediately obvious how to achieve it, but everything you need is there.

Go to second solution to see how I solved this issue.

First solution – the experimental one

Use experimental disks feature to define multiple disks.

$ cat Vagrantfile 

Vagrant.configure("2") do |config|
  config.vm.box = "debian/bullseye64"

  config.vm.provider :virtualbox

  config.vm.disk :disk, size: "2GB", name: "extra_storage1"
  config.vm.disk :disk, size: "1GB", name: "extra_storage2"
end

This solution has many drawbacks. It is experimental and doesn’t work with every vagrant box. It will work when using Debian box, but not CentOS.

Debian operating system.

$ VAGRANT_EXPERIMENTAL=disks vagrant up

==> vagrant: You have requested to enabled the experimental flag with the following features:                     
==> vagrant:                                             
==> vagrant: Features:  disks
==> vagrant: 
==> vagrant: Please use with caution, as some of the features may not be fully
==> vagrant: functional yet. 
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'debian/bullseye64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'debian/bullseye64' version '11.20210228.1' is up to date...
==> default: Setting the name of the VM: st_default_1615930633316_46471
==> default: Fixed port collision for 22 => 2222. Now on port 2200.
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 22 (guest) => 2200 (host) (adapter 1)
==> default: Configuring storage mediums...
    default: Disk 'extra_storage2' not found in guest. Creating and attaching disk to guest...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2200
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 6.0.0 r127566
    default: VirtualBox Version: 6.1
==> default: Configuring and enabling network interfaces...
==> default: Mounting shared folders...
    default: /vagrant => /home/milosz/Projects/vagrant/st 

==> default: Machine 'default' has a post `vagrant up` message. This is a message
==> default: from the creator of the Vagrantfile, and not from Vagrant itself:
==> default: 
==> default: Vanilla Debian box. See https://app.vagrantup.com/debian for help and bug reports

$ vagrant ssh -c lsblk

NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0  20G  0 disk 
`-sda1   8:1    0  20G  0 part /
sdb      8:16   0   2G  0 disk 
sdc      8:32   0   1G  0 disk 
Connection to 127.0.0.1 closed.

CentOS operating system.

$ VAGRANT_EXPERIMENTAL=disks vagrant up

==> vagrant: You have requested to enabled the experimental flag with the following features:
==> vagrant: 
==> vagrant: Features:  disks
==> vagrant: 
==> vagrant: Please use with caution, as some of the features may not be fully
==> vagrant: functional yet.
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'centos/7'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'centos/7' version '2004.01' is up to date...
==> default: Setting the name of the VM: st_default_1615930185225_39920
==> default: Fixed port collision for 22 => 2222. Now on port 2200.
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 22 (guest) => 2200 (host) (adapter 1)
==> default: Configuring storage mediums...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["storageattach", "a0f45474-df9e-4c6f-b1ed-326965b83525", "--storagectl", "SATA Controller", "--port", "0", "--device", "0", "--type", "hdd", "--medium", "/home/milosz/VirtualBox VMs/bullseye_default_1615923090380_12623/extra_storage1.vdi", "--comment", "This disk is managed externally by Vagrant. Removing or adjusting settings could potentially cause issues with Vagrant."]

Stderr: VBoxManage: error: Could not find a controller named 'SATA Controller'

The situation is obvious as this solution requires a specific box configuration.

Second solution – the proper one

This is where it gets interesting. This solution is universal. It will work with Debian, Ubuntu, and CentOS as it uses an additional storage controller and disks created using a trigger before the up operation.

I will save you some time, do not try to create disks inside the VirtualBox provider block as it is evaluated multiple times, so it is prone to unexpected race conditions. Also, there are cases where it does work properly, but not the way you would expect.

$ cat Vagrantfile

require 'fileutils'

# file operations needs to be relative to this file
VAGRANT_ROOT = File.dirname(File.expand_path(__FILE__))

# directory that will contain VDI files
VAGRANT_DISKS_DIRECTORY = "disks"

# controller definition
VAGRANT_CONTROLLER_NAME = "Virtual I/O Device SCSI controller"
VAGRANT_CONTROLLER_TYPE = "virtio-scsi"

# define disks
# The format is filename, size (GB), port (see controller docs)
local_disks = [
  { :filename => "disk1", :size => 1, :port => 5 },
  { :filename => "disk2", :size => 2, :port => 6 },
  { :filename => "disk3", :size => 1, :port => 25 }
]

Vagrant.configure("2") do |config|
  config.vm.box = "debian/bullseye64"
  #config.vm.box = "centos/7"
  #config.vm.box = "ubuntu/groovy64"

  disks_directory = File.join(VAGRANT_ROOT, VAGRANT_DISKS_DIRECTORY)

  # create disks before "up" action
  config.trigger.before :up do |trigger|
    trigger.name = "Create disks"
    trigger.ruby do
      unless File.directory?(disks_directory)
        FileUtils.mkdir_p(disks_directory)
      end
      local_disks.each do |local_disk|
        local_disk_filename = File.join(disks_directory, "#{local_disk[:filename]}.vdi")
        unless File.exist?(local_disk_filename)
          puts "Creating \"#{local_disk[:filename]}\" disk"
          system("vboxmanage createmedium --filename #{local_disk_filename} --size #{local_disk[:size] * 1024} --format VDI")
        end
      end
    end
  end

  # create storage controller on first run
  unless File.directory?(disks_directory)
    config.vm.provider "virtualbox" do |storage_provider|
      storage_provider.customize ["storagectl", :id, "--name", VAGRANT_CONTROLLER_NAME, "--add", VAGRANT_CONTROLLER_TYPE, '--hostiocache', 'off']
    end
  end

  # attach storage devices
  config.vm.provider "virtualbox" do |storage_provider|
    local_disks.each do |local_disk|
      local_disk_filename = File.join(disks_directory, "#{local_disk[:filename]}.vdi")
      unless File.exist?(local_disk_filename)
        storage_provider.customize ['storageattach', :id, '--storagectl', VAGRANT_CONTROLLER_NAME, '--port', local_disk[:port], '--device', 0, '--type', 'hdd', '--medium', local_disk_filename]
      end
    end
  end

  # cleanup after "destroy" action
  config.trigger.after :destroy do |trigger|
    trigger.name = "Cleanup operation"
    trigger.ruby do
      # the following loop is now obsolete as these files will be removed automatically as machine dependency
      local_disks.each do |local_disk|
        local_disk_filename = File.join(disks_directory, "#{local_disk[:filename]}.vdi")
        if File.exist?(local_disk_filename)
          puts "Deleting \"#{local_disk[:filename]}\" disk"
          system("vboxmanage closemedium disk #{local_disk_filename} --delete")
        end
      end
      if File.exist?(disks_directory)
        FileUtils.rmdir(disks_directory)
      end
    end
  end
end

Start box.

$ vagrant up

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Running action triggers before up ...
==> default: Running trigger: Create disks...
Creating "disk1" disk
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 30580ca1-b346-47d6-bcc9-09922b66b972
Creating "disk2" disk
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 91548a05-51e6-4345-bb6b-2f2cd12dabd0
Creating "disk3" disk
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Medium created. UUID: 13423cda-4934-49a2-a690-112800bf76f0
==> default: Importing base box 'debian/bullseye64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'debian/bullseye64' version '11.20210228.1' is up to date...
==> default: Setting the name of the VM: storage_default_1615931757454_98076
==> default: Fixed port collision for 22 => 2222. Now on port 2200.
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2200 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2200
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 6.0.0 r127566
    default: VirtualBox Version: 6.1
==> default: Mounting shared folders...
    default: /vagrant => /home/milosz/Projects/vagrant/storage

==> default: Machine 'default' has a post `vagrant up` message. This is a message
==> default: from the creator of the Vagrantfile, and not from Vagrant itself:
==> default: 
==> default: Vanilla Debian box. See https://app.vagrantup.com/debian for help and bug reports

Inspect disks inside it.

$ vagrant ssh -c lsblk

NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0   1G  0 disk 
sdb      8:16   0   2G  0 disk 
sdc      8:32   0   1G  0 disk 
sdd      8:48   0  20G  0 disk 
`-sdd1   8:49   0  20G  0 part /
Connection to 127.0.0.1 closed.

Stop box.

$ vagrant halt

==> default: Attempting graceful shutdown of VM...

Start box again. Disks are left alone as expected.

$ vagrant up

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Running action triggers before up ...
==> default: Running trigger: Create disks...
==> default: Checking if box 'debian/bullseye64' version '11.20210228.1' is up to date...
==> default: Clearing any previously set forwarded ports...
==> default: Fixed port collision for 22 => 2222. Now on port 2200.
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2200 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2200
    default: SSH username: vagrant
    default: SSH auth method: private key
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default: 
    default: Guest Additions Version: 6.0.0 r127566
    default: VirtualBox Version: 6.1
==> default: Mounting shared folders...
    default: /vagrant => /home/milosz/Projects/vagrant/storage
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.

==> default: Machine 'default' has a post `vagrant up` message. This is a message
==> default: from the creator of the Vagrantfile, and not from Vagrant itself:
==> default: 
==> default: Vanilla Debian box. See https://app.vagrantup.com/debian for help and bug reports

Destroy box. Remove disks.

$ vagrant destroy -f

==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...
==> default: Running action triggers after destroy ...
==> default: Running trigger: Cleanup operation...