Categories
SysOps

How to limit the number of available processors

Use boot parameters to limit the number of available processors.

There are two distinct boot options that affect the number of available processors.

The nr_cpus option which can be used as a hard limit.
The maxcpus option which can be used as a soft limit and altered later at your own discretion.

Generally, you want to use the first one when you are limited by some kind of licensing terms.

Preliminary information

I will use the following setup.

$ lscpu 
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              1
Core(s) per socket:              32
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           60
Model name:                      Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
Stepping:                        3
CPU MHz:                         2893.298
BogoMIPS:                        5786.59
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       1 MiB
L1i cache:                       1 MiB
L2 cache:                        8 MiB
L3 cache:                        192 MiB
NUMA node0 CPU(s):               0-31
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Unknown: Dependent on hypervisor status
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm con
                                 stant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 movbe popcnt a
                                 es xsave avx rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase avx2 invpcid md_clear flush_l1d

Hard limit

HARD limit the number of available processors to 16.

$ cat  /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=0 biosdevname=0"
GRUB_CMDLINE_LINUX="consoleblank=0 nr_cpus=16"

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

Update Grub.

$ sudo update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.10.0-6-amd64
Found initrd image: /boot/initrd.img-5.10.0-6-amd64
Found linux image: /boot/vmlinuz-5.10.0-3-amd64
Found initrd image: /boot/initrd.img-5.10.0-3-amd64
done

Reboot the operating system.

$ sudo reboot

Display information about the processors.

$ lscpu 
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          16
On-line CPU(s) list:             0-15
Thread(s) per core:              1
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           60
Model name:                      Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
Stepping:                        3
CPU MHz:                         2893.298
BogoMIPS:                        5786.59
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        4 MiB
L3 cache:                        96 MiB
NUMA node0 CPU(s):               0-15
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Unknown: Dependent on hypervisor status
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm con
                                 stant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 movbe popcnt a
                                 es xsave avx rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase avx2 invpcid md_clear flush_l1d

In this example you went down from 32 to 16 processors. This option is very useful when you are limited by licensing restrictions.

Soft limit

HARD limit the number of available processors to 8 and simultaneously change SOFT limit the number of available processors to 4.

$ cat  /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=0 biosdevname=0"
GRUB_CMDLINE_LINUX="consoleblank=0 nr_cpus=8 maxcpus=4"

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

Update Grub.

$ sudo update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.10.0-6-amd64
Found initrd image: /boot/initrd.img-5.10.0-6-amd64
Found linux image: /boot/vmlinuz-5.10.0-3-amd64
Found initrd image: /boot/initrd.img-5.10.0-3-amd64
done

Reboot the operating system.

$ sudo reboot

Display information about the processors.

$ lscpu 
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-3
Off-line CPU(s) list:            4-7
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           60
Model name:                      Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
Stepping:                        3
CPU MHz:                         2893.298
BogoMIPS:                        5786.59
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        1 MiB
L3 cache:                        24 MiB
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Unknown: Dependent on hypervisor status
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm con
                                 stant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 movbe popcnt a
                                 es xsave avx rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase avx2 invpcid md_clear flush_l1d

This time you went down from 32 processors to 8 available processors, but only 4 online.

$ find /sys/devices/system/cpu/cpu?/online -exec bash -c "echo -n '{}: '; cat {}"  \;
/sys/devices/system/cpu/cpu1/online: 1
/sys/devices/system/cpu/cpu2/online: 1
/sys/devices/system/cpu/cpu3/online: 1
/sys/devices/system/cpu/cpu4/online: 0
/sys/devices/system/cpu/cpu5/online: 0
/sys/devices/system/cpu/cpu6/online: 0
/sys/devices/system/cpu/cpu7/online: 0

Note, you cannot disable cpu0.

You can enable (1) or disable (0) these processors.

$ echo 1 | sudo tee /sys/devices/system/cpu/cpu6/online
$ lscpu 
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-3,6
Off-line CPU(s) list:            4,5,7
Thread(s) per core:              1
Core(s) per socket:              5
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           60
Model name:                      Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
Stepping:                        3
CPU MHz:                         2893.298
BogoMIPS:                        5786.59
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       160 KiB
L1i cache:                       160 KiB
L2 cache:                        1.3 MiB
L3 cache:                        30 MiB
NUMA node0 CPU(s):               0-3,6
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Unknown: Dependent on hypervisor status
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm con
                                 stant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 movbe popcnt a
                                 es xsave avx rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase avx2 invpcid md_clear flush_l1d

Additional notes

Inspect The kernel’s command-line parameters for more details.

        nr_cpus=        [SMP] Maximum number of processors that an SMP kernel
                        could support.  nr_cpus=n : n >= 1 limits the kernel to
                        support 'n' processors. It could be larger than the
                        number of already plugged CPU during bootup, later in
                        runtime you can physically add extra cpu until it reaches
                        n. So during boot up some boot time memory for per-cpu
                        variables need be pre-allocated for later physical cpu
                        hot plugging.
        maxcpus=        [SMP] Maximum number of processors that an SMP kernel
                        will bring up during bootup.  maxcpus=n : n >= 0 limits
                        the kernel to bring up 'n' processors. Surely after
                        bootup you can bring up the other plugged cpu by executing
                        "echo 1 > /sys/devices/system/cpu/cpuX/online". So maxcpus
                        only takes effect during system bootup.
                        While n=0 is a special case, it is equivalent to "nosmp",
                        which also disables the IO APIC.

Inspect boot parameters.

$ cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.10.0-6-amd64 root=UUID=1f09a8a9-d1d4-444e-aac9-61a1af04f221 ro consoleblank=0 nr_cpus=8 maxcpus=4 elevator=noop scsi_mod.use_blk_mq=Y net.ifnames=0 biosdevname=0

I should probably write CPUs instead of processors to keeps things in line with lscpu output.