When I recently got the chance to install a new VxRail cluster for a EUC project but ran into a “Not enough space on the bootbank” error. I wrote this blog on how I approached and solved the problem.
The cluster was used for a Windows 10 VDI project using NSX-T micro-segmentation in combination with vGPU for graphical acceleration.
Some info on the VxRail cluster and setup:
4x V570F nodes with NVIDIA M10
NSX-T 2.5
Horizon 7.10 (LSB release)
The initial deployment went without a glitch, everything was up and running perfectly. VxRail Manager had deployed and configured all ESXi hosts and vCenter server correctly. As well as the cluster,vSAN, and VxRail status were healthy.
So to finalize the installation of the cluster, I only had to install NVIDIA and NSX on each ESXi node.
I initially was able to install the NSX vibs without any issues as you can see on the screenshot below:
But continuing with the NVIDIA VIB, I encountered an issue where the ESX did not have enough space free on the bootbank. It gave me the following error: The pending transaction requires 245 MB free space, however the maximum supported size is 239 MB.
VMware does have a KB article on the issue.
Resolution: remove non-required vibs
How to approach removing non-required vibs:
To illustrate how to approach the resolution VMware states, I wanted to give you a step by step example. As this may seem like a daunting task in figuring out which are active and non-active vibs on your ESXi host.
Note: Please do not remove vibs without any validation, this can have a negative impact on the stability of your system. Validate twice, remove once!
1. Inventory in-use drivers
Before we start removing any drivers we need to get a better look at which drivers are in use and which aren’t.
This can be done with the following commands:
esxcfg-scsidevs -a
This will provide an overview of all SCSI devices and which driver they are using. This can be seen in the picture below, the second column.
esxcli network nic list
Provides an overview of all detected network interfaces in the host and the active drivers.
These 2 commands will provide an overview of all drivers in use. Meaning does NOT remove these, or you will have some major issues.
2. Next, we can proceed with removing non-essential drivers.
These are most of the time drivers for additional network / Fiber Channel cards that are shipped with every ESXi deployment.
Like in our example. We only have Intel NICs, so we could remove the Broadcom (BCM), Mellanox (MEL).
I did not remove the DELL based drivers as we where using VxRail, this requires the PTA agent and ISM service to be installed.
This can be similar for other vendors like Cisco, HPE,…
To remove the drivers, the following command needs to be used:
esxcli software vib remove -n vib_name
As you can see we removed both Mellanox drivers: NMLX5-core and rdma from the ESXi bootbank.
The remove isn’t fully committed as long as you don’t reboot the host.
So removing multiple drivers can better be done at once, to prevent multiple reboots.
Result
In the end, I was able to remove enough non-essential drivers from the ESXi host, to be able to install the NVIDIA VIB.
I hope this blog post has helped you resolve the “not enough space on the bootbank” error on VxRail or ESXi.
Would you like to read some more: VCAP7-DTM design study guide