Getting Hyper-V to work with Gen 2, EFI Linux VMs

So I recently ran into an issue at work where Red Hat Linux 8, Oracle Linux 8, CentOS 7, and Ubuntu machines were refusing to boot Gen 2, EFI virtual machines on Hyper-V that were previously on another popular hypervisor. These VMs would no longer boot correctly when migrated over. Long story short, it turns out some very important kernel modules are not included in initrd/initramfs by default. Microsoft used to provide something called Linux Integration Services that you would have to install on Red Hat Enterprise Linux 6 or CentOS 6 in order to get these kernel modules and get Hyper-V support. ("Support" in this case means modules that would facilitate host hypervisor -> guest OS communications.)

At some point, the 3.x line of the Linux kernel started including these Hyper-V modules natively, and this now carries over into Linux kernels 4.x and 5.x. The bad news (for Microsoft/Linux integration advocates) is that these are not enabled by default (which thankfully keeps the Linux kernel lean), but the good news is that you can start including these by default on your existing hypervisor infrastructure to ensure that migrations to Microsoft Hyper-V give you Linux VMs that will boot and understand the hypervisor's VM, storage and networking bus for guest OSes.

Here is a summary of the kernel modules you need:

Let's get started!

To do this on RHEL-based systems (see Ubuntu below), we use our old friend dracut to ensure the modules are included in your initramfs and thus get loaded as part of your system initialization:

# You should see nine (9) modules. Use the scroll bar to scroll right. 
$ sudo dracut -fv --add-drivers="hv_vmbus hv_utils hv_balloon hv_storvsc hv_netvsc hyperv-keyboard hid-hyperv hyperv_fb hyperv_drm" --regenerate-all

Of course, this only applies the change to all currently installed kernels. To get this to be included on ALL Linux kernel updates moving forward, add the add_drivers directive to a file under /etc/dracut.conf.d/. Any file name ending in .conf will do. Enforce the existence of this file on all hosts that you intend to have on Hyper-V virtual infrastructure with Puppet or Ansible for good measure.

$ vi /etc/dracut.conf.d/hv-modules.conf 

add_drivers+=" hv_vmbus hv_utils hv_balloon hv_storvsc hv_netvsc hyperv-keyboard hid-hyperv hyperv_fb hyperv_drm "

Note the spaces after and before the start and end quotes respectively. Not including this will guarantee that dracut will hate you forever.

NOTE: If you're an Ubuntu user, you'll be used to using the update-initramfs command to do the above. To include this there, edit /etc/initramfs-tools/modules

# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
hv_vmbus
hv_utils
hv_balloon
hv_storvsc
hv_netvsc
hyperv-keyboard
hid-hyperv
hyperv_fb
hyperv_drm

Now run:

# Use -k 'all' or target a single kernel
$ sudo update-initramfs -u -k 'all'

Congratulations! You are done.

FAQ (okay, maybe not frequently, but how about anticipatory?):

Q: Is it hyperv_keyboard or hyperv-keyboard? Same for hid-hyperv.

A: Yeah, I know. It's weird. I've seen the references to hyperv_keyboad, but the kernel source C file is hyperv-keyboard.c (see here) and the module that's found under your /lib/modules/$(uname -r) should read hyperv-keyboard.ko when unxz'd.)

Q: Is the list of nine Hyper-V modules exhaustive? Do I need to have any more?

A: Unfortunately it's not completely exhaustive. A couple more exist (e.g. pci-hyperv and hyperv_drm - which I almost excluded - for leveraging GPUs on the hypervisor) that could be useful for your use case and organization to support additional buses and capabilities. I've only covered the ones that prevent you from being completely dead in the water. :)

References:

https://www.microsoft.com/en-us/download/details.aspx?id=55106

https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/manage-hyper-v-integration-services

https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/5/html/virtualization/chap-virtualization-hyperv#sect-Virtualization-HyperV-Drivers