Paul Mekhedjian

CUDA: Unified and Managed Memory

2025-02-23T23:10:00-08:00

TL;DR / skip reading and go here: https://github.com/pmekhedjian/cuda-device-properties

These are CUDA notes for what will eventually be a comprehensive guide for myself on referring to memory management for programming in CUDA C and CUDA Fortran. Why CUDA, you might ask? Well, if you have an Nvidia GPU sitting in your desktop or server right now, you're sitting on a lot of untapped potential. Used intelligently and cleverly, GPU acceleration has the potential to significantly speed up your code. It's quite simple, provided you can understand and access the inner workings of the GPU via the CUDA runtime API.

I won't lie to you. To code CUDA correctly, you need to have a computer science major's level of understanding of software development, memory management, and knowledge of how code defines and accesses information in memory during runtime. As an SRE with a physics background, this doesn't come naturally to me either. Frankly, I suck at coding, but I'm willing to get my hands dirty and learn something new along the way.

A friend recommended I check out "unified memory" in CUDA as something that could help me more easily juggle the paradigm of host vs. device memory. In CUDA speak, the word "host" refers to anything in system memory (RAM), number-crunched by the CPU, and "device" is anything relating to the GPU's video memory (VRAM), number-crunched by the GPU cores themselves. This is my naive attempt at figuring this out, and I'm probably not going to get a lot right on the first go (so forgive me). I look forward to unpacking more as I learn.

As a physicist with exposure to Fortran and Python, it seemed natural to want to dive into a part of the pool I was more familiar with, and started looking at CUDA Fortran via samples referenced in the Nvidia HPC SDK, and then found writing(s) and posts by Greg Ruetsch, who also wrote the book on it. I became enamored. The more I read, the more I realized I should be learning both CUDA C and CUDA Fortran, as there are a ton of differences in the two languages regarding the CUDA implementations, and because they're both useful for what I'm interested in. Though superficially similar, Fortran and C/C++ are not one-to-one in how the CUDA API is exposed in their respective languages, and recognizing the nuances and limits of each would make me a better CUDA programmer in the long run.

Long story short, the CUDA C Programming Guide on the Nvidia website also documents a lot of useful constraints and definitions of the API from the C/C++ perspective. One major point is that of unified memory (see this post here as a starting point: Unified Memory for CUDA Beginners), which allows you to define, access, and utilize data that is usable by both the CPU (host) and GPU (device) compute, without having to ferry information back and forth.

I developed a short CUDA C++ and CUDA Fortran code that outputs what the current setting of the device parameters is in your Nvidia GPU. Is your system using basic CUDA managed memory, or does it have full CUDA unified memory support, to allow you to seamless use data across CPUs and GPUs on the system?

If you own a CUDA-capable Nvidia GPU, I invite you to clone this repo and try it out yourself. It does require you download and install the Nvidia HPC SDK in order to use nvfortran and nvcc, but it's free and a high-quality SDK you should have in your toolbox (instructions and code in repo):

$ git clone https://github.com/pmekhedjian/cuda-device-properties

Or visit the page here: https://github.com/pmekhedjian/cuda-device-properties

I do hope this piques your interest in CUDA and GPU computing!

Next up (probably): Why heterogenous memory management (HMM) in Linux will turn your CUDA Managed Memory GPU into a CUDA Unified Memory GPU. Is that really possible?!

References:

https://docs.nvidia.com/cuda/cuda-c-programming-guide/

https://developer.nvidia.com/blog/unified-memory-cuda-beginners

https://developer.nvidia.com/blog/easy-introduction-cuda-fortran

https://docs.nvidia.com/hpc-sdk/compilers/cuda-fortran-prog-guide

https://github.com/romerojosh/CUDA-Fortran-Tutorial

https://github.com/Koushikphy/Intro-to-CUDA-Fortran

Tips for Running Gen 2 EFI Linux VMs on Hyper-V

2025-01-25T22:22:00-08:00

So I recently ran into an issue at work where Red Hat Linux 8, Oracle Linux 8, CentOS 7, and Ubuntu machines were refusing to boot Gen 2, EFI virtual machines on Hyper-V that were previously on another popular hypervisor. These VMs would no longer boot correctly when migrated over. Long story short, it turns out some very important kernel modules are not included in initrd/initramfs by default. Microsoft used to provide something called Linux Integration Services that you would have to install on Red Hat Enterprise Linux 6 or CentOS 6 in order to get these kernel modules and get Hyper-V support. ("Support" in this case means modules that would facilitate host hypervisor -> guest OS communications.)

At some point, the 3.x line of the Linux kernel started including these Hyper-V modules natively, and this now carries over into Linux kernels 4.x and 5.x. The bad news (for Microsoft/Linux integration advocates) is that these are not enabled by default (which thankfully keeps the Linux kernel lean), but the good news is that you can start including these by default on your existing hypervisor infrastructure to ensure that migrations to Microsoft Hyper-V give you Linux VMs that will boot and understand the hypervisor's VM, storage and networking bus for guest OSes.

Here is a summary of the kernel modules you need:

Linux kernel module	Description
hv_vmbus	Host-guest communication
hv_utils	Host-guest support tools
hv_balloon	Dynamic memory expansion
hv_storvsc	IDE/SCSI storage driver
hv_netvsc	Network driver
hyperv-keyboard	Keyboard
hid-hyperv	USB mouse support
hyperv_fb	Video framebuffer
hyperv_drm	GPU passthrough

Let's get started!

To do this on RHEL-based systems (see Ubuntu below), we use our old friend dracut to ensure the modules are included in your initramfs and thus get loaded as part of your system initialization:

# You should see nine (9) modules. Use the scroll bar to scroll right. 
$ sudo dracut -fv --add-drivers="hv_vmbus hv_utils hv_balloon hv_storvsc hv_netvsc hyperv-keyboard hid-hyperv hyperv_fb hyperv_drm" --regenerate-all

Of course, this only applies the change to all currently installed kernels. To get this to be included on ALL Linux kernel updates moving forward, add the add_drivers directive to a file under /etc/dracut.conf.d/. Any file name ending in .conf will do. Enforce the existence of this file on all hosts that you intend to have on Hyper-V virtual infrastructure with Puppet or Ansible for good measure.

$ vi /etc/dracut.conf.d/hv-modules.conf 

add_drivers+=" hv_vmbus hv_utils hv_balloon hv_storvsc hv_netvsc hyperv-keyboard hid-hyperv hyperv_fb hyperv_drm "

Note the spaces after and before the start and end quotes respectively. Not including this will guarantee that dracut will hate you forever.

NOTE: If you're an Ubuntu user, you'll be used to using the update-initramfs command to do the above. To include this there, edit /etc/initramfs-tools/modules

# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
hv_vmbus
hv_utils
hv_balloon
hv_storvsc
hv_netvsc
hyperv-keyboard
hid-hyperv
hyperv_fb
hyperv_drm

Now run:

# Use -k 'all' or target a single kernel
$ sudo update-initramfs -u -k 'all'

Congratulations! You are done.

FAQ (okay, maybe not frequently, but how about anticipatory?):

Q: Is it hyperv_keyboard or hyperv-keyboard? Same for hid-hyperv.

A: Yeah, I know. It's weird. I've seen the references to hyperv_keyboad, but the kernel source C file is hyperv-keyboard.c (see here) and the module that's found under your /lib/modules/$(uname -r) should read hyperv-keyboard.ko when unxz'd.)

Q: Is the list of nine Hyper-V modules exhaustive? Do I need to have any more?

A: Unfortunately it's not completely exhaustive. A couple more exist (e.g. pci-hyperv and hyperv_drm - which I almost excluded - for leveraging GPUs on the hypervisor) that could be useful for your use case and organization to support additional buses and capabilities. I've only covered the ones that prevent you from being completely dead in the water. :)

Q: Why am I getting errors about hyperv_drm not building on my Ubuntu or CentOS 7 machine?

A: hyperv_drm is only available on Linux kernels 5.14 and above. See Wikipedia - Direct Rendering Manager.

References:

https://www.microsoft.com/en-us/download/details.aspx?id=55106

https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/manage-hyper-v-integration-services

https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/5/html/virtualization/chap-virtualization-hyperv#sect-Virtualization-HyperV-Drivers

FIPS 140-2 Compliance on Amazon Linux 2

2022-06-05T21:04:00-07:00

Enabling the Federal Information Processing Standard (FIPS) 140-2 on Amazon Linux 2 is incredibly simple. You would do this in order to comply with DISA STIGs, PCI DSS, NIST CMVP, FISMA, or FedRAMP standards, and would be one additional way to give your security posture on AWS a bit of a boost.

sudo yum install -y dracut-fips
sudo dracut -fv
sudo /sbin/grubby --update-kernel=ALL --args="fips=1"
sudo reboot

Better yet, you can replace dracut-fips with dracut-fips-aesni, to take advantage of and enable support for CPUs with AES New Instructions (AES-NI), supported by most Intel/AMD x64 CPUs released after 2008. This will aide in performance of decryption at boot/run time:

sudo yum install -y dracut-fips-aesni

Fortunately, dracut-fips will actually get installed along with it, as *-aesni is dependent on the parent package. Then, of course, it's good to verify that FIPS is enabled after you reboot:

sudo sysctl crypto.fips_enabled

This should return a value of 1. If a value of 0 is returned, something, like the best laid plans of mice and men, has gone awry or you haven't rebooted the system yet.

The referenced AWS blog can provide additional detail, but that sums it up. You're done! Simple, but would be ideal and pretty slick to place into user data (via AWS CloudFormation or HashiCorp Terraform)!

You should also compare your organization's cipher policy (via CIS, DISA STIG, etc.) and make those revisions in /etc/ssh/sshd_config for OpenSSH server.

References:

https://aws.amazon.com/blogs/publicsector/enabling-fips-mode-amazon-linux-2/

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html

SSL Certificates on Shared Hosting in Five Minutes

2022-02-10T19:20:00-08:00

Security... Ah, security...

There was a time when websites didn't need to have an SSL/TLS certificiate (colloquially SSL, TLS, and SSL/TLS are interchangable). If you're hosting an e-commerce platform, or are running API calls/interactions, you're obviously going to want to have an SSL certificate bound to your traffic. No excuses! As for the rest of us with private sites not selling anything, there's still no reason not to! (I'm talking to you, person who lets their users' experience on their website be suboptimal by having to bypass an invalid / missing cert message!)

If you're like me and are still paying for shared hosting, you don't really want to pay extra for certs (minimize ALL t3h c0stz). In all reality, I should switch over to one of those sweet S3 static websites on AWS for this site, but I like NameCheap and don't mind throwing a little money their way. Plus, they use Cloud Linux and I can't not support that. swoon

Disclaimer: I've only tested this on NameCheap, but it should work fine on other shared hosting, provided you have SSH access.

The following steps make use of the ACME protocol for certificate management. You can find the GitHub project (acme.sh) by acmesh-official used that makes it all possible here.

Pull up a terminal (either SSH or via cPanel). Curl and install:

$ curl https://get.acme.sh | sh

This will create a little home for itself in a local hidden dir

Next, source the environment variable shell script (this will ensure you can type acme.sh anywhere in your session):

$ source ~/.bashrc

Register your email (to stay posted on deadlines, lapses, and/or renewals with the ZeroSSL folks) acme.sh --register-account --accountemail email@example.com

Issue test certificate (note the --staging flag):

$ acme.sh --issue --webroot ~/public_html -d yourdomain.com --staging

Issue actual certificate (note the --force flag):

$ acme.sh --issue --webroot ~/public_html -d yourdomain.com --force

Deploy certificate using cPanel hook:

$ acme.sh --deploy --deploy-hook cpanel_uapi --domain yourdomain.com

Update (10/02/2022): In case the cpanel_uapi hook is temperamental, you can navigate to the SSL/TLS certificates list in cPanel and see if it appears there. If not, you can use SSH to navigate to the ~/.acme.sh/ directory and enter the file contents of the private and public certificates into the cPanel SSL/TLS tab manually.

You should be good. You should always verify that the certificate has been propagated with the full certificate chain using openssl:

$ openssl s_client -connect yourdomain.com:443

You're now the proud owner of a free ZeroSSL certificate that will re-new automatically! You can also ensure that the cronjob has "taken" by inspecting crontab"

$ crontab -l | grep acme.sh

If you have issues with HTTP -> HTTPS redirection, check out the second reference (shout out, dude) for more info!

References:

https://www.globalsign.com/en/blog/ssl-vs-tls-difference

https://dev.to/atomar/let-s-encrypt-ssl-certificate-in-namecheap-autorenewal-verified-working-using-acme-sh-4m7i

https://devops.egyan.space/applying-lets-encrypt-ssl-certificate-in-namecheap-in-2020-with-autorenewal-verified-working-using-acme-sh/

Packet Capture via Remote Virtual Interfaces (macOS)

2021-03-26T21:04:00-07:00

The simplest way to get output from a packet capture is via tcpdump, but the popularity of Wireshark and its GUI make it a very attractive option, too. My ultimate goal was to capture network traffic from an iPhone directly over USB, since iOS doesn't come with a native packet inspector, and because my home router isn't running OpenWrt (womp, womp) for me to shell from and get more granular logs there. Thankfully, Apple has developed a method to do this via Xcode for developers. By using the rvictl utility in CLI, you can associate the USB connection with a virtual network interface, but I was running into issues that numerous Google searches weren't able to remedy.

Spoiler alert: If you're running macOS v11.2 or greater, this bug has been fixed. If you wish to avoid an upgrade, you can find a way to upgrade Xcode to at least 12.2, where the ACTUAL fix is made (see below for link to the release notes)

To check your version of macOS, run the following in terminal:

$ sw_vers
ProductName:    macOS
ProductVersion: 11.1
BuildVersion:   20C69

Since the initial release of Big Sur 11.0.1 (first release for Intel Macs - 11.0 for Silicons), these have been the following releases:

macOS Big Sur 11.2.3 - Latest at time of writing
macOS Big Sur 11.2.2
macOS Big Sur 11.2.1
macOS Big Sur 11.2 (where Xcode 12.2 is released)
macOS Big Sur 11.1
macOS Big Sur 11.0.1

As described in the article by Andy Davies (referenced below), it's possible to link the iPhone via USB as a network device using Remote Virtual Interfaces (a la rvi -> rvictl - a CLI utility from Xcode):

$ rvictl -s <Unique Device Identifier - UDID>
Starting device <Unique Device Identifier - UDID> [FAILED]

If you keep receiving the above, you're most likely running lower than v11.2, an older version of Xcode, and/or aren't running com.apple.rpmuxd (see references below and check your process list via ps or top).

How did I this get resolved? I checked for updates and got the latest build of Big Sur, of course! :) If this is affecting you and your networking tasks, do consider upgrading.

After upgrade:

$ sw_vers
ProductName:    macOS
ProductVersion: 11.2.3
BuildVersion:   20D91

Starting device <Unique Device Identifier - UDID> [SUCCEEDED] with interface rvi0

You're now free to use tcpdump, tshark, or load up Wireshark where rvi0 will show up as a network interface where you can enjoy the wonderful world of packet inspection.

References:

https://developer.apple.com/documentation/network/recording_a_packet_trace/troubleshooting_packet_traces

https://developer.apple.com/forums/thread/655329

https://andydavies.me/blog/2019/12/12/capturing-and-decrypting-https-traffic-from-ios-apps/

https://useyourloaf.com/blog/remote-packet-capture-for-ios-devices/

Fluid Dynamics: Rayleigh-Bénard Convection

2021-02-16T21:04:00-08:00

One problem that piqued my interest early on was that of (you guessed it) Rayleigh-Bénard convection. This is really a rather simple experiment involving a fluid at rest between two solid parallel plates, one heated from below and the other either held at room temperature or cooled. When the temperature difference is low, heat transfers in a linear manner and eventually reaches equilibrium.

But what if you ramped up the heat? Heat rises and cooler mass elements will come down due to the battle between buoyancy and gravity as microchanges in the localized density profile occur (due to temperature variations). As the theory goes (read more on Wikipedia), this convection leads to characteristic rolls and potential hydrodynamic instabilities as the heat differential (i.e. the dimensionless Rayleigh number) increases.

In a course I took at UC Santa Cruz with Gary Glatzmaier in 2007 (Physics 227: Advanced Fluid Dynamics), we worked with a Fortran code that we developed over time, first starting with a linear version and built up to one with nonlinear effects. I've cataloged this code here. Glatzmaier also provides instructions in Appendix B of his book on how to produce movies from this code as well:

You can check out the Github repository for this here:

$ git clone https://github.com/pmekhedjian/thermalconvection2d.git

Refer to the README there to run this on your own system.

One magical aspect of this, is that normally you would need to have IDL (the Interactive Data Language) to parse and compile the .pro files, but fortunately, thanks to the GNU Data Language open source project, you won't need to purchase a very expensive license from the NV5 folks (the current maintainers of IDL). Although virtually unknown in some circles, physicists, astronomers, and remote sensing engineers & analysts have used IDL extensively for the last few decades.

$ git clone https://github.com/gnudatalanguage/gdl.git

Once installed (either via apt, yum, zypper or clean source, you simply need to run gdl in the directory where the Fortran code you compiled earlier produced its PPM image files:

$ gdl

GDL> .r loadct.pro reverse.pro write_ppm.pro cmcongrid.pro ka2.pro
% Compiled module: LOADCT.
% Compiled module: REVERSE.
% Compiled module: WRITE_PPM.
% Compiled module: CMCONGRID.
% Compiled module: LOADDAT.
% Compiled module: MOV.

GDL> mov,1,'mov.000001'