Nynaeve

Note: If you are just looking for the high speed VMware kernel debugging program, then you can find it here. This post series outlines the basic design principles behind VMKD.

One of the interesting talks at Blue Hat was one about virtualization and security. There’s a lot of good stuff that was touched on (such as the fact that VMware still implements a hub, meaning that VMs on the same VMnet can still sniff eachother’s traffic).

Anyways, watching the talk got me thinking again about how much kernel debugging VMs is a slow and painful experience today, especially if you’ve used 1394 debugging frequently.

While kd-over-1394 is quite fast (as is local kd), you can’t do that in any virtualization software that I’m aware of today (none of them virtualize 1394, and furthermore, as far as I know none of them even support USB2 debugging either, even VMware Workstation 6).

This means that if you’re kernel debugging a VM in today’s world, you’re pretty much left high and dry and have to use the dreaded virtual serial port approach. Because the virtual serial port has to act like a real serial port, it’s also slow, just like a real serial port (otherwise, timings get off and programs that talk to the serial port break all over the place). This means that although you might be debugging a completely local VM, you’re still throttled to serial port speeds (115200bps). Although it should certainly technically be possible to do better in a VM, none of the virtualization vendors support the virtual hardware required for the other, faster KD transports.

However, that got me thinking a bit. Windows doesn’t really need a virtual serial port or a virtual 1394 port to serve as a kernel debugging target because of any intrinsic, special property of serial or 1394 or USB2. Those interfaces are really just mechanisms to move bits from the target computer to the debugger computer and back again, while requiring minimal interaction with the rest of the system (it is important that the kernel debugger code in the target computer be as minimalistic and self-contained as possible or many situations where you just can’t debug code because it is used by the kernel debugger itself would start cropping up – this is why there isn’t a TCP transport for kernel debugging, among other things).

Now with a VM (as opposed to a physical computer), getting bits to and from an external kernel debugger and the kernel running in the VM is really quite easy. After all, the VM monitor can just directly read and write from the VM’s physical memory, just like that, without a need for indirecting through a real (or virtual) I/O interconnect interface.

So I got to be thinking that it should theoretically be possible to write a kernel debugger transport module that instead of talking to a serial, 1394, or USB2 port, talks to the local VMM and asks it to copy memory to and from the outside world (and thus the kernel debugger). After the data is safely out of the VM, it can be transported over to the kernel debugger (and back) with the mechanism of choice.

It turns out that Windows KD support is implemented in a way that is fairly conducive to this approach. The KD protocol is divided up into essentially two different parts. There’s the high level half, which is essentially a command set that allows the kernel debugger to request that the KD stub in the kernel perform an operation (like change the active register set, set a breakpoint, write memory, or soforth). The high level portion of the KD protocol sits on top of what I call the low level or (framing) portion of the KD protocol, which is a (potentially hardware dependant) transport interface that provides for reliable delivery of high level KD requests and responses between the KD program on a remote computer and the KD stub in the kernel of the target computer.

The low level KD protocol is abstracted out via a set of kernel debugger protocol modules (which are simple kernel mode DLLs) that are used by the kernel to talk to the various pieces of hardware that are supported for kernel debugging. For example, there is a module to talk to the serial port (kdcom.dll), and a module to talk to the 1394 controller (kd1394.dll).

These modules export a uniform API that essentially allows the kernel to request reliable (“mostly reliable”) transport of a high level KD request (say a notification that an exception has occured) from the kernel to the KD program, and back again.

This interface is fortunate from the perspective of someone who might want to, say, develop a high speed kernel debugger module for a VM running under a known VM monitor. Such a KD protocol module could take advantage of the fact that it knows that it’s running under a specific VM monitor, and use the VM monitor’s built-in VM exit / VM enter capabilities to quickly tell the VM monitor to copy data into and out of the VM. (Most VMs have some sort of “backdoor” interface for optimized drivers and enhanced guest capabilities, such as a way for the guest to tell the host when its mouse pointer has left the guest’s screen. For example, in the case of VMware, there is a “VMware Tools” program that you can install which provides this capability through a special “backdoor” interface to that allows the VM to request a “VM exit” for the purposes of having the VM monitor perform a specialized task.)

Next time: Examining the KD module interface, and more.

This entry was posted on Thursday, October 4th, 2007 at 2:08 am and is filed under Debugging, NT Internals, Windows. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.