Enabling the local kernel debugger on Vista RTM

If you’re a kernel developer, and you’ve upgraded to Vista, then one of the changes that you may have noticed is that you can’t perform local kernel debugging anymore.

This is true even if you elevate WinDbg. If you try, you’ll get an error message stating that the debugger failed to get KD version information (error 5), which corresponds to the Win32 ERROR_ACCESS_DENIED error code.

This is due to a change from Vista RC2 to Vista RTM, where the kernel function responsible for supporting much of the local KD functionality in WinDbg (KdSystemDebugControl) was altered to require the system to be booted with /DEBUG. This is apparent if we compare RC2 to RTM.

RC2 has the following check in KdSystemDebugControl (one comparison against KdpBootedNodebug):

nt!KdSystemDebugControl:
push    0F4h
push    81841938
call    nt!_SEH_prolog4
xor     ebx,ebx
mov     dword ptr [ebp-28h],ebx
mov     dword ptr [ebp-20h],ebx
mov     dword ptr [ebp-24h],ebx
cmp     byte ptr [nt!KdpBootedNodebug],bl
je      nt!KdSystemDebugControl+0x2c ; Success
mov     eax,0C0000022h ; STATUS_ACCESS_DENIED

On Vista RTM, two additional checks were added against nt!KdPitchDebugger and nt!KdDebuggerEnabled (disregard the fact that the RTM disassembly is from the x64 version; both the x86 and x64 Vista versions have the same checks):

nt!KdSystemDebugControl:
mov     qword ptr [rsp+8],rbx
mov     qword ptr [rsp+10h],rdi
push    r12
sub     rsp,170h
mov     r10,rdx
and     dword ptr [rsp+44h],0
and     qword ptr [rsp+48h],0
and     qword ptr [rsp+50h],0
cmp     byte ptr [nt!KdpBootedNodebug)],0
jne     nt!KdSystemDebugControl+0x8b7 ; Fail
cmp     byte ptr [nt!KdPitchDebugger],0
jne     nt!KdSystemDebugControl+0x8b7 ; Fail
cmp     byte ptr [nt!KdDebuggerEnabled],0
je      nt!KdSystemDebugControl+0x8b7 ; Fail

The essence of these checks is that you need to be booted with /DEBUG enabled in order for local kernel debugging to work.

There is a simple way to accomplish this, however, without the usual painful aspects of having a kernel debugger attached (e.g. breaking on user mode exceptions or breakpoints).

All you have to do is enable kernel debugging, and then disable user mode exception handling. This requires the following options to be set via BCDEdit.exe, the Vista boot configuration database manager:

  1. bcdedit /debug on. This enables kernel debugging for the booted OS configuration.
  2. bcdedit /dbgsettings <type> /start disable /noumex (where type corresponds to a usable KD type on your computer, such as 1394). This disables user mode exception handling for the kernel debugger. You should still be able to boot the system without a kernel debugger attached.

After setting these options, reboot, and you should be set. You’ll now be able to use local KD (you must still remember to elevate the debugger, though), but you won’t have user mode programs try to break into the kernel debugger when they crash without a user mode debugger attached.

Note, however, that you’ll still be able to break in to the system with a kernel debugger after boot if you choose these options (and if the box crashes in kernel mode, it’ll freeze waiting for a debugger to attach). However, at least you will not have to contend with errant user mode programs causing the system to break into the kernel debugger.

14 Responses to “Enabling the local kernel debugger on Vista RTM”

  1. Pavel Lebedinsky says:

    I have my machines configured like you described. With debugstart=disable I think the system will crash as usual (create a crash dump and reboot) rather than freeze waiting for kernel debugger. This is good, because it makes it easier to use lkd in production environments (most of the time people don’t want a production server to just freeze on bugchecks).

    In order to use local kd (or connect with a “real” kd) I have to enable kernel debugger at runtime with kdbgctrl.exe. Does kd -kl work for you after a reboot without having to run kdbgctrl?

  2. Skywing says:

    Pavel: No, I don’t need to use kdbgctrl before I can connect with this configuration.

    When I tried to initiate a bugcheck, I think the result was a hang until I attached the real (1394) kd (when I was experimenting with this on Friday). I am planning on re-running my tests, though, as I observed some really bizzare behavior that I think is related to a handled breakpoint in kernel mode with this kd configuration:

    I built and signed the following simple driver to test what would happen (i.e. if the system would hang waiting for kd or bluescreen, and at either the breakpoint or the KeBugCheck call) before I made this post:

    NTSTATUS
    NTAPI
    DriverEntry(
    __in PDRIVER_OBJECT DriverObject,
    __in PUNICODE_STRING RegistryPath
    )
    {

    __try
    {
    DbgBreakPoint();
    }
    __except(EXCEPTION_EXECUTE_HANDLER)
    {
    }

    KeBugCheck(0);

    return STATUS_SUCCESS;
    }

    Now, when I started the driver after configuring my machine as described in this post, something incredibly bizzare happens: the system and system idle processes together take 100% CPU (about 50/50 each; I am running on a dual core system). The box becomes extremely slow (although – barely – still usable), and the 1394 kernel debugger connection just completely hangs and becomes unresponsive.

    I was going to investigate just what was going on here with lkd (since doing this hosed the 1394 kd connection somehow), but it was taking too long to come up and I had to leave the office for a meeting, so that got cut short for the moment. The behavior appears to be reproducable though, so I am planning on investigating further just what happens in that scenario. It is definitely not the expected outcomes of either 1) hang until kd is connected, or 2) blue screen – having the box keep running (in an extremely slow state) is about the last thing I would have expected to happen.

  3. Skywing says:

    Oh, and the box where I was testing is running Vista x64, but to my knowledge there really shouldn’t be any differences to Vista x86 in this scenario.

  4. Lukas says:

    This article seemes for developers.
    However it may be interesting for “normal” people that happen to come across certain bluescreens.
    I did: I got a Bug Check 0x8E: KERNEL_MODE_EXCEPTION_NOT_HANDLED
    with 0x80000003: STATUS_BREAKPOINT as first parameter, telling that
    “A breakpoint or ASSERT was encountered when no kernel debugger was attached to the system.”

    I got this with Vista RTM (German version) – typical install, after installation of vista updates and via vista drivers (for harddisk and raid).
    The bluescreens started already while the installation was still running and hardware detection/installation was done. It is impossible to use the system for anything as I get a bluescreen either while booting or on logging in or at latest a few minutes after using vista.

    Anyway, I was unable to find the source of the trouble. No driver was specified. Often the source of the problem is stated to be ntkrpamp.exe, but this time just “hardware”.

    Could you give a few hints in general on how you would deal with such a problem?

  5. Alex Ionescu says:

    Note that DVD/MPEG2 playback won’t work with /DEBUG enabled — this isn’t a bug, but by design.

  6. Aaron Craelius says:

    I have tried everything that has been suggested on this page and I haven’t been able to get a local kernel debugger working.

    I still get the message Win32 error 5 when I try to run kd -kl. When I run kdbgctrl -e, I get the following error message:

    Unable to enable kernel debugger, NTSTATUS 0xC0000354
    An attempt to do an operation on a debug port failed because the port is in
    the process of being deleted.

    When I try to run the local debugger from WinDbg I get the message “Debuggee not connected”.

    I am running the command prompt as an administrator and I have the boot configuration set up as mentioned in this post. Any ideas on how to proceed?

    Thanks,
    Aaron

  7. Vignesh Jayaraman says:

    Aaron,

    Try the following:

    After Vista reboots, run [under Admin]
    kdbgctrl -db
    kdbgctrl -e
    Then
    kd -kl

  8. Shane says:

    I havent been playing around with this lately on Vista, it could be fairly simple answer but, on Vista SP1, x64, if you enable kd, and hit the F8 on bootup just so you get the extra live kd enabled w/o any fuss in windbg.

    What’s odd though is, you can not debug a CLR application EXE? I find that very perculier. You can debug a native application of course, and it’s no problem if that application does CorBindToRuntime(); Sort of odd..
    Anyhow, also, the system is much more unstable and I frequently get kernel dump’s during exception handling of userland applications.

  9. Shane says:

    Oh ya, I just thoguht while I was rebooting just now…. I guess the crashes I was getting during debugging of exception handlers is a big hint. The heavy use of exception’s by the CLR must cause some issue, only guess is that the kernel debugger reduces the kernel stack size available and either underflow’s some record keeping or just resolved as won’t fix…

    Seems to me that there will be other instances of this type of fault.

  10. Mark says:

    I have done the

    bcdedit /debug on
    bcdedit /dbgsettings 1394 /start active /noumex

    If I run kd -kl or windbg (local kernel) without elevation I get E_NOTIMPL (I mention this for use below). With an elevated command prompt is works fine.

    Now, I have this code (below, error handling stripped for brevity) that returns E_NOTIMPL on the AttachKernel line even when elevated and I am stumped. Any help would be appreciated.

    ::CoInitialize(NULL);
    IDebugClient *g_ExtClient;
    IDebugSymbols2 * g_ExtSymbols2;

    DebugCreate(__uuidof(IDebugClient),
    (void **)&g_ExtClient);

    g_ExtClient->QueryInterface(__uuidof(IDebugSymbols2),
    (void **)&g_ExtSymbols2);

    g_ExtClient->AttachKernel(DEBUG_ATTACH_LOCAL_KERNEL, NULL);

  11. Mark says:

    P.S. The code works fine on XP and I am trying it on Vista Home Premium.

  12. Umang Desai says:

    Hello,

    I am trying to use the microsoft kernel debugger on my machine.

    It has a Vista Home Premium 32 bit installed on AMD Turion64.
    I have installed the 32-bit package of kernel debugger.

    I performed the two bcdedit steps and rebooted. It still did not work.

    When I perform Mr. Vignesh’s steps

    (I run the command prompt as admin always)

    kdbgctrl -db

    o/p—->Kernel debugger block-enable set to: false

    kdbgctrl -e
    (When i press enter, my laptop restarts everytime)

    after this if I do

    kd -kl

    Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
    Copyright (c) Microsoft Corporation. All rights reserved.

    Debugger can’t get KD version information, Win32 error 0n5

    Can anyone help me solve the issue?

    I have tried using the 64-bit package as well but it does not install giving me the error that the package is not supported by this processor type

    thanks in advance for all the help

    Umang

  13. Skywing says:

    Did you run kd.exe as an elevated process (i.e. launched from an elevated cmd.exe)?

  14. Umang Desai says:

    hey skywing thnx for the reply. yes. the way i run it is..i have a shortcut to the cmd on my desktop. i right click on it and do “Run as admin..”

    it still gives me problems. In the same way i also run windbg as admin and tried running kd in that, it gives me the same error