Nynaeve

If you are working on a custom unhandled exception filter (perhaps to implement your own crash dump / error reporting mechanism), then you have probably run into the frustrating situation where you can’t debug the filter itself in case there is a crash bug in it.

The symtoms are that when a crash occurs, your crash reporting logic doesn’t work at all and the process just silently disappears.  When you attach a debugger and reproduce the crash to figure out what went wrong, the debugger keeps getting the exceptions (even if you continue after the first chance exception), and your unhandled exception filter is just never called.

Well, the reason for this is that UnhandledExceptionFilter tries to be clever and hides any last chance exception filter handling when a debugger is active.  Here’s why: the default implementation of the unhandled exception filter launches the JIT debugger and then restarts the exception.  When the JIT debugger attaches itself and the exception is restarted, you clearly want the exception to go to the debugger and not the unhandled exception filter, which would try to launch the JIT debugger again.

If, however, you have a custom unhandled exception filter that is crashing, then you probably don’t want this behavior so that you can debug the problem with the exception filter.  To disable this behavior and let the exception filter be called even if there is a debugger attached, you will need to patch kernel32!UnhandledExceptionFilter in a debugger.  If you unassemble it, you should eventually see a call to NtQueryInformationProcess that looks like this:

NtQueryInformationProcess(GetCurrentProcess(), 7, &var, 4, 0)

…followed by a comparison of that local variable passed in to NtQueryInformationProcess against 0.  You will need to patch the comparison to treat the local set by NtQueryInformationProcess as if it were zero, perhaps turning it into an unconditional jmp with the “eb” or “a” commands.

This comparison is checking for something called the “process debug port”, which is a handle to an LPC port that is used to communicate with the debugger attached to the current process.  The kernel returns a null handle value if there is no debugger attached, which is how UnhandledExceptionFilter knows whether to work its magic and forward the exception on to the debugger or call the registered unhandled exception filter.

After patching out the check, then exceptions should be forwarded to your unhandled exception filter as if no debugger were there, giving you a chance to inspect the operation of your crash handling code.

Last post, I discussed some of the major extensions available in SDbgExt.  This post is a continuation that explains the remaining major / interesting extensions available in the extension library.

 Another set of extensions that may prove useful if you do frequent C++ development are the STL datastructures display functions.  These allow you to interpret many of the complicated STL container datastructures and format them in more understandable forms from the debugger.  At present, the extension DLL only understands the Visual C++ 7.0 / 7.1 STL structure layout (no support for VC6 or VC8 as of yet, sorry).  This complements the built-in WinDbg support for STL, which does not cover all of VC7 (or at least did not the last time I checked).  The SDbgExt STL type display functions do not rely on type information contained in symbols, so you can use them to help debug third party programs that you do not have symbols for.  However, some extensions will require you to have a bit of knowledge about the structures contained in an STL container (usually the size of a container item).  These extensions will work on local or remote 32-bit targets.

The main functions that you might find useful in this grouping are these:

!stlmap allows you to traverse an std::map (binary tree) and optionally dump some bytes from each of the key and value types.  Due to the layout of the std::map type, you will need to tell SDbgExt about the size of the key and value types.

!stlset allows you to traverse an std::set (binary tree) optionally dump some bytes from each of value types.  It is very similar to the !stlmap extension, except that it works on std::set structures and thus only needs information about a value type and not an additional key type.

!stllist and !stlvector allow you to display the contents of an std::list or std::vector, respectively.  Optionally, you may provide the size of an element to dump some bytes from each element to the debugger.

!stlstring and !stlwstring allow you to display std::string and std::wstring structures.  If you are displaying very long strings (>16K characters), then you will need to provide a second argument specifying the maximum number of characters to display.  This limit is always capped at 64K characters.

Most of the STL datastructures traversal functions have only minimal protection against damaged or corrupted datastructures.  If you attempt to use them on a datastructure that is broken (perhaps it has a link that references itself, causing an infinite loop), then you will need to break out of the extension with Ctrl-C or Ctrl-Break depending on which debugger you use.  To ensure that your system remains responsive enough to have the option of breaking out, SDbgExt will lower its priority to at least normal (WinDbg runs at high priority by default) temporarily, for the duration of the call to the extension (the original priority is restored before the extension returns).

The last major category of functions supported by SDbgExt are those that are related to Windows UI debugging.  These extensions generally always require a live 32-bit target on the local computer in order to function (no remote debugging).  They work by directly querying internal window structures or by calling user32/gdi32 APIs about a particular UI object.  These can be used as a handy replacement for Spy++, which has a nasty tendancy to break badly (and break every GUI application on the same desktop with it) when it encounters a GUI program that is frozen in the debugger.

The !hwnd extension is the primary UI debugging extension supported by SDbgExt.  It will dump many of the interesting attributes about a window given its handle (HWND) to the debugger console.  Additionally, it can be used to enumerate the active windows on the current thread.  This extension is particular useful for programs that store things like class object pointers at GWLP_USERDATA or DWLP_USERDATA, which are normally hard to get at from WinDbg.

The !getprop extension can be used to enumerate window properties associated with a particular window, or to query a single specific window property associated with a particular window.  These are useful if a program stores information (like a class object pointer) in a window property and you need to get at it from the debugger, which is something that you cannot easily do from WinDbg normally.

The !msg extension will display information about a MSG structure given a pointer to it.  It has a limited built in set of message names for some of the most common standard window messages.  It will also display information about the window with which the window message is associated with (if any).

Finally, there are a couple of misc. functions that SDbgExt supports which don’t fit cleanly into any specific category.  Many of these are niche extension that are only useful for very specific scenarios.

The !switchdesk extension will switch the active desktop for the computer hosting the debugger.  This can be useful if you are debugging a full screen DirectX program using a GUI debugger on an alternate desktop and a console debugger in full screen mode connected to the GUI debugger using remote debugging on the same desktop as the program being debugged.

The !lmx extension will allow you to view the loaded module list in the various forms that the loader maintains it (in-load-order, in-memory-order, in-initialization-order).  When used on kernel targets, there is only one loaded module list, so the list identification parameter is unused in that usage case.

!findwfptr (courtsey of skape) allows you to scan the address space of the debuggee for function pointers that are located in writable memory.  It willl work on all target types.  For live targets, it can also optionally place breakpoints at all of the functions pointed to by pointers residing in writable memory.  This extension is useful if you are auditing a program for potential security risks; many of the static addresses (i.e. global variables) in standard system DLLs that contain function pointers have been changed to encode the pointer values based on a random cookie to prevent exploits from overwriting them to gain control over the program.  This extension can help you identify which function pointers might be used by an attacker to compromise your program when used in conjunction with certain types of security flaws.

!cmpmem allows you to compare ranges of memory over time, with the ability to exclude modified ranges.  This is particularly useful if you want to watch a data structure (or even the global variables of an entire module) and quickly determine what changes when a particular operation happens in the context of the debuggee.  For example, if you are trying to reverse engineer a function and want to understand some of the side effects it has on data structures or global variables, this function can help quickly identify modified areas without requiring you to analyze the entire function in a disassembler.

 That’s all for this series.  There are a couple of extensions that I didn’t mention, but they are either very obvious or not generally useful enough to be worth mentioning.  The online help (!help) provides basic syntax and a very brief description about all extensions supported by SDbgExt, so you can find parameter information about all of the extension I mentioned there.

One of the programs up on my homepage (does anyone call their website a “homepage” anymore?) is SDbgExt, a WinDbg-compatible extension DLL.  It’s a little collection of various debugging tools that I have put together over time that you might find useful.  There is some minimal documentation included, but not a whole lot to tell you where and when certain extensions are useful – hence, the topic of today’s blog post.

 This posting series assumes that you have already installed WinDbg, installed the Visual C++ 2005 Runtimes, and placed SDbgExt.dll into your Debugging tools for Windows\WinExt directory.  At present, SDbgExt can only be loaded into the 32-bit WinDbg package, although some extensions do support 64-bit targets (such as 64-bit kernel debugging targets).

To get started using SDbgExt, you’ll need to load it into your debugger.  For WinDbg, ntsd, cdb, and kd, you can use the “.load SDbgExt” command to do this.  If you then use the “!sdbgext.help” command, you should be presented with a list of the available extensions supported by SDbgExt.  Most of the extensions are targetted at live 32-bit processes on the local computer (such as the extensions dealing with displaying HWND information).  The documentation does not specify which targets are supported by each extension yet, so if you aren’t sure whether an extension is supported against your current debugging target, just try it; if not, it will harmlessly fail in an obvious manner.

Many of the SDbgExt extensions have very specific purposes, so I’ll try to several of them  address them individually and describe what they are best used for.  Additionally, most of the SDbgExt extensions can be broken down into several categories, such as symbol management, UI debugging, STL datastructures manipulation, kernel object support, and debuggee state manipulation.  I’ll be skipping some of the extensions that are either obvious or not generally useful, but I’ll try to cover most of the interesting ones.

To start off, I’ll talk about some of the debuggee state manipulation extensions.  These extensions allow you to control the state of a live 32-bit target process that is running on the same computer as the debugger (remote debugging and 64-bit targets are not supposed yet by these extensions).  The extensions that fall under this category include:

• !valloc, !vallocrwx, !heapalloc, !heapfree: Allocate memory within the address space of the target.
• !remotecall, !remotecall64: Call a function in the target, using the currently active thread (symbols are not required, unlike “.call”).
• !loaddll, !unloaddll: Load or unload a .dll within the address space of the target, using the currently active thread.
• !close: Close a kernel object handle in the targets handle space.
• !killthread: Terminate a thread in the target process.
• !adjpriv: Alter privileges of the target process or currently active thread.
• !ret: Effect a virtual function return in the context of the currently active thread.

Many of these extensions are useful if you are doing some runtime patching of a target or are trying to see how the target will react to specific circumstances.  If you are trying to reverse engineer or modify the behavior of a target, in particular, you might find several of these extensions very useful.

The first group of extensions are used for managing memory allocation in the targets address space – either by directly allocating pages within the target or allocating heap memory.  The latter requires that you resume execution of the target as internally the remote heap manipulation functions use the remote function call support to call the heap manager in the target process.  Perhaps the most common use for this family of functions is if you need to allocate some space on the fly if you are adding some code to the debuggee on the fly (maybe you need to patch a function and add several instructions, in which case you could allocate memory with !vallocrwx, and then patch a jump instruction to refer to the newly allocated memory block).

The next two groups of functions are used for directly calling functions in the target, from the context of the currently active thread in the debuggee.  Be warned that this is an invasive operation and may cause undesirable side effects in the context of the debuggee.  The main advantage of the !remotecall family of extensions over the built-in .call command is that you do not need to have private symbols for the target, which makes it particularly useful if you are reverse engineering something or need to call a Win32 API function in the context of the debuggee from the debugger.  These functions can allow you to do complicated things that are difficult or impossible to do remotely (from the debugger), such as calling SetHandleInformation to mark a handle closable or nonclosable from WinDbg. You can use !loaddll and !unloaddll as shortcuts (as opposed to using !remotecall on kernel32!LoadLibraryA/kernel32!FreeLibrary manually) for DLL management in the target process space.

!close is mostly analagous to the .closehandle built-in command, and is used to close a handle in the context of a debuggee.

!killthread is useful if you need to instantly terminate a thread in the debuggee process, for whatever reason.  You can manually achieve something like this with a command like “r eip=kernel32!ExitThread;g” for Win32 targets, but this extension provides a more elegant means to killing debuggee threads (for instance, you might want to kill a thread that is crashed so that it doesn’t take down the rest of the process in the default SEH handler, for certain scenarios).

!adjpriv is useful if you are debugging problems related to the privileges that are enabled in a primary or impersonation token.  You can use the built-in !token extension to determine what privileges are currently present, enabled, or disabled in a token, and the !adjpriv extension to manipulate these privileges from the debugger itself.  This can also be used to work around buggy programs that don’t properly enable privileges before they try to do certain privileged operations (such as things written for Win9x).

!ret is primarily useful in conditional breakpoints if you want to return from the middle of a function at a breakpoint location based on a particular condition.  It alters the context of the currently active thread (modifying the stack pointer, instruction pointer, and optionally return address registers) according to its arguments.

The next group of extensions that I’d like to describe are the symbol management extensions.  These are extremely useful if you are reverse engineering a program and want to synchronize your work between a disassembler (such as IDA) and the debugger.  The two extensions that fall into this category are !loadsym and !unloadsym.

These two extensions allow you to either create or remove custom virtual symbols in the target.  A virtual symbol allows you to name an address (although it does not allow you to convey type information, unfortunately).  This can be extremely useful if you are debugging a third-party program that has no symbols, and you want to name certain addresses to make them easier to recognize.

Both extensions can operate on two different types of symbol files: a custom format that is specific to SDbgExt and allows you to specify all possible attributes that are supported by virtual symbols (primarily the size of the symbol, name, and its offset from a base module), and a standard linker .map file.  The latter is generally the most useful of the two formats, as there are many things that can write symbol information to a .map file which you can then load into SDbgExt and access through WinDbg.  For instance, IDA allows you to dump all names in a database (disassembly project) to a .map file, which you could then load using SDbgExt and have names in WinDbg that match the work you have done in IDA.  These commands can also be useful if the only symbols you have for a particular binary are the linker map files (which has happened to me once or twice, on rare occasions).

Both extensions require a 32-bit target, although the target may be a remote target and can be either a user mode or kernel mode target.  For kernel targets, the symbol loading support will apply to modules in the kernel mode loaded module list primarily.  Virtual symbols are automatically unloaded whenever you reload symbols (such as with the “.reload” command), so you may find yourself needing to re-apply the custom symbols more than once in a session.  Additionally, due to a bug / limitation in how DbgHelp and DbgEng manage virtual symbols, the process of creating virtual symbols unfortunately gets exponentially slower relative to how many virtual symbols are currently in existance.  As a result, creating more than a couple thousand virtual symbols may take a while.

The last group of extensions that I am going to cover in the first installment of this series is the kernel object support extension group.  These extensions are intended to complement the built-in support (such as !handle or !token) for querying kernel objects by allowing access to things that are otherwise not easily queryable from the debugger.  Although the information available from the built-in debugger support is usually sufficient, in special cases you may need additional information (such as security descriptor information).  Most of these extensions require a live 32-bit target on the local computer to operate correctly.

The !objname extension takes an object handle relative to the handle space of the debuggee and returns the full name for it.  This is similar to the built-in !handle extension, except that it works on all kernel object types (unlike !handle, which does not work on some object types, such as file object handles).

!tokeninfo will allow you to inspect some additional information about an access token object, beyond that which the built-in !token extension makes available to you (either given a token handle, or by operating on the primary or impersonation token that is effective for the currently active thread).  The most useful pieces of information available from this extension are the TokenId (uniquely identifying a token object throughout the system) and the ModifiedId (which increments whenever a tokens attributes are altered in any way).

The !objsec extension is useful for displaying detailed information about the security descriptor of an object given an object handle.  In kernel mode, you can use the !sd extension based on the security descriptor pointer embedded in a kernel object header, but this extension allows you to perform a similar function from user mode.  It has built-in knowledge about the object specific access rights supported by all of the kernel object types (as of Windows Server 2003 SP1) and will automatically translate access right values in access masks to more human readable values.

If you are dealing with a raw access mask directly (perhaps passed to a security related function as a parameter), and you want to know what it means given a particular object type, then you can use the !amask extension to have SDbgExt interpret the access mask bits as they apply to a particular object type.  The !objtypes extension lists the object type names that are supported by !amask.  If you do not supply an object type argument to !amask, it will only interpret generic and standard access rights.

The !sidname extension can be used to convert a SID into an account name.  Unlike most extensions, this extension does not operate on the debuggee at all; instead, it simply takes a string SID as a single argument and attempts to resolve it against the security database of the computer hosting the debugger.  This is a shortcut for command line utilities (like PsGetSid) that could do the same for you, since many of the access token related functions will give you back a raw string SID and will not translate it into a more understandable account name.

The !threadinfo extension will display some basic information about a thread running in the debuggee.  It will only work on local targets on the same computer as the debugger.  This extension allows you to view a couple of rarely-used fields that aren’t easily viewable from the debugger in user mode, like processor affinity or thread and exit times.

 That’s all for this post.  The next post in this series will cover the remaining major extensions in SDbgExt.