Archive for the ‘NT Internals’ Category

Newer Entries »

Programming against the x64 exception handling support, part 3: Unwind internals (RtlUnwindEx interface)

Sunday, January 7th, 2007

Previously, I provided a brief overview of what each of the core APIs relating to x64’s extensive data-driven unwind support were, and when you might find them useful.

This post focuses on discussing the interface-level details of RtlUnwindEx, and how they relate to procedure unwinding on Windows (x64 versions, specifically, though most of the concepts apply to other architecture in principle).

The main workhorse of unwind support on x64 Windows is RtlUnwindEx. As previously described, this routine encapsulates all of the work necessary to restore execution context to a prior point in the call stack (relying on RtlVirtualUnwind for this task). RtlUnwindEx also implements all of the logic relating to interactions with unwind/exception handlers during the unwind process (which is essentially the value added by RtlUnwindEx on top of what RtlVirtualUnwind implements).

In order to understand the inner workings of how unwinding works, it is first necessary to understand the high level theory behind how RtlUnwindEx is used (as RtlUnwindEx is at the heart of unwind support on Windows). Although there have been previously posted articles that touch briefly on how unwind is implemented, none that I have seen include all of the details, which is something that this segment of the x64 exception handling series shall attempt to correct.

For the moment, it is simpler to just consider the unwind half of exception handling. The nitty-gritty, exhaustive details of how exceptions are handled and dispatched will be discussed in a future posting; for now, assume that we are only interested in the unwind code path.

When a procedure unwind is requested, by any place within the system, the first order of business is a call to RtlUnwindEx. The prototype for RtlUnwindEx was provided in a previous posting, but in an effort to ensure that everyone is on the same page with this discussion, here’s what it looks like for x64:

VOID
NTAPI
RtlUnwindEx(
   __in_opt ULONG64               TargetFrame,
   __in_opt ULONG64               TargetIp,
   __in_opt PEXCEPTION_RECORD     ExceptionRecord,
   __in     PVOID                 ReturnValue,
   __out    PCONTEXT              OriginalContext,
   __in_opt PUNWIND_HISTORY_TABLE HistoryTable
   );

These parameters deserve perhaps a bit more explanation.

  1. TargetFrame describes the stack pointer (rsp) value for the target of the unwind operation. In normal circumstances, this is always the EstablisherFrame argument to an exception handler that is handling an exception. In the context of an exception handler, EstablisherFrame refers to the stack pointer of the caller of the function that caused the exception being inspected. Likewise, in this context, TargetFrame refers to the stack pointer of the function that the call stack should be unwound to. Although given the fact that with data-driven unwind semantics, one might initially think that this argument is unnecessary (after all, one might assume that RtlUnwindEx could simply invoke RtlVirtualUnwind in order to determine the expected stack pointer value for the next function on the call stack), this argument is actually required. The reason is that RtlUnwindEx supports unwinding past multiple procedure frames; that is, RtlUnwindEx can be used to unwind to a function that is several levels down in the call stack, instead of the immediately lower function in the call stack. Note that the TargetFrame argument must match exactly the expected stack pointer value of the target function in the call stack.

    Observant readers may pick up on the SAL annotation describing the TargetFrame argument and notice that it is marked as optional. In general, TargetFrame is always supplied; it can be omitted in one specific circumstance, which is known as an exit unwind; more on that later.

  2. TargetIp serves a similar purpose as TargetFrame; it describes the instruction pointer value that execution should be unwound to. TargetIp must be an instruction in the same function on the call stack that corresponds to the target stack frame described by TargetFrame. This argument is supplied as a particular function may have multiple points that could be resumed in response to an exception (this typically the case if there are multiple try/except clauses).

    Like TargetFrame, the TargetIp argument is also optional (though in most cases, it will be present). Specifically, if a frame consolidation unwind operation is being executed, then the TargetIp argument will be ignored by RtlUnwindEx and may be set to zero if desired (it will, however, still be passed to unwind handlers for use as they see fit). This specialized unwind operation will be discussed later, along with C++ exception support.

  3. ExceptionRecord is an optional argument describing the reason for an unwind operation. This is typically the same exception record that was indicated as the cause of an exception (if the caller is an exception handler), although it does not strictly have to be as such. If no exception record is supplied, RtlUnwindEx constructs a default exception record to pass on to unwind handlers, with an exception code of STATUS_UNWIND and an exception address referring to an instruction within RtlUnwindEx itself.
  4. ReturnValue describes a pointer-sized value that is to be placed in the return value register at the completion of an unwind operation, just before control is transferred to the newly unwound context. The interpretation of this value is entirely up to the routine being unwound into. In practice, the Microsoft C/C++ compiler does not use the return value at all in typical cases. Usually, the Microsoft C/C++ compiler will indicate the exception code that caused the exception as the return value, but due to how unwinding across functions works with try/except, there is no language-level support for retrieving the return value of a function that has been unwound due to an exception. As a result, in most circumstances, the return value placed in the unwound execution context based on this argument is ignored.
  5. OriginalContext describes an out-only pointer to a context record that is updated with the execution context as procedure call frames are unwound. In practice, as RtlUnwindEx does not ever “return” to its caller, this value is typically only provided as a way for a caller to supply its own storage to be used as scratch space by RtlUnwindEx during the intermediate unwind operations comprimising an unwind to the target call frame. Typically, the context record passed in to an exception handler from the exception dispatcher is supplied. Because the initial contents of the OriginalContext argument are not used, however, this argument need not necessarily be the context record passed in from the exception dispatcher.
  6. HistoryTable describes a cache used to improve the performance of repeated function entry lookups via RtlLookupFunctionEntry. Under normal circumstances, this is the same history table passed in from the exception dispatcher to an exception handler, although it could also be a caller-allocated structure as well. This argument can also be safely omitted entirely, although if a non-trivial set of call frames are being unwound, passing in even a newly-initialized history table may improve performance.

Given all of the above information, RtlUnwindEx performs a procedure call unwind by performing a successive sequence of RtlVirtualUnwind calls (to determine the execution context of the next call frame in the call stack), followed by a call to the registered language handler for the call frame (if one exists and is marked for unwinding support). In most cases where there is a language unwind handler, it will point to _C_specific_handler, which internally searches all of the internal exception handling scopes (e.g. try/except or try/finally constructs), calling “finally” handlers as need be. There may also be internal unwind handlers that are present in the scope table for a particular function, such as for C++ destructor support (assuming asynchronous C++ exception handling has been enabled). Most users will thus interact with unwind handlers in the form of a “finally” handler in a try/finally construct in a function whose language handler refers to _C_specific_handler.

If RtlUnwindEx encounters a “leaf function” during the unwind process (a leaf function is a function that does not use the stack and calls no subfunctions), then it is possible that there will be no matching RUNTIME_FUNCTION entry for the current call frame returned by RtlLookupFunctionEntry. In this case, RtlUnwindEx assumes that the return address of the current call frame is at the current value of Rsp (and that the current call frame has no unwind or exception handlers). Because the x64 calling convention enforces hard rules as to what functions without RUNTIME_FUNCTION registrations can do with the stack, this is a valid assumption for RtlUnwindEx to make (and a necessary assumption, as there is no way to call RtlVirtualUnwind on a function with no matching RUNTIME_FUNCTION entry). The current call frame’s value of Rsp (in the context record describing the current call frame, not the register value of rsp itself within RtlUnwindEx) is dereferenced to locate the call frame’s return address (Rip value), and the saved Rsp value is then adjusted accordingly (increased by 8 bytes).

When RtlUnwindEx locates the endpoint frame of the unwind, a special flag (EXCEPTION_TARGET_UNWIND) is set in the ExceptionFlags member of the EXCEPTION_RECORD passed to the language handler. This flag indicates to the language handler (and possibly any C-language scope handlers) that the handler is being called as the “final destination” of the unwind operation. The Microsoft C/C++ compiler does not expose functionality to detect whether a “finally” handler is being called in the context of a target unwind or if the “finally” handler is simply being called as an intermediate step towards the unwind target.

After the last unwind handler (if applicable) has been called, RtlUnwindEx restores the execution context that has been continually updated by successive calls to RtlVirtualUnwind. This restoration is performed by a call to RtlRestoreContext (a documented, exported function), which simply transfers a given context record to the thread’s execution context (thus “realizing” it).

RtlUnwindEx does not return a value to its caller. In fact, it typically does not return to its caller at all; the only “return” path for RtlUnwindEx is in the case where the passed-in execution context is corrupted (typically due to a bogus stack pointer), or if an exception handler does something illegal (such as returning an unrecognized EXCEPTION_DISPOSITION) value. In these cases, RtlUnwindEx will raise a noncontinuable exception describing the problem (via RtlRaiseStatus). These error conditions are usually fatal (and are indicative of something being seriously corrupted in the process), and virtually always result in the process being terminated. As a result, it is atypical for a caller of RtlUnwindEx to attempt to handle these error cases with an exception handler block.

In the case where RtlUnwindEx performs the requested unwind successfully, a new execution context describing the state at the requested (unwound) call frame is directly realized, and as such RtlUnwindEx does not ever truly return in the success case.

Although RtlUnwindEx is principally used in conjunction with exception handling, there are other use cases implemented by the Microsoft C/C++ compiler which internally rely upon RtlUnwindEx in unrelated capacities. Specifically, RtlUnwindEx implements the core of the standard setjmp and longjmp routines (assuming the exception safe versions of these are enabled by use of the <setjmpex.h> header file) provided by the C runtime library in the Microsoft CRT.

In the exception-safe setjmp/longjmp case, the jmp_buf argument essentially contains an abridged version of the execution context (specifically, volatile register values are omitted). When longjmp is called, the routine constructs an EXCEPTION_RECORD with STATUS_LONGJUMP as the exception code, sets up one exception information parameter (which is a pointer to the jmp_buf), and passes control to RtlUnwindEx (for the curious, the x64 version of the jmp_buf structure is described as _JUMP_BUFFER in setjmp.h under the _M_AMD64_ section). In this particular instance, the ReturnValue argument of RtlUnwindEx is significant; it corresponds to the value that is seemingly returned by setjmp when control is being transferred to the saved setjmp context as part of a longjmp call (somewhat similar in principal as to how the UNIX fork system call indicates whether it is returning to the child process or the parent process). The internal operations of RtlUnwindEx are identical whether it is being used for the implementation of setjmp/longjmp, or for conventional exception-handler-based triggering of procedure call frame unwinding.

However, there are differences that appear when RtlUnwindEx restores the execution context via RtlRestoreContext. There is special support inside RtlRestoreContext for STATUS_LONGJUMP exceptions with one exception information parameter; if this situation is detected, then RtlRestoreContext internally reinitializes portions of the passed-in context record based on the jmp_buf pointer stored in the exception information parameter block of the exception record provided to RtlRestoreContext by RtlUnwindEx. After this special-case partial reinitialization of the context record is complete, RtlRestoreContext realizes the context record as normal (causing execution control to be transferred to the stored Rip value). This can be seen as a hack (and a violation of abstraction layers; there is intended to be a logical separation between operating system level SEH support, and language level SEH support; this special support in RtlRestoreContext blurs the distinction between the two for C language support with the Microsoft C/C++ compiler). This layering violation is not the most egregious in the x64 exception handling scene, however.

This concludes the basic overview of the interface provided by RtlUnwindEx. There are some things that I have not yet covered, such as exit unwinds, collided unwinds, or the deep integration and support for C++ try/catch, and some of the highly unsavory things done in the name of C++ exception support. Next time: A walkthrough of the complete internal implementation of RtlUnwindEx, including undocumented, never-before-seen (or barely documented) corner cases like exit unwinds or collided unwinds (the internals of C++ exception support from the perspective of RtlUnwindEx are reserved for a future posting, due to size considerations).

Posted in NT Internals, Programming, Reverse Engineering, Windows | 2 Comments »

Programming against the x64 exception handling support, part 1: Definitions for x64 versions of exception handling support

Wednesday, December 13th, 2006

This is a series dealing with how to use the new x64 exception handling support from a programmatic perspective (that is, how to write programs that take advantage of the new support, instead of the perspective of how to understand it while reverse engineering or disassembling something. Those topics have been covered in the past on this site already.)

To get started with programming against the new x64 EH support, you’ll need to have the structure and prototype definitions for the standard x64 EH related functions and structures. One’s first instinct here is to go to MSDN. Be warned, that if you are dealing with the low-level SEH routines (such as RtlUnwindEx), the documentation on MSDN is still missing / wrong for x64. For the most part, excepting RtlVirtualUnwind (which is actually correctly documented now), the exception handler support is only properly documented for IA64 (so don’t be surprised if things don’t work out how you would hope when calling RtlUnwindEx with the MSDN prototype).

For a recent project, I had to do some in-depth work with the inner workings of exception handling support on x64. So, if you’ve been ever having to deal with the low-level EH internals on x64 and have been frustrated by documentation on MSDN that is either incomplete or just plain wrong, here’s some of the things that I have run into along the way as far as things that are either missing or incorrect on MSDN while relating to x64 EH support:

  1. When processing an UNWIND_INFO structure, if the UNW_FLAG_CHAININFO flag is set, then there is an additional undocumented possibility for how unwind information can be chained. Specifically, if the low bit is set in the UnwindInfoAddress of the IMAGE_RUNTIME_FUNCTION_ENTRY structure referring to by the parent UNWIND_INFO structure, UnwindInfoAddress is actually the RVA of another IMAGE_RUNTIME_FUNCTION_ENTRY structure after zeroing the first bit (instead of the RVA of an UNWIND_INFO structure). This is used to help more efficiently chain exception data across a binary with minimal waste of space (credits go to skape for telling me about this).
  2. The prototype on MSDN for RtlUnwindEx is only for IA64 and does not apply to x64. The correct prototype is something more on the lines of this: 
    VOID
NTAPI
RtlUnwindEx(
   __in_opt ULONG64               TargetFrame,
   __in_opt ULONG64               TargetIp,
   __in_opt PEXCEPTION_RECORD     ExceptionRecord,
   __in     PVOID                 ReturnValue,
   __out    PCONTEXT              OriginalContext,
   __in_opt PUNWIND_HISTORY_TABLE HistoryTable
   );
  3. MSDN’s definition of DISPATCHER_CONTEXT (a structure that is passed to the language specific handler) is incomplete. There are some additional fields beyond HandlerData, which is the last field documented in MSDN. You can see this if you disassemble _C_specific_handler, which uses the undocumented ScopeIndex field. Additional credits go to Alex Ionescu for information on a couple of the undocumented DISPATCHER_CONTEXT fields. Here’s the correct definition of this structure for x64: 
    typedef struct _DISPATCHER_CONTEXT {
    ULONG64               ControlPc;
    ULONG64               ImageBase;
    PRUNTIME_FUNCTION     FunctionEntry;
    ULONG64               EstablisherFrame;
    ULONG64               TargetIp;
    PCONTEXT              ContextRecord;
    PEXCEPTION_ROUTINE    LanguageHandler;
    PVOID                 HandlerData;
    PUNWIND_HISTORY_TABLE HistoryTable;
    ULONG                 ScopeIndex;
    ULONG                 Fill0;
} DISPATCHER_CONTEXT, *PDISPATCHER_CONTEXT;
  4. Not all of the flags passed to an exception handler (primarily relating to unwinding) are properly documented on MSDN. These additional flags are included in winnt.h, however, and are actually the same for both x86 and x64. Here’s a listing of the missing flags that apply to the ExceptionFlags member of the EXCEPTION_RECORD structure (only the EXCEPTION_NONCONTINUABLE flag value is documented on MSDN): 
    #define EXCEPTION_NONCONTINUABLE   0x0001
#define EXCEPTION_UNWINDING        0x0002
#define EXCEPTION_EXIT_UNWIND      0x0004
#define EXCEPTION_STACK_INVALID    0x0008
#define EXCEPTION_NESTED_CALL      0x0010
#define EXCEPTION_TARGET_UNWIND    0x0020
#define EXCEPTION_COLLIDED_UNWIND  0x0040
#define EXCEPTION_UNWIND           0x0066

    In particular, EXCEPTION_UNWIND is a bitmask of other flags that indicates all possible flags that are used to signify an unwind operation. This is probably the most interesting bitmask/flag to you, as you’ll need it if you are distinguishing from an exception or an unwind operation from the perspective of an exception handler.

  5. The definition for the C scope-table information emitted by CL for __try/__except/__finally and implicit exception handlers is not documented. Here’s the definition of the scope table used for C exception handling support: 
    typedef struct _SCOPE_TABLE {
	ULONG Count;
	struct
	{
		 ULONG BeginAddress;
		 ULONG EndAddress;
		 ULONG HandlerAddress;
		 ULONG JumpTarget;
	} ScopeRecord[ 1 ];
 } SCOPE_TABLE, *PSCOPE_TABLE;

    This structure was briefly documented in a beta release of the WDK, although it has since disappeared from the RTM build. The ScopeRecord field describes a variable-sized array whose length is given by the Count field.
    You’ll need this structure definition if you are interacting with _C_specific_handler, or implementing assembler routines that are intended to use _C_specific_handler as their language specific handler.
    All of the above addresses are RVAs. BeginAddress and EndAddress are the RVAs for which the current scope record is effective for. HandlerAddress is the RVA of a C-specific exception handler (more on that below) that implements the __except filter routine in C exception support, or the hardcoded value 0x1 to indicate that this is the __except filter unconditionally accepts the exception (this is also set to 0x1 for a __finally block). The JumpTarget member is the RVA of where control is transferred if the C exception handler indicates the address of the body of an __except block (or a __finally block).

  6. The C exception handler routine whose RVA is given by the HandlerAddress of the C scope table for a code block is defined as follows: 
    typedef
LONG
(NTAPI * PC_LANGUAGE_EXCEPTION_HANDLER)(
   __in    PEXCEPTION_POINTERS    ExceptionPointers,
   __in    ULONG64                EstablisherFrame
   );

    The ExceptionPointers argument is the familiar EXCEPTION_POINTERS structure that the GetExceptionInformation macro returns. The EstablisherFrame argument contains the stack pointer value for the routine associated with the C exception handler in question at the point in which the exception occured. (If the exception occured in a subfunction called by the function that the exception is now being inspected at, then the stack pointer should be relative to the point just after the call to the faulting function was made.) The EstablisherFrame argument is typically used to allow transparent access to the local variables of the current function from within the exception filter, even though technically the exception filter is not part of the current function but actually a completely different function itself. This is the mechanic by which you can access local variables within an __except expression.
    The function definition deserves a bit more explanation than just the parameter value meanings, however, as it is really dual-purpose. There are two modes for this routine, exception handling mode and unwind handling mode. If the low byte of the ExceptionPointers argument is set to the hardcoded value 0x1, then the handler is being called for an unwind operation. In this case, the rest of the ExceptionPointers argument is meaningless, and only the EstablisherFrame argument holds a meaningful value. In addition, when operating in unwind mode, the return value of the exception handler routine is ignored (the compiler often doesn’t even initialize it for that code path). In exception handling mode (where the ExceptionPointers argument’s low byte is not equal to the hardcoded value 0x1), both arguments are significant, and the return value is also used. In this case, the return value is one of the familiar EXCEPTION_EXECUTE_HANDLER, EXCEPTION_CONTINUE_SEARCH, and EXCEPTION_CONTINUE_EXECUTION constants that are returned by an __except filter expression. If EXCEPTION_EXECUTE_HANDLER is returned, then control will eventually be transferred to the JumpTarget member of the current scope table entry.

  7. The definition of the UNWIND_HISTORY_TABLE structure (and associated substructures) for x64 is as follows (this structure is used as a cache to speed up repeated exception handling lookups, and is typically optional as far as usage with RtlUnwindEx goes – though certainly recommended from a performance perspective): 
    #define UNWIND_HISTORY_TABLE_SIZE 12

typedef struct _UNWIND_HISTORY_TABLE_ENTRY {
        ULONG64           ImageBase;
        PRUNTIME_FUNCTION FunctionEntry;
} UNWIND_HISTORY_TABLE_ENTRY,
*PUNWIND_HISTORY_TABLE_ENTRY;

#define UNWIND_HISTORY_TABLE_NONE 0
#define UNWIND_HISTORY_TABLE_GLOBAL 1
#define UNWIND_HISTORY_TABLE_LOCAL 2

typedef struct _UNWIND_HISTORY_TABLE {
        ULONG                      Count;
        UCHAR                      Search;
        ULONG64                    LowAddress;
        ULONG64                    HighAddress;
        UNWIND_HISTORY_TABLE_ENTRY
           Entry[ UNWIND_HISTORY_TABLE_SIZE ];
} UNWIND_HISTORY_TABLE, *PUNWIND_HISTORY_TABLE;
  8. There are inconsistencies regarding the usage of RUNTIME_FUNCTION and IMAGE_RUNTIME_FUNCTION in various places in the documentation. These two structures are synonymous for x64 and may be used interchangeably.

Most of the other x64 exception handling information on the latest version of MSDN is correct (specifically, parts dealing with dealing with function tables, such as RtlLookupFunctionTableEntry.) Remember that the MSDN documentation also includes IA64 definitions on the same page, though (and the IA64 definition is typically the one presented at the top with all of the arguments explained, where you would expect it). You’ll typically need to scroll through the remarks section to find information on the x64 versions of these routines. Be wary of using your locally installed Platform SDK help with the functions that are correctly documented on MSDN, though, as to my knowledge only the very latest SDK version (e.g. the Vista SDK) actually has correct information for any of the x64 exception handling information; older versions, such as the Platform SDK that shipped with Visual Studio 2005, only include IA64 information for routines like RtlVirtualUnwind or RtlLookupFunctionTableEntry. In general, anywhere you see a reference to a FRAME_POINTERS or Gp structure or value in the documentation, this is a good hint that the documentation is talking exclusively about IA64 and does not directly apply to x64.

That’s all for this installment. More on how to use this information from a programmatic perspective next time…

Posted in NT Internals, Programming, Windows | 3 Comments »

Debugger internals: How loaded module names are communicated to the debugger

Monday, December 11th, 2006

If you’ve ever used the Win32 debugging API, you’ll notice that the WaitForDebugEvent routine, when returning a LOAD_DLL_DEBUG_EVENT style of event, gives you the address of an optional debuggee-relative string pointer containing the name of the DLL that is being loaded. In case you’ve ever wondered just where that string comes from, you’ll be comforted to know that this mechanism for communicating module name strings to the remote debugger is built upon a giant hack.

To give a bit of background information on how loading of DLLs works, most of the heavy-lifting with respect to loading DLLs (referred to as “mapping an image”) is done by the memory manager subsystem in kernel mode – specifically, in the “MiMapViewOfImageSection” internal routine. This routine is responsible for taking a section object (known as a file mapping object in the Win32 world) that represents a PE image on disk, and setting up the in-memory layout of the PE image in the specified process address space (in the case of Win32, always the address space of the caller). This includes setting up PE image subsections with the correct alignment, zero-filling “bss”-style sections, and setting up the protections of each PE image subsection. It is also responsible for supplying the “magic” necessary to allow shared PE subsections to work. All of this behavior is controlled by the SEC_IMAGE flag being passed to NtMapViewOfSection (this behavior is visible by Win32 via passing SEC_IMAGE to MapViewOfFile, and can be used to achieve the same result of “just” mapping an image in-memory without going through the loader). Internally, the loader routine in NTDLL (LdrLoadDll and its associated subfunctions, which are called by the LoadLibrary family of routines in kernel32) utilizes NtMapViewOfSection to create the in-memory layout of the DLL being requested. After performing this task, the user-mode NTDLL-based loader then performs tasks such as applying base relocations, resolving imports to other modules (and loading dependent modules if necessary), allocating TLS data slots, making DLL initializer callouts, and soforth.

Now, the way that the debugger is notified of module load events is via a kernel mode hook that is called by NtMapViewOfSection (DbgkMapViewOfSection). This hook is responsible for detecting if a debugger (user mode or kernel mode) is present, and if so, forwarding the event to the debugger.

This is all well and good, but there’s a catch here. Both the user mode and kernel mode debuggers display the full path name to the DLL being loaded, but we’re now at the wrong level of abstraction, so to speak, to retrieve this information. All MiMapViewOfSection has is a handle to a section object (in actuality, a PSECTION_OBJECT and not a handle at this point). Now, the section object *does* have a reference to the PFILE_OBJECT associated with the file backing the section object (the reference is stored in the CONTROL_AREA of the section object), but there isn’t necessarily a good way to get the original filename that was passed to LoadLibrary out of the FILE_OBJECT (for starters, at this point, that path has already been converted to a native path instead of a Win32 path, and there is some potential ambiguouity when trying to convert native paths back to Win32 paths).

To work around this little conundrum, the solution the developers chose is to temporarily borrow a field of the NT_TIB portion of the TEB of the calling thread for use as a way to signal the name of a DLL that is being loaded (if SEC_IMAGE is being passed to NtMapViewOfSection). Specifically, NT_TIB.ArbitraryUserPointer is temporarily replaced with a string pointer (in Windows NT, this is always a unicode string) to the original filename passed to LdrLoadDll. Normally, the ArbitraryUserPointer field is reserved exclusively for use by user mode as a sort of “free TLS slot” that is available at a known location for every thread. Although this particular value is rarely used in Windows, the loader does make the effort to preserve its value across calls to LdrLoadDll. This works (since the loader knows that none of the code that it is calling will use NT_TIB.ArbitraryUserPointer), so long as you don’t have cross-thread accesses to a different thread’s NT_TIB.ArbitraryUserPointer (to date, I have never seen a program that tries to do this – and a good thing to, or it would randomly fail when DLLs are being loaded). Because the original value of NT_TIB.ArbitraryUserPointer is restored, the calling thread is typically none-the-wiser that this substitution has been performed.

Disassembling the part of the NTDLL loader responsible for mapping the DLL into the address space via NtMapViewOfSection (a subroutine named “LdrpMapViewOfDllSection” on Windows Vista), we can see this behavior in action:

ntdll!LdrpMapViewOfDllSection:
[...]
;
; Find the TEB address for the current thread.
; esi = NtCurrentTeb()->NtTib.Self
;
77f0e2ee 648b3518000000  mov     esi,dword ptr fs:[18h]
77f0e2f5 8365fc00        and     dword ptr [ebp-4],0
77f0e2f9 57              push    edi
77f0e2fa bf00000020      mov     edi,20000000h
77f0e2ff 857d18          test    dword ptr [ebp+18h],edi
77f0e302 c745f804000000  mov     dword ptr [ebp-8],4
77f0e309 0f85ce700400    jne     LdrpMapViewOfDllSection+0x26

ntdll!LdrpMapViewOfDllSection+0x42:
77f0e30f 8b4514          mov     eax,dword ptr [ebp+14h]
;
; Save away the previous ArbitraryUserPointer value.
;
; ebx = Teb->NtTib.ArbitraryUserPointer
77f0e312 8b5e14          mov     ebx,dword ptr [esi+14h]
77f0e315 6a04            push    4
77f0e317 ff7518          push    dword ptr [ebp+18h]
;
; Set the ArbitraryUserPointer value to the string pointer
; referring to the DLL name passed to LdrLoadDll.
; Teb->NtTib.ArbitraryUserPointer = (PVOID)DllNameString;
; 
77f0e31a 894614          mov     dword ptr [esi+14h],eax
77f0e31d 6a01            push    1
77f0e31f ff7510          push    dword ptr [ebp+10h]
77f0e322 33c0            xor     eax,eax
77f0e324 50              push    eax
77f0e325 50              push    eax
77f0e326 50              push    eax
77f0e327 ff750c          push    dword ptr [ebp+0Ch]
77f0e32a 6aff            push    0FFFFFFFFh
77f0e32c ff7508          push    dword ptr [ebp+8]
;
; Call NtMapViewOfSection to map the image and perform the
; debugger notification.
;
77f0e32f e830180300      call    NtMapViewOfSection
77f0e334 857d18          test    dword ptr [ebp+18h],edi
77f0e337 5f              pop     edi
;
; Restore the previous value of
; Teb->NtTib.ArbitraryUserPointer.
;
77f0e338 895e14          mov     dword ptr [esi+14h],ebx
77f0e33b 5e              pop     esi
77f0e33c 894514          mov     dword ptr [ebp+14h],eax
77f0e33f 5b              pop     ebx
77f0e340 0f85bc700400    jne     LdrpMapViewOfDllSection+0x75

Sure enough, the user mode loader uses the current thread’s NT_TIB.ArbitraryUserPointer to communicate the DLL name string pointer (in this context, the “eax” value loaded into NT_TIB.ArbitraryUserPointer is the dll name string.) We can easily verify this in the debugger:

Breakpoint 0 hit
eax=0017ecfc ebx=00000000 ecx=0017ecd8
edx=774951b4 esi=c0000135 edi=0017ed80
eip=773fe2e5 esp=0017ec10 ebp=0017ed18
iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b
gs=0000             efl=00000246
ntdll!LdrpMapViewOfDllSection:
773fe2e5 8bff            mov     edi,edi
0:000> g 773fe31a 
eax=001db560 ebx=00000000 ecx=0017ecd8
edx=774951b4 esi=7ffdf000 edi=20000000
eip=773fe31a esp=0017ebf0 ebp=0017ec0c
iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b
gs=0000             efl=00000246
ntdll!LdrpMapViewOfDllSection+0x4d:
773fe31a 894614          mov     dword ptr [esi+14h],eax
0:000> du @eax
001db560  "C:\\Windows\\system32\\CLBCatQ.DLL"

Looking in the kernel, we can clearly see the call to DbgkMapViewOfSection:

ntoskrnl!NtMapViewOfSection+0x21a:
0060a9b6 50              push    eax
0060a9b7 8b55e0          mov     edx,dword ptr [ebp-20h]
0060a9ba 8b4dd8          mov     ecx,dword ptr [ebp-28h]
0060a9bd e86e1c0100      call    ntoskrnl!DbgkMapViewOfSection

Additionally, we can see the references to NT_TIB in DbgkMapViewOfSection:

ntoskrnl!DbgkMapViewOfSection+0x65:
;
; Load eax with the address of the current thread's
; KTHREAD object.
;
; Here, fs refers to the KPCR.
;    +0x120 PrcbData         : _KPRCB
;  (in KPRCB)
;    +0x004 CurrentThread    : Ptr32 _KTHREAD
;
0061c695 64a124010000    mov     eax,dword ptr fs:[00000124h]
;
; Load esi with the address of the current thread's
; user mode PTEB.
;
; Here, we have the following layout in KTHREAD:
;    +0x084 Teb              : Ptr32 Void
;
0061c69b 8bb084000000    mov     esi,dword ptr [eax+84h]
0061c6a1 eb02            jmp     DbgkMapViewOfSection+0x75
ntoskrnl!DbgkMapViewOfSection+0x75:
0061c6a5 3bf3            cmp     esi,ebx
0061c6a7 7421            je      DbgkMapViewOfSection+0x9a
0061c6a9 3b8a44010000    cmp     ecx,dword ptr [edx+144h]
0061c6af 7519            jne     DbgkMapViewOfSection+0x9a
0061c6b1 56              push    esi
0061c6b2 e82c060200      call    DbgkpSuppressDbgMsg
0061c6b7 85c0            test    eax,eax
0061c6b9 0f85bf000000    jne     DbgkMapViewOfSection+0x144
0:000> u
ntoskrnl!DbgkMapViewOfSection+0x8f:
;
; Recall that 14 is the offset of the
; ArbitraryUserPointer member in NT_TIB,
; and that NT_TIB is the first member of TEB.
;
;    +0x000 NtTib            : _NT_TIB
;  (in NT_TIB)
;    +0x014 ArbitraryUserPointer : Ptr32 Void
;
0061c6bf 83c614          add     esi,14h
;
; [ebp-90h] is now the current thread's value of
; NtCurrentTeb()->NtTib.ArbitraryUserPointer
;
0061c6c2 89b570ffffff    mov     dword ptr [ebp-90h],esi

Thus is the story of how the filename that you pass to LoadLibrary ends up being communicated to the debugger, in a rather round-about and hackish way.

It is also worth noting that the kernel cannot trust the user mode supplied filename for use with opening the file handle to the DLL passed to the debugger process. This is because the kernel uses ZwOpenFile which bypasses normal security checks. As a result, the kernel needs to retrieve the filename via querying the section’s associated PFILE_OBJECT anyway, although for different purposes than providing the filename to the debugger.

Posted in Debugging, NT Internals, Windows | 1 Comment »

The kernel object namespace and Win32, part 3

Wednesday, November 22nd, 2006

Recently, I have posted about various features of the kernel object namespace. Here’s the table of contents for this series:

  1. The kernel object namespace and Win32, part 1.
  2. The kernel object namespace and Win32, part 2.
  3. The kernel object namespace and Win32, part 3.

This posting (the last in the series) attempts to focus on the remaining two parts of the kernel namespace that are visible to Win32. These two parts are broken up as follows:

  1. DOS devices. DOS device names are object names that can be manipulated with the file management functions, such as CreateFile or DefineDosDevice. The set of DOS devices also includes traditional DOS device names (hence the name), such as NUL, CON, AUX, and soforth. This object namespace is “sessionized” by Terminal Server.
  2. Window station and desktop objects. These are infact named kernel objects, although this fact is only barely made visible to Win32. Like DOS devices, window stations and desktops have a namespace that is “sessionized” in the presence of Terminal Server.

Frist, I’ll cover how DOS devices are “sessionized”. This information is as of how the namespace is implemented in Windows XP and later.

In Windows XP, DOS devices are virtualized based off of the LSA logon session ID a token has associated with it (otherwise known as an authentication ID). This value is actually a bit more granular than the Terminal Server session ID.

While a Terminal Server session ID represents a complete “terminal” (essentially, a complete input/output stack with all of the software support that goes with that – a “session”), an LSA logon session ID simply represents an instance of a user that is logged on to the local computer.

Now, in normal circumstances, there is typically a very similar correlation between Terminal Server sessions and LSA logon sessions. For each Terminal Server session, there is typically at least one LSA logon session, which corresponds to the user that is logged on interactively to that Terminal Server session. However, this is not necessarily a one-to-one relationship; for instance, a Terminal Server session that is idle does not really have an LSA logon session ID associated with it (other than perhaps the system logon session, under which winlogon and csrss operate). Or, a Terminal Server session could have many LSA logon session IDs associated with it; think of the case where a user starts a program under a different account with the “RunAs” command. To add to that, some LSA logon session IDs may not even correspond to a local user at all; for example, a remote network logon from a different computer that corresponds to an attempt to access a share hosted on the local machine may be given a logon session ID, even though it corresponds to no user physically logged on to the local machine.

The reason why DOS devices must be virtualized is fairly obvious; different users may have different drive maps, and you need to be able to map the same drive letter as someone else in a different session. The way this was handled needed to be more granular than Terminal Server sessions, though (at least in Windows XP); this is due to how RunAs interacts with drive share access and network drive mappings.

Specifically, the behavior of associating DOS devices with an LSA logon session ID is used because the credentials used to access a remote share are associated with the LSA logon session of the calling user. This means that if you start a program under RunAs, it won’t have access to the same credentials as the parent LSA logon session. As a result, a program started via RunAs doesn’t really have access to network drive mappings (in particular, drive letter based network drive mappings). Because of this, it makes more sense to isolate DOS devices based on LSA logon sessions than Terminal Server sessions, as a separate LSA logon session cannot access drive share connections made by a different LSA logon session anyway.

The resulting architecture that isolates DOS devices on a per-LSA-logon-session basis thus operates as follows:

For each Terminal Server session ID, there is an object directory of the name \Sessions\<TS-session-ID>\DosDevices. Although this object directory is created under the session-named directory for each Terminal Server logon session, only the object directory \Sessions\0\DosDevices is actually used in Windows XP and beyond.

In the the \Sessions\0\DosDevices directory, there is a set of subdirectories that are each named by a LSA logon session id, in the form of a 64-bit hexadecimal number with two 32-bit halves separated by a dash, with each 32-bit half padded out to 8 characters by leading zeros (Terminal Server session IDs used to name the \Sessions\<TS-session-ID> object directories are decimal with no leading zeroes). So, you might see a directory named in the form \Sessions\0\DosDevices\00000000-00b73dfe for an LSA logon session ID 0x0000000000b73dfe.

Each of these object directories named by an LSA logon session ID contains any DOS devices that were created by that LSA logon session. The way the system evaluates DOS device names is by first trying the LSA logon session local object directory, and then trying the global DOS device object directory (which is named \GLOBAL??, and has DOS devices that are associated with no one session in particular, such as physical hard drive DOS device names like C:, or predefined DOS devices like NUL or PhysicalDrive0). To override this lookup behavior and specify that a DOS device name only references the corresponding object in the global DOS device directory, the “Global\” prefix may be used with the DOS device name (similar to named kernel objects, such as mutex objects). This support is handled by a symbolic link named “Global” that is created in each LSA-logon-session ID-named object directory. This symbolic link refers to the “\GLOBAL??” object directory. (Of course, only \DosDevices is publicly documented, to my knowledge.)

If you’ve been reading the previous articles, you may be wondering how the “\DosDevices” and “\??” object directories relate to the “\GLOBAL??” object directory that I just said contained all global DOS device symbolic links (after all, the DDK documents these symbolic links as being present in “\DosDevices”). Well, the way it ends up working out is that there is a symbolic link named “\DosDevices” which points to “\??”. The “\??” name is a bit of a hack; it is not actually a real object name, but instead a hardcoded special case in the object manager’s parser routines that is interpreted as a reference to the global DOS device directory. (This was done for performance reasons; it is very common to look up objects in the \DosDevices directory, and adding a special case check first, before the normal object directory traversal speeds up name translations for Win32 names a bit.) As a result, you can actually use “\??, “\DosDevices”, and “\GLOBAL??” to refer to global DOS devices; they all end up pointing to the same underlying location (eventually).

The other part of Terminal Server session instancing on the kernel object namespace deals with window stations and desktops. Window stations for a particular Terminal Server session are placed in a directory named in the form of \Sessions\<TS-Session-ID>\Windows\WindowStations\<Window-Station-Name> (desktop objects have names that treat a window station as an object directory, such as \Sessions\2\Windows\WindowStations\WinSta0\Default). Note that for session zero, this convention is slightly different; instead, window stations for session zero reside in \Windows\WindowStations.

That wraps up the series on the kernel object namespace (as it relates to Win32). By now, you should have a working understanding of how to use the kernel object namespace from Win32, and how the “magic” behind Terminal Server and LSA logon session isolation is actually implemented. (It is worth noting that the abstraction layer that Win32 provides added real value here, as there were no gross hacks required to be added on to the kernel object namespace to support Terminal Server. Instead, everything is implemented as plain object directories and symbolic links.)

Terminal Server session isolation for the rest of Win32 (namely, the input/output portion) is much more complicated and is hardly as clean as the separation imposed on the object namespace is. That, however, is a story for a different series…

Posted in NT Internals, Windows | 5 Comments »

The kernel object namespace and Win32, part 2

Wednesday, November 15th, 2006

Last time, I talked about how the kernel object namespace intersects with the Win32 world as of how things stood in NT4 (and Windows 2000 when Terminal Server is disabled).

Although the object namespace model used in these operating systems worked well, some cracks began to appear in it with the introduction of Terminal Server as a mainstream product (and later, RunAs).

There are basically two problems that are imposed by Terminal Server:

  1. Many programs are designed to run as “singleton” programs, but in multi-session environments like Terminal Server, it is desirable to allow each session to run an instance of such programs. The classic way to enforce singleton behavior like this is to create a named kernel object at startup, and check to see if the object already existed before the program tried to create it. Without alterations to how the object namespace presented to Win32 functions, this would prevent singleton applications from working under Terminal Server.
  2. Drive letters and other “DOS Devices” symbolic links need to become “sessionized”, because the “glass terminal” (physical computer console) typically has things like COM1 or LPT1 pointing to physical serial ports or printer ports, whereas Terminal Server sessions might be using device redirection to point those device names to serial ports or printer ports on the Terminal Server client machine.

The solution to this problem was partitioning the view of the kernel object namespace provided to Win32 based on Terminal Server session id. This is done with the use of a set of symbolic links and object directories.

The basic idea is that session zero continues to use the \BaseNamedObjects object directory, as how things used to work on downlevel systems. In this object directory, there are a couple of new symbolic links:

  • \Local, which points to the session local namespace. For session zero, this symbolic link typically points to \BaseNamedObjects. For other sessions, it points to a “sessionized” namespace, such as \Sessions\<Terminal-Server-Session-ID>\BaseNamedObjects.
  • \Global, which always points to the session zero namespace (the “global” namespace). This always points to \BaseNamedObjects for all sessions.
  • \Session, which points to \Sessions\BNOLINKS. This latter symbolic link is not documented (except to say that it is “reserved for system use”), but its function is to allow one session (with the appropriate access granted to it) to create objects in an arbitrary session local namespace, by using a path in the form \Session\<Terminal-Server-Session-ID>\ObjectName. \Sessions\BNOLINKS is an object directory which contains a set of symbolic link objects, each named after a Terminal Server session ID. These links point to the appropriate “sessionized” BaseNamedObjects directory for each session (such as \BaseNamedObjects for session zero, \Sessions\1\BaseNamedObjects for session 1, and soforth).

For sessions other than session zero, a BasedNamedObjects directory named in the form of \Sessions\<Terminal-Server-Session-ID>\BaseNamedObjects is created. This is the session local namespace for that session, and it is what is linked to via the “\Local” symbolic link. Additionally, a corresponding symbolic link in \Sessions\BNOLINKS is created so that the (undocumented) “\Session” link works for the new session.

This scheme allows for maximum compatibility with pre-Terminal Server applications which are not aware of the “session isolation” concept, and need some help in order to have their named objects placed in a “sessionized” location where they will not conflict with other user sessions. A means for programs that truly need globally-accessible object names is also provided (the magical \Global prefix symbolic link), which is typically used by services and user-level UI applications that need to communicate with their privileged service counterpart.

In addition to the session isolation of kernel object names, “DOS device” names also became isolated based on Terminal Server session ID. The way this works differs between Windows 2000 and future OS versions, though; I’ll cover it in the next installment of this series. The basic idea for Windows 2000 is that each session got its own directory for DOS device names, but Windows XP and beyond go one step further to better accomodate the “runas” case.

Posted in NT Internals, Windows | 2 Comments »

Debugger flow control: More on breakpoints (part 2)

Wednesday, November 8th, 2006

Last time, I discussed the two types of breakpoints that you’ll see in a debugger (hardware and sfotware) at a high level. I didn’t really explain when it was best to use one instead of the other, though, besides a couple hints relating to hardware breakpoints being limited in number and good for tracking down memory corruption issues at times.

By taking a closer look at how each of the two breakpoints work, we can get some idea as to when we’ll prefer one to another. Both types of breakpoints alter the target in some way, but to differing degrees.

The primary concern with software breakpoints is that they actually involve patching memory in the target to set the breakpoint. This is usually fine; the debugger uses it as its default breakpoint strategy when you give an end address to g, for instance. However, it begins to break down if the target both executes a region that you are setting a breakpoint on, and also reads that same region.

This particular concern is a real problem when you are dealing with self-modifying code, or certain protection schemes (such as code that attempts to checksum itself in memory). In these cases, you might accidentally break self-modifying code, or trip a protection scheme, simply by virtue of setting a breakpoint (since the act of setting a software breakpoint actually modifies the address space of the target).

In cases like this, hardware breakpoints can come to the rescue. Since setting an execute hardware breakpoint does not actually modify the underlying instruction, anything that reads the memory backing that instruction will not get back an 0xCC opcode instead of the real first byte of the instruction opcode. Granted, you can only have four enabled hardware breakpoints at a time, but usually you can get by with that many (or at least, a “sliding window” of hardware breakpoints, assuming you have breakpoints over a well-defined execution sequence. In this case, you could have breakpoints disable themselves and enable the next breakpoint, thus conserving the number of active breakpoints).

There is also the other obvious advantage to hardware breakpoints which I touched on earlier: the ability to set a breakpoint on a memory fetch for a particular address. This obviously has a great deal of uses, whether you’re reverse engineering something or are tracking down a corruption problem. Memory-access breakpoints are an excellent way to very quickly figure out which piece of code is modifying a variable, without having to trace through an arbitrarily large set of code to find the access that you were looking for. One thing to consider about memory-access breakpoints on x86 and x64, though, is that there is only support for setting memory-access breakpoints on regions that are 1) a power of 2 in length, and 2) have a length that is less than or equal to the native pointer size (8 for x64, or 4 for x86). (If you are lucky enough [or perhaps unlucky enough, as Itanium isn’t exactly the most friendly thing to view from an assembler perspective] to be debugging on an Itanium platform, this restriction does not exist; you can set a length of any power of 2 between 1 byte and two gigabytes). As a result, you’ll have to plan where to set your breakpoints carefully, as on x86, you can only cover at most 16 bytes with this kind of “memory guard” access. You might or might not be able to use the same kind of “sliding breakpoint window” idea I mentioned above, if the memory locations you are setting breakpoints on are accessed in a particular sequence (or at least, the accesses that you are interested in).

Hardware breakpoints are typically less invasive than software breakpoints, but there are still ways that they can be interfered with. The most common case for this happening is if you try to set a hardware breakpoint while DLL initializers are being called during process startup (such as at the initial create process breakpoint). If you try to do this, you’ll get a warning from the debugger advising you that your breakpoints won’t stick:

0:000> ba e1 kernel32!CreateFileA
        ^ Unable to set breakpoint error
The system resets thread contexts after the process
breakpoint so hardware breakpoints cannot be set.
Go to the executable's entry point and set it then.
 'ba e1 kernel32!CreateFileA'

The reason why this is the case is that there is a context set that occurs between the initial process breakpoint being hit and the requested thread start address / process start address being executed. I’ll go into just how this works at process startup in a future posting, but to keep it simple, the basic idea is that an APC is queued to the new usermode thread that runs the loader component in NTDLL. One of the arguments to the APC is a context record describing the register context that was requested for the new thread by CreateProcess, CreateThread, and soforth. The loader component runs process (or thread) DLL initializers, and then calls NtContinue to continue execution at the specified context record, which kicks off execution at the user requested thread start address. We can see this in action easily by looking at the arguments that the APC dispatcher supplies to the loader initializer APC:

0:000> kv
ChildEBP RetAddr  Args to Child              
0013fb1c 7c93edc0 7ffdf000 ntdll!DbgBreakPoint
0013fc94 7c921639 0013fd30 ntdll!LdrpInitializeProcess+0xffa
0013fd1c 7c90eac7 0013fd30 ntdll!_LdrpInitialize+0x183
00000000 00000000 00000000 ntdll!KiUserApcDispatcher+0x7
0:000> .cxr 0013fd30 
eax=4ad05056 ebx=7ffdd000 ecx=00f2faa8 edx=00090000
esi=7c9118f1 edi=00011970
eip=7c810665 esp=0013fffc ebp=7c910570 iopl=0
         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=0038  gs=0000
             efl=00000200
kernel32!BaseProcessStartThunk:
7c810665 33ed            xor     ebp,ebp

If you have been paying attention so far, it should be clear why hardware breakpoints set at the initial process breakpoint do not appear to work like you might expect when you set them at the initial process breakin: When the APC that runs loader initializers returns, it restores a previously saved register context image via NtContinue. Since hardware breakpoints are part of the register context, they are wiped away after the context is restored, and so your breakpoints would appear to simply disappear after DLL initializers were finished.

This limitation also implies that calling SetThreadContext on a thread can interfere with hardware breakpoints if care is not taken to preserve the value of the Dr series of registers. Indeed, some protection schemes utilize such a trick in an attempt to defeat hardware breakpoints.

Fortunately, it is easy to work around such limitations using the debugger. There is a little-used command called “.apply_dbp” that allows you to instruct the debugger that it should re-apply hardware breakpoints, either to the current register context, or a saved register context image in-memory (supplied by the /m Context argument). With the use of this command, you can quickly restore your hardware breakpoints even after something attempts to trash them. Combined with a conventional breakpoint on, say, kernel32!SetThreadContext, this can be used to quickly re-enable the use of hardware breakpoints on such cases. You can also use this trick to persist hardware breakpoints in the process startup case, by using .apply_dbp /m <address-of-context-record-argument-from-APC-dispatcher> to enforce any hardware breakpoints you set in the register context image that will eventually be restored by NtContinue. For instance, in the case of the example that I gave above, you might use the following to apply hardware breakpoints to the context that NtContinue will restore:

0:000> .apply_dbp /m 0013fd30 
Applied data breakpoint state

Next up, some more tricks that you can do to get the most out of controlling the target in the debugger.

Posted in Debugging, NT Internals, Reverse Engineering, Windows | 1 Comment »

The kernel object namespace and Win32, part 1

Thursday, October 26th, 2006

The kernel object namespace is partially exposed by various Win32 APIs. Everything that allows you to create a named object that returns a kernel handle is interacting with the kernel object namespace in some form or another, and many Win32 APIs internally use the object namespace under the hood.

The kernel object namespace is fairly similar to a filesystem; there are object directories, which contain named objects. Objects can be of various different types, such as a Device object (created by a kernel driver) or an Event object, a Semaphore object, and soforth. Additionally, there are symbolic link objects, which (like filesystem links on a UNIX-based system) allow you to create one name that simply refers to another named object in the system.

Until the introduction of Windows 2000, the part of the kernel object namespace that Win32 exposed was a fairly limited and simple subset of the full object namespace available to drivers and programs using the native system call interfaces.

First, file-related APIs interact with the \DosDevices object directory (otherwise known as \??). This is the object directory that holds anything that you might open with CreateFile() and related calls, such as drive letter links (say, C:), serial ports (COM1), other standard DOS devices, and custom devices created by kernel drivers. This is why, if you are a driver, you need to explicitly specify \DosDevices\DeviceName instead of that being automatically assumed (as it is in Win32, if you call CreateFile). Otherwise, the created object name will not be easily accessible to Win32.

Secondly, there is the \BaseNamedObjects object directory. This object directory is where named Event, Mutex, Semaphore, and Section (file mapping) objects are based at when created with the Win32 API.

\BaseNamedObjects is managed and created by the Base API server dll (basesrv.dll) running in the context of CSRSS at boot time. This means that, in particular, boot start drivers cannot rely on \BaseNamedObjects as being present early in the boot process (which can be a problem if you want to share a named event object with a user mode program, from a boot start driver). \DosDevices, however, is created by the kernel itself at boot time and is generally always accessible.

In general, that is the limit to how much of the kernel namespace is directly exposed to (and used to support) Win32 prior to Windows 2000. (This is technically not quite true. There is a little used pair of kernel32 APIs called DefineDosDevice and QueryDosDevices that allow limited manipulation of symbolic links based within the \DosDevices object directory. Using these APIs, you can discover the native target names of many of the internal symbolic links (for example, C: -> \Device\HarddiskVolume2). You can also create symbolic links based in \DosDevices that point to other parts of the NT object namespace with the DDD_RAW_TARGET_PATH flag using DefineDosDevice.).

Next time I’ll go into a bit more detail as to how some of the changes to the object manager namespace work with Windows 2000, and then Windows XP, which both introduce some significant changes to how Win32 interacts with object names (first with improved multi-session support for Terminal Server and Fast User Switching, and then with how mapped drive letters work with LSA logon sessions).

Posted in NT Internals, Programming, Windows | 5 Comments »

Newer Entries »