Archive for the ‘Windows’ Category

« Older Entries
Newer Entries »

“Find all references” in VS 8

Monday, August 21st, 2006

There is a cool new feature in Visual Studio 8’s editor UI that is pretty useful if you are trying to determine the scope of a particular change or crossreference a variable or function to determine where it is used: “find all references”.

You can activate this by right clicking a symbol in the source code editor and selecting “find all references”.  A window will open near the build output that lists all source references to the symbol in the current solution (read: workspace).  This is also pretty useful in debugging, if you want to determine where a particular variable might be modified in a particular way.  For example, I used this recently to pull up a list of all locations that might set a particular class pointer to null when debugging a null pointer dereference bug.

Another good use for this is if you are working in unfamiliar code and want to get a feel for the scope of a change you make.  Of course, this utility has some limitations (it only searches within the current workspace), but it can be very handy to quickly gauge how impactful a change might be.

Posted in Programming, Windows | 3 Comments »

You might be using unhandled exception filters without even knowing it.

Friday, August 18th, 2006

In a previous posting, I discussed some of the pitfalls of unhandled exception filters (and how they can become a security problem for your application). I mentioned some guidelines you can use to help work around these problems and minimize the risk, but, as I alluded to earlier, the problem is actually worse than it might appear on the surface.

The real gotcha about unhandled exception filters is that you have probably used them before in programs or DLLs and not even known that you were using them, which makes it very hard to not use them in dangerous situations. How can this be, you might ask? Well, it turns out that the Microsoft C runtime library uses an unhandled exception filter to catch unhandled C++ exceptions and call the terminate handler registered by set_terminate.

This unhandled exception filter is setup by the internal CRT functions _cinit (via _initterm_e). If you have the CRT source handy, this lives in crt0dat.c. The call looks like:

/*
* do initializations
*/
initret = _initterm_e( __xi_a, __xi_z );

Here, “__xi_a” and “__xi_z” define the bounds of an array of function pointers to initializers called during the CRT’s initialization. There is a pointer to a function (_CxxSetUnhandledExceptionFilter) that sets up the unhandled exception filter for C++ exceptions in this array. Unfortunately, source code for the function used to setup _CxxUnhandledExceptionFilter is not present, but you can find it by looking at the CRT in a disassembler.

push    offset CxxUnhandledExceptionFilter
call    SetUnhandledExceptionFilter
mov     lpTopLevelExceptionFilter, eax
xor     eax, eax
retn

This is pretty standard; it is just saving away the old exception filter and registering its new exception filter. The unhandled exception filter itself checks for a C++ exception – if found, it calls terminate, otherwise it tries to verify that the previous exception filter points to executable code, and if so, it will call it.

push    esi
mov     esi, [esp+arg_0]
mov     eax, [esi]
cmp     dword ptr [eax], 0E06D7363h
jnz     short not_cpp_except
cmp     dword ptr [eax+10h], 3
jnz     short not_cpp_except
mov     eax, [eax+14h]
cmp     eax, 19930520h
jz      short is_cpp_except
cmp     eax, 19930521h
jnz     short not_cpp_except 

is_cpp_except:
call    terminate

not_cpp_except:
mov     eax, lpTopLevelExceptionFilter
test    eax, eax
jz      short old_filter_unloaded
push    eax             ; lpfn
call    _ValidateExecute
test    eax, eax
pop     ecx
jz      short old_filter_unloaded
push    esi
call    lpTopLevelExceptionFilter
jmp     short done

old_filter_unloaded:
xor     eax, eax

done:
pop     esi
retn    4

The problem with the latter validation is there is no way to tell if the code is part of a legitimate DLL, or part of the heap or some other allocation that has moved over where a DLL had previously been unloaded, which is where the security risk is introduced.

So, we have established that the CRT potentially does bad things by installing an unhandled exception filter – so what? Well, if you link to the DLL version of the CRT, you are probably fine. The CRT DLL is unlikely to be unloaded during the process lifetime and will only be initialized once.

The kicker is if you linked to the static (non-DLL) version of the CRT. This is where things start to get dicey. The dangerous combination here is that each image linked to the static version of the CRT will have its own copy of _cinit, and its own copy of _CxxSetUnhandledExceptionFilter, its own copy of _CxxUnhandledExceptionFilter, and soforth. What this boils down to is that every image linked to the static version of the Microsoft C runtime installs an unhandled exception filter. So, if you have a DLL (say one that hosts an ActiveX object) which links to the static CRT (which is pretty attractive, as for plugin type DLLs you don’t want to have to write a separate installer to ensure that end users have that cumbersome msvcr80.dll), then you’re in trouble. Since this is an especially common scenario (plugin DLL linking to the static CRT), you have probably ended up using an unhandled exception filter without knowing it (and probably without realizing the implications of doing so) – simply by making an ActiveX control usable by Internet Explorer, for example. This really turns into a worst case scenario when it comes to DLLs that host ActiveX objects. These are DLLs that are going to be frequently loaded and unloaded, are controllable by untrusted script, and are very likely to link to the static CRT to get out of the headache of having to manage installation of the DLL CRT version. If you put all of these things together and throw in any kind of crash bug, you’ve got a recipie for remote code execution. What is even worse is that this isn’t just quick-fixable with a patch to the CRT, as the vulnerable CRT version is compiled into your binaries and not in its own hotfixable standalone DLL.

So, in order to be truly safe from the dangers of unhandled exception filters, you also need to rid your programs of the static CRT. Yes, it does make setup more of a pain, but the DLL CRT is superior in many ways (not to mention that it doesn’t suffer from this security problem!).

Posted in Programming, Reverse Engineering, Security, Windows | 4 Comments »

Win32 calling conventions: __cdecl in assembler

Thursday, August 17th, 2006

Continuing on the series about Win32 calling conventions, the next topic of discussion is how the various calling conventions look from an assembler level.

This is useful to know for a variety of reasons; if you are reverse engineering (or debugging) something, one of the first steps is figuring out the calling convention for a function you are working with, so that you know how to find the arguments for it, how it deals with the stack, and soforth.

For this post, I’ll concentrate primarily on __cdecl.  Future posts will cover the other major calling conventions.

As I have previously described, __cdecl is an entirely stack based calling convention, in which arguments are cleaned off the stack by the caller.  Given this, you can expect to see all of the arguments for a function placed onto the stack before a function call is main.  If you are using CL, then this is almost always done by using the “push” instruction to place arguments on the stack.

Consider the following simple example function:

__declspec(noinline)
int __cdecl CdeclFunction1(int a, int b, int c)
{
 return (a + b) * c;
}

First, we’ll take a look at what calls to a __cdecl function look like. For example, if we look at a call to the function described above like so:

CdeclFunction1(1, 2, 3);

… we’ll see something like this:

; 119  : 	int v = CdeclFunction1(1, 2, 3);

  00000	6a 03		 push	 3
  00002	6a 02		 push	 2
  00004	6a 01		 push	 1
  00006	e8 00 00 00 00	 call	 CdeclFunction1
  0000b	83 c4 0c	 add	 esp, 12

There are basically three different things going on here.

  1. Setting up arguments for the target function. This is what the three different “push” instructions do. Note that the arguments are pushed in reverse order – you’ll always see them in reverse order if they are placed on the stack via push.
  2. Making the actual function call itself. After all the arguments are in place, the “call” instruction is used to transfer execution to the target. Remember that on x86, the call instruction implicitly pushes the return address on the stack. After the function call returns, the return value of the function is stored in the eax (or edx:eax) registers, typically.
  3. Cleaning arguments off the stack after the function returns. This is the purpose of the “add esp, 0xc” instruction following the “call” instruction. Since the target function does not adjust the stack to remove arguments after the call, this is up to the calller. Sometimes, you may see multiple __cdecl function calls be made in rapid succession, with the compiler only cleaning arguments from the stack after all of the function calls have been made (turning many different “add esp” instructions into just one “add esp” instruction).

It is also worth looking at the implementation of the function to see what it does with the arguments passed in and how it sets up a return value. The assembler for CdeclFunction1 is as so:

CdeclFunction1 proc near

a= dword ptr  4
b= dword ptr  8
c= dword ptr  0Ch

mov     eax, [esp+8]    ; eax = b
mov     ecx, [esp+4]    ; ecx = a
add     eax, ecx        ; eax = eax + ecx
imul    eax, [esp+0Ch]  ; eax = eax * c
retn                    ; (return value = eax)
CdeclFunction1 endp

This function is fairly straightforward. Since __cdecl is stack based for argument passing, all of the parameters are on the stack. Recall that the “call” instruction pushes the return address onto the stack, so the stack will begin with the return value at [esp+0] and have the first argument at [esp+4]. A graphical view of the stack layout (relative to “esp”) of this function is thus:

+00000000  r              db 4 dup(?)      ; (Return address)
+00000004 a               dd ?
+00000008 b               dd ?
+0000000C c               dd ?

In this case, there is no frame pointer in use, so the function accesses all of the arguments directly relative to “esp”. The steps taken are:

  1. The function fills eax with the value of the second argument (b), located at [esp+8] according to our stack layout.
  2. Next, the function loads ecx with the value of the first argument (a), which is located at [esp+4].
  3. Next, the function adds to eax the value of the first argument (a), now stored in the ecx register.
  4. Finally, the function multiplies eax by the value of the third argument (c), located at [esp+c].
  5. After finishing with all of the computations needed to implement the function, it simply returns with a “retn” instruction. Since the caller cleans the stack, the “retn” intruction (with a stack adjustment) is not used here; __cdecl functions never use “retn <displacement>“, only “retn”. Additionally, because the result of the “mul” instruction happened to be stored in the eax register here, no extra instructions are needed to set up the return value, as it is already stored in the return value register (eax) at the end of the function.

Most __cdecl function calls are very similar to the one discussed above, although there will typically be much more code to the actual function and the function call (if there are many arguments), and the compiler may play some optimization tricks (such as deferring cleaning the stack across several function calls). The basic things to look for with a __cdecl function are:

  • All arguments are on the stack.
  • The return instruction is “retn” and not “retn <displacement>“, even when there are a non-zero number of arguments.
  • Shortly after the function call returns, the caller cleans the stack of arguments pushed. This may be deferred later, depending on how the compiler assembled the caller.

Note that if you have been paying attention, given the above criteria, you’ve probably noticed that a __cdecl function with zero arguments will look identical to an __stdcall function with zero arguments. If you don’t have symbols or decorated function names, there is no way to tell the two apart when there are no arguments, as the semantics are the same in that special case.

That’s all for a basic overview of __cdecl from an assembler perspective. Next time: more on the other calling conventions at an assembly level.

Posted in Debugging, Programming, Reverse Engineering, Windows | 5 Comments »

Beware of custom unhandled exception filters in DLLs

Wednesday, August 16th, 2006

Previously, I had discussed some techniques for debugging unhandled exception filters.  There are some more gotchas relating to unhandled exception filters than just debugging them, though.

The problem with unhandled exception filters is that they are broken by design.  The API (SetUnhandledExceptionFilter) used to install them allows you to build a chain of unhandled exception filters as multiple images within a process install their own filter.  While this may seem fine in practice, it actually turns out to be a serious flaw.  The problem is that there is no support for removing these unhandled exception filters out of order.  If you do so, you often end up with a previous unhandled exception filter pointer used by some DLL that points to a now-unloaded DLL, because some DLL with an unhandled exception filter was unloaded, but the unhandled exception filter registered after it still has a pointer to the previous filter in the now unloaded DLL.

This turns out to (at best) cause your custom crash handling logic to appear to randomly fail to operate, and at worst, introduce serious security holes in your program.  You can read more about the security hole this introduces in the paper on Uninformed.org, but the basic idea is that if unhandled exception filters are unregistered out of order, you have a “dangling” function pointer that points to no-mans-land.  If an attacker can fill your process address space with shell code and then cause an exception (perhaps an otherwise “harmless” null pointer dereference that would cause your program to crash), he or she can take control of your process and run arbitrary code.

Unfortunately, there isn’t a good way to fix this from an application perspective.  I would recommend just not ever calling the previous unhandled exception filter, as there is no way to know whether it points to the real code that registered it or malicious code that someone allocated all over your process address space (called “heap spraying” in exploitation terminology).

You still have to deal with the fact that someone else might later install an unhandled exception filter ahead of yours, though, and then cause the unhandled exception filter chain to be broken upstream of you.  There is no real good solution for this; you might investigate patching SetUnhandledExceptionFilter or UnhandledExceptionFilter to always call you, but you can imagine what would happen if two functions try to do this at the same time.

So, the moral of the story is as follows:

  1. Don’t trust unhandled exception filters, as the model is brittle and easily breaks in processes that load and unload DLLs frequently.
  2. If you must register an unhandled exception filter, do it in a DLL that is never unloaded (or even the main .exe) to prevent the unhandled exception filter from being used as an attack vector.
  3. Don’t try to call the previous unhandled exception filter pointer that you get back from SetUnhandledExceptionFilter, as this introduces a security risk.
  4. Don’t install an unhandled exception filter from within a plugin type DLL that is loaded in a third party application, and especially don’t install an unhandled exception filter in a plugin type DLL that gets unloaded on the fly.

Unfortunately, it turns out to be even harder than this to not get burned by unhandled exception filter chaining.  More on that in a future posting.

Posted in Programming, Security, Windows | 12 Comments »

Win32 calling conventions: Usage cases

Tuesday, August 15th, 2006

Last time, I talked about some of the general concepts behind the varying calling conventions in use on Win32 (x86).  This posting focuses on the implications and usage cases behind each of the calling conventions, in an effort to provide a better understanding as to when you’ll see them used.

When looking at the different calling conventions, we can see that there are a number of differences between them.  Stack usage vs register parameters, caller vs callee cleans stack, member function calls vs “plain C” function calls, and soforth.  These differences lend each calling convention to specific cases where they are best suited.

To begin with, consider the __cdecl calling convention.  We know that it is a stack based calling convention where the caller cleans the stack.  Furthermore, we know that it is the default calling convention for CL (big hint as to when you’ll see this being used!).  These attributes make it well suited for a couple of cases:

  • Variadic functions, or functions with an ellipsis (…) terminating the argument list.  These functions have a variable number of arguments, which is not known at compile time of the callee.  __cdecl is useful for these functions because the compiler needs to implement a stack displacement to clean arguments off of the stack, but it doesn’t know how many arguments there are.  By leaving the argument-disposal up to the caller, who does know the number of arguments at compile time, the compiler doesn’t need special help from the programmer to correctly adjust the stack when the variadic function is going to return – it “just works”.  __cdecl is the only calling convention on Win32 x86 that supports variadic functions when used with CL.
  • Old-style C functions without prototypes.  For compatibility with legacy C code, the C compiler needs to support making function calls to unprototyped functions.  These must be treated as if they were variadic functions, because the compiler doesn’t know whether the function takes a fixed number of arguments or not (because there is no prototyped argument list).
  • Any other case where the programmer does not explicitly override the calling convention.  The default Visual Studio build environment will use the compiler default calling convention if you do not explicitly tell it otherwise, and this goes to __cdecl.  Some build environments (the DDK/build.exe platform in particular) default to different calling conventions, but Visual Studio built programs will always default to __cdecl if you are using CL.

Next, we’ll take a look at __stdcall.  This calling convention is the standard for Win32 APIs; virtually all system APIs are __stdcall (typically decorated as “WINAPI”, “NTAPI”, or “CALLBACK” in the headers, which are macros that expand to __stdcall).  Here are the typcial usage cases for __stdcall (and the “why” behind them):

  • Library functions.  Excepting the C runtime libraries, virtually all Microsoft-shipped Win32 libraries use __stdcall.  The main reason for this (if you discount the “that’s the way it has always been”) is that you save some instruction code space by using __stdcall and not __cdecl for library functions.  The reason for this is that for __cdecl functions, the caller typically needs to adjust the stack pointer after every “call” instruction to a __cdecl function (which takes up instruction code space – typically an “add esp, imm8” opcode).  For __stdcall functions, you only pay this penality once, in the “retn imm16” opcode at the end of the function (as opposed to once for every caller).  For frequently called functions (say, ReadFile), this begins to add up.  You also theoretically save a bit of processor time and cache space, as there is one less instruction to be executed per “call”.
  • COM functions.  COM uses __stdcall with the “this” pointer being the first argument, which is a required part of the COM API contract for publicly accessible functions.
  • Functions that need to be called from a language other than C/C++.  This also ties back into the COM and library function purposes, but of all of the calling conventions discussed here, only __stdcall has practically universal support among non-Microsoft or non-C/C++ compilers for x86 Win32 (such as Visual Basic, or Delphi).  As a result, it is advantageous to use __stdcall if you are expecting to be called from other languages.
  • Microsoft-built programs.  Microsoft defaults their programs to __stdcall and not __cdecl virtually everywhere, even in images that don’t export functions, or in internally, non-exported funtions within a system library.  This also applies to Microsoft kernel mode code, such as the HAL and the kernel itself.
  • Programs built with the DDK.  The DDK defaults to __stdcall and not __cdecl.
  • NT kernel drivers.  These are always (or at least should always be!) built with the DDK, which again, defaults to __stdcall.

There yet remains __fastcall to discuss.  This calling convention is not used as extensively as the other two (no Microsoft build environment that I am aware of defaults to it), so most of the cases for it being used are the result of a programmer explicitly requesting it.

  • Functions that do not call other functions (“leaf functions”).  These are good candidates for __fastcall because the register arguments are passed in volatile registers, so there is a penalty associated with __fastcall functions that call subfunctions and need to use their arguments across those function calls, as this requires the __fastcall function to save arguments to somewhere nonvolatile (i.e. the stack), and that defeats the whole purpose of __fastcall entirely.
  • Functions that do not use their arguments after the first subfunction call.  These can still benefit from __fastcall without the penalty mentioned above relating to preserving arguments across function calls.
  • Short functions that call other functions and then return.  If you can make both functions __fastcall, then sometimes the compiler can be clever and not need to re-load the argument registers when a __fastcall function calls a __fastcall subfunction.  This can be useful for “wrapper” functions in some cases.
  • Functions that interface with assembly code.  Sometimes it can be more convenient to make a C function called by assembler code __fastcall, because this can save you the work of manually tracking stack displacements.  It can also sometimes be more convenient to make assembler functions called by C code __fastcall as well, for similar reasons.

In general, __fastcall is relatively rare.  There are a couple of kernel functions that use it (KfRaiseIrql, for instance).  A couple of software vendors (such as Blizzard Entertainment) seem to like to ship things compiled with __fastcall as the default calling convention, but this not the common case, and usually not a good idea.

Finally, there is __thiscall.  This calling convention is only used if you are using the default calling convention for member functions.  Note that for member functions that are not accessible cross-program (e.g. not exported somehow), the compiler will sometimes replace ecx with ebx for the “this” pointer as a custom calling convention, depending on your optimization settings.

That’s all for this installment.  In the next posting, I’ll discuss what the various calling conventions look like at a low level (assembly), and what this means to you.

Posted in Debugging, Reverse Engineering, Windows | 4 Comments »

Win32 calling conventions: Concepts

Monday, August 14th, 2006

If you have done debugging work on Win32 (x86) for any length of time, you probably know that there are many different calling conventions.  While I have already covered the Win64 (X64) calling convention, I haven’t yet gone into details about how to work with the various calling conventions present on x86 Windows.  This miniseries is going to go into some detail relating to how the calling conventions work from the perspective of debugging and reverse engineering.

There are three major calling conventions in use in modern Win32 (on x86, anyway): __stdcall, __cdecl, and __fastcall.  All three have different characteristics that are important if you are debugging or reverse engineering something, and it is important to be able to recognize and work with functions written against all three calling conventions – even if you don’t have access to symbols.  To start off, it’s probably best to get a feel for what all of the different calling conventions do, how they work, and soforth.

Most of the calling conventions have some things in common.  All of the calling conventions have the same set of volatile registers, which are not required to remain the same across a call site: eax, ecx, and edx.  Additionally, all three calling conventions use eax for 32-bit return values, and eax:edx for 64-bit return values (high 32 bits in edx).  For large return values (e.g. functions that return a structure, and not a pointer to a structure), a hidden argument is passed to a hidden local variable in the caller’s stack frame that represents the large return value, and the return value is filled in using the hidden argument (that is, large return values are actually implemented as a hidden pointer parameter).

  • __cdecl is the default calling convention for the Microsoft C/C++ compiler.  This is an entirely stack based calling convention for parameter passing; the caller cleans the arguments off of the stack when the call returns.
  • __stdcall has the same semantics as __cdecl, except that the callee (target function) cleans the arguments off of the stack instead of the caller.
  • __fastcall passes the first two register-sized arguments in ecx and edx, with the remaining arguments passed on the stack like __stdcall.  If any stack based arguments were present, the callee cleans them off of the stack.

Additionally, there is a fourth calling convention implemented by CL, known as __thiscall, which is like __stdcall except that there is a hidden pointer argument representing a class object (“this” in C++) that is passed in ecx.  As you might imagine, __thiscall is used for non-static class member function calls (by default).  If you explicitly override the calling convention of member functions, then “this” becomes a hidden (first) argument that is passed according to the conventions of the overriding calling convention.  In general, __thiscall is not really used in the Windows API.

That’s a general overview of the basic concepts behind all of the major Win32 x86 calling convention.  In some upcoming posts, I’ll explore the implications behind the different attributes of these calling conventions, their usage cases, and how they apply to you when you are debugging or reverse engineering something.

Posted in Debugging, Reverse Engineering, Windows | Comments Closed

Checking if a binary exists on a symbol repository

Wednesday, August 9th, 2006

This question came up on the microsoft.public.windbg newsgroup and turned out to be more complicated to solve than it might appear, so I’m posting about it here.

Someone asked if there is a way to use the DTW tools to detect if a binary is present on a symbol repository.  Now, you might naively assume that you can just use symchk.exe on the binary and it will work, and indeed if you try this, you might be fooled into thinking that it does work.  However, it really doesn’t – if you do this, all that symchk will do is verify that the symbols for the binary are on the symbol repository.

If you need to make sure that the binary is there (i.e. so you can do dump file debugging using the symbol repository), then you need to use the “/id filename” command line parameter with symchk.  This tells symchk that you want to verify that symbols for a dump file exist on the symbol server.  Because DbgEng lets you load a single PE image (.dll/.exe/.sys/etc) as a dump file, and because dump files require loading the actual binary itself from the symbol path, this forces symchk to verify that the binary (and not just the symbols) exist on the symbol repository.

Posted in Debugging, Windows | Comments Closed

Debugging a custom unhandled exception filter

Tuesday, August 8th, 2006

If you are working on a custom unhandled exception filter (perhaps to implement your own crash dump / error reporting mechanism), then you have probably run into the frustrating situation where you can’t debug the filter itself in case there is a crash bug in it.

The symtoms are that when a crash occurs, your crash reporting logic doesn’t work at all and the process just silently disappears.  When you attach a debugger and reproduce the crash to figure out what went wrong, the debugger keeps getting the exceptions (even if you continue after the first chance exception), and your unhandled exception filter is just never called.

Well, the reason for this is that UnhandledExceptionFilter tries to be clever and hides any last chance exception filter handling when a debugger is active.  Here’s why: the default implementation of the unhandled exception filter launches the JIT debugger and then restarts the exception.  When the JIT debugger attaches itself and the exception is restarted, you clearly want the exception to go to the debugger and not the unhandled exception filter, which would try to launch the JIT debugger again.

If, however, you have a custom unhandled exception filter that is crashing, then you probably don’t want this behavior so that you can debug the problem with the exception filter.  To disable this behavior and let the exception filter be called even if there is a debugger attached, you will need to patch kernel32!UnhandledExceptionFilter in a debugger.  If you unassemble it, you should eventually see a call to NtQueryInformationProcess that looks like this:

NtQueryInformationProcess(GetCurrentProcess(), 7, &var, 4, 0)

…followed by a comparison of that local variable passed in to NtQueryInformationProcess against 0.  You will need to patch the comparison to treat the local set by NtQueryInformationProcess as if it were zero, perhaps turning it into an unconditional jmp with the “eb” or “a” commands.

This comparison is checking for something called the “process debug port”, which is a handle to an LPC port that is used to communicate with the debugger attached to the current process.  The kernel returns a null handle value if there is no debugger attached, which is how UnhandledExceptionFilter knows whether to work its magic and forward the exception on to the debugger or call the registered unhandled exception filter.

After patching out the check, then exceptions should be forwarded to your unhandled exception filter as if no debugger were there, giving you a chance to inspect the operation of your crash handling code.

Posted in Debugging, Reverse Engineering, Windows | 3 Comments »

Save time when you are debugging by rebasing your DLLs

Monday, August 7th, 2006

If you are working with a process that has many of your DLLs loaded in it, and your DLLs tend to be loaded and unloaded dynamically, then you can sometimes save yourself a lot of trouble when debugging a problem by making sure that each of your DLLs has a unique base address.

That way, if you have a bug where one of your DLLs is called after unloaded, you can easily figure out which DLL the call was supposed to go to and which function it should have gone to by loading the DLL using the dump file loading support (start WinDbg as if you were going to debug a dump file, but select the .DLL in question instead of a .dmp file) and unassembling the address that was referenced.

On Windows Server 2003 and later, NTDLL maintains a list of the last few unloaded modules and their base addresses in user mode (accessible using “lm” in WinDbg) which can make debugging this kind of problem a bit more manageable even if you don’t rebase your DLLs, but rebasing is easy, improves loading performance (and especially scalability under Terminal Server), so I would highly recommend going the rebasing route anyway.

If you weren’t on Windows Server 2003 and didn’t rebase your DLL, then chances are the loader relocated it at load time to some not-predictable location, which makes finding the actual DLL being called and which function / global / etc in the DLL was being referenced when the crash occured much more difficult than if the DLL always loads at its preferred base address and you can simply look up the address in the DLL directly.

Posted in Debugging, Windows | Comments Closed

Using the symbol proxy in cross-domain scenarios with UncAccessFilter

Friday, August 4th, 2006

If you have a symbol proxy setup and you need to have it talk to a symbol server path that references a UNC share on a server that isn’t on the same domain as the IIS webserver hosting symproxy, then you may need to do some hackery to get the symbol proxy to work properly without prompting for credentials to use on the network.

 Specifically, the problem here is that there is no way to tell IIS to try and map a UNC share with a particular username and/or password before processing a request unless the request itself points to a network share.

One way to work around this is to use a simple ISAPI filter that I wrote (UncAccessFilter) to make sure any required UNC paths are mapped before the symproxy ISAPI filter is called.  After installing the ISAPI filter in the usual way, make sure that it is prioritized above the symproxy filter.  To configure it, you will need to manually set up some values in the registry.

 Create the key “HKEY_LOCAL_MACHINE\Software\Valhalla’s Legends\Skywing\UncAccessFilter” and ensure that the user web requests will be running in has read access to it.  You will probably want to ensure that only the web access user, administrators, and the system account have read access to this key because it will have passwords stored in it (be aware of this as a potential security risk if someone gets access to the registry key, as the passwords are not obfuscated in any way).  Then, for each share, create a REG_SZ value whose name is the share path you want to map (e.g. \\fileserver\fileshare) and whose contents are of the format “username;password”, for instance, “fileserver\symbolproxyuser;mypassword”.

To debug the filter, you can create a REG_DWORD value in that key named “DebugEnabled” and set it to 1, in which case the IIS worker process under which the ISAPI filter is running in will do some diagnostic OutputDebugString calls about what operations it is performing if you have a debugger attached to the process.  Assuming you configured the filter properly, on startup, you should see a series of messages listing the configured UNC shares (you may need to attach to the svchost process that creates the w3wp worker processes and use `.childdbg 1′ to catch this message for the new worker processes on startup).

If you are using the prebuilt binaries then make sure to install the VC++ 8 runtimes on the IIS server first.  Note that the prebuilt binaries are 32-bit only at this time, you’ll need to rebuild the ISAPI filter from source if you want to use the filter in 64-bit mode.

Be aware that the ISAPI filter is fairly simple and is not extraordinarily robust (and may be a bit slow if you have high traffic volumes, since it enumerates mapped network shares on every incoming request).  Additionally, be aware that if one of the servers referenced in the registry is down, it can make web requests that you have configured to be filtered by UncAccessFilter take a long time as the filter tries unsuccessfully to reconnect to the configured share on that server.  However, when properly configured, it should get the job done well enough in most circumstances.

Note that if you can get away with using the same account for all of your shares, a better solution is to simply change the account the web application associated with the symbol proxy is running under.  If you need to use multiple accounts, however, this doesn’t really do what you need.

Update: It would help if I had posted the download url.

Posted in Debugging, Windows | 1 Comment »

« Older Entries
Newer Entries »