Introduction to x64 debugging, part 4

Last time, I talked about how exception handling and unwinding works in x64, what it means to you when debugging, and how you can access exception handlers from the debugger. In this installment, I’ll be covering some more of the common pitfalls that can sneak up and bite you when doing Wow64 debugging with the native x64 debugger.

As I had alluded to in the first installment of this series, debugging Wow64 programs with the x64 debugger introduces a lot of extra complexity. I had already illustrated one of the major annoyances – that you need to manually toggle between the x86 and x64 contexts in many places.

The problems don’t end there, though. Many extensions, especially legacy extensions that were written long before x64 was introduced do not handle the Wow64 case gracefully. This results from extensions not properly checking the current effective processor (IDebugControl::GetEffectiveProcessorType). This is something to watch out for if you are writing a debugger extension of your own, as it is no longer enough to just see if the target uses 64-bit pointers or not, since with Wow64 debugging, the target processor type can change rapidly within the debugging session as the user switches modes with the “.effmach” command.

One example of a very useful extension that breaks like this is “!locks”, which analyzes the list of critical sections in a process (maintained by NTDLL) in order to help provide information about deadlocks. The !locks extension will always currently operate on the 64-bit critical section list, which makes it difficult to debug deadlocks in Wow64 programs with the native debugger.

Another common cause for confusion with Wow64 debugging is that references to NTDLL may not actually do what you expect. Under Wow64, there are actually two copies of NTDLL in every 32-bit process; the native 64-bit NTDLL, used by the Wow64 layer itself, and a modified version of the original 32-bit NTDLL (which thunks to Wow64 instead of making system calls itself). The problem here is that if you reference the name “ntdll”, you will tend to get the 64-bit version of ntdll back, even if you are in x64 mode. For instance, consider the following:

0:026:x86> u ntdll!NtClose ntdll!ZwClose:
00000000`78ef1350 4c      dec     esp
00000000`78ef1351 8bd1    mov     edx,ecx
00000000`78ef1353 b80c000000 mov  eax,0xc
00000000`78ef1358 0f05    syscall
00000000`78ef135a c3      ret
00000000`78ef135b 666690  nop
00000000`78ef135e 6690    nop
ntdll!NtQueryObject:
00000000`78ef1360 4c      dec     esp
0:026:x86> .effmach .
Effective machine: x64 (AMD64)
0:026> u ntdll!NtClose
ntdll!ZwClose:
00000000`78ef1350 4c8bd1           mov     r10,rcx
00000000`78ef1353 b80c000000       mov     eax,0xc
00000000`78ef1358 0f05             syscall
00000000`78ef135a c3               ret
00000000`78ef135b 666690           nop
00000000`78ef135e 6690             nop
ntdll!NtQueryObject:
00000000`78ef1360 4c8bd1           mov     r10,rcx
00000000`78ef1363 b80d000000       mov     eax,0xd

Here, we got the same address back even if we switched to x86 mode, and as a result the code we tried to disassemble wasn’t valid (because of the new instruction prefixes added by x64). This can get particularly insidious if you are trying to set a breakpoint in the middle of an ntdll function, since if you are not careful, you might set a breakpoint in the wrong copy of ntdll (and probably in the middle of an instruction, which would likely lead to a crash later on instead of the expected stop at a breakpoint exception). If you want to reference the 32-bit ntdll, then you have to use a special name that is a concatenation of the string “ntdll_” and the base address at which the 32-bit ntdll was loaded. For instance:

0:026:x86> u ntdll_7d600000!NtClose
ntdll_7d600000!NtClose:
00000000`7d61c917 b80c000000 mov  eax,0xc
00000000`7d61c91c 33c9    xor     ecx,ecx
00000000`7d61c91e 8d542404 lea    edx,[esp+0x4]
00000000`7d61c922 64ff15c0000000 call dword ptr fs:[000000c0]
00000000`7d61c929 c20400  ret     0x4
00000000`7d61c92c 8d4900  lea     ecx,[ecx]
ntdll_7d600000!NtQueryObject:
00000000`7d61c92f b80d000000 mov  eax,0xd
00000000`7d61c934 33c9    xor     ecx,ecx

Another common gotcha is forgetting that you are in the wrong processor mode for the code you are disassembling. The disassembler operates in the current effective processor as set by “.effmach”, regardless of whether you are disassembling code in a 32-bit or 64-bit module. This can be confusing if you forget to change the processor type, as you can end up looking at something that is almost valid code, but not quite (due to some subtle differences in the 32-bit and 64-bit instruction sets).

Finally, one other source of confusion can be filenames. Remember that under Wow64, programs have an altered view of cetain filesystem locations, such as %SystemRoot%\System32. Some filenames (especially for loaded modules) may refer to %SystemRoot%\system32, and some may refer to %SystemRoot%\syswow64. Despite the difference in apparent filenames, if you are debugging a Wow64 process, these two directories are the same (and both refer to %SystemRoot%\SysWOW64 on the actual filesystem as viewed from 64-bit programs).

Next time: Tricks for catching 64-bit portability problems with the debugger.

2 Responses to “Introduction to x64 debugging, part 4”

  1. mattd says:

    Great series man. You certainly know your stuff inside and out.

  2. Yuhong Bao says:

    “This can be confusing if you forget to change the processor type, as you can end up looking at something that is almost valid code, but not quite (due to some subtle differences in the 32-bit and 64-bit instruction sets).”
    Be especially suspicious if you see, for example, register INC or DEC instructions (these are REX prefixes in 64-bit mode) or ARPL instructions (these are MOVSXD instructions in 64-bit mode.