Nynaeve

(Warning: Long post about Exchange 2007 setup woes ahead.)

Recently, I’ve had the chance (or the misfortune, some might say) of having some time with poking around in an Exchange 2007 configuration. Here’s a brief list of some of the more annoying things I’ve ran into along the way, and how I resolved them (in no particular order):

1. Offline Address Book (OAB) doesn’t work in RPC-HTTP (Outlook Anywhere) mode unless you have autodiscovery set up. This one took me several weeks to figure out. It was a pretty annoying problem because first I didn’t even know what was the problem – when I configured Outlook 2007 to talk to Exchange in native RPC mode, and then configured it to work over Outlook Anywhere (RPC-HTTP), some indeterminant time after that I would occasionally get complaints from Outlook that send/receive had failed because an item was not found. Now, I enabled full Outlook debug logging, but of course, there was absolutely no mention of this anywhere at all in Outlook’s logs on disk – in fact, there was barely anything useful there at all. Only by searching in Google for a long time did I learn that it might be related to the OAB (or Offline Address Book). After learning this, I narrowed it down to the OAB by determining that doing a send/receive just for purposes of downloading the address book would always break (you can do this from the send/receive menu).

   However, fixing the problem was quite another story. The thing that was most frustrating is that, of course, RPC-HTTP runs over SSL and so you can’t do a packet capture of what was going on. So I tried checking IIS logs, but there weren’t any hits outside of the RPC-HTTP proxy URL (nothing at all apparently related to OAB). Most of the information I had found on Google / Microsoft.com related to things like the OAB distribution URL not being set, the OAB virtual directory not being created in IIS, and a variety of other problems, all of which I could rule out as not applicable to me.

   The whole time, the Exchange address book worked fine over OWA too. And to make things even stranger, I seem to recall that it magically appeared to work if I switched Outlook off of RPC-HTTP and back to direct RPC connectivity.

   It turns out that the real problem here was that I didn’t have autodiscovery working correctly for one of the mail domains in the Exchange environment. I only thought to look at autodiscovery after reading this post about somebody else’s OWA woes. Apparently, Outlook wanted to talk to https://example.com/autodiscover or https://autodiscover.example.com/autodiscover (where “example.com” is the mail domain in use) in order to determine the OAB download URL. This explained the lack of hits in my IIS logs, as the mail server for this domain happened to be on a completely different box from anything else on that domain, so hits on example.com/autodiscover would never show up. Because the mail server wasn’t even on that domain, I decided to just go with autodiscover.example.com. However, this presented a problem, as I would need to acquire another cert and another IP address for that
   domain, just for Outlook to not complain about the OAB periodically. Ugh!

   After doing that (I ended up using a subject alternate name (SAN) certificate), the OAB magically began working in Outlook 2007. Hooray.

2. Don’t specify a list of domain names (-DomainName) for New-ExchangeCertificate in quotes if you are requesting a SAN certificate. I spent about 10 minutes staring at my command line wondering just what was possibly wrong with it before I learned that the way New-ExchangeCertificate is written, it expects a list of command line arguments and not a command line argument that is a list (subtle distinction, eh?). This one turned out to be my not paying close enough attention to the documentation examples.
3. Exchange will not accept a certificate with an “E=” in the subject name field for the TLS listeners for IMAP4 / POP3 / SMTP. This one ate up a good chunk of time trying to work through as well. I had filled in an email field out of habit when requesting a cert for Exchange from the domain CA, and everything else in the world besides Exchange had no problem with it. That is to say, IIS liked it, browsers liked it, RDP-SSL liked it, and pretty much everything else I tried with it worked. However, as soon as I gave it to Exchange to use for IMAP4 / POP3 / SMTP, it would barf with an extremely unhelpful (and totally misleading!) event log message:
   

   A certificate for the hostname “example.com” could not be found. SSL or TLS encryption cannot be made to the IMAP service.

   Which was of course, completely wrong. The certificate was there in the cert store for the computer account, and nothing else had any trouble recognizing it. Even the Exchange console recognized it just fine, but the service just would not take it on start.

   Now, to make things even worse, the Event ID of that event log message happened to be “2007“. Try searching in Google for “Exchange 2007 event id 2007” and you’ll see what a wonderful thing that is for getting useful information on the subject. (Hint: You’ll get pages talking about any Exchange 2007 event log message.)

   Finally, I ended up taking the “sledgehammer approach” and just made a new cert, without an “E=” in the subject name, and it magically worked. Grrrrr…

4. Something doesn’t work right with the “-MemberOfGroup” filter when used in conjunction with an email address policy (EAP). For some reason, I could never get this to work. The bizzare thing was, the exact same filter string would work great for an address book policy. Furthermore, when dumping the EAP out, if I ran the LDAP query that the filter OPATH got translated into in the AD management console, it returned the expected results. Even more baffling, if I used any other custom filter besides “-MemberOfGroup”, the EAP would work, which didn’t make any sense at all given that the exact same OPATH filter worked fine with an address book policy. I never got to the bottom of this unfortunately, and finally gave up and used a filter off of one of the AD properties for a user instead (which, by the way, worked fine as both a custom or precanned filter).

   I’m guessing that this one has got to be something misconfigured or broken on my end, but for the life of me I have absolutely no idea what. The particular Exchange install was a fresh one on a completely clean Active Directory, all pretty much the most extreme simple case possible (both of which were, as far as I know, done by the books).

5. Exchange requires that the root certificate authority issuing a non-self-signed certificate be trusted for a certificate to be used for IMAP4 / POP3 / SMTP. Another fun one, unlike everything else in Windows (including RDP-SSL, IIS, etc), Exchange barfs on a certificate for usage as a server if the root CA is not trusted. If you used an external CA to save yourself the headache of setting up a domain CA just for Exchange testing, make sure that Exchange is configured to trust it, or IMAP4 / POP3 / SMTP will all fall over when given a cert issued by that CA. This holds true for both Hub Transport / Client Access / Edge Transport roles in my observation.
6. The Exchange mangement console and Exchange command shell use a ton of memory. In my experience, the management console (MMC applet) displaced something on the order of 150-200MB* of commit if you watch the memory counters closely before it finished loading. The command shell (PowerShell-based) clocked in at a cool 80MB or so. Wow. Whatever happened to Bill Gate’s quote on memory usage:
   

   For DOS LM 1.0, the redirector took up 64K of RAM.

   And Bill went ballistic.

   “What do you mean 64K? When we wrote BASIC, it only took up 8K of RAM. What the f*k do you think idiots think you’re doing? Is this thing REALLY 8 F*ing BASIC’s?”

   Yeah, I’m a programmer, and I realize that more complicated software tends to make memory-performance trade offs. Still, it’s a mail server management UI and a command shell. I find it rather amazing that Visual Studio 2005 Team Suite in all of its glory and .NET-ness still manages to clock in at less memory usage (after loading a project) than the MMC console for a mail server. Oh, and to add insult to injury, the MMC GUI can’t even do about half of the administrative tasks out there, despite its humongous memory footprint (to be fair, the GUI is supposed to be enhanced to cover most of the “missing spots” in the SP1 timeframe, from what I can see). Times change, I guess…

   * Note: To be fair (and more precise), dumping the address space and totalling MEM_COMMIT | MEM_PRIVATE regions turned up ~160MB of unshared memory after spawning one instance of the MMC console when the commit charge for the process was ~200MB.

7. The Exchange 2007 management console has friendly, owner-drawn windows with pretty gradient custom background bitmaps. In other words, the GUI looks nice and polished… until you try and use it over an Internet link instead of a LAN. Then you end up waiting 10 seconds for the “New Mailbox” page to blit its pretty gradient background over to mstsc.exe, block by block. Whoops. To Microsoft’s credit, there is an option (View->Visual Effects->Never) to turn this off. Too bad that it seems to be stuck on maximum graphical awesomeness (otherwise read as excruciatingly slowness over Terminal Server) by default, and that the option is arguably rather well hidden.

Okay, enough bashing on Exchange (and to be fair, the last two items are more things that annoyed me about Exchange than something I’d consider a “problem” in some sense of the word). I like it for all of the cool things it offers, and knowing what I do now, I could probably have gotten a simple Exchange 2007 configuration running in half the time or less than it took the first time around. Furthermore, I’m sure that most of the other large, competing integrated messaging solutions also have their fair share of skeletons in the closet, too.

But that doesn’t change the fact that despite going over the documentation available to me, setting up Exchange was an exercise in banging my head against bizzare problem after bizzare problem, while intermittantly either waiting on pretty dialogs to take forever to transfer over RDP (before I figured out how to “de-prettyify” the GUI), or waiting for the MMC console to finish loading.

Yes, yes, I know, I should be using the PowerShell applet to do all of the work instead of the clunky GUI. Sorry, but I don’t fancy typing out long strings like “First Storage Group”, “CN=First Storage Group,CN=InformationStore”, etc over and over and over again until my fingers bleed. Sometimes, having a GUI is a good thing. Both tools are useful, but sometimes it is more convenient to use the GUI, and sometimes it is more convenient to use the command line. On the plus side, the fact that Exchange 2007 appears to completely support full administration exclusively via a scriptable command line is a big step forward.

Enough of my Exchange meanderings for now. Back to more regular topics next time…

This entry was posted on Tuesday, August 28th, 2007 at 10:40 pm and is filed under Windows. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.