Nynaeve

Last time, I described (at a high level) the interface used by the Verizon connection manager to talk to the V740, and how it related to my goal of coercing the V740 to re-upgrade to EVDO after failing over to 1xRTT, without idling the connection long enough for dormant mode to take over.

As we saw previously, it turned out that the V740’s abstraction layer module (WmcV740.dll) was static linked to a debug version of a Novatel SDK that encapsulated the mechanism to talk to the V740’s firmware and instruct the device to perform various tasks.

Now, the WmcV740.dll module contains code that is specific to the V740 (or at least specific to Novatel devices), which knows how to talk to a V740 that is connected to a computer. Internally, the way this appears to work is that the Novatel driver creates a virtual serial port (conveniently named Novatel Wireless Merlin CDMA EV-DO Status Port (COMxx)), which is then used by the DLL to send and receive data from the firmware interface. In other words, the virtual serial port is essentially a “backdoor” control channel to talk to the firmware, separate from the dial-up modem aspect of the device that is also presented by the driver in the form of a modem device that can be controlled via standard “AT” commands. (The advantage to taking this approach is that a serial port is typically only accessible by one program at a time, and if one is using the standard RAS/dial-up modem support in Windows to dial the connection, then this precludes being able to use the modem serial port to perform control functions on the device, as it’s already being used for the data session.)

By simply looking at the debug print strings in the binary, it’s possible to learn a fair amount at what sort of capabilities the SDK functions baked into the binaries might hold. Most functions contained at the very least a debug print at the start and end, like so:

Diag_Get_Time   proc near

push    ebp
mov     ebp, esp

[...]

 ; "Diag_Get_Time: Begin\\n"
push    offset aDiag_get_timeB
push    2               ; int
call    DebugPrint
add     esp, 8
cmp     IsNewFirmware, 0
jnz     short FirmwareOkay1

cmp     FirmwareRevision, 70h
jnb     short FirmwareOkay1

 ; "Diag_Get_Time: End: Old Firmware\\n"
push    offset aDiag_get_timeE
push    2               ; int
call    DebugPrint
add     esp, 8
mov     ax, 14h
jmp     retpoint

[...]

Although most of the routines named in debug prints didn’t appear all that relevant to what I was planning on doing, at least one stood out as worth further investigation:

strings WmcV740.dll | find /i "Begin"
[..]
Diag_ERI_Clear: Begin
Diag_Call_Origination: Begin
Diag_Call_End: Begin
Diag_Read_DMU_Entry: Begin
[...]

(Strings is a small SysInternals utility to locate printable strings within a file.)

In particular, the Diag_Call_End routine looked rather interesting (recall that at least for my handset, pressing the “end call” button on the device while a data connection is active places it into dormant mode). If this routine performed the same function as my handset’s “end call” button, and if “ending” the data call also put the V740 into dormant mode (such that it would re-select which network to use), then it just might be what I was looking for. There were several other functions that looked promising as well, though experimentation with them proved unhelpful in furthering my objective in this particular instance.

At this point, there were essentially two choices: Either I could try and re-implement the code necessary to speak the custom binary protocol used by WmcV740.dll to communicate with the device, or I could try and reuse as much of the already-existing code in the DLL as possible to achieve the task. There are trade-offs to both approaches; reimplementing the protocol would allow me to bypass any potential shortcomings in the DLL (it actually turns out that the DLL is slightly buggy, and the commands to reset power on the device will result in heap corruption on the process heap – ick! – fortunately, those functions were not required for what I wanted to do). Additionally, with a “from-scratch” implementation, assuming that I was correct about my theory that the the driver’s virtual serial port is just a pass-through to the firmware, such a re-implementation would possibly be more portable to other platforms for which Novatel doesn’t supply drivers (and might be extendable to do things that the parts of the SDK linked to WmcV740.dll don’t have support for).

However, taking the approach of reimplementing a client for the firmware control interface also would require a much greater investment of time (while not extremely complicated, extra work would need to be done to reverse enough of the protocol to get to the point of sending the desired commands as performed by the “Diag_Call_End” function that we’re interested in) compared to simply reusing the already existing protocol communications code in the DLL. Furthermore, reimplementing the protocol client from scratch carries a bit more risk here than if you were just reverse engineering a network protocol, because instead of talking to another program running on a conventional system, in this case, I would be talking to the firmware of a (non-serviceable-by-me) device. In other words, if I managed to botch my protocol client in such a way as to cause the firmware on the device to do something sufficiently bad as to break the device entirely, I’d have a nice expensive paperweight on my hands (without knowing a whole lot more about the firmware and the device itself, it’s hard to predict what it’ll do when given bad input, or the like – although it might well be quite robust against such things, that’s still a relatively risky decision to make, because it could just as well fall over and die in some horribly bad way on malformed input). Not fun, by any means, that. To add to that, at this point, I didn’t even know if the Diag_Call_End function would pan out at all, so if I spent all the time to go the full nine yards just to try it out, I might be blowing a lot of effort on another dead end.

Given that, I decided to go the more conservative route of trying to use the existing code in WmcV740.dll, at least initially. (Although I did research the actual protocol a fair bit after the fact, I didn’t end up writing my own client for it, merely just reusing what I could from the WmcV740.dll module). However, there’s a minor sticking point here; the DLL doesn’t provide any way to actually reach the code in Diag_Call_End that is externally visible to code outside the module. In other words, there are no exported functions that lead to Diag_Call_End. Actually, the situation was even more grim than that; after a bit of analysis by IDA, it became immediately clear that there weren’t any callers of Diag_Call_End present in the module, period! That meant that I wouldn’t have a “working model” to debug at runtime, as I did with the interface that WmcV740.dll exports for use by the Verizon connection manager GUI.

Nonetheless, the problem is not exactly an insurmountable one, at least not if one is willing to get their hands dirty and use some rather “under-handed tricks”, assuming that we can afford to tie ourselves to one particular version of WmcV740.dll.

Next time: Zeroing in on Diag_Call_End as a potential solution.

Continuing from the previous article, the first step in determining a possible way to kick the V740 back into EVDO mode is to understand how the Verizon connection manager software interfaces with it. In order to do this, I used WinDbg and IDA for debugging and code analysis, respectively.

The software that Verizon ships presents a, uninform user interface regardless of what type of device you might be using. This means that it likely has some sort of generic abstraction layer to communicate with any connected devices (as the connection manager software supports a wide variety of cards and tethered phones, from many different manufacturers).

Indeed, it turns out that there is an abstraction layer, in the term of a set of DLLs present in the connection manager installation directory, which are named based off of the device they support (e.g. WMCLGE_VX8700.dll, WMC_NOK_6215i.dll, WmcV740.dll). These DLLs implement a high level interface in the form of a (fairly simple) exported C API, which is then used by the connection manager software to control the connected device.

The abstraction layer is fairly generic across devices, and as such does not support a whole lot of advanced device-specific features. However, it does provide support for operations like querying information about the network and over the air protocols in use, link quality (e.g. signal levels), sending and receiving SMS messages, manipulating the phone book (for some devices, not the V740 as it would have it), powering on and off the device, performing over the air activation, and a handful of other operations. In turn, each device-specific module translates requests from the standard exported C API into calls to their associated devices, using whatever device-specific mechanism that device offers for communication with the computer. This is typically done by building a sort of adapter from the connection manager’s expected C API interface to a vendor-specific library or SDK for communicating with each device.

As most of the potentially APIs in question have only one or two parameters (and could be easily inspected on the fly by debugging the connection manager) it was fairly trivial to gain a basic working understanding of the device abstraction layer API. The approach that I took to accomplish this was to simply set a breakpoint on all of the exported functions (in WinDbg, bm wmcv740!*), and from there, record which functions were invoked by what parts of the connection manager GUI and inspect in/out parameter values, in conjunction with analysis in IDA. Although there are some fairly tantalyzingly named functions (e.g. WMC_SetPower, WMC_SetRadioPower), none of the APIs in question turned out to do just what I wanted. (The calls to set low power mode / normal mode / on / off for the device cause any active PPP dialup connection over the device to be reset, thereby ruling them out as useful for my purposees.)

The fact that none of the APIs did what I set out to look for (though I did determine how to do some interesting things like determine signal strengths and what sort of coverage is present) is not all that surprising, as there doesn’t appear to be any sort of support for forcing a device to change protocols on-the-fly in the connection manager app (or support for forcing a dormant mode transition). Because the abstraction layer essentially implements only the minimum amount of functionality for the connection manager software to function, extra device-specific functionality that isn’t used by the connection manager isn’t directly supported by the standard exported C API. All of this essentially translates into there being no dice on using the exported APIs to do what I wanted with forcing the card to upgrade back to EVDO on-the-fly.

While this might initially seem like a pretty fatal roadblock (at least as far as using the connection manager and its libraries to perform the task at hand), it turned out that there was actually something of value in the V740 module after all, even if it didn’t directly expose any useful functionality of this sort through the exported interface. In a stroke of luck, it just so happened that the V740 module was linked to what appeared to be a debug version of the Novatel SDK. In particular, two important points arose from this being the case:

First, the debug version of the Novatel SDK would appear to have handy debug prints all over the place, many naming the function they are called from, providing a fair amount of insight into what internal functions in the module are named. Essentially, the debug prints made it essentially the same as if public symbols had been made available for the Novatel SDK part of the module, as most non-trivial functions in the SDK appear to contain at least one debug print, and virtually all the debug prints named the function they were called from.

Second, the SDK itself and the abstraction layer module appear to have been built without a certain type of linker optimization (/OPT:REF) that removes unreferenced code and data from a final binary. (Disabling this optimization is a default setting for debug builds.) This meant that for the most part, a large part of the Novatel SDK (with debug prints included) happened to be present in the V740 abstraction layer module. As a result of this, there existed the possibility that there might be something of use still there inside the V740 module, even if it wasn’t directly exposed.

At this point, I decided to continue down the road of examining the abstraction layer module, in the hopes that some portion of the baked-in Novatel SDK might be of use.

Next time: Taking a closer look at the V740 abstraction layer module and the Novatel SDK embedded into it.

Recently, I got a V740 EVDO Rev.A ExpressCard for mobile Internet access. Aside from a couple of shortcomings, the thing has actually worked fairly well. However, there are some problems with it which prove fairly annoying in practice.

Aside from blinky lights gadget syndrome, the other major deficiency I’ve encountered with the V740 (really a rebranded Novatel Merlin X720) so far is a problem with the 1xRTT<->EVDO handoff logic.

(There’s a little bit of important background information about 1xRTT/EVDO networks here that is relevant for understanding the rest of the series, so bear with me for a minute. Most of this information is from Wikipedia, and I’ve tried to include links to relevant parts along the way.)

You see, the way the card works is that it supports both 1xRTT-based networks (which are fairly common as long as you’re not completely in the middle of nowhere) for rougly dialup modem speeds, and 1xEVDO-based networks, for low(er) latency, high(er) bandwidth. EVDO is still limited to populous areas – e.g. medium to large size cities and the surrounding areas, and isn’t as ubiqitous as 1xRTT coverage is, at least in the US.

Because EVDO is hardly universal in the US, most CDMA data devices supporting EVDO can actually use both 1xRTT or EVDO, depending the local coverage. Beyond that, most such devices also support (mostly seamless) handoff between EVDO and 1xRTT, such that when you move out of an EVDO coverage area and into a 1xRTT-only coverage area, the data session is transferred without any interruption other than a momentary blip in latency as the air interface is changed from EVDO to 1xRTT. This can happen even while data is being actively transferred on the link without dropping TCP connections or anything as intrusive as that, which is very nice; aside from an increase in latency and decrease in bandwidth as you move to 1xRTT, everything otherwise continues to magically “just work”.

However, there is a minor problem with this logic: Sometimes, if you happen to pass through an area with spotty cell coverage (bridge, elevator, whatnot), you’ll lose one or both of the EVDO or 1xRTT signals temporarily. Depending on whether the device reacquires the EVDO signal fast enough, this sometimes results in the call being handed over to 1xRTT even though you’re really still in an EVDO coverage area. The result is that all of a sudden, everything is high latency and low bandwidth, as the device has decided to use the lower quality network. Unfortunately, there doesn’t seem to be support in the V740 to automatically “fail upwards” from 1xRTT back to EVDO if the device re-enters an area with EVDO signal. This means that your data session is now stuck in “slow mode” due to a minor coverage blip – not so good.

The typical solution that people take to this problem is to instruct their device to disable 1xRTT support and only use EVDO, which does work fine if you are really ever only in an EVDO coverage area. However, I really didn’t want to take this approach, as it means manually futzing around with the card’s configuration when you travel. Worse, it makes you less robust against situations where there really is better connectivity to the 1xRTT network than the EVDO network, even if you’re in an EVDO coverage region.

It turns out that there is actually a way to cause the device to re-select the best network after it has failed over to 1xRTT (other than re-dialing the connection), but it is unfortunately not all that seamless: When the device enters “dormant mode” and then transitions back to an active state, it will rescan available networks and pick the best quality network with which to re-engage the link. (Dormant mode is a mode where the device essentially relinquishes its over-the-air slot, freeing it up for another caller, until either end detects activity on the link. Then, the link is seamlessly re-established, again without TCP connections or the like being dropped. Having the device let go of its over-the-air slot is good for network operators, as it allows the system to essentially “borrow back” the capacity used for an otherwise idle session until the session needs it again.)

Normally, dormant mode is only activated when both ends of the link have been idle for some pre-set period of time, usually several seconds. While this means that if you have a fairly idle connection, you’ll eventually get switched back over to EVDO, if your connection is active, it will not enter dormant mode and will thus continue to run on 1xRTT.

Now, if you’re using a conventional handset in tethered mode (Bluetooth or via USB cable), you can often forcibly enter dormant mode on the device with the “end call” button, and as soon as either end of the link tries to send data, the device will pick up the call on the best network, essentially offering a manual way to force the device to consider switching back to EVDO. (At least, that’s how it works on my LG VX8100 – your mileage my vary based on your device, of course.) While that’s not ideal (it’s a completely manual process to hit the end call button, of course), it does get the job done.

With an ExpressCard, however, the problem is a bit more severe, simply because there is no “end call” button. (Sure, you can disconnect the session, but that kind of defeats the whole point – the goal is to re-establish the high speed network connection without dropping your active TCP connections). Unfortunately, as an end user here, you’re pretty much out of luck without the device vendor (or the network operator) providing a tool to coax the device into reconsidering which data network to use (and as far as I know, neither Verizon nor Novatel provide such a utility).

Not satisified with this, I decided to take matters into my own hands, so to speak, and see if there might be a way to programmatically tell the card to switch back to EVDO, or at least enter dormant mode on-demand without idling all data traffic, so that it can automatically select EVDO when resuming from dormant mode. The end desired result would be a program that I could either manually run or that could run continually while a session is active, and try to re-establish the EVDO link where appropriate. While Novatel does appear to make some mention of an SDK for their devices available on their website, it doesn’t appear to be readily available to individual users, which unforunately rules it out. That leaves reverse engineering the way the V740 is controlled from the computer in the hopes of finding something to achieve the desired results.

Fortunately, reversing things is something that I do happen to have a little bit of knowledge of…

Next time: Popping the hood on Verizon’s connection manager app (“VZAccess Manager”), and how it talks to the V740 ExpressCard.