The Kernel Will See You Now

A few days ago, just after installing some new software, I ran into a curious bug that prevented one of my favorite programs from loading after Software X had been installed (names have been changed to protect the guilty).  The error I was getting way up in userland informed me that a driver could not load due to the fact that it failed to register one or more system notification routines (their words, not mine).  This was news to me, and in fact, I wasn't even sure what it was telling me initially, so I decided to do a little troubleshooting and see if it was something I could fix on my own.

If you do a Google search for "System notification routine," the first two results which come up, as documented by MS are PsSetLoadImageNotifyRoutine and PsSetCreateProcessNotifyRoutine.  Per the documentation here and here:

The PsSetLoadImageNotifyRoutine routine registers a driver-supplied callback that is subsequently notified whenever an image is loaded (or mapped into memory).

The PsSetCreateProcessNotifyRoutine routine adds a driver-supplied callback routine to, or removes it from, a list of routines to be called whenever a process is created or deleted.

So at a glance, we've at least found the two calls that are likely failing (for unknown reasons), and it's most likely that this is a compatibility issue that cannot be resolved by the user, but it never hurts to try!  In reading the documentation about PsSetCreateProcessNotifyRoutine, the folks at Microsoft also note that "the system can register up to 64 process-creation callbacks."  Based on the error message, my initial hypothesis was that I had reached the system limit which would explain why uninstalling other pieces of software made the problem go away.  After discussing this with some other folks, it seemed unlikely, but it was really all I had to go with which means the next task was to enumerate all registered process-creation callbacks to either confirm or eliminate this as the cause.

When all you have is a hammer, everything looks like a nail.

My initial approach was to avoid heading straight for the debugger find a lazy solution using either process monitor or Volatility, neither of which ended up doing what I needed.  Since I think it's important to document failures as well as successes, let's quickly cover why Volatility didn't do what I wanted.  First, it was my fault for using a tool for something other than its intended purpose and expecting good results (I should have known better after this long in the security industry...).  Second, I used the "enumfunc" plugin from Volatility 2.2 which did exactly what it was supposed to do, but ended up not being what I needed. The closest this output got me was:

<KERNEL> Import aswVmm.sys 904 0x000000008062deb2 ntoskrnl.exe!PsSetCreateProcessNotifyRoutine

While that's very helpful, the numbers didn't add up when it came to imported functions so I couldn't see anything definitive based on the output.  Lastly, the memory image I was supplying Volatility with was a 64-bit version of Windows 7 SP1 which isn't supported by all of the plugins such as "callbacks" and others which may have helped me put this to bed a long time ago.

In the end, kd.exe was the only way out of this and thankfully, I found a Windbg/KD script written by Scott Noone back in November of 2010 that did exactly what I wanted.  After configuring the machines for kernel debugging over a 1394 cable, I ran the script and sure enough, it worked flawlessly.  Unfortunately, it turns out only 15 process creation-callbacks were registered on the system, so I quickly went from having a hypothesis, to having no idea =/  However, on the bright side, I did get to learn a bit and I found the answer to my question, which was how to enumerate process-creation callbacks!  So, if anyone comes here looking for ways to enumerate callbacks or list registered notification routines, this post should get you started.

In the mean time, I'll see if I can identify why this driver still fails to load and post any additional findings.