Windows Hooks

We will learn about Windows hooks in this tutorial. Windows hooks are very powerful. With them,
you can poke inside other processes and sometimes alter their behaviors.

Theory:
Windows hooks can be considered one of the most powerful features of Windows. With them,
you can trap events that will occur, either in your own process or in other processes. By
"hooking", you tell Windows about a function, filter function also called hook procedure, that will
be called everytime an event you're interested in occurs. There are two types of them: local and
remote hooks.
• Local hooks trap events that will occur in your own process.
• Remote hooks trap events that will occur in other process(es). There are two types of
remote hooks
• thread-specific traps events that will occur in a specific thread in other process.
In short, you want to observe events in a specific thread in a specific process.
• system-wide traps all events destined for all threads in all processes in the
system.
When you install hooks, remember that they affect system performance. System-wide hooks are
the most notorious. Since ALL related events will be routed through your filter function, your
system may slow down noticeably. So if you use a system-wide hook, you should use it
judiciously and unhook it as soon as you don't need it. Also, you have a higher chance of
crashing the other processes since you can meddle with other processes and if something is
wrong in your filter function, it can pull the other processes down to oblivion with it. Remember:
Power comes with responsibility.!!You have to understand how a hook works before you can use
it efficiently. When you create a hook, Windows creates a data structure in memory, containing
information about the hook, and adds it to a linked list of existing hooks. New hook is added in
front of old hooks. When an event occurs, if you install a local hook, the filter function in your
process is called so it's rather straightforward. But if it's a remote hook, the system must inject the
code for the hook procedure into the address space(s) of the other process(es). And the system
can do that only if the function resides in a DLL. Thus , if you want to use a remote hook, your
hook procedure must reside in a DLL. There is two exceptions to this rule: journal record and
journal playback hooks. The hook procedures for those two hooks must reside in the thread that
installs the hooks. The reason why it must be so is that: both hooks deal with the low-level
interception of hardware input events. The input events must be recorded/playbacked in the order
they appeared. If the code of those two hooks is in a DLL, the input events may scatter among
several threads and it is impossible to know the order of them. So the solution: the hook
procedure of those two hooks must be in a single thread only i.e. the thread that installs the
hooks.
There are 14 types of hooks:
• WH_CALLWNDPROC called when SendMessage is called
• WH_CALLWNDPROCRET called when SendMessage returns
• WH_GETMESSAGE called when GetMessage or PeekMessage is called
• WH_KEYBOARD called when GetMessage or PeekMessage retrieves WM_KEYUP or
WM_KEYDOWN from the message queue
• WH_MOUSE called when GetMessage or PeekMessage retrieves a mouse message
from the message queue
• WH_HARDWARE called when GetMessage or PeekMessage retrieves some hardware
message that is not related to keyboard or mouse.
 • WH_MSGFILTER called when a dialog box, menu or scrollbar is about to process a
message. This hook is local. It's specifically for those objects which have their own
internal message loops.
• WH_SYSMSGFILTER same as WH_MSGFILTER but system-wide
• WH_JOURNALRECORD called when Windows retrieves message from the hardware
input queue
• WH_JOURNALPLAYBACK called when an event is requested from the system's
hardware input queue.
• WH_SHELL called when something interesting about the shell occurs such as when the
task bar needs to redraw its button.
• WH_CBT used specifically for computer-based training (CBT).
• WH_FOREGROUNDIDLE used internally by Windows. Little use for general applications
• WH_DEBUG used to debug the hooking procedure
Now that we know some theory, we can move on to how to install/uninstall the hooks.
To install a hook, you call SetWindowsHookEx which has the following syntax:
SetWindowsHookEx proto HookType:DWORD, pHookProc:DWORD,
hInstance:DWORD, ThreadID:DWORD

• HookType is one of the values listed above, e.g., WH_MOUSE,

WH_KEYBOARD

• pHookProc is the address of the hook procedure that will be called to process
the messages for the specified hook. If the hook is a remote one, it must reside in
a DLL. If not, it must be in your process.

• hInstance is the instance handle of the DLL in which the hook procedure
resides. If the hook is a local one, this value must be NULL

• ThreadID is the ID of the thread you want to install the hook to spy on. This
parameter is the one that determines whether a hook is local or remote. If this
parameter is NULL, Windows will interpret the hook as a system-wide remote
hook that affects all threads in the system. If you specify the thread ID of a thread
in your own process, this hook is a local one. If you specify the thread ID from
other process, the hook is a thread-specific remote one. There are two
exceptions to this rule: WH_JOURNALRECORD and
WH_JOURNALPLAYBACK are always local system-wide hooks that are not
required to be in a DLL. And WH_SYSMSGFILTER is always a system-wide
remote hook. Actually it is identical to WH_MSGFILTER hook with ThreadID==0.

If the call is successful, it returns the hook handle in eax. If not, NULL is returned. You
must save the hook handle for unhooking later.
You can uninstall a hook by calling UnhookWindowsHookEx which accepts only one parameter,
the handle of the hook you want to uninstall. If the call succeeds, it returns a non-zero value in
eax. Otherwise, it returns NULL.
Now that you know how to install/uninstall hooks, we can examine the hook procedure.
The hook procedure will be called whenever an event that is associated with the type of hook you
have installed occurs. For example, if you install WH_MOUSE hook, when a mouse event occurs,
your hook procedure will be called. Regardless of the type of hook you installed, the hook
procedure always has the following prototype:
HookProc proto nCode:DWORD, wParam:DWORD, lParam:DWORD
• nCode specifies the hook code.
• wParam and lParam contain additional information about the event
HookProc is actually a placeholder for the function name. You can name it anything you like so
long as it has the above prototype. The interpretation of nCode, wParam and lParam is
dependent on the type of hook you install. So as the return value from the hook procedure. For
example:
WH_CALLWNDPROC

• nCode can be only HC_ACTION which means there is a message sent to a

window

• wParam contains the message being sent, if it's not zero

• lParam points to a CWPSTRUCT structure

• return value: not used, return zero

WH_MOUSE

• nCode can be HC_ACTION or HC_NOREMOVE

• wParam contains the mouse message

• lParam points to a MOUSEHOOKSTRUCT structure

• return value: zero if the message should be processed. 1 if the message should
be discarded.
The bottom line is: you must consult your win32 api reference for details about the meanings of
the parameters and return value of the hook you want to install.
Now there is a little catch about the hook procedure. Remember that the hooks are chained in a
linked list with the most recently installed hook at the head of the list. When an event occurs,
Windows will call only the first hook in the chain. It's your hook procedure's responsibility to call
the next hook in the chain. You can choose not to call the next hook but you'd better know what
you're doing. Most of the time, it's a good practice to call the next procedure so other hooks can
have a shot at the event. You can call the next hook by calling CallNextHookEx which has the
following prototype:
CallNextHookEx proto hHook:DWORD, nCode:DWORD, wParam:DWORD,
lParam:DWORD

• hHook is your own hook handle. The function uses this handle to traverse the
linked list and search for the hook procedure it should call next.

• nCode, wParam and lParam you can just pass those three values you receive
from Windows to CallNextHookEx.
An important note about remote hooks: the hook procedure must reside in a DLL which will be
mapped into other processes. When Windows maps the DLL into other processes, it will not map
the data section(s) into the other processes. In short, all processes share a single copy of code
but they will have their own private copy of the DLL's data section! This can be a big surprise to
the unwary. You may think that when you store a value into a variable in the data section of a
DLL, that value will be shared among all processes that load the DLL into their process address
space. It's simply not true. In normal situation, this behavior is desirable since it provides the
illusion that each process has its own copy of the DLL. But not when Windows hook is concerned.
We want the DLL to be identical in all processes, including the data. The solution: you must mark
the data section as shared. You can do this by specifying the section(s) attribute in the linker
switch. For MASM, you need to use this switch:
 /SECTION:<section name>, S
The name of the initialized data section is .data and the uninitialized data is .bss. For example if
you want to assemble a DLL which contains a hook procedure and you want the uninitialized data
section to be shared amoung processes, you must use the following line:
link /section:.bss,S /DLL /SUBSYSTEM:WINDOWS ..........
S attribute marks the section as shared.
Example:
There are two modules: one is the main program which will do the GUI part and the other is the
DLL that will install/uninstall the hook.
;--------------------------------------------- This is the source code of the main program -----------------------
.386
.model flat,stdcall
option casemap:none
include \masm32\include\windows.inc
include \masm32\include\user32.inc
include \masm32\include\kernel32.inc
include mousehook.inc
includelib mousehook.lib
includelib \masm32\lib\user32.lib
includelib \masm32\lib\kernel32.lib
wsprintfA proto C :DWORD,:DWORD,:VARARG
wsprintf TEXTEQU <wsprintfA>
.const
IDD_MAINDLG equ 101
IDC_CLASSNAME equ 1000
IDC_HANDLE equ 1001
IDC_WNDPROC equ 1002
IDC_HOOK equ 1004
IDC_EXIT equ 1005
WM_MOUSEHOOK equ WM_USER+6
DlgFunc PROTO :DWORD,:DWORD,:DWORD,:DWORD
.data
HookFlag dd FALSE
HookText db "&Hook",0
UnhookText db "&Unhook",0
template db "%lx",0
.data?
hInstance dd ?
hHook dd ?
.code
start:
invoke GetModuleHandle,NULL
mov hInstance,eax
invoke DialogBoxParam,hInstance,IDD_MAINDLG,NULL,addr DlgFunc,NULL
invoke ExitProcess,NULL
DlgFunc proc hDlg:DWORD,uMsg:DWORD,wParam:DWORD,lParam:DWORD
LOCAL hLib:DWORD
LOCAL buffer[128]:byte
LOCAL buffer1[128]:byte
LOCAL rect:RECT
.if uMsg==WM_CLOSE
.if HookFlag==TRUE
invoke UninstallHook
.endif
invoke EndDialog,hDlg,NULL
.elseif uMsg==WM_INITDIALOG
invoke GetWindowRect,hDlg,addr rect
invoke SetWindowPos, hDlg, HWND_TOPMOST, rect.left, rect.top, rect.right, rect.bottom,
SWP_SHOWWINDOW
.elseif uMsg==WM_MOUSEHOOK
invoke GetDlgItemText,hDlg,IDC_HANDLE,addr buffer1,128
invoke wsprintf,addr buffer,addr template,wParam
invoke lstrcmpi,addr buffer,addr buffer1
.if eax!=0
invoke SetDlgItemText,hDlg,IDC_HANDLE,addr buffer
.endif
invoke GetDlgItemText,hDlg,IDC_CLASSNAME,addr buffer1,128
invoke GetClassName,wParam,addr buffer,128
invoke lstrcmpi,addr buffer,addr buffer1
.if eax!=0
invoke SetDlgItemText,hDlg,IDC_CLASSNAME,addr buffer
.endif
invoke GetDlgItemText,hDlg,IDC_WNDPROC,addr buffer1,128
invoke GetClassLong,wParam,GCL_WNDPROC
invoke wsprintf,addr buffer,addr template,eax
invoke lstrcmpi,addr buffer,addr buffer1
.if eax!=0
invoke SetDlgItemText,hDlg,IDC_WNDPROC,addr buffer
.endif
.elseif uMsg==WM_COMMAND
.if lParam!=0
mov eax,wParam
mov edx,eax
shr edx,16
.if dx==BN_CLICKED
.if ax==IDC_EXIT
invoke SendMessage,hDlg,WM_CLOSE,0,0
.else
.if HookFlag==FALSE
invoke InstallHook,hDlg
.if eax!=NULL
mov HookFlag,TRUE
invoke SetDlgItemText,hDlg,IDC_HOOK,addr UnhookText
.endif
.else
invoke UninstallHook
invoke SetDlgItemText,hDlg,IDC_HOOK,addr HookText
mov HookFlag,FALSE
invoke SetDlgItemText,hDlg,IDC_CLASSNAME,NULL
invoke SetDlgItemText,hDlg,IDC_HANDLE,NULL
invoke SetDlgItemText,hDlg,IDC_WNDPROC,NULL
.endif
.endif
.endif
.endif
.else
mov eax,FALSE
ret
.endif
mov eax,TRUE
ret
DlgFunc endp
end start
;----------------------------------------------------- This is the source code of the DLL ----------------------------
.386
.model flat,stdcall
option casemap:none
include \masm32\include\windows.inc
include \masm32\include\kernel32.inc
includelib \masm32\lib\kernel32.lib
include \masm32\include\user32.inc
includelib \masm32\lib\user32.lib
.const
WM_MOUSEHOOK equ WM_USER+6
.data
hInstance dd 0
.data?
hHook dd ?
hWnd dd ?
.code
DllEntry proc hInst:HINSTANCE, reason:DWORD, reserved1:DWORD
.if reason==DLL_PROCESS_ATTACH
push hInst
pop hInstance
.endif
mov eax,TRUE
ret
DllEntry Endp
MouseProc proc nCode:DWORD,wParam:DWORD,lParam:DWORD
invoke CallNextHookEx,hHook,nCode,wParam,lParam
mov edx,lParam
assume edx:PTR MOUSEHOOKSTRUCT
invoke WindowFromPoint,[edx].pt.x,[edx].pt.y
invoke PostMessage,hWnd,WM_MOUSEHOOK,eax,0
assume edx:nothing
xor eax,eax
ret
MouseProc endp
InstallHook proc hwnd:DWORD
push hwnd
pop hWnd
invoke SetWindowsHookEx,WH_MOUSE,addr MouseProc,hInstance,NULL
mov hHook,eax
ret
InstallHook endp
UninstallHook proc
invoke UnhookWindowsHookEx,hHook
ret
UninstallHook endp
End DllEntry
;---------------------------------------------- This is the makefile of the DLL ----------------------------------------
NAME=mousehook
$(NAME).dll: $(NAME).obj
Link /SECTION:.bss,S /DLL /DEF:$(NAME).def /SUBSYSTEM:WINDOWS
/LIBPATH:c:\masm\lib $(NAME).obj
$(NAME).obj: $(NAME).asm
ml /c /coff /Cp $(NAME).asm
Analysis:
The example will display a dialog box with three edit controls that will be filled with the class
name, window handle and the address of the window procedure associated with the window
under the mouse cursor. There are two buttons, Hook and Exit. When you press the Hook button,
the program hooks the mouse input and the text on the button changes to Unhook. When you
move the mouse cursor over a window, the info about that window will be displayed in the main
window of the example. When you press Unhook button, the program removes the mouse hook.
The main program uses a dialog box as its main window. It defines a custom message,
WM_MOUSEHOOK which will be used between the main program and the hook DLL. When the
main program receives this message, wParam contains the handle of the window that the mouse
cursor is on. Of course, this is an arbitrary arrangement. I decide to send the handle in wParam
for the sake of simplicity. You can choose your own method of communication between the main
program and the hook DLL.
.if HookFlag==FALSE
invoke InstallHook,hDlg
.if eax!=NULL
mov HookFlag,TRUE
invoke SetDlgItemText,hDlg,IDC_HOOK,addr UnhookText
.endif
The program maintains a flag, HookFlag, to monitor the state of the hook. It's FALSE if the hook
is not installed and TRUE if the hook is installed.
When the user presses Hook button, the program checks if the hook is already installed. If it is
not, it call InstallHook function in the hook DLL to install it. Note that we pass the handle of the
main dialog as the parameter of the function so the hook DLL can send the WM_MOUSEHOOK
messages to the right window i.e. our own.
When the program is loaded, the hook DLL is loaded too. Actually, DLLs are loaded immediately
after the program is in memory. The DLL entrypoint function is called before the first instruction in
the main program is execute even. So when the main program executes the DLL(s) is/are
initialized. We put the following code in the DLL entrypoint function of the hook DLL:
.if reason==DLL_PROCESS_ATTACH
push hInst
pop hInstance
.endif
The code just saves the instance handle of the hook DLL itself to a global variable named
hInstance for use within the InstallHook function. Since the DLL entrypoint function is called
before other functions in the DLL are called , hInstance is always valid. We put hInstance in .data
section so that this value is kept on per-process basis. Since when the mouse cursor hovers over
a window, the hook DLL is mapped into the process. Imagine that there is already a DLL that
occupies the intended load address of the hook DLL, the hook DLL would be remapped to
another address. The value of hInstance will be updated to those of the new load address. When
the user presses Unhook button and then Hook button, SetWindowsHookEx will be called again.
However, this time, it will use the new load address as the instance handle which will be wrong
because in the example process, the hook DLL's load address hasn't been changed. The hook
will be a local one where you can hook only the mouse events that occur in your own window.
Hardly desirable.
InstallHook proc hwnd:DWORD
push hwnd
pop hWnd
invoke SetWindowsHookEx,WH_MOUSE,addr MouseProc,hInstance,NULL
mov hHook,eax
ret
InstallHook endp
The InstallHook function itself is very simple. It saves the window handle passed as its parameter
to a global variable named hWnd for future use. It then calls SetWindowsHookEx to install a
mouse hook. The return value of SetWindowsHookEx is stored in a global variable named hHook
for use with UnhookWindowsHookEx.
After SetWindowsHookEx is called, the mouse hook is functional. Whenever a mouse event
occurs in the system, MouseProc ( your hook procedure) is called.
MouseProc proc nCode:DWORD,wParam:DWORD,lParam:DWORD
invoke CallNextHookEx,hHook,nCode,wParam,lParam
mov edx,lParam
assume edx:PTR MOUSEHOOKSTRUCT
invoke WindowFromPoint,[edx].pt.x,[edx].pt.y
invoke PostMessage,hWnd,WM_MOUSEHOOK,eax,0
assume edx:nothing
xor eax,eax
ret
MouseProc endp
The first thing it does is to call CallNextHookEx to give the other hooks the chance to process the
mouse event. After that, it calls WindowFromPoint function to retrieve the handle of the window at
the specified screen coordinate. Note that we use the POINT structure in the
MOUSEHOOKSTRUCT structure pointed to by lParam as the current mouse coordinate. After
that we send the window handle to the main program via PostMessage with WM_MOUSEHOOK
message. One thing you should remember is that: you should not use SendMessage inside the
hook procedure, it can cause message deadlock. PostMessage is recommended. The
MOUSEHOOKSTRUCT structure is defined below:
MOUSEHOOKSTRUCT STRUCT DWORD
pt POINT <>
hwnd DWORD ?
wHitTestCode DWORD ?
dwExtraInfo DWORD ?
MOUSEHOOKSTRUCT ENDS
• pt is the current screen coordinate of the mouse cursor
• hwnd is the handle of the window that will receive the mouse message. It's usually the
window under the mouse cursor but not always. If a window calls SetCapture, the mouse
input will be redirected to that window instead. Because of this reason, I don't use the
hwnd member of this structure but choose to call WindowFromPoint instead.
• wHitTestCode specifies the hit-test value. The hit-test value gives more information about
the current mouse cursor position. It specifies on what part of window the mouse cursor
is. For complete list, check your win32 api reference under WM_NCHITTEST message.
• dwExtraInfo contains the extra information associated with the message. Normally this
value is set by calling mouse_event and retrieved by calling GetMessageExtraInfo.
When the main window receives WM_MOUSEHOOK message, it uses the window handle in
wParam to retrieve the information about the window.
.elseif uMsg==WM_MOUSEHOOK
invoke GetDlgItemText,hDlg,IDC_HANDLE,addr buffer1,128
invoke wsprintf,addr buffer,addr template,wParam
invoke lstrcmpi,addr buffer,addr buffer1
.if eax!=0
invoke SetDlgItemText,hDlg,IDC_HANDLE,addr buffer
.endif
invoke GetDlgItemText,hDlg,IDC_CLASSNAME,addr buffer1,128
invoke GetClassName,wParam,addr buffer,128
invoke lstrcmpi,addr buffer,addr buffer1
.if eax!=0
invoke SetDlgItemText,hDlg,IDC_CLASSNAME,addr buffer
.endif
invoke GetDlgItemText,hDlg,IDC_WNDPROC,addr buffer1,128
invoke GetClassLong,wParam,GCL_WNDPROC
invoke wsprintf,addr buffer,addr template,eax
invoke lstrcmpi,addr buffer,addr buffer1
.if eax!=0
invoke SetDlgItemText,hDlg,IDC_WNDPROC,addr buffer
.endif
To avoid flickers, we check the text already in the edit controls and the text we will put into them if
they are identical. If they are, we skip them.
We retrieve the class name by calling GetClassName, the address of the window procedure by
calling GetClassLong with GCL_WNDPROC and then format them into strings and put them into
the appropriate edit controls.
invoke UninstallHook
invoke SetDlgItemText,hDlg,IDC_HOOK,addr HookText
mov HookFlag,FALSE
invoke SetDlgItemText,hDlg,IDC_CLASSNAME,NULL
invoke SetDlgItemText,hDlg,IDC_HANDLE,NULL
invoke SetDlgItemText,hDlg,IDC_WNDPROC,NULL
When the user presses Unhook button, the program calls UninstallHook function in the hook DLL.
UninstallHook just calls UnhookWindowsHookEx. After that, it changes the text of the button back
to "Hook", HookFlag to FALSE and clears the content of the edit controls.
Note the linker switch in the makefile.
Link /SECTION:.bss,S /DLL /DEF:$(NAME).def /SUBSYSTEM:WINDOWS
It specifies .bss section as a shared section to make all processes share the same uninitialized
data section of the hook DLL. Without this switch, your hook DLL will not function correctly.