Sentinel HASP Protection System Analysis

Ive decided to start a little bit of research on this subject, disregarding all existing articles, if any. Keep in mind that I'm writing this for educational purposes only, so there won't be any commercial application involved in the process. I'll be using a test target enveloped with latest Developer Kit (v5.10), acquired from SafeNet's web-page. There's a high probability that a hardware key (a dongle) may be required for parts of this process. We'll see as I progress. Article is to be divided in parts, based on how much time I got on my hands to pursue this objective. Before we continue, I will also say that I am currently operating under Windows XP (x32), just to eliminate any confusion regarding future questions I believe you know what those questions will be

| TEST TARGET DEVELOPMENT |

Using Microsoft Visual Studio 2010 (or compiler of your choice), Ive coded this small test application, entitled HASP_TEST:

#include <windows.h> void _declspec(noinline) __stdcall ShowMessageBox() { MessageBox( NULL, TEXT( "Sentinel HASP Protection System: TEST RUN." ), TEXT( "Info" ), MB_OK|MB_ICONINFORMATION ); } int __stdcall WinMain( HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nShowCmd ) { ShowMessageBox(); return 0; }

The _declspec(noinline) specifier was used to prevent MSVC from inlining the function. Considering Ive chosen small-code optimization in projects settings, inlining was bound to happen. Application may evolve as I progress, in case of SDK usage (HASP APIs). Create a new project, set build type to Release, add in a new source file (I called it main.cpp), copy-paste above code in your newly added source file, then change project properties as follows:

Configuration Properties Character Set Use Multi-Byte Character Set; C++ Optimization Optimization Minimize Size (/O1); C++ Code Generation Runtime Library Multi-threaded (/MT); Linker Advanced Randomize Base Address No (/DYNAMICBASE:NO);
Once youve done that, compile the application. If using MSVC 2010, default location of the output file is here: %HOMEPATH%\My Documents\Visual Studio 2010\Projects\HASP_TEST\Release. A reference to Windows Environment Variables can be found at this link: http://vlaurie.com/computers2/Articles/environment.htm.
1

Sentinel HASP Protection System Analysis

| SOFTWARE DEVELOPMENT KIT (SDK) |

Acquire Sentinel HASP Protection System SDK from SafeNet's website: http://dlm.safenet-inc.com/Sentinel_HASP_5.10.zip. Note that its size is 0.98 GB. Install it using defaults I didn't change a thing while installing, so user and password for logging on to Envelope is HASP. You will find the Envelope product here: %PROGRAMFILES%\SafeNet\Sentinel HASP\VendorTools\VendorSuite\envelope.exe. As an option, make a shortcut of the executable on your desktop, for faster use.

| APPLYING THE ENVELOPE |

Let's protect our test target. a. Open up Envelope and choose Work Offline.

b. c.

Drag and drop the test target (HASP_TEST.exe) over main application's window. Click on its reference on the left-side menu (under Programs).

d.

Find the Protect button to the lower right-side pane, underneath General block. Hit it (and our test target will be enveloped (check out Output file field to learn where it was saved). By default, its at this location: %HOMEPATH%\My Documents\Aladdin\Sentinel HASP 5.10\VendorTools\VendorSuite\Protected.
2

Sentinel HASP Protection System Analysis

Thats about it. Close Envelope, without saving the project. Running the enveloped application shows these messages:

Great, we now have a HASP protected application (compiled application is 30 KB in size, while output envelope is ~2 MB).

| DEBUGGER DETECTED |
Extract OllyDbg from the provided archive to a directory of your choice and let's begin our quest. NOTE: We will NOT use any plugins today Goal is to study HASP and how it behaves, not a quick bypass over its antidebug features. Fire up Olly, drag over HASP_TEST.exe and click OK on the message prompt, regarding EP not in code sections scope. Lets give it a run, shall we? Hit Shit+F9 combination of keys. Result:

Sentinel HASP Protection System Analysis

What we want to achieve now is running software, bypassing the debugger, and reaching that demo message we saw earlier. But this time, I wont go directly to the code responsible of that error message. Instad, lets pick it apart one step at a time. I know its a long, tedious path, but its all worth it. Believe me. Or.. you can skip to the part of your interest

| CODE ANALYSIS : INIT |

At its prologue, HASP creates a stack frame and saves state of all R32 registers:
0040B286 0040B287 0040B289 60 8BC4 A3 F4BE4000 PUSHAD MOV EAX,ESP MOV DWORD PTR DS:[40BEF4],EAX

This move is done for later use, where, before entering main envelope code, this address is retrieved from its placeholder, 40BEF4. (1) Next up, we see:
0040B28E 0040B293 0040B299 B8 18B54000 2B05 3CB54000 A3 3CB54000 MOV EAX,0040B518 SUB EAX,DWORD PTR DS:[40B53C] MOV DWORD PTR DS:[40B53C],EAX

; HASP_TES.0040B518

Considering EAX register is not used further along, I can only say that this is a decoy piece of code. Result is obviously 0 and is stored at 40B53C. (2) Comparison at 40B29E succeeds, since content of 40BEF0 is NULL, after which HASP starts collecting info. It first retrieves handles of kernel32.dll and user32.dll libraries and stores them for later use:
0040B29E 0040B2A5 0040B2A7 0040B2AD 0040B2AE 0040B2B4 0040B2B7 0040B2BC 0040B2C1 0040B2C7 0040B2CC 0040B2D1 0040B2D7 833D F0BE4000 00 74 15 8B0D F4BE4000 51 FF15 F0BE4000 83C4 04 E9 A5000000 68 DCBE4000 FF15 40BF4000 A3 F8B44000 68 E8BE4000 FF15 40BF4000 A3 FCB44000 CMP DWORD PTR DS:[40BEF0],0 JE SHORT 0040B2BC (taken) MOV ECX,DWORD PTR DS:[40BEF4] PUSH ECX CALL DWORD PTR DS:[40BEF0] ADD ESP,4 JMP 0040B361 PUSH 0040BEDC CALL DWORD PTR DS:[<&KERNEL32.GetModuleHandleA>] MOV DWORD PTR DS:[40B4F8],EAX PUSH 0040BEE8 CALL DWORD PTR DS:[<&KERNEL32.GetModuleHandleA>] MOV DWORD PTR DS:[40B4FC],EAX

; ASCII "kernel32" ; kernel32.GetModuleHandleA ; ASCII "user32" ; kernel32.GetModuleHandleA

Notice how value of 40BEF0 is NULL. For now. Turns out its filled in a tad later. (3) Two API addresses are further stored in placeholders:
0040B2DC 8B15 44BF4000 0040B2E2 8915 00B54000 0040B2E8 A1 4CBF4000 0040B2ED A3 04B54000 MOV EDX,DWORD PTR DS:[<&KERNEL32.GetProcAddress>] MOV DWORD PTR DS:[40B500],EDX MOV EAX,DWORD PTR DS:[<&USER32.MessageBoxA>] MOV DWORD PTR DS:[40B504],EAX

Next up, HASP NULLs data at two locations:

4

Sentinel HASP Protection System Analysis 0040B2F2 C705 08B54000 00000000 0040B2FC C705 0CB54000 00000000 MOV DWORD PTR DS:[40B508],0 MOV DWORD PTR DS:[40B50C],0

Now, if you remember when I spoke about that decoy code at (2), HASP will make use of it in the following lines:
0040B306 0040B30C 0040B312 0040B318 0040B31E 0040B31F 0040B324 0040B325 0040B32B 0040B331 0040B332 8B0D 3CB54000 030D 10B54000 890D F0BE4000 8B15 18B54000 52 A1 48B54000 50 8B0D 3CB54000 030D 40B54000 51 E8 42010000 MOV ECX,DWORD PTR DS:[40B53C] ADD ECX,DWORD PTR DS:[40B510] MOV DWORD PTR DS:[40BEF0],ECX MOV EDX,DWORD PTR DS:[40B518] PUSH EDX MOV EAX,DWORD PTR DS:[40B548] PUSH EAX MOV ECX,DWORD PTR DS:[40B53C] ADD ECX,DWORD PTR DS:[40B540] PUSH ECX CALL 0040B479 ; HASP_TES.0060B6D5

; HASP_TES.0060AED0

Up in the code, content of 40B53C is 0. HASP then adds content of 40B510, and stores that result at 40BEF0, thus filling in what I said earlier, at (3) ;-) | [40B510] == 60B6D5h Then it retrieves a hardcoded DWORD from 40B518 in EDX and pushes it as parameter to the stack. | [40B518] == 9D7587F8h A second DWORD is retrieved from 40B548 into EAX register and is pushed to stack as well. | [40B548] == 46C0h A similar calculation is done, this time grabbing an address from 40B540 into ECX register. Value is pushed to stack as parameter. | [40B540] == 60AED0h The way I see it, the CALL at 40B332 can be summarized in what follows: sub_40B479( 60AED0h, 46C0h, 9D7587F8h ); A function with 3 parameters. From the looks of it, what this function does is to decrypt code at location 60AED0, sized 46C0h, having start seed 9D7587F8h. Thats an assumption. Lets see if were right. Click in dump window, press Ctrl+G, type in 60AED0 and hit Enter/click OK. Right-click in this window and choose Backup

Update backup. Now that we are at CALL address (EIP == 40B332) with our tracing, we can press F8 to step over the call.
Notice how, in dump window, code at 60AED0 gets red in color ;-) Looks like we were right after all. We were also right about the 3 parameters (3 parameters mean 3 DWORDs; DWORD is 4 bytes; 3*4=12, 0xC in hexa):
0040B337 83C4 0C ADD ESP,0C

Next up we have another CALL to a function, at location 40B33A. If you enter inside it with F7 and trace it a bit, youll see that whats executed is the following:
0040B366 0040B367 0040B369 0040B36C 0040B371 55 8BEC 83EC 14 A1 3CB54000 0305 F0B44000 PUSH EBP MOV EBP,ESP SUB ESP,14 MOV EAX,DWORD PTR DS:[40B53C] ADD EAX,DWORD PTR DS:[40B4F0]

; HASP_TES.0060F3F0 5

Sentinel HASP Protection System Analysis 0040B377 0040B37A 0040B381 .. .. 0040B473 0040B475 0040B477 0040B478 8945 F0 833D 3CB54000 00 0F84 EC000000 MOV DWORD PTR SS:[EBP-10],EAX CMP DWORD PTR DS:[40B53C],0 JE 0040B473 (taken)

33C0 8BE5 5D C3

XOR EAX,EAX MOV ESP,EBP POP EBP RETN

; HASP_TES.0060F3F0

Value of 40B53C remained 0. Thus, what function does is to retrieve an address stored at 40B4F0 (40B4F0 acts as a pointer address pointing to another address) and place it in stack at EBP-10. Well find out later what its used for. Then it copies 6*4 DWORDs from 40B4F8 to [40B520] == 60ECF8h:
0040B33F 0040B344 0040B349 0040B34F B9 06000000 BE F8B44000 8B3D 20B54000 F3:A5 MOV ECX,6 MOV ESI,0040B4F8 MOV EDI,DWORD PTR DS:[40B520] REP MOVS DWORD PTR ES:[EDI],DWORD PTR DS:[ESI]

; HASP_TES.0060ECF8

Result in dump window:

0060ECF8 00 00 80 7C 00 00 41 7E 40 AE 80 7C EA 07 45 7E ..|..A~@|E~ 0060ED08 00 00 00 00 00 00 00 00 ........

Last part before entering the REAL protection is fetching stack frame containing original registers state which I mentioned, if you remember, at (1):
0040B351 0040B357 0040B358 8B15 F4BE4000 52 FF15 F0BE4000 MOV EDX,DWORD PTR DS:[40BEF4] PUSH EDX CALL DWORD PTR DS:[40BEF0]

; HASP_TES.0060B6D5

| CODE ANALYSIS : JUNK |

The tough part begins now. We are at 40B358 with our EIP. Lets enter the funky maze of HASP with a press of F7. EIP is now 60B6D5. Ill mark in red junk code, with blue complementary code and with green redundant code. All of this can be removed and what remains, is good, executable code, in black. Ill keep the colors for later markings as well.
0060B6D5 /E9 6C240000 .. 0060DB46 0060DB49 0060DB4A 0060DB4C 0060DB4F .. 0060DB50 0060DB52 0060DB54 0060DB58 0060DB5A 0060DB5C 0060DB5F .. 0060E5B8 8F4424 FC POP DWORD PTR SS:[ESP-4] 6 FFF6 FFF7 66:C1ED 40 86E3 86DC 8D6D 00 E8 540A0000 PUSH ESI PUSH EDI SHR BP,40 XCHG BL,AH XCHG AH,BL LEA EBP,DWORD PTR SS:[EBP] CALL 0060E5B8 ; HASP_TES.0040B510 66:8BDB 55 8BEC 83EC 2C EB FF MOV BX,BX PUSH EBP MOV EBP,ESP SUB ESP,2C JMP SHORT 0060DB50 JMP 0060DB46

Sentinel HASP Protection System Analysis 0060E5BC C745 FC 00000000 MOV DWORD PTR SS:[EBP-4],0 SHL DI,20 JA 0060CDB2 CMP DWORD PTR DS:[60EE40],0 CALL 0060E03F LEA ESP,DWORD PTR SS:[ESP+4] JE 0060B558 CMP BP,BP SUB ECX,0 CMP DWORD PTR DS:[60ED08],0 (4) XCHG EBX,EBP XCHG EBP,EBX JE 0060E6DF MOV ECX,DWORD PTR DS:[60ED00] MOV DWORD PTR DS:[60EE54],ECX MOV DWORD PTR DS:[60ED00],0060DFDF JNO 0060E259 CALL 0060BA1A ; kernel32.GetProcAddress

0060E5C3 66:C1E7 20 0060E5C7 ^ 0F87 E5E7FFFF .. 0060CDB2 833D 40EE6000 00 0060CDB9 E8 81120000 0060E03F 8D6424 04 0060E043 ^\0F84 0FD5FFFF .. 0060B558 66:3BED 0060B55B 83E9 00 0060B55E 833D 08ED6000 00 0060B565 87EB 0060B567 87DD 0060B569 0F84 70310000 .. 0060E6DF 8B0D 00ED6000 0060E6E5 890D 54EE6000 0060E6EB C705 00ED6000 DFDF6000 0060E6F5 ^\0F81 5EFBFFFF 0060E259 E8 BCD7FFFF

< we are here with EIP

Lets take a break for a moment, shall we? Ive traced code up to location mentioned above, 60E259, following flow of execution. As you can see, all the red colored opcodes can be removed. Regarding the complementary ones, they work as follows: a CALL adds 4 to the stack, so we have ADD ESP,4; POP DWORD PTR SS:[ESP-4] will do the exact opposite, SUB ESP,4. Therefore these 2 instructions can easily be considered as a NOP (NoOPeration). Also, Ive marked in bold black, at heading (4), a comparison which, if it fails, it will BYPASS whole HASP detection of the debugger. At least the part where program runs without a dongle. Dont know later if there are extra checks, after application logs on the hardware key. Ill show you what I mean in what follows. Lets proceed ;-)