Waiting to Start

Performance Comparison of AccessDatas Forensic Toolkit and Guidance Softwares EnCase Forensic software. 1 by Joel Snyder, Opus One2
Executive Summary
The greatest resource constraint facing the digital investigator is time. Getting answers faster offers the capability to investigate more cases, more thoroughly. Globally, two products are category leaders in the world of digital and computer investigation forensics: AccessDatas Forensic Toolkit (FTK) and Guidance Softwares EnCase Forensic (EnCase). Our testing of six different combinations of dataset size and hardware capabilities (see Figure 1, below) shows that FTK is as much as 8 times faster in analyzing evidence. While analysis time is only one aspect to consider when selecting a computer forensic product, the tremendous performance advantage of AccessDatas FTK should be a prime factor product buyers include in their evaluation of these products. www.opus1.com

90 80 70 60 50 40 30 20 10 0
16.1

81.6

FTK v4.2 EnCase v7.0

13.0 3.5 3.6

16.4 6.0 0.7 75 GB DS / 16-core HW 1.3 75 GB DS / 8-core HW 6.0

1 TB Dataset / 16-core HW

250 GB DS / 16-core HW

250 GB DS / 8-core HW

Figure 1 - Time in Hours to Process Evidence (Shorter Bars are Better)

Trademarks, company names, and product names used in this white paper are the property of their respective trademark holders. All trademarks remain property of their respective holders, and are used only to directly describe the products mentioned. Their use in no way indicates any relationship between Opus One, Inc. and the holders of the trademarks. 2 Authors contact information: Joel Snyder, Senior Partner, Opus One, jms@opus1.com, http://www.opus1.com/jms
1

Analysis of Results
Processing of forensic evidence from computers, disk drives and mobile devices is a difficult and intensive operation. Both FTK and EnCase not only process all standard files in the dataset, but also look through the file system to find files and fragments of files that have been deleted or corrupted. These files are categorized by type, embedded files and other artifacts are extracted, and all the data is indexed. For compressed and compound files, such as ZIP archives, or for files containing multiple embedded objects, such as email mailbox files (Microsoft PST), the files are broken up into pieces and individually analyzed and indexed. This process results in an explosion of information. Unfortunately, the digital investigator cannot begin their analysis of the target data until the processing (pre-analyzing and exposing) of the digital evidence completes. As additional information is discovered, the investigator may even need to make additional passes over the evidence, searching for particular types of data or file types. This means that the time to complete the initial (and any subsequent) extraction, enumeration, and indexing of files and file fragments is a significant barrier to the efficiency and effectiveness of the investigator. In our testing, FTK performs these tasks significantly faster than EnCase. Although each products processing algorithm and procedures remain proprietary, a simple examination of the system performance metrics during the initial analysis reveals that FTK is faster because it makes better and more effective use of hardware resources. Figure 2 below summarizes our observations about the use of resources by the two products.

Resource
CPU

Use by FTK
Uses all cores in a multi-CPU system, maximizing use of hardware Stresses disks significantly by

Disk

queuing I/O to all spindles, improving overall performance by maximizing disk throughput Uses all available memory

Memory

Use by EnCase Generally uses a single core for most of the processing operations, leaving hardware resources underutilized Does not keep disks busy, no obvious disk queuing and elevated latency, indicating underutilization of I/O subsystem and underutilized disk resources Uses all available memory

Figure 2 - Resource Usage (observed) in FTK and EnCase

FTK

EnCase

The screen shots to the left, showing Microsofts Windows Task Manager for both FTK and EnCase operating on the same hardware platform and data set, clearly show the advantage FTK has with multiprocessor systems. FTK leverages all available CPU resources in the multi-processor system, while EnCase is only keeping one CPU busy. This observation was typical for almost all of the processing of the evidence observed throughout the testing we conducted. To better understand the factors that affect performance of these two products, we selected two different hardware platforms. (see How We Tested, below, for exact specifications) and three different datasets, then tested all combinations of datasets and hardware platform.3

Since time is critically important in digital investigations, it makes sense to select the fastest tools which do not compromise breadth or depth of analysis. By leveraging high-performance tools such as AccessDatas FTK, investigators avoid case-killing delays and make the most effective use of the time available resources.

Comparing Apples to Apples

Both FTK and EnCase have a wealth of options that can be selected as part of the processing of evidence. In running this benchmark, we tried to make the product comparison as objective as possible by selecting the same options in each product. Because these products are not identical and matching up options is not a question of one-to-one mapping of check boxes, we chose to rely on a combination of the EnCase documentation from Guidance Software and the documentation and technical support staff at AccessData. In some cases, we had a clear mismatch in capabilities. For example, FTK offers an option to apply OCR (optical character recognition) to any images discovered in an attempt to extract available text. OCR represents a huge computational burden, but also is a capability we couldnt find in the EnCase product. Another mismatch was in file carver default options.4 When the file carving is enabled in both products, the default set of file types is very different; FTK recognizes many more file types than EnCase. In EnCase, for example, the common PDF (Adobe Acrobat) file type is not carved by default, while it is in FTK. This could result in dramatically different results in a forensic investigation. To make the comparison as real world as possible (the way a normal customer would run these products), we did not dive into every sub-menu, but let each product run with the vendor-recommended defaults in each major subsystem even when we determined that the testing unfairly favored EnCase. In cases where options were ambiguous, we erred in favor of EnCase.

One strange conclusion from our testing is that Guidance Softwares recommendation of a Quad Core processor as better than a Dual Core doesnt actually make any difference in reducing the elapsed time of initial data acquisition and indexing. Multi-core processors are cheap nowadays, but in this case, its largely a waste of money. 4 File carver is a term of art in the forensics community. From Wikipedia, File carving is the process of reassembling computer files from fragments in the absence of filesystem metadata. The carving process makes use of knowledge of common file structures, information contained in files, and heuristics ... [to] infer which fragments belong together. http://en.wikipedia.org/wiki/File_carving , retrieved 16/January/2013
3

We built on the work that Guidance Software did in a competitive performance test they published in March 2012 (EnCase Forensic A Development Perspective)5 giving their own apples-to-apples settings. In that article, Guidance staff had identified specific areas of comparison between EnCase and FTK. We worked with FTKs technical support staff to create an equivalent settings list so that the same set of major modules (such as the Instant Messenger parser and the File Carver) were enabled in both products. The table in Figure 3 summarizes the major settings for each product:

Settings

Metacarve

EnCase

Flag Bad Extensions

Enabled Enabled Enabled Enabled MD5,SHA1 Enabled

FTK

Enabled Enabled Enabled Enabled MD5,SHA1 Enabled

Comments

Called Recover Folders in EnCase Called File sig in EnCase

Protected File Analysis Thumbnail Creation Hash Analysis

Compound Files Processing

SHA-256 was not available in EnCase, so it was disabled in FTK

Find email Find Internet artifacts

Enabled Enabled

Enabled Enabled

FTK gives the user options on the types of archives that can be expanded. In addition it has support for more archives, such as PDF and 7zip EnCase supports fewer email types. All email-specific settings left at defaults.

indexing

Max word length: 32

Max word length: 32

Both products support multiple browser types, including IE, Chrome, Firefox, Safari and Opera

Registry Reports IM Parser

Enabled

Enabled for all available (Yahoo, MSN, AOL) Enabled

Enabled

EnCase default word length is 64, but our experience is that this does not improve overall analysis and bloats indices with unnecessary entries. Reduced to 32 in both products Called System Info Parser in EnCase

File Carver

Enabled for FTK supports an additional 5 IM types Yahoo, MSN, not available in EnCase, but only the four AOL and listed here were enabled Facebook Enabled Default settings for EnCase only include 5 (out of 314 possible) types to be carved; FTK defaults include 10 file types

Win Event Logs

Win Artifact Parser Unix Login, Syslog OCR on images

Enabled Enabled n/a Disabled

Enabled Enabled Disabled Disabled

Source data sets were from Windows Not supported in EnCase

Figure 3 - Processing Configuration Settings Selections in FTK and EnCase

http://encase-forensic-blog.guidancesoftware.com/2012/03/encase-forensic-development-perspective.html, retrieved 16/January/2013

How We Tested
To test computer forensic analysis tools, we identified three unique data sets and two hardware configurations. We configured each tool to have similar analysis functions (see Apples to Apples above). Then, we ran through the six different scenarios (two hardware configurations and three data sets) with each tool, timing the actual run-time of the complete processing sequence. Three different data sets were selected to represent different types of evidence that investigators would be confronted with in the field. Figures 4 and 4a provide detailed data set characteristics.

Data Set Size

75 GB 250 GB 1 TB

Characteristics
Windows XP boot drive 1176604 items (images, documents, spreadsheets, databases, emails, archives, temporary internet files, chat sessions) Windows NTFS data drive with 2297197 items Windows Vista boot drive with 8473455 items

Figure 4 - Data Set Characteristics

75 GB
Archives Databases Email 5369 58 71264 149 36103 13375 5510 1114 8128 1302 139 220 26 39856 336289 221013 436689

250 GB
Archives Databases Email 7894 147 97916 8244 91903 19190 17587 3210 360 15414 167678 3256 3783 58 463989 276482 1120086

1TB
Archives Databases Email 90760 223 1855907 79894 246238 374619 3750 2162649 26195 38640 11548 337161 62046 56661 42 1041985 2085089

Documents Executable Folders Graphics

Documents Executable Folders Graphics

Documents Executable Folders Graphics

Internet/Chat Files Multimedia OS/File System Files Other Known Types Presentations Spreadsheets User Types Total Slack/Free Space Unknown Types

Internet/Chat Files Multimedia OS/File System Files Other Known Types Presentations Spreadsheets User Types Total Slack/Free Space Unknown Types

Internet/Chat Files Multimedia OS/File System Files Other Known Types Presentations Spreadsheets User Types Total Slack/Free Space Unknown Types

Other Encryption Files

Other Encryption Files

Other Encryption Files

1176604

2297197

8473455

Figure 4 a - Item Type Breakout

Testing was done on two similar hardware configurations representing servers that would commonly be available to forensic investigators. All of these systems meet or exceed the recommended specifications from both AccessData and Guidance Software for the products tested.

Specification
CPU Memory I/O Subsystem Operating System Approximate list price

8 Core Configuration
2 sockets with Intel E5-2643 4-core CPUs @3.3 GHz each, total 8 cores 32 GB 4x 1TB SATA 7200 RPM drives (6 GB/sec direct-to-motherboard connection) Windows 2008 $4,000

16 Core Configuration
2 sockets with Intel E5-2470 8-core CPUs @2.3 GHz each, total 16 cores 96 GB 8x 600 GB SAS 15K RPM drives, 6GB/ sec, RAID0 configured into 4 RAID0 (stripe) volumes Windows 2008 $11,000

Each of the hardware configurations had current (as of January, 2013) full production versions (not demonstration) the software installed according to the vendors instructions Product Version
v4.2 v7.0.5.01.10

AccessData FTK EnCase Forensic

We configured the products for optimal performance as advised by both vendors in their configuration guidelines and best practices. In the case of FTK, we spread the I/O across three separate drives (boot, database/page file, and input evidence/case folder). With EnCase, we did a preliminary test to determine whether the product performed better when the use base case folder for primary evidence was selected or not. In our preliminary testing, we found that selecting this option doubled performance (halved run-time). We spread EnCase I/O across four separate drives (boot, evidence, evidence cache, and case folder). For testing on the faster hardware, instead of raw hard drives we used RAID0 virtual volumes (stripe sets).

Disclaimer
The research in this white paper was funded by AccessData. All testing occurred at AccessDatas facilities. The design of the test was developed by Opus One in consultation with AccessData technical staff. Testing was conducted under the supervision of Opus One staff.

www.opus1.com