Benchmarking

TheThemisproductbenchmarksprovideasimplemethodofcomparingtheperformanceofproductandsystemswith
similarpropertiesacrosscomparablearchitectures.ThebenchmarkingcanbehelpfulinunderstandinghowtheSIM
managercorrespondstovariousproducts,bothontermsofcompetitivenessbutalsointermsofintegrationbetween
modernheterogeneousenvironments.

ThepurposeoftheThemisproductbenchmarkingistoaddresstypicalquestionsthatwegetasked,seebelow:

-Arethetoolsuserfriendly?
-Isthedocumentationclearandunderstandable?
-Isinstallationstraightforward?
-Whatisthe"outofthebox"experience?
-Isiteasttounderstandthetechnicalarchitecture?
-Howdothetoolscomparetowhatthecompetitorsareoffering?

Theproductsaregradedbyacarefulevaluationoffunctionalityandfeasibilityinanumberofselectedareas.Allsystems
havebeentestedonthesameplatformsandunderthesameenvironmentalconditions.

TheBenchmarkingTrianglebelowillustratesthecommonparadigmusedforthebenchmarkingtests

Quality


User
Experience

Input Output

Thegradingisdoneinalmostabinaryway-givingtheevaluatedfunctiononlya(+)ora(-)–thusidentifyingwhether
thegradingisthereornotandthereforebeconsideredasnon-complianttoamoredetailedandformalcapacity
measurementprocess.

Thetoolshavebeenevaluatedwithrespecttoanon-educatedfirst-timeuser’sperceptionandhis/heractualexperiences
withthedifferenttools,andwehavethereforefoundthesinglegradingmostefficient.Itishoweverrecognisedthat
continuedexperiencesandformaleducationintherespectivetoolscouldchangethefunctionalityintheproducts,butitis
believedtherewillbemarginalimpactonthefinalbenchmarkingresults.

Inafullevaluationprocessthesubjectivityofanybenchmarkingprocessmustbetakenintoaccount.Theprocesscanand
willonlybaseitsevaluationcriteriaontheactualtestedanddocumentedfunctionalities.Therebytheprocessrecognizes,
butdoesnottakeintoaccountpossibleother‘non-tested’similarfunctionalityinthevariousdifferentsoftware-products.
Equallytherecouldbeothertoolsnotincludedinthisbenchmarkingprocess,whichcouldpossiblydelivermorebreadth
anddepthintotheprocess,butduetotime-limitationsthesebenchmarkshaveconcentratedonthe,toourbest
knowledge,2006globallymarketleadingsoftwareproducts.Allofthesebenchmarkingresultsshouldalwaysbeusedas
onlyaminorpartofamorecompletecorporateevaluationandselectionprocess,inordertoensuretherightinvestmentin
therightprotectiontool.
ThemisSoftwareLtd2007

Benchmarks

Total Scores Points Evaluation

Themis System Information Manager Suite 60

IBM Consul Risk Management Suite 39

Arcsight ESM Security Suite 37

Trinamo Exaprotect Security Management Suite 36

NetIQ Security Manager Suite 32

Sophos end Point Security and Control suite 32

McAfee System Security Solution Suite 29

Symantec SSIM and Client Security Suite 27

OperationalTimeandFeasibilityexample
OperationalTimeandFeasibilityexample

McAfee Sophos Symantec NetIQ Themis

Log-On Full scan of C: Drive

7.25min 8.44min 9.29min 6.41min 1.22min
(system snapshot)

Second On Demand Scan of C drive

7.21min 3.14min 4.37min 2.12min 12sec
(post cache)

On Demand Scan single folder

29 sec 27 sec 37 sec 44sec 7 sec
(544MB, no zip folders)
TheThemisbenchmarkingprocessismadewithaThemisSIMSuitereferenceandaccordingtonormalstandard-
processes*

Themis
IBM Arcsight
NetIQ Sophos Symantec McAfee Trinamo SIM
Consol ESM
Suite
General
Detailed Capacities 5 4 3 4 3 5 3 4
Capabilities
Ease of
up and initial configuration
Installation
Ease of Manager and Clients
Configuration discovery - population etc
Management ability and
Ease of Usage
overview
Stress-Performance under
heavy workload
Help and Understandable and easy
Documentation documentation and help
Available and competent
Support
support structure

Asset Overall Asset

10 10 9 9 13 11 12 22
Management management capability
Hardware Assets System data management
Hardware Assets BIOS data management
Mainboard data
Hardware Assets
management
Hardware Assets Chassis management
Memory controller
Hardware Assets
management
Hardware Assets Processor management
Hardware Assets Cache management
Memory module
Hardware Assets
management
Memory device
Hardware Assets
management
Hardware Assets Port management
Hardware Assets Slot management
On Board device
Hardware Assets
management
Hardware Assets Driver management
Printer and device
Hardware Assets
management
Applications and path
Software assests
management
Software assests Startup management
Software assests Processes management
Software assests Services management
Software assests Engines management
 System Keys (MS)
Software assests
management
Policy
Application manifest tool
Compliances
License Control License meetering tools

Security Overall Security

10 13 10 10 14 13 13 24
Management management capability
User Behaviour File & folder access and
capacity content
application and system
access
data access and
manipultaion
registry access and change
share and web (inter-intra-
extranet) access
remote storage access and
filestructures
USB/Firewire/CD-DVD
access and filestructures
User Analysis Meta Data management
capacity (snapshots)
User Assist Analysis
Stream MRU analysis
Shellbags analysis
COMdlg 32 analysis
Windows Protected
Storage analysis
Windows Prefetch analysis
Webbrowser analysis
Recycle Bin analysis
Drive Content analysis
OS analysis
Network settings analysis
Key Log scanning full intext content scanning
Forensic scanning full behavioural scanning
System scanning full system asset scanning
full drive structure and
Drives scanning
content scanning
Rapid metadata file
File Compare compare and change
functions

Compliance Overall Compliance

2 1 1 1 3 3 3 4
Management management capability
Patch Hotfixes and update
Management documentation
Application Application versions and
Version Control date documentation
Realtime USB connection
USB connections
and content documentation
complex user profiling
User Behaviours
documentation

Real Time Overall Monitoring

2 2 2 2 3 3 3 3
Monitoring capability
realtime client scanning
 realtime network scanning
triggered alerting

Overall Reporting
Reporting 3 2 2 3 3 2 2 3
capability
Summary report
Report wizard
Common XML formats

Total Total Benchmark figure 32 32 27 29 39 37 36 60

*http://en.wikipedia.org/wiki/BenchmarkingandHarrington,H.J.TheCompleteBenchmarkingImplementationGuide:TotalBenchmarking
Management.NewYork:McGraw-Hill,1996.



Contact information
PleasefeelfreetocontactTHEMISforfurtherinformationorcomments.

sales@themissoftware.com

ExecutivePartner
ExecutivePartner
Mr.SeanKane
sean@themissoftware.com
+44(773)4392210

ExecutivePartner
ExecutivePartner
Mr.AndersSørensen
anders@themissoftware.com
+4528191112