Vous êtes sur la page 1sur 264

Foglight ®

5.5.2
Foglight Experience Viewer
User Guide
© 2009 Quest Software, Inc. ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in this guide is furnished
under a software license or nondisclosure agreement. This software may be used or copied only in accordance with
the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any
means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser's
personal use without the written permission of Quest Software, Inc.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
www.quest.com
email: legal@quest.com

Refer to our Web site for regional and international office information.

Trademarks
Quest, Quest Software, the Quest Software logo, AccessManager, ActiveRoles, Aelita, Akonix, AppAssure,
Benchmark Factory, Big Brother, BusinessInsight, ChangeAuditor, ChangeManager, DeployDirector,
DirectoryAnalyzer, DirectoryTroubleshooter, DS Analyzer, DS Expert, ERDisk, Foglight, GPOADmin, Imceda,
IntelliProfile, InTrust, Invirtus, iToken, I/Watch, JClass, Jint, JProbe, LeccoTech, LiteSpeed, LiveReorg, LogADmin,
MessageStats, Monosphere, NBSpool, NetBase, NetControl, Npulse, NetPro, PassGo, PerformaSure, Quest Central,
Quest vToolkit, Quest vWorkSpace, ReportADmin, RestoreADmin, SelfServiceADmin, SharePlex, Sitraka,
SmartAlarm, Spotlight, SQL LiteSpeed, SQL Navigator, SQL Watch, SQLab, Stat, StealthCollect, Storage Horizon,
Tag and Follow, Toad, T.O.A.D., Toad World, vAutomator, vControl, vConverter, vFoglight, vOptimizer Pro, vPackager,
vRanger, vRanger Pro, vSpotlight, vStream, vToad, Vintela, Virtual DBA, VizionCore, Vizioncore vAutomation Suite,
Vizioncore vBackup, Vizioncore vEssentials, Vizioncore vMigrator, Vizioncore vReplicator, Vizioncore vTraffic,
Vizioncore vWorkflow, WebDefender, Webthority, Xaffire, and XRT are trademarks and registered trademarks of Quest
Software, Inc in the United States of America and other countries. Other trademarks and registered trademarks used
in this guide are property of their respective owners.

Disclaimer
The information in this document is provided in connection with Quest products. No license, express or implied, by
estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of
Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE
LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT,
CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION,
DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF
THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY
OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of
the contents of this document and reserves the right to make changes to specifications and product descriptions at any
time without notice. Quest does not make any commitment to update the information contained in this document.

License Credits and Third Party Information


To view license credit information, click the License Credits link on the Welcome to Foglight online help page.

User Guide
August 2009
Version 5.5.2
Table of Contents

Introduction to this Guide ...................................................................................................................................9


About Foglight ................................................................................................................................................................ 10
About Foglight Experience Viewer ................................................................................................................................. 10
About this Guide............................................................................................................................................................. 10
Foglight Documentation Suite ........................................................................................................................................ 12
Core Documentation Set ....................................................................................................................................... 13
Cartridge Documentation Sets .............................................................................................................................. 13
Foglight Experience Viewer Documentation Set ................................................................................................... 13
Feedback on the Documentation........................................................................................................................... 14
Text Conventions ........................................................................................................................................................... 14
About Quest Software, Inc. ............................................................................................................................................ 15
Contacting Quest Software.................................................................................................................................... 15
Contacting Quest Support ..................................................................................................................................... 15

Introducing the Foglight Experience Viewer (FxV) .........................................................................................17


Architecture Overview .................................................................................................................................................... 18
How Are Hits Processed?...................................................................................................................................... 18
What Are Hit Filters and Transaction Filters?........................................................................................................ 19
How Is Hit Analysis Performed? ............................................................................................................................ 20
How Do You Perform a Search? ........................................................................................................................... 20
How Do You Examine a Session?......................................................................................................................... 21
Data Model Overview ..................................................................................................................................................... 22
Custom Fields........................................................................................................................................................ 23
Metrics ................................................................................................................................................................... 24
Hits ........................................................................................................................................................................ 24
Sessions ................................................................................................................................................................ 26
Transactions .......................................................................................................................................................... 27
4 Foglight Experience Viewer
User Guide

FxV Browser Interface .................................................................................................................................................... 28


Logging In .............................................................................................................................................................. 29
Automatic Login via Foglight.................................................................................................................................. 29
FxV Screen Elements ............................................................................................................................................ 30

Performing Searches ........................................................................................................................................ 35


Time Constraints............................................................................................................................................................. 37
Session Search............................................................................................................................................................... 38
Transaction Search......................................................................................................................................................... 40
Hit Search ....................................................................................................................................................................... 43
Search Result Limits....................................................................................................................................................... 48
Saved Searches ............................................................................................................................................................. 48
Custom Search Screens................................................................................................................................................. 49

Analyzing User Sessions.................................................................................................................................. 51


Session Explorer Toolbar ............................................................................................................................................... 52
Details View .................................................................................................................................................................... 53
Timeline View ................................................................................................................................................................. 54
Hit Inspector View........................................................................................................................................................... 55
Content Replay View ...................................................................................................................................................... 57
Transactions View .......................................................................................................................................................... 61
Combined Sessions View ............................................................................................................................................... 63

Defining Custom Search Screens.................................................................................................................... 65


Creating and Editing Saved Searches............................................................................................................................ 66
Creating and Editing Custom Search Screens ............................................................................................................... 67

Setting User Preferences.................................................................................................................................. 71


Account Information........................................................................................................................................................ 72
Session Explorer Settings............................................................................................................................................... 72
Search Settings .............................................................................................................................................................. 75

Configuring the Hit Analysis Process ............................................................................................................. 79


Hit Filters......................................................................................................................................................................... 80
Hit Filter Match Conditions..................................................................................................................................... 80
Hit Filter Actions..................................................................................................................................................... 81
Table of Contents 5

Hit Filter Configuration............................................................................................................................................90


Hit Filter Additional Information ..............................................................................................................................92
Transaction Filters...........................................................................................................................................................92
Transaction Events.................................................................................................................................................93
Transaction Event Match Conditions......................................................................................................................94
Transaction Event Actions......................................................................................................................................97
Transaction Filter Storage Configuration..............................................................................................................104
Transaction Filter Configuration ...........................................................................................................................105
Transaction Filter Additional Information..............................................................................................................106
Special Events ..............................................................................................................................................................106
Special Events Configuration ...............................................................................................................................107
Special Events Additional Information..................................................................................................................108
Custom Fields ...............................................................................................................................................................108
Custom Fields Attributes ......................................................................................................................................109
Custom Fields Configuration ................................................................................................................................113
Custom Fields Additional Information...................................................................................................................115
Metrics...........................................................................................................................................................................115
Metrics Configuration ...........................................................................................................................................115
Metrics Additional Information ..............................................................................................................................117
Captured Metadata .......................................................................................................................................................117
Scripts ...........................................................................................................................................................................118
Scripts Configuration ............................................................................................................................................119
Scripts Additional Information...............................................................................................................................120
Sensitive Data Protection..............................................................................................................................................121
Sensitive Hit Details .............................................................................................................................................121
Sensitive Response Content Expressions ...........................................................................................................123
Hit Analysis Configuration Options................................................................................................................................125
Session Configuration ..........................................................................................................................................125
Archiver Configuration..........................................................................................................................................126
Script Configuration..............................................................................................................................................127
Other Configuration ..............................................................................................................................................127
Hit Analysis Configuration Change Log ........................................................................................................................128
Hit Analysis Import/ Export Configuration .....................................................................................................................130

Hit Analysis Examples.....................................................................................................................................133


6 Foglight Experience Viewer
User Guide

“Simple Data Extraction” Example................................................................................................................................ 134


“Event Occurrence Counting” Example ........................................................................................................................ 134
“Session-Level Event Occurrence Counting” Example................................................................................................. 135
“Parsing HTML Content” Example................................................................................................................................ 136
“Parameterized Metrics” Example ................................................................................................................................ 137
“Buy Tunnel” Example .................................................................................................................................................. 138
“Buy Tunnel”: Hit and Transaction Filters Overview ............................................................................................ 139
Transaction Event: Add to Cart............................................................................................................................ 139
Transaction Event: Checkout............................................................................................................................... 140
Transaction Event: Shipping ................................................................................................................................ 141
Transaction Event: Payment................................................................................................................................ 141
Transaction Event: Confirmation.......................................................................................................................... 143
Transaction Event: Cart Abandoned.................................................................................................................... 145
Hiding Sensitive Data........................................................................................................................................... 147

Configuring the Analysis Repository ............................................................................................................ 149


Overview....................................................................................................................................................................... 150
Architecture................................................................................................................................................................... 151
The Session Table ............................................................................................................................................... 151
The Transaction Tables ....................................................................................................................................... 153
Configuration ................................................................................................................................................................ 154
Analysis Repository Initial Setup.......................................................................................................................... 154
Analysis Repository Detailed Configuration......................................................................................................... 159
Data Transfer................................................................................................................................................................ 160
Access .......................................................................................................................................................................... 161
Accessing the Analysis Repository from Foglight ................................................................................................ 161
Accessing the Analysis Repository from Toad for MySQL .................................................................................. 161

Performing Advanced FxV Administration ................................................................................................... 165


Archivers....................................................................................................................................................................... 166
Collectors...................................................................................................................................................................... 167
Superuser Tasks........................................................................................................................................................... 168
Appliance Maintenance........................................................................................................................................ 168
Metrics ................................................................................................................................................................. 172

Managing User Account Settings .................................................................................................................. 175


Table of Contents 7

Users and User Groups ................................................................................................................................................176


User Preference Settings .....................................................................................................................................176
Pre-Defined User Accounts..................................................................................................................................176
Managing User Accounts .....................................................................................................................................177
User Group Settings.............................................................................................................................................178
Pre-Defined User Groups.....................................................................................................................................181
Managing User Groups ........................................................................................................................................182
Resource Groups ..........................................................................................................................................................183
Preference Groups........................................................................................................................................................184

Appendix: Java Regular Expressions in FxV Hit Analysis ..........................................................................187


Regular Expressions Overview .....................................................................................................................................188
Regular Expressions in FxV..........................................................................................................................................192
Looking for “Matches” (Yes or No) .......................................................................................................................192
Extracting a Single Value .....................................................................................................................................193
Identifying Sensitive Response Content...............................................................................................................195
FxV Regular Expression Tester ....................................................................................................................................198

Appendix: FxV Metrics ....................................................................................................................................199


Archiver Metrics ............................................................................................................................................................200
Hit Summary Metrics ............................................................................................................................................200
Analysis Repository Metrics .................................................................................................................................208
Archiver Capacity Metrics.....................................................................................................................................210
Archiver Health Metrics ........................................................................................................................................215
Archiver Appliance Metrics...................................................................................................................................224
Archiver Search Usage Metrics............................................................................................................................228
Script Metrics........................................................................................................................................................233
Collector Metrics ...........................................................................................................................................................234
Collector Health....................................................................................................................................................234
Server Metrics ...............................................................................................................................................................236
Server Health .......................................................................................................................................................237
Server Sessions ...................................................................................................................................................239
Server Replay Cache ...........................................................................................................................................239
Server Requests to Archivers...............................................................................................................................240
Error/Warning Metrics ...................................................................................................................................................241
8 Foglight Experience Viewer
User Guide

Errors ................................................................................................................................................................... 242


Warnings.............................................................................................................................................................. 242

Appendix: Glossary ........................................................................................................................................ 243


Component Architecture Terms.................................................................................................................................... 244
Data Model Terms ........................................................................................................................................................ 250
Product Feature Terms................................................................................................................................................. 253
Networking and Database Terms ................................................................................................................................. 256

Index ................................................................................................................................................................. 259


Introduction to this Guide

This chapter provides information about what is contained in the FxV User Guide. It
also provides information about the Foglight documentation suite and Quest Software.

This chapter contains the following sections:


About Foglight .............................................................................................................................10
About Foglight Experience Viewer ..............................................................................................10
About this Guide..........................................................................................................................10
Foglight Documentation Suite .....................................................................................................12
Text Conventions .........................................................................................................................14
About Quest Software, Inc...........................................................................................................15
10 Foglight Experience Viewer
User Guide

About Foglight
Foglight is an application management solution that reduces or eliminates service
disruptions to unify IT and the business. Unlike other solutions, it provides a correlated,
360 degree view of your applications from end user to database and from service levels
to infrastructure—to source the root cause of every incident impacting your business
and fix them quickly. Foglight correlates data from multiple perspectives into a single
version of the truth to provide deep insight into the service relationships that exist
between end users, the business and infrastructure components. Its unique adaptive
technology rapidly adjusts to change for improved application performance and service
levels, reduced operational cost and risk, and enhanced visibility for all stakeholders.

About Foglight Experience Viewer


Foglight Experience Viewer (FxV) is a network-based appliance that allows for capture,
storage, analysis, and replay of actual end user sessions in their entirety. Data is
provided to the Foglight Experience Viewer by the Foglight Experience Monitor (FxM),
which gathers the raw Web traffic from a switch span port or other network tap.
Foglight Experience Viewer allows customers to:
• Understand the impact of infrastructure problems on end users and business.
• Identify the business impact associated with a poor user experience.
• Determine the affected end users in order to better prioritize incident and problem
resolution.

About this Guide


The FxV User Guide provides in-depth coverage of the Foglight Experience Viewer
features, focusing on strategies, tactics, and real-world examples.
The intended audience for this document are all users who need a deeper understanding
of the Foglight Experience Viewer appliance.
The FxV User Guide is organized as follows:
Chapter 1, Introducing the Foglight Experience Viewer (FxV)—provides an
overview of the FxV data model used for hit analyzing, hit searching, and session replay
via the FxV browser interface.
Introduction to this Guide 11
About this Guide

Chapter 2, Performing Searches—presents the FxV functionality available to users


when performing data searches, and outlines the types of data searches and the results
they provide.
Chapter 3, Analyzing User Sessions—describes how FxV users can analyze a session
of interest, and outlines the analysis methods that are currently available.
Chapter 4, Defining Custom Search Screens—describes the settings that can be
configured when configuring a Custom Search Screen.
Chapter 5, Setting User Preferences—outlines the preferences that users can view and
edit (depending on their privileges); that is, account information, replay settings, and
search settings.
Chapter 6, Configuring the Hit Analysis Process—presents the resources available to
users for configuring the FxV Hit Analysis process (that is, resources defining the
conditions and actions used to analyze hits and sessions, as they are captured).
Chapter 7, Hit Analysis Examples—provides several examples illustrating common
uses of Hit Filters and Transactions Filters that FxV users can define via the FxV
browser interface.
Chapter 8, Configuring the Analysis Repository—presents the operations that an
administrator can perform to configure a secondary database for long-term storage of
session and transaction data captured by FxV. This database is configured via the FxV
user interface, and the data stored can be accessed through Foglight or via direct SQL
queries.
Chapter 9, Performing Advanced FxV Administration—presents the operations that
an administrator can perform to configure and administer the system via the FxV
browser interface.
Chapter 10, Managing User Account Settings—presents the concepts an
administrator must know in order to manage the FxV users and the capabilities they
have when using the FxV resources.
Appendix A, Java Regular Expressions in FxV Hit Analysis—provides guidance on
how to create regular expressions and how to use them with in FxV.
Appendix B, FxV Metrics—provides the complete list of FxV system metrics.
Appendix C, Glossary—provides complete Foglight Experience Viewer terminology.
12 Foglight Experience Viewer
User Guide

Foglight Documentation Suite


The Foglight documentation suite is made up of the core documentation set, plus the
documentation set for each Foglight cartridge that you deploy. Documentation is
provided in a combination of online help, PDF and HTML formats.
• Online Help: You can open the online help by selecting the Help tab from
Foglight’s action panel.

• PDF: The complete Foglight documentation set is available in Adobe PDF from
SupportLink. The PDF documentation can also be found in the Documentation
folder on the Foglight DVD. The following subset is available from the computer
Foglight is installed on: Administration and Configuration Guide, User Guide,
Command-Line Reference Guide, Web Component Guide and the Web
Component Tutorial. In addition, the cartridges ship with PDF guides. To view
the installed PDF guides, in Windows go to Start > Programs > Quest Software
> Foglight 5.5.2 > Documentation. The default location of the documentation
after installation is <foglight_home>/docs. Adobe Reader is required.
• HTML: Release Notes are provided in HTML format.
• Videos: Tutorial videos are not provided with the product, but are available on
SupportLink. They provide an easy and accessible way to learn about key
features and help you get started with Foglight.
Introduction to this Guide 13
Foglight Documentation Suite

Core Documentation Set


The core documentation set consists of the following documents:
• Release Notes (HTML)
• Middleware Release Notes (HTML)
• Getting Started Guide (PDF)
• What’s New Guide (PDF)
• System Requirements and Platform Support Guide (PDF)
• Upgrade Guides (PDF)
• Installation and Setup Guide set (all in PDF format)
• Administration and Configuration Guide (PDF and online help)
• Foglight User Guide (PDF and online help)
• Command-Line Reference Guide (PDF and online help)
• Transition Guide (PDF)
• Web Component Guide (PDF and online help)
• Web Component Tutorial (PDF and online help)
• Web Component Reference (online help)

Cartridge Documentation Sets


When you deploy a cartridge, the documentation set for it is installed. The online help
for the cartridge is integrated automatically with the core Foglight help. When you open
the help, the name of the cartridge is displayed in a top level entry within the table of
contents.
Most cartridges include additional PDF guides, which may be one or more of the
following: a Getting Started Guide, an Installation Guide, a User Guide, and a
Reference Guide. Cartridges also generally include their own Release Notes.

Foglight Experience Viewer Documentation Set


The Foglight Experience Viewer documentation set consists of the following
documents:
• Release Notes (HTML)
14 Foglight Experience Viewer
User Guide

• Quick Installation Guide for Dell PowerEdge Systems (PDF)


• Installation and Administration Guide (PDF)
• User Guide (PDF)
• Upgrade Field Guide (PDF)
• Security and Compliance Field Guide (PDF)
• Dell Remote Access Controller Guide (PDF)

Feedback on the Documentation


We are interested in receiving feedback from you about our documentation. For
example, did you notice any errors in the documentation? Were any features
undocumented? Do you have any suggestions on how we can improve the
documentation? All comments are welcome. Please submit your feedback to the
following email address:
am.docfeedback@quest.com
Please do not submit Technical Support requests to this email address.

Text Conventions
The following table summarizes how text styles are used in this guide:

Convention Description

Code Monospace text represents code, code objects, and command-


line input. This includes:
• Java language source code and examples of file contents
• Classes, objects, methods, properties, constants, and events
• HTML documents, tags, and attributes

Variables Monospace-plus-italic text represents variable code or


command-line objects that are replaced by an actual value or
parameter.

Interface Bold text is used for interface options that you select (such as
menu items) as well as keyboard commands.
Introduction to this Guide 15
About Quest Software, Inc.

Convention Description

Files, components, Italic text is used to highlight the following items:


and documents • Pathnames, file names, and programs
• The names of other documents referenced in this guide

About Quest Software, Inc.


Quest Software, Inc., a leading enterprise systems management vendor, delivers
innovative products that help organizations get more performance and productivity from
their applications, databases, Windows infrastructure and virtual environments.
Through a deep expertise in IT operations and a continued focus on what works best,
Quest helps more than 100,000 customers worldwide meet higher expectations for
enterprise IT. Quest provides customers with client management as well as server and
desktop virtualization solutions through its subsidiaries, ScriptLogic and Vizioncore.
Quest Software can be found in offices around the globe and at www.quest.com.

Contacting Quest Software

Email info@quest.com

Mail Quest Software, Inc.


World Headquarters
5 Polaris Way
Aliso Viejo, CA 92656
USA

Web site www.quest.com

Refer to our web site for regional and international office information.

Contacting Quest Support


Quest Support is available to customers who have a trial version of a Quest product or
who have purchased a commercial version and have a valid maintenance contract. Quest
16 Foglight Experience Viewer
User Guide

Support provides around the clock coverage with SupportLink, our web self-service.
Visit SupportLink at: http://support.quest.com.
From SupportLink, you can do the following:
• Quickly find thousands of solutions (Knowledgebase articles/documents).
• Download patches and upgrades.
• Seek help from a Support engineer.
• Log and update your case, and check its status.
View the Global Support Guide for a detailed explanation of support programs, online
services, contact information, and policy and procedures. The guide is available at:
http://support.quest.com/pdfs/Global Support Guide.pdf.
1
Introducing the Foglight
Experience Viewer (FxV)

Foglight Experience Viewer allows for capture, storage, analysis, and replay of actual
end user sessions in their entirety. Data is provided to the Foglight Experience Viewer
by the Foglight Experience Monitor, which gathers the Web traffic from a switch span
port or other network tap.
Foglight Experience Viewer allows customers to:
• Understand the impact of infrastructure problems on end users and business
• Identify the business impact associated with a poor user experience
• Determine affected end users in order to better prioritize incident and problem
resolution
This chapter is an overview of the FxV data model used for hit analyzing, hit searching,
and session replay via the FxV browser interface.

This chapter contains the following sections:


Architecture Overview .................................................................................................................18
Data Model Overview ..................................................................................................................22
FxV Browser Interface .................................................................................................................28
18 Foglight Experience Viewer
User Guide

Architecture Overview
This section is intended for FxV’s first-time users, and provides information about the
following topics:
• “How Are Hits Processed?” on page 18
• “How Is Hit Analysis Performed?” on page 20
• “What Are Hit Filters and Transaction Filters?” on page 19
• “How Do You Perform a Search?” on page 20
• “How Do You Examine a Session?” on page 21

How Are Hits Processed?


The Foglight Experience Viewer (FxV) and the Foglight Experience Monitor (FxM)
components work together for data capture and storage, as illustrated in the following
figure.
Logical Architecture Model Figure 1
Introducing the Foglight Experience Viewer (FxV) 19
Architecture Overview

The FxM Agent consumes raw HTTP packets, determines the session ID for each hit,
and passes raw hit data to its local Collector (through a local, shared memory interface).
The FxM Collector accumulates these raw hits and then sends them across a network
socket, to the FxV Tcpbatches listener. The FxV Tcpbatches process opens and listens
on a TCP socket (FxV’s port 7623), and when hit data is received, it forwards it to its
local Archiver, through a shared memory interface.
The FxV Archiver reads each hit, performs hit processing and analysis, then updates the
searchable database (for the hit, the session that contains the hit, and any related
transactions). The Archiver maintains a subset of the capture database on local disk, and
manages the transfer of database segments to the storage tier (if present).
A secondary Analysis Repository can be implemented as part of the Storage service, to
provide access to session and transaction data over extended periods of time.

What Are Hit Filters and Transaction Filters?


A Hit Filter is a collection of conditions and actions used to analyze hits as they are
captured. Hit Filters can be used to detect and alert on any per-hit condition, to mark
interesting hits for later searching, and to manage hit storage. Hit Filters are powerful in
their ability to populate custom fields and increment metrics. Hit Filters can also be used
to define events within Transaction Filters.
A Transaction Filter is a collection of conditions and actions used to analyze sessions,
as they are captured, and to identify the distinct transactions within those sessions.
Transaction Filters can be used to detect and alert on any activity within a session, and
to calculate custom metrics across aggregated sessions or transactions. A Transaction
Filter can detect multiple events that provide information about site problems and client
activity within a related set of pages. Like Hit Filters, Transaction Filters give relevant
context to the captured data to make searching and reporting easier.

Important A key distinction between Hit Filters and Transaction Filters is that Hit Filters work on a
single hit, while Transaction Filters can work across multiple hits in the same session.

Hit Filters and Transaction Filters can also invoke Groovy scripts to perform additional
capture-time processing, such as evaluating complex conditional expressions or regular
expressions, transforming hit data for use in searchable fields or metrics, or interacting
with external systems through a generic HTTP interface.
20 Foglight Experience Viewer
User Guide

How Is Hit Analysis Performed?


The Archiver performs real-time analysis with Hit Filters and Transaction Filters that
profile and alert upon end user behaviors and problems.
The Archiver takes the following actions for each hit:
• Validates and parses the hit into a hit model object.
• Determines the type of the hit (HTML versus non-HTML) by scanning response
headers and content.
• For HTML hits, creates a string representation of the content for content analysis
(which may require decompression).
• For HTML hits, extracts the page title from the response content.
• Finds and updates the session model for the hit, creating a new session model if
necessary.
• Applies Hit Filters, which calculate the final hit status and optionally update the
hit/session or metrics.
• Applies Transaction Filters, which identify specific transactions within the
session, calculate transaction status, and optionally update the session/
transaction(s) or metrics.
• Optionally for HTML hits, breaks the response content down into keywords for
full-text searching (prepares to update inverted index).
• Writes hit information to the capture database for later searching and replay.

How Do You Perform a Search?


The user can perform a hit, session, or transaction search using the FxV browser
interface, or as the result of a drilldown from Foglight.
When performing a search, the FxV Server accesses data stored on various Archiver and
Storage components in the appliance group, assembles the results, and returns the
merged view of the search results. If a storage tier is present, the Storage components
are searched first, to avoid any possible work for the Archivers.
The data stored in the Analysis Repository is not accessible directly from the FxV user
interface. Instead, this data can be accessed from within Foglight, via a Foglight data
source (as part of the Foglight Experience Viewer cartridge). It can also be queried
directly via SQL, using Quest’s Toad® for MySQL or any other third-party reporting
tool.
Introducing the Foglight Experience Viewer (FxV) 21
Architecture Overview

How Do You Examine a Session?


After users have searched and found a session of interest, they can inspect trace-level
details for the user session, or visually replay the session (presenting the Web
applications as the end user originally saw them). Session exploring provides visibility
into every aspect of the user experience:
• Complete hit details for the entire HTTP request and response for any hit in the
session
• A visual presentation of each page as originally served by the Web server
• A visual presentation of each page with user input (forms/links)
• The original HTML source for each page generated by the Web server
• Any errors (HTML or other source) detected by Hit Filters and Transaction
Filters
• Session and transaction durations, hit counts, and other aggregate details
The FxV Session Explorer offers several ways of analyzing the hit/session/transaction
search results, via the following types of views: Details, Timeline, Hit Inspector,
Content Replay, Transactions, and Combined Sessions.
The following figure illustrates an example of a Timeline view for a user session.
User Session Visual Replay Example Figure 2
22 Foglight Experience Viewer
User Guide

Data Model Overview


All data captured and stored by FxV starts with the individual hits processed by the
FxM Collectors and passed to the FxV Archivers. As illustrated in the following figure,
hits are grouped into sessions based on their session ID. Every hit is part of one session,
at most. A session may contain any number of transactions, as defined by Transaction
Filters.
Hit-Session-Transaction Diagram Figure 3

The Hit, Session, and Transaction data types represent the core of the FxV data model.
Other elements include Custom Fields and Metrics. The interactions between the
elements of the data model are complex, as illustrated in the following diagram.
Introducing the Foglight Experience Viewer (FxV) 23
Data Model Overview

Data Model Diagram Figure 4

For additional details about the FxV data model, see these sections:
• “Custom Fields” on page 23
• “Metrics” on page 24
• “Hits” on page 24
• “Sessions” on page 26
• “Transactions” on page 27

Custom Fields
The FxV Hit Analysis process is highly customizable by virtue of user-defined Hit
Filters and Transaction Filters. These filters allow the captured data to be customized by
adding custom fields to any hit, session, or transaction. A custom field can consist of
24 Foglight Experience Viewer
User Guide

any piece of data that might be useful as a search term or a search result. The values
stored in custom fields can come from a variety of sources. Typically, they are some
variation on a value extracted from a hit. In many cases, it is useful to extract data from
a specific hit and store it as a custom field on the enclosing session and/or transaction, in
order to allow the session to be found based on that data (for example, a username).
Custom fields on individual hits are less common, since most of the data typically used
to populate a custom field value is already part of the hit. However, it may be useful to
extract a header, field, or cookie value and store it as a hit-scoped custom field, in order
to allow it to appear directly in hit search results.

Metrics
FxV metrics are distinct from any hit, session, or transaction. However, like custom
fields, metrics are updated by Hit Filters and Transaction Filters. Essentially, FxV user-
defined metrics are global numeric custom fields, whose values are not set explicitly
(they are only incremented or decremented).
Although metrics are not directly associated with hits, sessions, or transactions, metric
updates are. This means that for any hit, session, or transaction, it is possible to see any
metrics that were modified during its processing, specifically:
• If a Hit Filter updates a metric, that update is associated with the hit being
processed, as well as with the active session. However, that update is NOT
associated with any transactions.
• If a Transaction Filter updates a metric, that update is associated with the active
transaction, as well as with the active session. However, that update is NOT
associated with any hits.

Hits
A hit is a single request made by a client browser and the corresponding response from
the Web server. FxV captures and stores the following data for each hit:
• Overall Hit Status (determined by Hit Filters)
• Timing Details
• End to End Time
• Client Time
• Processing Time
Introducing the Foglight Experience Viewer (FxV) 25
Data Model Overview

• Request Details
• Client IP address
• HTTP Method (GET, POST, etc.)
• Full URL
• Server
• Path
• Request Headers (for example, User-Agent and Referer)
• Request Fields (including form field values)
• Response Details
• HTML Title
• HTML Base HREF
• Timestamp
• Server IP
• HTTP Version
• HTTP Code
• Total Response Length
• Response Headers (for example, Content-Type)
• Cookies
• Custom Fields values (determined by Hit Filters)
• Metric Updates caused (determined by Hit Filters)
• Archiver Details
• (FxM) Collector ID
• Collector Version
• Hit Identifier
• Batch ID
• Session ID
• FxM URL ID
• FxM Referer ID
• Exception(s)
• Response Contains HTML Tag
• Response Contains Frameset Tag
26 Foglight Experience Viewer
User Guide

• Hit Filter Matches (determined by Hit Filters)


• Response Content (which include the Response Content Keywords)

Note The following values are available on each hit for hit analysis, but are only stored at the
session level (therefore, you can’t search for hits based on them):
• City
• Region
• Country
• ISP
• Subnet
• Browser Type
• Username

Sessions
A session is a list of hits (with the same session ID) that were submitted by the same
browser client in the same visit to the site. The following data is calculated and stored
for each session:
• Details
• Session ID
• Start Time
• Last Hit Time
• Duration
• Session Active (In Progress)
• Stop Reason
• Time of First Error
• Time of Last Error
• Client Details
• Client IP
• Browser Type
• ISP
• City
• Region
Introducing the Foglight Experience Viewer (FxV) 27
Data Model Overview

• Country
• Subnet
• Username
• Hit Counts
• Total Hits
• Stored Hits
• HTML Hits
• Total Errors
• Total Warnings
• HTML Errors
• HTML Warnings
• Transaction Counts
• Total Completed Transactions
• Completed Transaction Errors
• Completed Transaction Warnings
• Timings
• Total Client Time
• Total End to End Time
• Total Processing Time
• Custom Field values (determined by Hit Filters and/or Transaction Filters)
• Metric Updates caused (determined by Hit Filters and/or Transaction Filters)
• Hit Filter Match Counts (determined by Hit Filters)

Transactions
A transaction is a portion of a session that matched a Transaction Filter. Transactions
are considerably different than hits and sessions, in that they only exist as a result of
specific Transaction Filter definitions. The following data is stored for each transaction:
• Details
• Transaction Filter Name
• Session ID
• Start Time
28 Foglight Experience Viewer
User Guide

• Last Hit Time


• Duration
• Transaction Active (In Progress)
• Stop Reason
• Time of First Error
• Time of Last Error
• Transaction Status (determined by Transaction Filters)
• Hit Counts
• Total Hits
• Stored Hits
• HTML Hits
• Total Errors
• Total Warnings
• HTML Errors
• HTML Warnings
• Timings
• Total Client Time
• Total End to End Time
• Total Processing Time
• Event occurrences (determined by Transaction Filters)
• Custom Field values (determined by Transaction Filters)
• Metric Updates caused (determined by Transaction Filters)

FxV Browser Interface


The FxV browser interface is the main visual interface to the Foglight Experience
Viewer appliance. It is a local Web application that runs as part of the FxV Server
software component. The browser interface is used for configuring the appliance and for
user session reporting and visualizations (including visual replay). The browser
interface also supports workflows where customers drill down from the Foglight or
Foglight Experience Monitor interfaces.
Introducing the Foglight Experience Viewer (FxV) 29
FxV Browser Interface

This section provides details about the FxV browser interface and its elements, how to
log into the GUI, and how to get quick tips about the GUI elements (via the tips). For
details about these topics, see the following sections:
• “Logging In” on page 29
• “Automatic Login via Foglight” on page 29
• “FxV Screen Elements” on page 30

Logging In
To access the Foglight Experience Viewer browser interface:
1 Enter one of the following addresses in your Web browser:
• http://[myappliance]/console
• https://[myappliance]/console (for SSL access)
Note To force all browser interface traffic to use https, log into the browser interface,
navigate to Configure > Superuser Tasks > Server Configuration, and set
SSL Redirection Enabled to Yes.

where [myappliance] is the IP address allocated to the eth0/Gb1 port of the


appliance configured as Server (for information about how to allocate an IP
address, see the FxV Installation and Administration Guide).
Note If your system is configured to not use the default http/https ports (that is, 80/443),
you need to add port numbers to the URLs, as follows:
http://[myappliance]:7630/console
https://[myappliance]:7643/console

2 Enter your username and password at the Foglight Experience Viewer login
screen, then click Login. For information about default accounts, see the FxV
Installation and Administration Guide.

Automatic Login via Foglight


Foglight users that want to drill down in to an FxV appliance can automatically follow
an URL to a remote FxV appliance, without the need to re-authenticate themselves. In
order to avoid explicitly logging in to the FxV appliance, an FxV administrator must
enable the Automatic Login via Foglight feature for the specific Foglight users. For
details, refer to the FxV Installation and Administration Guide.
30 Foglight Experience Viewer
User Guide

FxV Screen Elements


The FxV browser interface is illustrated in the following figure.
FxV Screen Elements Figure 5

The FxV screen consists of the following elements:


• Menu bar—contains application-specific menus which provide access to the FxV
functions and help information.
Note Some menu options are displayed only for users with full administrative privileges.

• Action panel—contains the various actions that you can perform, depending on
the menu option selected from the menu bar (for example, define search
conditions for a hit/session/transaction search, display the results of a data search,
edit the replay preferences, configure the FxV system, manage users and group
accounts, etc.).
For a complete list of menu options, see the following table.
Introducing the Foglight Experience Viewer (FxV) 31
FxV Browser Interface

Menu Option Description

Search Transactions Performs a search for transactions in the capture


database. Once an interesting transaction has
been found, the transaction or the entire session
can be analyzed from the search results screen.
For more information, see “Transaction Search”
on page 40.

Sessions Performs a search for sessions in the capture


database. Once an interesting session has been
found, it can be analyzed from the search results
screen. For more information, see “Session
Search” on page 38.

Hits Performs a search for hits in the capture


database. Once an interesting hit has been found,
the entire session can be analyzed from the
search results screen. For more information, see
“Hit Search” on page 43.

Custom Search Performs a hit search based on search conditions


specified by a Custom Search Screen. For more
information, see “Custom Search Screens” on
page 49.
Note This menu option is available only after a
Custom Search Screen has been defined.

Preferences Provides account information, a list of replay


settings, and a list of search settings. These fields
can be edited or just viewed, depending on the
user’s privileges. For more information, see
“Setting User Preferences” on page 71.
32 Foglight Experience Viewer
User Guide

Menu Option Description

Configure1 Hit Analysis Creates, configures, or displays:


• Hit Filters. See “Hit Filters” on page 80.
• Transaction Filters. See “Transaction Filters”
on page 92.
• Special Events. See “Special Events” on
page 106.
• Custom Fields. See “Custom Fields” on
page 108.
• Metrics. See “Metrics” on page 172.
• Captured Metadata. See “Captured Metadata”
on page 117.
• Scripts. See “Scripts” on page 118.
• Sensitive Hit Details. See “Sensitive Hit
Details” on page 121.
• Sensitive Response Content Expressions. See
“Sensitive Response Content Expressions” on
page 123.
• Hit Analysis Configuration Options. See “Hit
Analysis Configuration Options” on page 125.
• Analysis Repository. See “Configuring the
Analysis Repository” on page 149.
• Hit Analysis Configuration Change Log. See
“Hit Analysis Configuration Change Log” on
page 128.
Allows importing and exporting the Hit Analysis
configuration. See “Hit Analysis Import/ Export
Configuration” on page 130.

Custom Search Creates and configures Custom Search Screens.


Screens For more information, see “Defining Custom
Search Screens” on page 65.
Note The first step towards creating a Custom Search
Screen is to create a Saved Search. Once a
saved search has been created, it can be used
to build a Custom Search Screen.
Introducing the Foglight Experience Viewer (FxV) 33
FxV Browser Interface

Menu Option Description

Users Creates and configures User accounts for persons


who need to log into the FxV browser interface.
For more information, see “Users and User
Groups” on page 176.

User Groups Creates and configures User Groups.


A User Group is a group of users sharing the
same permissions. For more information, see
“Users and User Groups” on page 176.

Preference Creates and configures Preference Groups.


Groups A Preference Group is a group of users with
common default user preference settings. For
more information, see “Preference Groups” on
page 184.

Resource Creates and configures Resource Groups.


Groups A Resource Group is a collection of resources
(Custom Search Screens and Saved Searches)
that can be associated with one or more User
Groups, in order to grant access to those
resources. For more information, see “Resource
Groups” on page 183.

Archivers Adds and configures new Archivers to the


system. For more information, see “Archivers”
on page 166.

Collectors Displays the list of Collectors that have made


contact with the FxV Server, and manages the
Collector Groups defined in the system. For
more information, see “Collectors” on page 167.

Superuser Provides access to the Appliance Maintenance


Tasks settings and allows operations with metrics
(viewing metrics, resetting metrics, and running
metric-based diagnostics). For more information,
see “Superuser Tasks” on page 168.
34 Foglight Experience Viewer
User Guide

Menu Option Description

Help Contents Provides the list of documents available via the


FxV browser interface, in PDF or HTML format:
• FxV User Guide
• FxV Installation and Administration Guide
• Groovy Scripting API

About Provides information about the product version,


technical support contact details, and Quest’s
copyright information.

Logout Logs you out of the FxV browser interface.


1
This menu is available only for users with the necessary privileges.

By default, the display area shows tips about each page element. These tips contain
supplementary information regarding their corresponding element.
To view a tip:
• Click the button associated with it.
To turn tips on/off:
• Click Hide Tips/Show Tips, on the upper right side of the screen.
2
Performing Searches

This chapter presents the FxV functionality available to users when performing data
searches, and outlines the types of searches that can be performed and the results they
provide.
Data captured by FxV can be retrieved via the search screens. Three types of searches
are available in the FxV browser interface. They are distinguished by the types of data
that they return: individual hits, user sessions, and transactions.
For each of these data types, two types of search interfaces are available: default search
screen and custom search screens.
Each row in the search results screen represents a single session, transaction, or hit,
depending on the type of search that was performed. The columns that appear in the
search results depend on two factors: custom fields and user preferences. In addition to a
default set of column headings, a column is added for each custom field (with the
appropriate scope) that has been configured to appear in the search results.

To hide any of these columns, click the Edit Hidden Columns icon in the upper-left
corner of the search results screen. Alternatively, configure the set of hidden columns
via the Preferences screen, by editing the Hit/ Session/ Transaction Search Results
Hidden Columns settings.
The column by which the results are sorted can be changed by clicking any column
heading.
The full search results can be exported as a comma-separated text file (CSV) by clicking
the Export Results (CSV) button in the search results screen.
This chapter contains the following sections:
36 Foglight Experience Viewer
User Guide

Time Constraints..........................................................................................................................37
Session Search............................................................................................................................38
Transaction Search......................................................................................................................40
Hit Search....................................................................................................................................43
Search Result Limits....................................................................................................................48
Saved Searches ..........................................................................................................................48
Custom Search Screens..............................................................................................................49
Performing Searches 37
Time Constraints

Time Constraints
This section presents the generic time constraints imposed on all types of data searches.
All searches are constrained to a user-specified time window. Depending on the type of
search, several parameters define the search window, as follows:
• Time Range Mode—for session/transaction searches only, defines the type of
sessions and transactions to be searched for, in terms of their status (“active”
versus “completed”). The options available for this parameter include:
• Sessions/Transactions that started within (a specified time range)
• Sessions/Transactions that ended within (a specified time range)
• Sessions/Transactions that were in progress at (a specified time)
• Sessions/Transactions that are currently in progress (no additional time
constraints are specified)
• Time Range—restricts the search to the specified time window. It ranges from
“the last minute” to “the last 72 hours”. Alternatively, you can choose to specify
start and end times (rather than defaulting the end time to “now”).
Note The maximum (72 hours) is determined by the user’s Preference Group (and
possibly, though not typically, by the user preference settings).

• Timestamp—for session/transaction searches only, specifies a specific time for


session/transaction searches when the Time Range Mode is set to “...were in
progress at...”.
• Start Time—defines the time to start the search, when Time Range is set to
“specified start and end times...”. It is specified as a date and a time (down to the
minute).
• End Time—restricts how much captured data can be included in the search, when
Time Range is set to “specified start and end times...”. It is specified by selecting
a search window duration (such as five minutes, one hour, etc.). The UI displays
the actual end time based on the start time and the selected window.
• Sessions/Transactions to Search—for session/transaction searches only, specifies
whether to include active or completed sessions/transactions. The options
available for this parameter are:
• Search completed and active sessions/transactions
• Search completed sessions/transactions only
• Search active sessions/transactions only
38 Foglight Experience Viewer
User Guide

Note If any options are not available for these generic parameters defining the search window, it
is due to the currently selected value of one of the other time-related fields.
Certain combinations are not allowed by the user interface. For example, if you select
“Sessions that are currently in progress” for the Time Range Mode parameter, the option
“Search completed sessions only” for the Sessions to Search parameter is disabled, since
these two values are contradictory.

Session Search
This section presents the parameters available to define a session search and the type of
information displayed in session search result screens.
Session searches return information about completed and/or active user sessions, based
on search conditions that are grouped into seven basic categories (in addition to the time
constraints presented in section “Time Constraints” on page 37):
• General Conditions—for session search, the only field in this category is the
Session ID. This provides direct access to a known session, based on its unique
ID.
• Custom Field Conditions—any number of session custom fields can be defined
by Hit Filters and Transaction Filters. These filters then populate custom field
values for each session. The value of any session-level custom field can be used
as a search condition.
Note Custom fields defined as numeric can be searched using operators such as “>” and
“<=”.

• Metric Update Conditions—Hit Filters and Transaction Filters can be configured


to update metrics when a particular condition is met. When searching for
sessions, the search can be constrained to only include sessions during which a
specified metric increased by a specified amount. Comparison operators allow for
full control over the types of increases to be included (for example, increase > 0
or increase <= 10, etc.).
• Client Detail Conditions—restricts the session search by one or more captured
session details relating to the end-user client for the session, including Browser
Type and client location related details.
Performing Searches 39
Session Search

• Client IP—restricts the session search based on the IP address of the client
browser. An asterisk (“*”) can be used as a wildcard character: “66.35*”
would search for any IP addresses that begin with “66.35”.
Note An IP address cannot be used to reliably identify a single client on public Web
sites. It is not uncommon for a single client visit to use multiple IP addresses, or
for traffic from simultaneous clients to be multiplexed onto a single originating IP
address.

• Browser Type—restricts the session search based on the client’s “Browser


Type”. An asterisk (“*”) can be used as a wildcard character. Click the Find
Browser icon to choose from a list of browser options.
• ISP—restricts the session search based on the ISP (Internet Service Provider)
of the client. An asterisk (“*”) can be used as a wildcard character. Click the
Find ISP icon to choose from a list of ISP options.
• City—restricts the session search based on the City of the client. An asterisk
(“*”) can be used as a wildcard character. Click the Find City icon to
choose from a list of city options.
• Region—restricts the session search based on the Region of the client (for
example, State). An asterisk (“*”) can be used as a wildcard character. Click
the Find Region icon to choose from a list of region options.
• Country—restricts the session search based on the Country of the client. An
asterisk (“*”) can be used as a wildcard character. Click the Find Country
icon to choose from a list of country options.
• Hit Filter Conditions—the match conditions defined for each Hit Filter in the
system are evaluated against every hit processed by FxV. This type of condition
allows you to search for sessions that contain a particular number of hits that
matched a specified Hit Filter.
For example, if you have defined a Hit Filter called “Failed Login” that matched
hits to the page displayed to a user when they entered incorrect login information,
to find sessions in which users have hit this page five times or more, you could
specify the “Failed Login” Hit Filter with a value of “>= 5” and all such sessions
would be returned by the session search.
• Hit Count Condition—these types of conditions allow the user to search for
sessions that contain a specified number of a particular category of hit.
• Transaction Count Conditions—Transaction Filters allow FxV to delineate
transactions as subsets of the hits that make up a session. These conditions allow
the user to search for sessions that contain a specified number of transactions.
40 Foglight Experience Viewer
User Guide

The session searches have the following limitation when searching on metric deltas or
hit filter matches: users can only match on zero when using the search criteria “= 0”. All
other searches, “>”, “>=”, “<“, “<=”, and “!=”, do not match on zero.
For example:
• If you use the criterion “< 5”, it returns sessions where the entry is 1, 2, 3, or 4,
but not 0.
• Also, the criterion “!= 3” returns sessions where this value is 1, 2, or >3 (not 0 or
3).

Note This limitation does not apply to transaction searches.

The session search results screen renders the information in table format. Each row in
the session search results table contains two icons that can be clicked to trigger activities
related to that session:
• Explore Session—click this icon to open the Session Explorer view. This is a
tabbed view that provides multiple ways to get details about the session: Details,
Transactions, Timeline, Hit Inspector, and Content Replay.
• View Session Details—click this icon to open a new window containing all
available details about the session.

Transaction Search
This section presents the parameters available to define a transaction search and the type
of information displayed in transaction search result screens.
Transaction searches return information about completed and/or active transactions of a
specific type. The first step in any transaction search is to specify the type of
transactions (that is, the Transaction Filter) to search for. Transactions can then be found
based on search conditions that are grouped into five basic categories (in addition to the
time constraints presented in section “Time Constraints” on page 37):
• General Conditions—for transaction search, the only field in this category is the
Session ID. This provides direct access to any transactions that occurred within a
known session based on its unique ID.
• Status Conditions—every transaction can have a status of OK, Warning, or Error,
as defined by the Transaction Events that make up the Transaction Filter. By
Performing Searches 41
Transaction Search

default, all transactions have a status of OK. The Transaction Filter may define
conditions that change this status.
• Custom Field Conditions—if the type of transaction being searched defines
transaction-scope custom fields, these fields can be used as search constraints.
Note Custom fields defined as numeric can be searched using operators such as “>” and
“<=”.

• Hit Count Conditions—these types of conditions allow the user to search for
transactions that contain a specified number of a particular category of hit.
• Event Conditions—every Transaction Filter is made up of one or more
Transaction Events. Some events may not occur in every transaction, and some
events may occur more than once, if allowed by the event definition. The
occurrence (or lack thereof) of a specific event can be used as a search condition.
For example: “Find transactions in which event A occurred, event B did not
occur, and event C occurred three or more times.”
The transaction search results screen renders the information in table format.
Transaction search results are similar to session search results, including the two icons
(Explore Session and View Session Details) described in session “Session Search” on
page 38.
Unique to the transaction search result screen is the display of the following Transaction
Event information:
• Each event that makes up the Transaction Filter definition (displayed above the
main result table).
• For each event, a count of the number of transactions in which the event occurred.
• The percentage of the total number of transactions returned by the search that
included the event.
Note For some types of transactions, this information is not meaningful. If all events are
required and therefore occur in every transaction, the number shown for each event
is the same and the percentage is always 100%. However, for a transaction like a
“buy tunnel”, the event counts and percentages can provide valuable information
about conversion rates and common transaction exit points.

One column unique to the transaction search result screen is Events. This column shows
a sequence of single-character labels for all Transaction Events that occurred within the
transaction, in the order that they occurred. This provides a quick summary of the
activities that took place (or did not take place) as part of the transaction.
Example:
Consider a Transaction Filter made up of the following events:
42 Foglight Experience Viewer
User Guide

• Add To Cart (A)


• Shipping (B)
• Billing (C)
• Confirmation (D)
• Orders Placed (E)
• Error Occured (R)
• Abandoned (X)
The Events column might contain values such as:
• ABCDE—user added one item to the shopping cart, calculated shipping,
calculated billing, got a confirmation for the transaction, and placed the order.
• ABCX—user added one item to the shopping cart, calculated shipping, calculated
billing, and abandoned the shopping cart.
• AX—user added one item to the shopping cart, then abandoned it.
• ABCDRX—user added one item to the shopping cart, calculated shipping,
calculated billing, got a confirmation for the transaction, then an error occured
and user abandoned the shopping cart.
Performing Searches 43
Hit Search

The following figure illustrates an example of results returned by the transaction search
matching the conditions imposed by this Transaction Filter.

Hit Search
This section presents the parameters available to define a hit search and the type of
information displayed in hit search result screens.
Hit searches return information about individual captured hits (HTTP Requests/
Responses) based on search conditions that fall into six basic categories (in addition to
the time constraints presented in section “Time Constraints” on page 37):
• General Conditions—includes a wide range of hit details, such as the page title
(for HTML hits), the Web server, the path, and response content keywords:
• Content Keywords—restricts the hit search based on keywords found within
page content. This allows searching for visible text displayed to clients, or for
strings hidden within HTML or JavaScript code.
Foglight Experience Viewer breaks each page down into a list of keywords,
single words that are more than four characters long and less than 64
characters long. The simplest search is for one of several words: green blue
44 Foglight Experience Viewer
User Guide

(find green or blue, any order). To find an exact phrase (several words in
order), use quotes around the keywords: “Quest Software” (exact phrase).
Advanced operators include: asterisk (*) for multiple-character wildcard, plus
sign (+) for required words/phrases, minus sign for words/phrases to omit:
+re* -read (find region and repeat but not read).
Note Keyword indexing must be enabled by one or more Hit Filters. If keyword search
is not working as expected, verify that such a Hit Filter exists and that its keyword
indexing is currently enabled.

• Client IP—restricts the hit search based on the IP address of the client
browser. An asterisk (“*”) can be used as a wildcard character: “66.35*”
would search for any IP addresses that begin with “66.35”.
Note IP address cannot be used to reliably identify a single client on public Web sites.
It is not uncommon for a single client visit to use multiple IP addresses, or for
traffic from simultaneous clients to be “multiplexed” onto a single originating IP
address.

• Server IP—restricts the hit search based on the IP address of the web server
that served the hit. An asterisk (“*”) can be used as a wildcard character:
“66.35*” would search for any IP addresses that begin with “66.35”.
• HTML Title—restricts the hit search based on the HTML page title. An
asterisk (“*”) can be used as a wildcard character: “Foo*” would search for
any HTML pages with titles that begin with “Foo”.
An HTML hit is defined as a hit with a “Content-Type” response header
starting with “text/html”, containing a proper HTML tag in the response
content, and having a response code of 200. Non-HTML hits do not have
searchable titles. Click the Find HTML Title icon to choose from a list of
captured titles.
• HTTP Method—restricts the hit search based on the HTTP request method.
The most commonly used methods are “GET” and “POST”, though there are
additional methods defined by the HTTP specification. An asterisk (“*”) can
be used as a wildcard character.
• Collector ID—restricts the hit search based on the Collector used to capture
the hit. This is especially useful if multiple Collectors are being used for
different applications or in different datacenters.
• Collector Group—restricts the hit search based on the Collector Group used to
capture the hit. This is especially useful if multiple Collectors are being used
for different applications or in different datacenters.
Performing Searches 45
Hit Search

• Web Server—restricts the hit search based on the Web server that handled the
hit request. Search values should include the protocol scheme (that is, HTTP
vs. HTTPS), port number, and either a DNS name or IP address.
http://mysite:8080.com
https://12.106.87.32
An asterisk (“*”) can be used as a wildcard character: “https*” would search
for any hits served over SSL. Click the Find Server icon to choose from a
list of captured web servers.
• Path—restricts the hit search to the specified path, often used to narrow the
search to a specific page. The path is defined as the portion of the request URL
after the hostname (and optional port), with any query parameters removed.
Paths always begin with a leading slash. An asterisk (“*”) can be used as a
wildcard character: “/index*” would search for any paths beginning with “/
index”. Click the Find Path icon to choose from a list of captured paths.
• Type—restricts the hit search to hits of the specified type. An HTML hit is
defined as a hit with a “Content-Type” response header starting with “text/
html”, containing a proper HTML tag in the response content, and having a
response code of 200.
• Session ID—restricts the hit search to hits with the specified Session ID.
• Exception—restricts the hit search based on network/protocol/capture
Exceptions noted by the FxV system when the hit was captured. Exceptions
are stored with the hit as a comma separated list (for example,
CorruptedResponse,MissingProperty). To search for a single value, the search
value should be surrounded with a wildcard characters (“*”); for example,
“*CorruptedResponse*”. Click the Find Exceptions icon to choose from a
list of possible exceptions.
• HTTP Header/Field Conditions—any cookie, request header, response header, or
request field captured as part of a hit can be used as a search constraint. Request
field searches are particularly useful for finding hits based on values entered into
form fields.
Note The most frequently captured header/field names are included in the list of captured
metadata. Less frequently captured data may be excluded form this list. This feature
applies only to fields, cookies, request headers, and response headers.

• Custom Field Conditions—Hit Filters can be defined to set custom field values on
any hit. These values can be used to constrain hit searches.
Note Custom fields defined as numeric can be searched using operators such as “>” and
“<=”.
46 Foglight Experience Viewer
User Guide

• Metric Update Conditions—Hit Filters can be configured to update metrics when


a particular condition is met. When searching for hits, the search can be
constrained to include only hits whose processing caused a specified metric to be
increased by a specified amount. Comparison operators allow for full control over
the types of increases to be included (for example, increase > 0 or increase <= 10,
etc.).
Note Hit searches have the same constraint that the session searches have. That is, hits
whose metric update value is “0” are not included in the results; hits whose metric
value is less then three match on “1” or “2” (but not 0). Also, hit searches have the
added constraint that they do not support searching for “0” (=0). The browser
interface displays an error message to the user if they attempt this.

• Status Conditions—every hit can have a status of OK, Warning, or Error, as


defined by any Hit Filter whose match conditions are met by the hit. By default,
all hits have a status of OK. It is up to the Hit Filters to define conditions that
change this status.
Note Any Hit Filter can change the status of the hit. The “worst” status recorded by any Hit
Filter is the hit's final status (for example, if one Hit Filter leaves the status as OK,
another sets it to Error, and another sets it to Warning, the final hit status is Error,
regardless of the order in which the Hit Filters were processed).

In addition to the overall hit status as defined by Hit Filters, it is also possible to
search for hits based on their HTTP Response Code. This status may not
(necessarily) correspond to a value such as OK, Warning, or Error, but it does
define the status of the hit from the perspective of the Web server.
• Hit Filter Conditions—hits can be found if they met the match conditions of a
specific Hit Filter.
• Timing Constraints—restricts the hit search based on the measured performance
of the individual hits. Hits can be found by performing hit searches on:
• End to End Time—total end-to-end time recorded for the hit.
• Client Time—time spent by the client (browser) acknowledging the response
sent from server.
• Processing Time—time spent processing the hit.
The hit search results screen renders the information in table format. Each row in the hit
search results table represents an individual hit (an HTML page, an image, a CSS file,
etc.). Each row in the hit search results table contains two icons that can be clicked to
trigger activities related to the session containing that hit:
• Explore Session containing Hit—click this icon to open the Session Explorer
view. This is a tabbed view that provides multiple ways to get details about the
Performing Searches 47
Hit Search

session containing that hit: Details, Transactions, Timeline, Hit Inspector, and
Content Replay.
• View Session Details—click this icon to open a new window containing all
available details about the session.
Two additional icons are located beside each hit’s URL:
• View Hit Details—click this icon to open a window containing all available
information about the selected hit.
• Hit Content Preview—click this icon to open, in the corner of the search
results table, a small window that shows a visual representation of the hit:
• For image hits, this preview shows the actual image.
• For text files, such as CSS or JavaScript files, the actual text is displayed.
• For HTML hits, the page is rendered as it appeared to the user.
Hit search results can also be viewed in Aggregate. In this mode (selected from the
Search Result View drop-down list, above the results table), the individual hits are not
displayed. Instead, each unique URL is displayed along with a count of the number of
times a hit with that URL appeared in the search results.

Note Since this view does not show individual hits, the details/replay/preview icons are not
displayed.

Two variations of the aggregate hit search results table are available:
• full URL—the entire URL is used when identifying common hits.
• ignore URL params—any query parameters that appear in the URL are ignored
when identifying common hits.
For example, the following two hits would be considered the same when using “ignore
URL params” mode, but would be considered different in “full URL” mode:
http://www.myveryownwebsite.com/somepage?a=foo&b=bar
http://www.myveryownwebsite.com/somepage?a=dog&b=baz
In “ignore URL params” mode, these URLs would be listed as:
http://www.myveryownwebsite.com/somepage
48 Foglight Experience Viewer
User Guide

Search Result Limits


This section presents the concept of limiting the number of results brought back for
visualization, and why these limits are necessary. It also presents how users (with
certain privileges) can modify the pre-defined limits.
When a search is performed, the FxV Server sends the query to all available Archivers
configured in the system. Each Archiver limits the number of results it returns to a
configurable maximum number of results. By default, the limits are set to 200 hits,
sessions, or transactions. These limits can be modified as needed, up to a maximum
value of 1000 per Archiver. If any Archiver reaches its limit, a message is displayed
together with the search results, indicating that some results that match the query
conditions have not been returned.

Important If a search is performed in parallel by several Archivers, the maximum search result limit
is not multiplied by the number of Archivers involved the search, but applies to each
Archiver individually.

These search result limits are configurable at the Preference Group level. Each user is a
member of a single Preference Group. By default, the search result limits are not
configurable by individual users, but it is possible to unlock these limits and allow each
user to configure their own limits via the Preferences menu on the menu bar. For more
information, see “Setting User Preferences” on page 71.

Saved Searches
Users that expect to perform repeatedly the same search have the option of saving these
search conditions as Saved Searches. Saved searches make searching easier because
there is no need to remember all the details for frequently performed searches. Saved
searches can also be shared between users, as shared resources.
To create or edit Saved Searches:
• See “Creating and Editing Saved Searches” on page 66.
To perform searches based on a Saved Search:
• In the Hit/Session/Transaction Search page, restore the saved search conditions
by selecting a saved search from the Load Saved Search list.
Note The Load Saved Search list is empty if no saved searches have been created yet.
Performing Searches 49
Custom Search Screens

Once loaded, these conditions can be modified as needed before executing the
search operation.
The second purpose of saved searches is to serve as the basis for the creation of Custom
Search Screens. The saved searches must specify search condition values that the
custom screen can use or override. For more information, see “Defining Custom Search
Screens” on page 65.

Custom Search Screens


A Custom Search Screen is a simplified search screen, usually defined for frequently
used searches or for less-technical users. This section presents how to use an already
defined Custom Search Screen, via the Custom Search menu on the FxV menu bar.
This menu is available only to users who have access to at least one Custom Search
Screen resource. Therefore, it is not displayed to any users until as least one custom
screen has been created.
To create or edit Custom Search Screens:
• See “Creating and Editing Custom Search Screens” on page 67.
To perform a search based on a Custom Search Screen:
1 In the FxV browser interface, on the menu bar, click Custom Search.
A Custom Search Screen page appears.

2 Select a Custom Search Screen from the Search Screen list.


Each Custom Search Screen created in the system is unique. When you select a
Custom Search Screen to use, you are presented with the standard time controls
and the list of search fields that are defined for that Custom Search Screen.
50 Foglight Experience Viewer
User Guide

3 Fill out the form as needed, then click Search.


The search screen results page appears, listing all hits that matched the custom
search settings. Search results are consistent with the type of search being
performed (hit/session/transaction).
3
Analyzing User Sessions

This chapter describes how FxV users can analyze a session of interest, and outlines the
available analysis views. To access these views, click the Session Explorer icon on
the search results screen.

This chapter contains the following sections:


Session Explorer Toolbar.............................................................................................................52
Details View.................................................................................................................................53
Timeline View ..............................................................................................................................54
Hit Inspector View........................................................................................................................55
Content Replay View ...................................................................................................................57
Transactions View .......................................................................................................................61
Combined Sessions View............................................................................................................63
52 Foglight Experience Viewer
User Guide

Session Explorer Toolbar


The Session Explorer toolbar is located on the upper right corner of the screen. The
following table presents the functionality of the toolbar’s buttons.

Button Name Button Button Description


Icon

Combine this Enables FxV to combine the hits from multiple


Session with sessions for the Session Explorer.
others for When you click this button, the Combined Sessions
Session Explorer view becomes visible and displays the results of the
session combining.
Note The button is shown only when session combining
is enabled and when the active session has a value
for the session combining custom field. To
configure the Session Combining settings, see
“Session Explorer Settings” on page 72.

Return to un- Disables the display of combined sessions in the


combined Session Explorer. When you click this button, the
Session Combined Sessions view is hidden.
Note The button is shown only when viewing combined
sessions.

Return to Search Re-displays the page containing the results of your


last search operation.
Note The button is shown only when opening the
Session Explorer from an FxV search result
screen.

Refresh Refreshes the information displayed in the Session


Explorer views for sessions that are “in progress”.
Note This button is shown only when the session is “in-
progress”.

Edit Session Opens a dialog box which allows you to modify the
Explorer default Session Explorer settings. For more
Preferences information, see “Session Explorer Settings” on
page 72.
Analyzing User Sessions 53
Details View

Details View
The Details view presents all data elements (and their values) that define the session of
interest.

To access this view, click the Session Explorer icon on any search result screen,
then click the Details tab. The same information can also be accessed via the hit or
session search result screen, by clicking the View Session Details icon.
Details View Figure 1

The session details are grouped in several categories and presented in a tabbed format.
The following data is calculated and stored for each session (for a complete list of
elements see “Sessions” on page 26):
• Client, Counts and Timings—contains Details, Client Details, Transaction
Counts, Hit Counts, and Timings
• Custom Fields—contains custom fields set by Hit Filters and/or Transaction
Filters
• Metric Updates—contains metrics updated by Hit Filters and/or Transaction
Filters
• Hit Filter Matches—contains matches determined by Hit Filters
54 Foglight Experience Viewer
User Guide

Timeline View
The Timeline view presents a time-accurate visual representation of how long hits
within the session lasted, and their time-relation to other hits.

To access this view, click the Session Explorer icon on any search result screen,
then click the Timeline tab.
Timeline View Figure 2

The Timeline view displays individual “bars” that represent the hits. Each “bar” has
banded colors which further break down the duration of each hit into client time,
processing time, and latency. Additionally, any transactions that occurred during the
session are displayed in relation to the hits (and other transactions). This view gives
great visibility into the parallel nature of how hits occur in the “real world”.
Each hit bar can be accompanied by icons and labels, which are made visible/hidden by
clicking the Labels and Icons buttons, respectively. The icons are the same as in the hit
search results screen and in the Content Replay view. The hit labels are a shortened
version of the hit’s URL (which is usually the “filename” part); this shortening is due to
limited space available for hits’ display. The number within brackets is part of the label
(for example, [2.5]) and is intended to match up hits between the three views that deal
with hits (that is, Timeline, Hit Inspector, and Content Replay). These numeric labels are
only relevant to the session views and they may change within a particular session if
you refresh a session that is “in progress”.
Analyzing User Sessions 55
Hit Inspector View

To analyze a hit of interest:


1 In the Timeline view, click the hit icon or label.
The dialog box that appears presents a short list of hit details.
2 For additional information, do one of the following:
• To view the complete list of hit details, click View Full Hit Details.
A new window appears, displaying the data collected and stored for this hit
(for a complete list of elements see “Hits” on page 24).
• To view the Web page corresponding to this hit, click Show Hit in Content
Replay.
The Content Replay view opens, displaying the hit’s component Web page, as
the end user originally viewed it.
To display/hide non-HTML hits:
• In the Timeline view, click Non-HTML.
The display’s background is split into three sections: the upper section corresponds to
hours, the middle section to minutes, and the lower section to seconds. To zoom in/out
an area of interest, click the plus/minus icons on the right side of the view. To
move the focus of your interest on a different area of the time line, click anywhere on
the view and drag left or right.

Hit Inspector View


The Hit Inspector view presents timing details and other information for every hit that
occurred during the session.

To access this view, click the Session Explorer icon on any search result screen,
then click the Hit Inspector tab.
56 Foglight Experience Viewer
User Guide

Hit Inspector View Figure 3

The upper pane of the Hit Inspector view lists all hits that make up the session of
interest and a brief summary of their details. The column by which the hits are sorted
can be changed by clicking any column heading.
The lower pane of the Hit Inspector view provides complete details about the hit
selected in the upper panel.
The hit details are grouped in several categories and presented in a tabbed format. The
following data is collected and stored for each hit (for a complete list of elements see
“Hits” on page 24):
• Request—contains Request Details, Request Headers, and Request Fields
• Response—contains Response Details and Response Headers
• Cookies—contains cookies
• Status Analysis—contains status information as determined by Hit Filters
• Timings—contains Timing Details
• Hit Custom Fields—contains custom fields set by Hit Filters
• Metric Updates—contains metrics updated by Hit Filters
• Other—contains Archiver Details
Analyzing User Sessions 57
Content Replay View

Content Replay View


The Content Replay view allows FxV users to visualize a session of interest by
displaying its component Web pages as the end user originally viewed them.

Note The Content Replay view is available only to FxV users who are part of a User Group for
which the Hit Content Replay privilege has been enabled.

To access this view, click the Session Explorer icon on any search result screen,
then click the Content Replay tab. The following figure illustrates the Content Replay
GUI.
Content Replay GUI Figure 4

The screen consists of the following elements:


• Replay menu bar—provides access to the replay operations, replay settings, and
results display modes.
• Hit Selection pane—contains two tabs:
58 Foglight Experience Viewer
User Guide

• Captured Hits—displays the list of hits that were captured by FxV for the
session being replayed. This is essentially the list of hits that were actually
sent over the network to the end user’s browser.
• Page Visit Order {pv#}—displays the list of HTML Pages in the order in
which FxV determines the end user visited the pages.
Note Captured hits may be visited more than once during a session (for example, by
using the Back button) and thus they may appear in the Page Visit Order more
than once.

• Hit Details pane—displays brief details about the currently selected hit, along
with some end user navigation information.
Note You can hide/ display this pane by clicking the collapse / expand icons.

• User Input pane—contains the User Input tab, which displays field values
submitted with HTML forms from the currently selected hit, if applicable.
Note You can hide/ display this pane by clicking the collapse / expand icons.

• Hit Replay pane—displays data captured by FxV for the currently selected hit.
Depending on the display mode selected, this pane may “replay” pages as the end
user has originally seen them, as source code, or as a list of complete hit details.
The Hit Selection pane and the Hit Details pane use the same color scheme in presenting
the list of captured hits (that is, Referer = previous page visited, highlighted hit = current
page, and Navigated To = next page visited).
Analyzing User Sessions 59
Content Replay View

The following table presents the functionality of the Replay menu bar.

Menu Option Description

Navigates to the previous hit in the Page


Previous Page Visited Visit Order, and displays it in the Hit
Replay pane.

Navigates to the next hit in the Page


Next Page Visited Visit Order, and displays it in the Hit
Replay pane.

View Show Only HTML Hits In the Hit Selection pane, toggles
between these options:
• ON—displays only the HTML hits in
the selected session.
• OFF—displays all hits in the selected
session, including non-HTML pages
such as images and style sheets.

Display Page Titles for In the Hit Selection pane, toggles


HTML Hits between these options:
• ON—displays the HTML page title
for hit labels (when available).
• OFF—displays the URL for the
HTML hit labels.

Display Full URL for Hit In the Hit Selection pane, toggles
Labels between these options:
• ON—displays the full URL for hit
labels, including the parameters (e.g.,
http://foo.com/bar?p1=v1).
• OFF—displays only a the URL path
for the hit labels (e.g., /bar for the
URL http://foo.com/bar).

Expand All Expands all nested elements in the Hit


Selection pane > Captured Hits tab.
60 Foglight Experience Viewer
User Guide

Menu Option Description

Expand Errors/Warnings Expands all nested elements marked as


errors or warnings in the Hit Selection
pane > Captured Hits tab.

Collapse All Collapses the tree of nested elements in


the Hit Selection pane > Captured Hits
tab.

Enable Tooltips Enables the display of tooltips in the


Content Replay view.

Tools Export to ZIP Exports the session selected for replay


(file is saved in ZIP format).

Display Hit Replay In the Hit Replay pane, displays a


Mode “Replay” of currently selected hit’s
response content. For HTML Pages, this
is an approximation of how the page
was originally displayed to the end user.
Note When replaying HTML Pages, any
page elements (for example, images,
css, etc.) that were not captured with
the session may be requested from
the “live” Web server, depending on
the user’s Replay preferences.

Hit Replay (Highlight User Same as “Hit Replay” mode, except that
Input) any form fields where the user entered
data are populated with the data that
was entered, and the links clicked are
highlighted.

Hit Source In the Hit Replay pane, for HTML


pages, displays the HTML source code
captured for the selected hits.
Note If a hit that is not an HTML page is
selected, the equivalent of the “Hit
Replay” mode is displayed.
Analyzing User Sessions 61
Transactions View

Menu Option Description

Hit Details In the Hit Replay pane, displays details


about the selected hit. These details are
stored by FxV during the data capture
process.

Important Visual replay is most reliable when applied to Web 1.0 applications that make light use of
JavaScript. Heavy use of JavaScript and frames can negatively impact FxV’s ability to
replay the user experience with complete accuracy.

Transactions View
The Transactions view presents information about any transactions that took place
during the session being explored.

To access this view, click the Session Explorer icon on any search result screen,
then click the Transactions tab. This tab is selected automatically when entering from a
transaction search result page.
62 Foglight Experience Viewer
User Guide

Transactions View Figure 5

The upper pane of the Transactions view lists all transactions that occurred during the
selected session, and a brief summary of their details. The column by which the
transactions are sorted can be changed by clicking any column heading.
The lower pane of the Transactions view, named Transaction Details, provides
complete details about the transaction selected in the upper panel.
The information presented in the Transaction Details pane can also be accessed via the
transaction search’s result screen, by clicking the View Transaction Details icon.
The transaction details are grouped in several categories and presented in a tabbed
format. The following data is calculated and stored for each transaction (for a complete
list of elements see “Transactions” on page 27):
• Client, Counts and Timings—contains Details, Hit Counts, and Timings
• Events—contains information about the Transaction Events that occurred as
defined by the Transaction Filter
• Custom Fields—contains custom fields set by Transaction Filters
• Metric Updates—contains metrics updated by Transaction Filters
Analyzing User Sessions 63
Combined Sessions View

Combined Sessions View


The “combined sessions” feature allows FxV to combine hits from multiple sessions for
the Session Explorer. This functionality should only be used when proper sessionizing
(of hits into unified sessions) is not possible. By default, this feature is disabled.
The Combined Sessions view presents information about all the combined sessions,
when displaying a “combined” session. This view is visible on the Session Explorer
only when the “combined sessions” feature is enabled. To configure the Session
Combining Settings, see “Session Explorer Settings” on page 72 and “Preference
Groups” on page 184.
To access the Combined Sessions view:

1 Click the Session Explorer icon on any search result screen.


2 Click the Combine this Session with others for Session Explorer icon on the
Session Explorer toolbar.
3 Click the Combined Sessions tab.
Combined Sessions View Figure 6

The information presented in this view is grouped into several categories:


• Primary Session details—includes Session ID, Started, Last Hit, Combined Start,
and Combined Last Hit.
Note In this case, the other Session Explorer views show values which are relevant only
to the Primary Combined Session.
64 Foglight Experience Viewer
User Guide

• Combined Sessions criteria—includes Custom Filed Name, Custom Field Value,


Time Range, and a list of links to all other sessions combined with the Primary
Session.
Note When you click one of the links to other sessions, the Session Explorer views show
values relevant to the selected session, and the Combined Sessions view
disappears.
4
Defining Custom Search Screens

A Custom Search Screen is a simplified search screen, usually defined for frequently
used searches or for less-technical users. They allow searches to be performed using
simple screens that contain just a few familiar fields that apply specifically to the
business (for example, Name, Address, Account Number, etc.). The first step towards
defining a Custom Search Screen is to create a Saved Search.

This chapter contains the following sections:


Creating and Editing Saved Searches.........................................................................................66
Creating and Editing Custom Search Screens ............................................................................67
66 Foglight Experience Viewer
User Guide

Creating and Editing Saved Searches


Users that expect to repeatedly perform the same search have the option of saving these
search conditions as Saved Searches.
The purpose of the Saved Search is to serve as a foundation for the Custom Search
Screen. Every search performed using a Custom Search Screen uses the values defined
on its underlying Saved Search. For example, if you want to define a custom search that
looks for sessions in which a particular metric was incremented by at least one, your
Saved Search would include a condition such as “myMetric > 0”. If you then want to
allow users to search for these sessions based on the user’s login (stored in a custom
field), you would not specify the login field in the Saved Search (since you want that
value to be entered by the person performing the search). Instead, you would add that
field in the next step when you create the Custom Search Screen.
The Saved Search must specify search condition values that the custom screen can use
or override.
To create a Saved Search:

Note This procedure illustrates the example of a session saved search. For hit and transaction
saved searches, similar procedures apply.

1 In the FxV browser interface, on the menu bar, click Search > Sessions.
The Search Sessions page appears.
2 Set the search conditions as needed for your session search.
3 In the Save Search As text box, specify a unique name for the search, then click
Save.
The Select Resource Groups page appears.
4 Select the check box(es) beside the resource groups to which this resource must
belong, then click OK.
The Search Sessions page re-appears. The search conditions are now saved and
the name of the Saved Search appears in the Load Saved Search list.
Note The Select Resource Group screen appears if you have access to more that one
Resource Group configured to store saved searches. This screen may not appear
depending on the configuration of your system. For more information about
Resource Groups, see“Resource Groups” on page 183.
Defining Custom Search Screens 67
Creating and Editing Custom Search Screens

To edit Saved Searches:


1 In the Hit/Session/Transaction Search page, click Saved Searches.
The Saved Searches page opens, listing the already defined saved searches.
2 Perform one of the following operations, as needed:
• To view the set of conditions defined for a Saved Search, using the main
search screen, click the View icon.
• To change the Resource Groups a Saved Search is associated with, click the
Share icon.
• To delete a Saved Search from the system, click the Delete icon.
Note Depending on the state of the Saved Search and your user privileges, some of these
icons may not appear for certain Saved Searches.

3 Click OK.

Creating and Editing Custom Search Screens


A Custom Search Screen is a simplified search screen, usually defined for frequently
used searches or for less-technical users.

Note The first step towards creating a Custom Search Screen is to create a Saved Search. For
more information, see “Creating and Editing Saved Searches” on page 66.

To create Custom Search Screens:


1 In the FxV browser interface, on the menu bar, click Configure > Custom
Search Screens.
The Custom Search Screens page appears.
2 Click Create Custom Search Screen.
The Custom Search Screen page appears.
3 Select the type of search to be performed (Hit, Session, or Transaction) from the
Search Type list.
Note These options are available if at least one Saved Search of the same type has been
created and is available to this user.
68 Foglight Experience Viewer
User Guide

4 Fill in the Custom Search Screen Name. Upon creation, this name will be
displayed in the list of available screens, in the Custom Search menu, on the
menu bar.
Note The Custom Search menu is only displayed to users who have access to at least
one custom search screen. Therefore, it is not displayed to any users until at least
one custom screen has been created.

5 Add search fields and order them using the Move Up and Move Down
arrows. The names of these fields will appear (in the same order) on the Custom
Search Screen. These names must be meaningful to the screen’s users (for
example, Login, First Name, Account Number, etc.).
6 Select one or multiple search phases from the Search Phases list and order them
using the Move Up and Move Down arrows.
When a user submits a search via a Custom Search Screen, one or more Saved
Searches are executed. While it is often sufficient to execute a single search, it is
possible to specify multiple searches (phases) to be executed in sequence, until at
least one result is found. This can be useful, for example, if there are multiple
pages where the user can log in. Each page can be searched by its own Saved
Search in order to cover all possible entry points.
For each Search Phase, mappings are defined from the fields entered by the user
of the custom search screen to fields and/or cookies sent from the client.
7 Click OK.
The Custom Search Screen is now saved and appears in the list of screens
available for this user.
Note If this is the first Custom Search Screen created for this user, then the Custom
Search menu appears on the menu bar.

To edit Custom Search Screens:


1 In the FxV browser interface, on the menu bar, click Configure > Custom
Search Screens.
The Custom Search Screens page appears, showing the Custom Search Screens
available for this user.
Note If no custom screen has been defined yet, or the user has no access to a resource of
this type, the list is empty.

2 Perform one of the following operations, as needed:


• To view the settings defined for a Custom Search Screen, click the View
icon.
Defining Custom Search Screens 69
Creating and Editing Custom Search Screens

• To modify a Custom Search Screen, click the Edit icon, and update the
fields as needed.
• To delete a Saved Search from the system, click the Delete icon.
3 Click OK.
70 Foglight Experience Viewer
User Guide
5
Setting User Preferences

This chapter outlines the preferences that users can view and edit (depending on their
privileges): account information, replay settings, and search settings.

This chapter contains the following sections:


Account Information.....................................................................................................................72
Session Explorer Settings ...........................................................................................................72
Search Settings ...........................................................................................................................75
72 Foglight Experience Viewer
User Guide

Account Information
The fields in the Account Information section allow you to edit basic information about
your account.
To edit the Account Information settings:
1 In the FxV browser interface, on the menu bar, click Preferences.
The Preferences page appears.
2 In the Account Information area, identify the fields that need editing. The
following fields are provided: Login (not editable), Last Name, First Name,
Email, and Password.
Note The Password field is not directly editable on this page. To change your password,
click Change Password and follow the instructions.

To modify the settings for user accounts other than your own, see “Managing User
Account Settings” on page 175.

Session Explorer Settings


The Session Explorer settings are grouped in the following categories:
• Default Session Explorer Tab—allows you to specify the initial tab shown when a
session is displayed in the Session Explorer.
Note If the option Based on search context is selected, the initial tab is based on the
Search View from which the Session Explorer opens.

• Content Replay Settings—allow you to change the behavior of the Content


Replay view:
• Define the default replay parameters and display mode.
• Customize the way user input data is displayed in the Replay pane.
• Improve the way Web pages are rendered during a replay.
• Session Combining Settings—allow you to enable/disable and configure the
session combining feature.
To edit the Session Explorer settings, use one of the following methods:
• Via the FxV browser interface—on the menu bar, click Preferences.
The Preferences page appears; it provides a section dedicated to Session Explorer
Settings.
Setting User Preferences 73
Session Explorer Settings

• Via the Session Explorer screen—on the toolbar, click the Edit Session Explorer
Preferences icon.
The Session Explorer Settings dialog box appears.

Note An individual user can edit the Replay Settings only if this operation is enabled via the
Preference Group to which that user belongs to. Settings that are not locked in a Preference
Group serve as defaults, and may be changed by individual users. Locking a setting
immediately overrides any user-specified values. For more information about this topic, see
“Preference Groups” on page 184.

The following Content Replay Settings may be configured:


• Rewrite JavaScript—specifies if JavaScript should be enabled while replaying
captured HTML hits.
Caution Because JavaScript may sometimes cause problems during replay, this setting is
set to “NO” by default.

• Less Strict Replay With User Input (Forms)—when set to “Yes”, improves the
replay of user input data, in cases where JavaScript is used to manipulate form
field values.
• Less Strict Replay With User Input (Links)—when set to “Yes”, may improve
replay of user clicked links, in the cases where hit referrers are not present.
• Show Only HTML Hits—specifies if non-HTML hits (such as images and style
sheets) should be listed in the Hit Selection pane, when first opening the Replay
browser interface.
Note This setting may be changed during replay by clicking the Show Only HTML Hits
icon.

• Hit List Display Mode—specifies how the URLs are displayed in the Hit
Selection pane, when first opening the Replay browser interface. The following
options are available for selection:
• Show Title (or Path)—displays the HTML page title, or the path (when the
title is unavailable).
• Show Title (or Full URL)—displays the HTML page title, or the URL (when
the title is unavailable).
• Show Path Only—displays only the path to the hit (for example, “/bar” for
the URL “http://foo.com/bar”).
74 Foglight Experience Viewer
User Guide

• Show Full URL—displays the full URL for the hit, including any parameters
(for example, “http://foo.com/bar?p1=v1”).
Note This setting may be changed during replay by clicking the Display Page Titles for
HTML Hit Labels and the Display Full URL for Hit Labels icons.

• Display Mode—defines the default display mode to be used on the Replay pane,
when first opening the Replay browser interface. The following options are
available for selection:
• Hit Replay—displays hits as the user has seen them, including any errors.
Note If the content is unavailable (for example, if the browser used a cached page),
the replay system may attempt to obtain the content from other sessions or from
the live Web server, depending on the values of the “Use Elements From
Unrelated Sessions” and “When Element Content Not Available” settings.

• Hit Replay (with user form input)—displays hits similarly to “Hit Replay”
mode, except that any form fields where the user has entered data are
populated with the data that was entered, and links clicked are highlighted.
• Hit Details—displays network and HTTP details including response time, hit
analysis (including Hit Filter matches), request and response fields and
headers, cookies, and Archiver details.
• Hit Source—displays the source code that was captured for HTML hits, useful
for debugging dynamic Web applications.
• User Input Highlighting - Primary Color—specifies the primary color to be used
when highlighting user input. Any valid CSS color value may be used (for
example, #FC0, RGB(255,204,0), lime).
Note The color input field uses the specified color as its background color. If the field has a
blank background, this indicates an invalid color code (or white).

• User Input Highlighting - Secondary Color—specifies the secondary color to be


used when highlighting user input. This color is used for highlighting specific
items, such as the option a user selected from a pull-down menu. Any valid CSS
color value may be used (for example, #FF0, RGB(255,255,0), yellow).
Note The color input field uses your selected color as its background color. If the field has
a blank background, this indicates an invalid color code (or white).

• Use Elements From Unrelated Sessions—specifies if page elements (images,


stylesheets, etc.) from other unrelated sessions should be used for Replay.
• When Element Content Not Available—specifies what FxV should do during
replay if the content for a page element (images, stylesheets, etc.) is not available
in the session being replayed or in other unrelated sessions (when the Use
Setting User Preferences 75
Search Settings

Elements From Related Sessions option is enabled). The following options are
available:
• Redirect to ‘live’ web server—page elements may be used from the live Web
server.
Note This option is a bad choice if the live server is not reachable from the Replay
browser interface or the live Web server/application requires a login to reach
page elements.

• Send 404 Response—the replay system sends a “404” error code to the
browser replaying the session. For missing images, a “broken image” icon is
displayed; if style sheets or javascript are missing, then the HTML page may
not visually render correctly.
The following Session Combining Settings may be configured:
• Session Combining Custom Field—specifies the name of the custom field for
combining other sessions for the Session Explorer. For example, if “Client IP” is
the selected Session Combining Custom Field and the current session being
explored has the custom field name/value: “Client IP”/”10.0.0.10”, hits from
other sessions with the same custom field name/value pair should be combined
with the session currently being explored. By default, this field is disabled.
Caution This functionality should be avoided, unless proper sessionizing hits into unified
sessions is not possible.
Note Combining sessions only combines the hits from multiple sessions for Replay.
Session statistics/details and Transaction Filter matching are not affected by this
setting.

• Session Combining Time Range—specifies the number of minutes (measured


before and after the time range of the session currently being explored) to
combine for the Session Explorer other sessions that match the custom field
selected in Session Combining Custom Field.

Search Settings
The Search settings allow you to control the search conditions and the display of results,
as follows:
• Limit the amount of data to be included into a search and the maximum time
range over which to perform the search.
• Specify the default type of view for the search result screen.
• Define the type of information to be included by default in the result screen.
76 Foglight Experience Viewer
User Guide

To edit the complete list of Search settings:


• In the FxV browser interface, on the menu bar, click Preferences.
The Preferences page appears; it provides a section dedicated to Search Settings.

Note An individual user can edit the Search Settings only if this operation is enabled via the
Preference Group to which that user belongs to. Settings that are not locked in a Preference
Group serve as defaults, and may be changed by individual users. Locking a setting
immediately overrides any user-specified values. For more information about this topic, see
“Preference Groups” on page 184.

The following Search settings may be configured:


• Hit Search Result Limit—defines the maximum number of hits that can be
returned from a single Archiver in response to a single hit search. The default
value is 200; the maximum value allowed is 1000. Larger values may increase the
time required to complete a search.
Note This limit is applied per Archiver searched. For an installation with three Archivers
and a result limit of 200 hits per search, up to 600 hits may be returned in one
search.

• Session Search Result Limit—defines the maximum number of sessions that can
be returned from a single Archiver in response to a single session search. The
default value is 200; the maximum value allowed is 1000. Larger values may
increase the time required to complete a search.
Note This limit is applied per Archiver searched. For an installation with three Archivers
and a result limit of 200 sessions per search, up to 600 sessions may be returned in
one search.

• Transaction Search Result Limit—defines the maximum number of transactions


that can be returned from a single Archiver in response to a single transaction
search. The default value is 200; the maximum value allowed is 1000. Larger
values may increase the time required to complete a search.
Note This limit is applied per Archiver searched. For an installation with three Archivers
and a result limit of 200 transactions per search, up to 600 transactions may be
returned in one search.

• Search Time Range Limit—defines the largest possible (and default) time range
limit used when searching. All Search screens are constrained to use a specific
time range (for example, the last 48 hours).
Note The default value is 48 hours. The maximum value allowed is 72 hours. Larger
values may increase the time required to complete a search.
Setting User Preferences 77
Search Settings

• Search View—specifies the search screen to be used when logging into the FxV
browser interface, and when clicking Search on the menu bar.
Note The default value is “Find Transactions”. If “Find Transactions” is selected and there
are no Transaction Filters defined, then “Find Sessions” is used by default.

• Hit Search Result View—specifies the view first used when displaying hit search
results (default value is “Hits”). The options are the following:
• Hits—shows the individual hits found by the hit search. From this view, you
can preview the content of a hit and other hit details.
• Aggregate (full URL)—groups hits together based on URL (for example, if a
search returned several hits against the exact same URL, this would be
displayed as a single result row).
• Aggregate (ignore URL params)—aggregates hits by URL, but ignores URL
query parameters (the portion of the URL after the “?”).
Note The view type can also be changed via the search results screen, by selecting an
option from the Search Result View drop-down menu.

• Show Only HTML Search Results—specifies if non-HTML hits should be hidden


when first displaying hit search results (default value is “No”).
Note This setting can be toggled by clicking the HTML icon in the Type column on the
search results screen.

• Hit Search Results Hidden Columns—specifies the columns to be hidden on the


hit search results screen. All custom field columns and the pre-defined hit detail
columns are included in the list of columns that may be hidden. To edit the list of
hidden columns, click the Edit Hidden Columns icon, use the arrow icons to
select the columns that should be Displayed or Hidden in the search result screen,
and click OK.
Note The hidden columns can also be edited via the hit search results screen (when the
Hits view is selected), by clicking the Edit Hidden Columns icon in the top left
corner.

• Session Search Results Hidden Columns—specifies the columns to be hidden on


the session search results screen. All custom field columns and the pre-defined
session detail columns are included in the list of columns that may be hidden. To
edit the list of hidden columns, click the correspondent Edit Hidden Columns
icon, and use the arrow icons to select the columns that should be Displayed or
Hidden in the search result screen.
Note The hidden columns can also be edited via the hit search results screen (when the
Hits view is selected), by clicking the Edit Hidden Columns icon in the top left
corner.
78 Foglight Experience Viewer
User Guide

• Transaction Search Results Hidden Columns—specifies the columns to be hidden


on the transaction search results screen. All custom field columns and the pre-
defined transaction detail columns are included in the list of columns that may be
hidden. To edit the list of hidden columns, click the correspondent Edit Hidden
Columns icon, and use the arrow icons to select the columns that should be
Displayed or Hidden in the search result screen.
Note The hidden columns can also be edited via the hit search results screen (when the
Hits view is selected), by clicking the Edit Hidden Columns icon in the top left
corner.
6
Configuring the Hit Analysis
Process

This chapter presents the resources available to users for configuring the FxV Hit
Analysis process, that is, resources defining the conditions and actions used to analyze
hits or sessions as they are captured.

This chapter contains the following sections:


Hit Filters .....................................................................................................................................80
Transaction Filters .......................................................................................................................92
Special Events...........................................................................................................................106
Custom Fields............................................................................................................................108
Metrics ....................................................................................................................................... 115
Captured Metadata.................................................................................................................... 117
Scripts........................................................................................................................................ 118
Sensitive Data Protection ..........................................................................................................121
Hit Analysis Configuration Options ............................................................................................125
Hit Analysis Configuration Change Log.....................................................................................128
Hit Analysis Import/ Export Configuration..................................................................................130

For Hit Analysis examples see “Hit Analysis Examples” on page 133.
For details about how to configure a secondary database for long-term storage of
session and transaction data captured by FxV, see “Configuring the Analysis
Repository” on page 149.
80 Foglight Experience Viewer
User Guide

Hit Filters
A Hit Filter is a collection of match conditions and actions to be performed when a hit
matches those conditions. Every Hit Filter is evaluated against each hit as it enters an
FxV Archiver.
Hit Filters can be used to detect and alert on any per-hit conditions, to mark interesting
hits for later searching, and to manage hit storage. They can also be used to define
events within Transaction Filters.
Hit Filters are helpful because they provide a human context to the captured hits. For
example, it is easier to search and alert on Login Attempts using a Hit Filter than to
search and alert by the server, path, and fields used by the Web application.
Hit Filters are very powerful in their ability to extract custom fields and update metrics.
Using regular expressions, Hit Filters can pull data from fields, headers, cookies, or
from page content.
For detailed information about Hit Filters, see the following sections:
• “Hit Filter Match Conditions” on page 80
• “Hit Filter Actions” on page 81
• “Hit Filter Configuration” on page 90
• “Hit Filter Additional Information” on page 92

Hit Filter Match Conditions


In order for a Hit Filter to have an impact on a hit, the filter’s Match Conditions must
apply to the hit. Match conditions may include elements such as the request URL,
request field values, HTTP response code, etc. When necessary, the match conditions
may even evaluate the output of a custom script (written in the Groovy programming
language) or analyze the full content of the HTTP response.
A full range of comparison operators is available when defining match conditions.
These include “equals”, “contains”, “less than”, “matches” (regular expression), etc.

Note The most frequently captured header/field names are included in the list of captured
metadata. Less frequently captured data may be excluded from this list. This feature applies
only to fields, cookies, request headers, and response headers.
Configuring the Hit Analysis Process 81
Hit Filters

Defining Match Conditions


When defining a Hit Filter match condition, follow these guidelines:
• Make match conditions as specific as possible.
• Avoid the use of response content matches. Performing regular expression
matching on every hit can negatively impact the performance of the capture
system.
• Avoid the use of scripting. Executing a script on every hit can negatively impact
the performance of the capture system.
• Avoid creating hit filters that match every hit, unless you truly want to perform an
action on every hit.

Grouping Match Conditions


Multiple match conditions may be combined using arbitrary boolean logic operators
(for example, AND, OR, and NOT), as needed.
Example:
Request path contains “shopping” AND the value of the “cartValue” cookie is >0
AND the HTTP response code is >=500.
Hit Filter Match Conditions can be combined in complex grouping of match conditions.
When at least three match conditions are defined, you can specify an arbitrary
expression that defines how the conditions are to be evaluated.
Example:
With conditions 1, 2, and 3, you could write:
AND(1 OR(2 3))
This would cause a match to occur when condition 1 occurs AND either condition 2 OR
condition 3 occurs.

Hit Filter Actions


When a Hit Filter’s match conditions match a given hit, any number of actions may be
performed. For additional details, see the following sections:
• “Execute Scripts” on page 82
• “Hit Filters: Set Custom Fields” on page 82
82 Foglight Experience Viewer
User Guide

• “Hit Filters: Increment Metrics” on page 86


• “Configure Storage Settings” on page 87

Execute Scripts
FxV Hit Analysis supports the concept of script execution using the Groovy
programming language. Scripts can be used to perform complex operations which are
not supported directly by the Hit Analysis model, including the execution of complex
algorithms or remotely accessing external systems (for example, to look up user account
information based on data in the hit, such as a user ID).
Scripts can set output variables that can be referenced in other areas of the Hit Filter (for
example, custom fields and metrics). If the only purpose of a script is to set a particular
output variable, it doesn’t need to be explicitly executed as an action of the Hit Filter.
Instead, the script output variable can be referenced when updating the custom field or
metric, and the script is executed as needed.
For detailed information about scripting, see “Scripts” on page 118.

Hit Filters: Set Custom Fields


Custom Fields allow specialized information to be associated with individual hits,
sessions, and transactions. A Hit Filter whose match conditions match a given hit can
add (or update) custom field values for that hit or for the session that contains it
(transaction custom fields are only accessible in Transaction Filters).
You can add a custom field to a Hit Filter by creating a new Custom Field or by
selecting (and customizing, if necessary) an existing one. In the first case, the following
Custom Field attributes can be configured:
• “Name” on page 83
• “Description” on page 83
• “Storage (Scope)” on page 83
• “Storage (Type)” on page 84
• “Value Source” on page 85
• “Value Population Policy” on page 85
• “Value Assignment Mode” on page 85
• “Update If Blank” on page 86
Configuring the Hit Analysis Process 83
Hit Filters

In the latter case, you must specify only the value source and the value population
policy. Optionally, you may also override any of the following three attributes defined
for the Custom Field: Value Assignment Mode, value Separator (for append modes), and
Update If Blank. The other attributes get the default settings defined for that Custom
Field.

Name
This field defines the name of the Custom Field.

Note This name may contain only letters, numbers, and spaces. Other characters are not
allowed. Names must be less than 80 characters long, and cannot contain leading or trailing
spaces.

Description
This field is optional. If populated, it is used to build tooltips and reports, when the
custom field name appears as a column heading in a search results screen.

Storage (Scope)
A Hit Filter can set hit-scoped or session-scoped custom fields. Within these two
categories, there are additional options to be considered. The main thing to decide is
where the custom field needs to be seen.
The following table presents the values you can select for hit-scoped custom fields.

Value (Update Hit) Description

Show in Search Results For fields that should appear in search results.

Not in Search Results For fields that should not be in search results, but
can be seen when examining hit details for a
specific hit.

First Column in Search Results For identifying fields (for example, username)
that should appear first in search results (far left
side of the results screen).
84 Foglight Experience Viewer
User Guide

The following table presents the values you can select for session-scoped custom fields.

Value (Update Session) Description1

Available only during Hit In some cases, a piece of data extracted from a hit
Analysis is needed by a Transaction Filter. This setting
allows such data to be stored for transaction
processing purposes without exposing the value
at the session level.

Show in Search Results For fields that should appear in search results.

Not in Search Results For fields that should not be in search results, but
can be seen when examining session details for a
specific session.

First Column in Search Results For identifying fields (for example, username)
that should appear first in search results (far left
side of the results screen).

Show in Search Results (+ Audit For fields that should appear in search results, and
Log) be used for audit logging.

Not in Search Results (+ Audit For fields that should not be in search results, but
Log) can be seen when examining hit details for a
specific hit, and be used for audit logging.

First Column in Search Results For identifying fields (for example, username)
(+ Audit Log) that should appear first in search results (far left
side of the results screen). Also used for audit
logging.
1
The options that end with “(+ Audit Log)” can be used to flag custom fields for audit logging
when sessions are viewed by FxV users. This allows you to track the data seen by individual
FxV users.

Storage (Type)
A Custom Field must be declared as either a string (Store Values as Text) or numeric
(Stores Values as Numbers) field. Explicitly defining a custom field as numeric enables
searching based on numerical comparison operators (for example, is the value of custom
field “shopping cart value” greater than 100?). Numeric sorting is also enabled.
Configuring the Hit Analysis Process 85
Hit Filters

Value Source
A custom field’s value source defines where the value for the field comes from. The
following basic types of value sources are available:
• Hit Details—any data already associated with the hit (request field values, cookie
values, URL, etc.).
• Script Output—any data exposed as an “output variable” by a Groovy script.
Note The script is executed as needed in order to populate the custom field.

• Fixed Values—any hard-coded value.


A regular expression may be applied to the value source in order to extract only a
portion of the value. For example, if request field “price” contains values in the form
“123.00USD”, a regular expression could be used to extract the value “123.00” and
store it in a “numeric price” custom field.
To add a regular expression, click the two slashes (//) at the far right of the Value Source
entry section.

Value Population Policy


Custom field values are typically updated any time a Hit Filter’s match conditions apply
to a hit. However, it is possible to configure updates to occur conditionally based on the
hit status (either the final hit status, or the hit status determined by the current Hit
Filter).

Value Assignment Mode


This attribute controls how the Custom Field is updated, which is most useful for fields
added to sessions. The field value can be “reset” for each match, or “set on the first
match” to lock in just the initial value. String fields allow “appending” to an existing
field using a custom separator; numeric fields allow “incrementing”. When appending
strings, a maximum of 2 KB is stored in a Custom Field value.
When a custom field already has a value, several different options are available.

Value Description1

Reset value on each match Default mode in which the custom field value is
always set to the value defined by the current
value source [2].
86 Foglight Experience Viewer
User Guide

Value Description1

Set value on first match if not Once the custom field value is set, it is not
already set changed [1].

Append to value on each match Concatenates values as they are encountered (an
optional separator can be inserted between the
values) [1232].

Increment value on each match Increments a numerical custom field value by the
amount defined by the current the value source
[8].

Append to value on each match Similar to Append to value on each match, but
(if unique) only unique values are added [123].

Append to value on each match Similar to Append to value on each match, but
(if unique, ignoring case) values like “foo” and “Foo” are considered
identical [123].
1
The values in brackets represent the final custom field value in the case where three
assignments to the same field occur in the following sequence: 1, 2, 3, 2.

Update If Blank
Blank values (as defined by any value source) are ignored by default (that is, a previous
custom field value is not replaced by a blank value). This behavior can be changed by
setting the Update If Blank flag to “Yes”.

Hit Filters: Increment Metrics


Metrics are essentially numeric, global custom fields with a value assignment mode of
“increment value on match”. You can add a metric to a Hit Filter by creating a new
metric or by selecting (and customizing, if necessary) an existing one. Like custom
fields, metric values can be incremented by:
• Fixed Values—any hard-coded value (typical “counter” metrics are set to
increment by a hard-coded value of one).
• Script Outputs—any data exposed as an “output variable” by a Groovy script.
Note The script is executed as needed in order to determine the metric increment amount.
Configuring the Hit Analysis Process 87
Hit Filters

• Hit Details—any numeric data already associated with the hit (for example,
request field values, cookie values, extracted response content, etc.).
Like custom fields, metric values are typically updated any time a Hit Filter’s match
conditions apply to a hit. However, it is possible to configure updates to occur
conditionally, based on the hit status (either the final hit status, or the hit status
determined by the current Hit Filter).
When entering the name for a metric, it is sometimes useful to parameterize the name
based on some piece of data from the hit.
Example
You may want to define a series of metrics such as:
Errors from Spain
Errors from France
Errors from Italy
These can be defined via a single metric that contains a substitution token. The name of
the metric would be defined as Errors from $0.
When $0 is included in the metric name, the Hit Filter UI displays a field called
$0 Source used to identify the value source for the $0 token (that is, the data that should
be used to replace $0 in the metric name).
In this example, you would select Country from the list of hit details, and the hit’s
country would be used in the metric name.

Important All value sources available for populating custom field and metric values are available
here as well. In order to avoid the creation of an unusable number of metrics, ensure that
there is a reasonable number of unique values for the substitution token.

Configure Storage Settings


This section allows for the configuration of storage-related policies for the hits that
match this Hit Filter's match conditions.

Define Hit Storage Restrictions


The default storage policy is to store all hits, without filtering sensitive data and without
indexing the response content. A Hit Filter may specify that hits that match the filter’s
match conditions and (optionally) have a specific status value should be given special
88 Foglight Experience Viewer
User Guide

consideration in terms of how they are stored. Various combinations of the following
options may be selected:
• Do not store hit—hits matching the conditions are dropped completely (that is, no
data is stored), and they are not available for search/replay. This setting saves on
storage space, but results in complete loss of data for hits matching the Hit
Filter’s match conditions. This setting requires that the Do not store response
content setting is selected.
• Do not store response content—hit details are stored without response content.
• Index response content—response content is stored and indexed, to make the hit’s
content keyword-searchable. This setting requires that the Do not store hit and Do
not store response content settings are not selected.
• Do not store hit details marked “Always Sensitive”—hit is stored after applying
any Sensitive Hit Detail rules defined as “Always Sensitive” (to avoid storing
sensitive fields, headers, or cookie values). This setting requires that the Do not
store hit setting is not selected.
• Do not store response content marked “Always Sensitive”—response content is
stored after applying any Sensitive Response Content Expressions defined as
“Always Sensitive” (to hide sensitive data contained in the response content
before storing the hit). This setting requires that the Do not store hit and Do not
store response content settings are not selected.

Note If a hit matches multiple Hit Filters, any overrides to the default storage policy are honored.

Storage restrictions can be customized based on the final status of the hit (for all
matching filters), or just the status defined by this Hit Filter alone. You can choose to
keep all matching hits, or just the errors, or tailor your storage to make best use of the
available disk space.

Define Session Storage Restrictions


By default, all sessions are transferred to the Analysis Repository (if configured). This
section allows sessions to be excluded from the Analysis Repository based on a Hit
Filter match, by selecting the Do not store session in Analysis Repository check box.

Mark Hit
You can also determine (via Mark Hits menu) whether a Hit Filter match should be
stored in the capture database for later searching. The following options are available:
• Always—to be able to search on the Hit Filter.
Configuring the Hit Analysis Process 89
Hit Filters

• Never—to only support Transaction Filters or to manage hit storage.


• Conditional—to mark hits according to the status of the matched hits. This option
is useful when a large number of hit filter matches is expected and the disk space
is limited.
“Mark Hit” is the only action performed by default when a Hit Filter matches a hit.
Marking a hit means that the hit is annotated as having matched the Hit Filter. This
allows users to easily search for hits that matched a given Hit Filter.
In some cases, it may be desirable to only mark hits as matching a Hit Filter when the
hit has a particular status (for example, Error). In this case, hit marking can be defined
as “conditional” based on the status of the hit.
In the case where there is no need to search for hits that match a given Hit Filter and the
Hit Filter is not used by a Transaction Filter, hit marking may be set to Never. In this
case, the other actions defined by the Hit Filter are performed, but there is no explicit
way to see that the Hit Filter matched the hit.
Hit Filters that match every hit should usually be set to “do not mark the hits”, since no
additional information is added by doing so (marking every hit just wastes storage
space).
By default, Hit Filters configured to mark hits that meet their match conditions can be
used as search criteria when searching for hits. For example, you can search for hits that
occurred during a specified time range that matched Hit Filter X. If searching based on
Hit Filter matches is not desired, it can be disabled (that is, the Hit Filter does not appear
in the list of Hit Filter options on the search screen) by setting the “searchable” flag to
NO.

Note Hit Filters whose match conditions are set to match every hit should always be configured
as “non-searchable”, since every hit would be returned by a search looking for matches to
that Hit Filter.

Update Hit Status


Every hit carries a status of OK, Warning, or Error. Unless explicitly changed, a hit
status is always OK (the Hit Analysis process has no intrinsic definition of what
constitutes a Warning or Error condition). When a hit matches a Hit Filter’s match
conditions, any error or warning conditions are evaluated as well. These conditions are
defined just like the match conditions (that is, you can evaluate any aspect of the hit in
order to determine its status).
90 Foglight Experience Viewer
User Guide

In some situations, it is desirable to say that any time a hit matches a Hit Filter’s match
conditions, its status should be set to Error. In fact, the FxV user interface supports this
concept as the default when setting a non-OK status. If a status value must be set only in
certain situations, an additional step is necessary to define the error and warning
conditions.
Multiple Hit Filters can potentially evaluate differently the status of a given hit. The
basic rule prevailing in this case is “worst status wins”. That is, if any Hit Filter sets a
hit’s status to Error, the hit’s final status is Error. If one Hit Filter sets the status to
Warning and another sets it to Error, the hit’s final status is Error. There are situations
where it may be important to distinguish between the status of the hit according to Hit
Filter X and the final status of the hit (once all Hit Filters have evaluated the hit and set
the hit status). The Hit Filter and Transaction Filter interfaces make this distinction and
refer to the “final status of the hit” versus the “status recorded by Hit Filter X” where it
is useful to differentiate between these two types.
Example:
Hit Filter A does not set the status.
Hit Filter B sets the status to Warning.
Hit Filter C sets the status to Error.
In this case, the final status of the hit is Error. The status according to Hit Filter A
is OK, the status according to Hit Filter B is Warning, and the status according to
Hit Filter C is Error.

Hit Filter Configuration


You can create, edit, view, copy, and delete Hit Filters defined in your system, via the
FxV browser interface.
Several “default” Hit Filters are provided with the initial FxV configuration. These
filters can be modified as needed to meet the requirements of your installation. They
also provide examples of what can be accomplished by defining Hit Filters.
The following Hit Filters are installed by default:
• Default Code Exclusions—drops hits with an HTTP response code of 304. These
hits are generally uninteresting responses that tell the browser that it can use the
content (usually images) it already has cached from a previous hit. Dropping
these hits helps reduce confusion and clutter during replay.
Configuring the Hit Analysis Process 91
Hit Filters

• Default HTTP Error Codes—marks hits with an Error status value due to HTTP
Error Code; specifically, the status of hits with an HTTP response code of 400 or
greater (with the exception of 401) is set to Error.
• Default Session Custom Fields—adds a custom field called Initial Referer to
every session.
• Default Session Page Visit Custom Fields—adds an Entry Page and a Last Page
Visited custom field to every session with HTML hits.
• Default Type Exclusions—excludes hits by response content type. This Hit Filter
is disabled by default. It is provided as a template for customers wishing to reduce
the volume of data captured by the system.
• Optional Keyword Indexing—turns ON content keyword indexing for all HTML
hits. Hits can be indexed based on two factors:
• A hit filter has to “mark” the hit for indexing.
• The response content type has to start with either “text/html”, “text/xml”, or
“text/plain”.
To configure the Hit Filters in your system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Hit Filters section.
The Hit Filters already defined in your system are displayed in table format.

Column Name Description

Name Hit Filter name.


The Disabled icon following a filter name indicates that the Hit
Filter is currently disabled.

Used By Lists any Transaction Filters (or Special Events) currently using
this Hit Filter.
The Disabled icon following a filter name indicates that the
Transaction Filter is currently disabled.
If the hit filter match counts for this filter are stored in the Analysis
Repository, the Analysis Repository icon is also displayed.

3 Do one of the following:


92 Foglight Experience Viewer
User Guide

• To view an existing Hit Filter, click the View icon.


• To edit an existing Hit Filter, click the Edit icon.
• To copy an existing Hit Filter, click the Copy icon.
• To remove a Hit Filter from the system, click the Delete icon.
Note If a filter is “in use”, you can delete it from the system only after removing all
references to it (including removing its mapping in the Analysis Repository, if
applicable).

• To create a new Hit Filter, click Create Hit Filter, then fill in the information
requested.
4 Click OK.

Hit Filter Additional Information


This section is displayed every time you view or edit a hit filter defined in your system.
It contains the following attributes:
• Created—the name of the user who created this object and the creation date.
• Last Update—the name of the user who last updated this object and the update
date.
• Version—The version number for this object. Each time a change is made, the
version number is incremented.

Transaction Filters
A Transaction Filter is a collection of conditions and actions used to analyze sessions as
they are captured, and to identify the distinct transactions within those sessions. While
they are most commonly used to identify a series of hits that make up some sort of
logical transaction, Transaction Filters can be used to detect any activity within a
session. A key distinction between Hit Filters and Transaction Filters is that Hit Filters
can only examine the hit currently being processed, while Transaction Filters can act on
a series of hits within the active session.
A session is a set of hits with the same Session ID that were submitted by the same
browser client in the same visit to the site. Sessions are tracked based on simple timeout
rules and configurable Global Stop Events. A transaction is a portion of a session that
met the criteria specified by a Transaction Filter.
Configuring the Hit Analysis Process 93
Transaction Filters

A Transaction Filter can detect multiple events that provide information about site
problems and client activity within a related set of pages. Like Hit Filters, Transaction
Filters give a helpful and relevant context to the captured data, making the searching
and reporting easier.
For detailed information about Transaction Filters, see the following sections:
• “Transaction Events” on page 93
• “Transaction Event Match Conditions” on page 94
• “Transaction Event Actions” on page 97
• “Transaction Filter Storage Configuration” on page 104
• “Transaction Filter Configuration” on page 105
• “Transaction Filter Additional Information” on page 106

Transaction Events
Transaction Filters are made up of a series of Transaction Events (events). There are a
number of different types of events, but the most common type is “Hit Filter match”—
an event that occurs as the result of a Hit Filter’s match conditions matching an
incoming hit. Like Hit Filters, Transaction Events consist of matching conditions (for
example, the fact that Hit Filter X matched a hit) and a collection of actions to be
performed when those conditions are matched. The different types of match conditions
and events are described in the following sections.
Each Transaction Event has a name and an optional one-character label. The labels are
used to summarize the events that occurred in a given transaction.
Example:
A Transaction Filter has a start event, three intermediate steps, and a stop event, and
uses the labels S, 1, 2, 3, and F for these events. When viewing a list of transactions, the
events may appear as S123F / S12F / S321F etc., indicating which events occurred for
each individual transaction.

All transactions flow through the following series of states:


Not Started -> Active -> Stopped
Each event specifies the “entry state” required for the event to occur. Events that trigger
the beginning of a transaction specify an entry state of Not Started. Most other events
94 Foglight Experience Viewer
User Guide

specify an entry state of Active. Certain special events can be defined with an entry
state of Stopped. These events use special match conditions that only apply to post-
transaction processing.
When determining whether or not to evaluate an event’s match conditions, the Hit
Analysis system checks the value of the “allow multiples” flag, in addition to checking
the entry state. If a given event has already occurred for the current transaction and this
flag is set to “No”, the event is not evaluated again. When this flag is set to “Yes”,
multiple occurrences of a given event can be registered within a single transaction. Each
time the match conditions are met, the event’s actions are executed and the event is
recorded. Using the event labels described earlier, this may result in event label
sequences such as S112233123321F, if the events with labels 1, 2, and 3 are all
configured to allow multiple occurrences.

Transaction Event Match Conditions


A Transaction Event is fully processed only when its match conditions are met. Several
types of match conditions are available, the most common match condition for a
Transaction Event being the “Hit Filter match”.
For a description of each match condition type, see these sections:
• “Hit Filter Match” on page 94
• “Hit Count by Hit Filter” on page 95
• “Hit Filter Ordering” on page 95
• “Session Custom Field” on page 95
• “Transaction Custom Field” on page 95
• “Script Output” on page 95
• “Session/Transaction Hit Counts” on page 95
• “Session/Transaction Duration” on page 96
• “Always True” on page 96
• “Post Transaction: Stopped by Global Stop Condition” on page 96
• “Post Transaction: Stopped by Transaction Stop Condition” on page 97

Hit Filter Match


This type of match condition specifies a specific Hit Filter that triggers the event. If an
incoming hit matches the match conditions of the specified Hit Filter, the event is
Configuring the Hit Analysis Process 95
Transaction Filters

considered to have occurred. Optionally, a specific hit status can be specified (for
example, if the hit matches Hit Filter Foo and has a status of “Error”).

Hit Count by Hit Filter


This type of match condition specifies a specific Hit Filter as well as the number of
times the Hit Filter must match within a given session or transaction in order to trigger
the event. This type of condition can be used to identify recurring behavior, such as
failed login attempts (for example, if Hit Filter Failed Login Page occurs five times in
the same session). Optionally, a specific hit status can be specified (only Hit Filter
matches with the specified status are counted).

Hit Filter Ordering


This type of match condition specifies two Hit Filters and allows you to define things
such as “if (within a given session) a hit matching Hit Filter Bar occurs before a hit
matching Hit Filter Foo occurs”. Specific status values may be specified.

Session Custom Field


This type of match condition allows you to trigger an event based on the value of a
Session Custom Field (which may be set by a Hit Filter or a Transaction Filter).

Transaction Custom Field


This type of match condition allows you to trigger an event based on the value of a
Transaction Custom Field defined in this Transaction Filter. This event condition is not
typically used, since the condition that caused the custom field to be updated in the first
place is usually sufficient.

Script Output
This type of match condition allows you to trigger an event based on the value of an
output variable set by a script.

Session/Transaction Hit Counts


This type of match condition triggers based on the total number of hits since the session
started, or since the transaction started. Hit counts can include all hits or can be limited
to HTML hits only. Hit status may be taken into account as well. This allows for
conditions such as “if the total number of HTML hits in the session with status Error is
greater than 100”.
96 Foglight Experience Viewer
User Guide

Session/Transaction Duration
This type of match condition triggers based on the duration of the session or transaction.
The duration of a session is based on the time passed since the first hit in the session.
The duration of a transaction is based on the time passed since the transaction started, as
defined by some other Transaction Event (one which changed the state of the
transaction to In Progress).
The following timing values can be evaluated:
• Client Time—the amount of time spent on the client side during client-server
communication with multiple bursts. It is the sum of all the Client Time delays
that occur between each burst of server data.
• Total Time—total “wall clock” time for the session or transaction. In other words,
the amount of time that has passed since the session or transaction started.
• Processing Time—the total time spent by the Web server(s) processing hits that
occurred within the session or transaction.
• End-to-End Time—the total time spent downloading the contents of the URLs
processed within the session or transaction.

Always True
This type of match condition triggers on any hit. The purpose of this type of condition is
to allow for the creation of transactions that start as soon as the session starts. By using
an Always True condition to trigger a transaction state change to “In Progress”, the
Transaction Filter causes a new transaction to start with every new session.

Post Transaction: Stopped by Global Stop Condition


This is a special type of match condition used only for post-transaction processing. It
can only be triggered after a transaction has been stopped. It provides a final opportunity
to annotate the transaction (or perform other actions) based on the manner in which the
transaction was stopped. When this type of condition occurs, it indicates that a system-
wide global stop condition forced the transaction to stop. Such conditions include:
• The global stop-transaction event occurred
• The global stop-session event occurred
• The maximum session hit count was reached
• The maximum session duration was reached
• The “long session” timeout was exceeded
Configuring the Hit Analysis Process 97
Transaction Filters

• The “short session” timeout was exceeded

Post Transaction: Stopped by Transaction Stop Condition


This is a special type of match condition used only for post-transaction processing. It
can only be triggered after a transaction has been stopped. It provides a final opportunity
to annotate the transaction (or perform other actions) based on the manner in which the
transaction was stopped. This type of condition occurs only when a transaction is
stopped by a Transaction Event explicitly setting the transaction state to “Stopped”. In
most cases, the event that stopped the transaction can do any final processing itself.
However, if there are multiple events that can potentially stop the transaction, this
special condition allows you to define any post-transaction processing in a single
location.

Grouping Match Conditions


Like Hit Filter Match Conditions, Transaction Filter Match Conditions can be grouped
together as needed (with any combination of the AND, OR, and NOT logic operators).
This is rarely necessary for Transaction Event Match Conditions.
Transaction Filter Match Conditions can be combined in complex grouping of match
conditions. For more information, see “Grouping Match Conditions” on page 81.

Transaction Event Actions


When a Transaction Event’s match conditions are met, any number of actions may be
performed. These actions are grouped in the following categories:
• “Set Transaction Status” on page 97
• “Change Transaction/Session State” on page 98
• “Execute Scripts” on page 98
• “Transaction Event: Set Custom Fields” on page 99
• “Transaction Event Increment Metrics” on page 103

Set Transaction Status


By default, all transactions have a status of “OK”. Each Transaction Event has an
opportunity to set the transaction status to “Error” or “Warning”.
98 Foglight Experience Viewer
User Guide

Change Transaction/Session State


As described in section “Transaction Events” on page 93, all transactions flow through a
the following series of states:
Not Started -> Active -> Stopped
Certain event conditions identify the beginning or end of a transaction. These events are
responsible for changing the transaction state from Not Started to Active (that is, start
the transaction) or from Active to Stopped (that is, stop the transaction). In some cases, a
single event may start and stop a transaction (that is, take it from Not Started directly to
Stopped). This is useful for identifying activities such as repeated failed logins (for
example, “When Hit Filter FailedLogin occurs five times, start/stop a HackerAttempt
transaction and set a session custom field that flags the suspicious session”).
In addition to changing the state of a transaction, it is also possible to change the state of
a session (to Stopped). This allows events such as a user clicking on a logout link to
immediately end a session (rather than waiting for the session to eventually time out).

Note A session should only be marked as Stopped in cases where you are certain that the Web
application has ended the session.
Some events do not impact the state of the transaction (or session). Once a transaction is
active (that is, it has been started by some event), other interesting events may occur that
leave the transaction in the Active state.

Execute Scripts
FxV Hit Analysis supports the concept of script execution using the Groovy
programming language. Scripts can be used to perform complex operations not
supported directly by the Hit Analysis model, including the execution of complex
algorithms or the remote access of external systems (for example, to look up user
account information based on captured data such as a user ID).
Scripts can set output variables that can be referenced in others areas of the Transaction
Event (custom fields and metrics). If the only purpose of a script is to set a particular
output variable, the script does not have to be explicitly executed as an action of the
Transaction Event. Instead, the script output variable can be referenced when updating
the custom field or metric, and the script is executed as needed.
For detailed information about how scripting works, see “Scripts” on page 118.
Configuring the Hit Analysis Process 99
Transaction Filters

Transaction Event: Set Custom Fields


Custom Fields allow specialized information to be associated with individual hits,
sessions, and transactions. A Transaction Event whose match conditions have been met
can add (or update) custom field values for the current transaction or for the session that
contains it (hit custom fields are only accessible in Hit Filters).
You can add a custom field to a Transaction Event by creating a new Custom Field or by
selecting (and customizing, if necessary) an existing one. In the first case, the following
Custom Field attributes can be configured:
• “Name” on page 99
• “Description” on page 99
• “Storage (Scope)” on page 100
• “Storage (Type)” on page 101
• “Value Source” on page 101
• “Value Assignment Mode” on page 102
• “Update If Blank” on page 102
In the latter case, you must specify only the value source and the value population
policy. Optionally, you may also override any of the following three attributes defined
for the Custom Field: Value Assignment Mode, value Separator (for append modes), and
Update If Blank. The other attributes get the default settings defined for that Custom
Field.

Name
This field defines the name of the Custom Field.

Note This name may contain only letters, numbers, and spaces. Other characters are not
allowed. Names must be less than 80 characters long, and cannot contain leading or trailing
spaces.

Description
This field is optional. If populated, it is used to build tooltips and reports, when the
custom field name appears as a column heading in a search results screen.
100 Foglight Experience Viewer
User Guide

Storage (Scope)
A Transaction Filter can set transaction-level or session-level custom fields. Within
these two categories, there are additional options to be considered. The main thing to
decide is where the custom field needs to be seen.
The following table presents the values you can select for hit-scoped custom fields.

Value (Update Transaction) Description

Available only within this For fields that are only needed within the
Transaction Filter definition of the Transaction Filter (perhaps as a
value source for some other custom field or
metric).

Show in Search Results For fields that should appear in search results.

First Column in Search Results For identifying fields (for example, username)
that should appear first in search results (far left
side of the results screen).

Not in Search Results For fields that should not be in search results, but
can be seen when examining transaction details
for a specific transaction.

The following table presents the values you can select for session-scoped custom fields.

Value (Update Session) Description1

Available only for Transaction Makes the custom field value available to any
Filters Transaction Filter defined in the system, but the
value does not appear when viewing session
search results or session details.

Show in Search Results For fields that should appear in search results.

Not in Search Results For fields that should not be in search results, but
can be seen when examining session details for a
specific session.
Configuring the Hit Analysis Process 101
Transaction Filters

Value (Update Session) Description1

First Column in Search Results For identifying fields (for example, username)
that should appear first in search results (far left
side of the results screen).

Show in Search Results (+ Audit For fields that should appear in search results, and
Log) be used for audit logging.

Not in Search Results (+ Audit For fields that should not be in search results, but
Log) can be seen when examining hit details for a
specific hit, and be used for audit logging.

First Column in Search Results For identifying fields (for example, username)
(+ Audit Log) that should appear first in search results (far left
side of the results screen). Also used for audit
logging.
1
The options that end with “(+ Audit Log)” can be used to flag custom fields for audit logging
when sessions are viewed by FxV users. This allows you to track the data seen by individual
FxV users.

Storage (Type)
A Custom Field must be declared as either a string (Store Values as Text) or numeric
(Stores Values as Numbers) field. Explicitly defining a custom field as numeric enables
searching based on numerical comparison operators (for example, is the value of custom
field “shopping cart value” greater than 100?). Numeric sorting is also enabled.

Value Source
A custom field’s value source defines where the value for the field comes from. Three
basic types of value sources exist:
• Other Custom Fields—a custom field associated with the active session or
transaction.
• Script Outputs—any data exposed as an “output variable” by a Groovy script.
Note The script is executed as needed in order to populate the custom field.

• Fixed Values—any hard-coded value.


102 Foglight Experience Viewer
User Guide

Value Assignment Mode


This attribute controls how the Custom Field is updated. The field value can be “reset”
for each event, or “set on the first event” to lock in just the initial value. String fields
allow “appending” to an existing field using a custom separator; numeric fields allow
“incrementing”. When appending strings, a maximum of 2 K is stored in a Custom Field
value.
When a custom field already has a value, several different options are available.

Value Description1

Reset value each time this event Default mode in which the custom field value is
occurs always set to the value defined by the current
value source [2].

Set value the first time this event Once the custom field value is set, it is not
occurs changed [1].

Append to value each time this Concatenates values as they are encountered (an
event occurs optional separator can be inserted between the
values) [1232].

Increment value each time this Increments a numerical custom field value by the
event occurs amount defined by the current the value source
[8].

Append value on each match (if Similar to Append to value each time this event
unique) occurs, but only unique values are added [123].

Append value on each match (if Similar to Append to value each time this event
unique, ignoring case) occurs, but values like “foo” and “Foo” are
considered identical [123].
1
The values in brackets represent the final custom field value in the case where three
assignments to the same field occur in the following sequence: 1, 2, 3, 2.

Update If Blank
Blank values (as defined by any value source) are ignored by default (that is, a previous
custom field value is not replaced by a blank value). This behavior can be changed by
setting the Update If Blank flag to “Yes”.
Configuring the Hit Analysis Process 103
Transaction Filters

Transaction Event Increment Metrics


Metrics are essentially numeric, global custom fields with a value assignment mode of
“increment value on match”. You can add a metric to a Transaction Event by creating a
new metric or by selecting (and customizing, if necessary) an existing one. Similarly to
custom fields, metric values can be incremented by:
• Fixed Values—any hard-coded value (typical “counter” metrics are set to
increment by a hard-coded value of one).
• Script Outputs—any data exposed as an “output variable” by a Groovy script.
Note The script is executed as needed in order to determine the metric increment amount.

• Numeric Custom Fields—any numeric custom field associated with the active
session or transaction.
Metric updates defined by Transaction Events do no take place until the transaction
stops. Updates can be configured to take place unconditionally (as soon as the
transaction ends) or conditionally based on the Commit Pending Metric Update policy
defined for the Transaction Filter.
When entering the name for a metric, it is sometimes useful to parameterize the name
based on some piece of data from the transaction.
Example
You may want to define a series of metrics such as:
Errors from Spain
Errors from France
Errors from Italy
These can be defined via a single metric that contains a substitution token. The name of
the metric would be defined as Errors from $0.
When $0 is included in the metric name, the Transaction Filter UI displays a field called
$0 Source used to identify the value source for the $0 token (that is, the data that should
be used to replace $0 in the metric name).
In this example, you would select Country from the list of transaction details, and the
transaction’s country would be used in the metric name.

Important All value sources available for populating custom field and metric values are available
here as well. In order to avoid the creation of an unusable number of metrics, ensure that
there is a reasonable number of unique values for the substitution token.
104 Foglight Experience Viewer
User Guide

Transaction Filter Storage Configuration


The following settings default to “Always”, but can be configured to allow for
conditional storage and updates of certain data:
• Store Transactions—can be set to “Always”, “Never”, or “Conditional”, based on
the final status of the transaction. Some Transaction Filters may only truly define
a transaction when certain events occur (beyond the “start” event). For example,
if a session timeout occurs, the transaction may not be meaningful and should not
be stored.
• Store Custom Fields—can be set to “Always” or “Conditional”, based on the final
status of the transaction. This setting applies to transaction custom fields and
allows custom field values to be conditionally stored based on the final
transaction status.
• Store Events—can be set to “Always”, “Never”, or “Conditional”, based on the
final status of the transaction. In some cases, the occurrence of the individual
events provides no additional value. The most common such case would be a
Transaction Filter whose events always occur in the same order without variation.
Storing the fact that events A, B, C, and D occurred for every transaction is not
helpful.
• Commit Pending Metric Updates—can be set to “Always” or “Conditional”,
based on the final status of the transaction. Metric updates defined by the events
that make up the Transaction Filter can be set to update unconditionally as soon
as the transaction stops (by setting the metric’s increment policy to “Always
increment when transaction stops”). These metric updates are not impacted by
this higher-level setting. However, metric updates with an increment policy of
“Conditionally increment when transaction stops” are updated according to this
setting. This allows metrics that are only meaningful in certain situations (for
example, when the transaction completes successfully) to be updated only when
appropriate.
• Store Transaction in Analysis Repository—controls whether or not to store
transactions in the Analysis Repository (if enabled). It can be set to “Always” (for
transactions that should always be stored), “Conditional” (to base storage on the
final status of the transaction), or “Never” (to temporarily disable storage of these
transactions in the Analysis Repository).
Note Transactions must be stored in the Archiver database in order to be eligible for
transfer to the Analysis Repository. You must also configure the Analysis Repository
database schema to define a table for this Transaction Filter.
Configuring the Hit Analysis Process 105
Transaction Filters

• Store Session in Analysis Repository—controls whether or not to store sessions


in the Analysis Repository (if enabled).
Note Transaction Filters and Hit Filters can be configured to flag sessions for exclusion
from the Analysis Repository. If a session is marked for exclusion by any filter, it is
excluded regardless of the storage policy defined by any other filters.

Transaction Filter Configuration


You can create, edit, view, copy, and delete Transaction Filters defined in your system,
via the FxV browser interface. No “default” Transaction Filters are provided with the
initial FxV configuration.
To configure the Transaction Filters in your system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Transaction Filters section.
If Transaction Filters are already defined in your system, they are displayed in
table format.

Column Name Description

Name Transaction Filter name.

Used By Displays the Analysis Repository icon if transactions identified


by this filter are stored in the Analysis Repository.

3 Do one of the following:


• To view an existing Transaction Filter, click the View icon.
• To edit an existing Transaction Filter, click the Edit icon.
• To copy an existing Transaction Filter, click the Copy icon.
Note When you copy a Transaction Filter that has transaction-scoped custom fields,
you are implicitly creating new custom fields (with transaction scope, but for the
new Transaction Filter).

• To remove a Transaction Filter from the system, click the Delete icon.
Note If a filter is stored in the Analysis Repository, you can delete it from system only
after you remove its mapping to the Analysis Repository.
106 Foglight Experience Viewer
User Guide

• To create a new Transaction Filter (using the full-featured user interface), click
Create Transaction Filter, then fill in the information requested.
• To create a new Transaction Filter and new Hit Filters in a single step, click
Quick Create, then fill in the information requested. This interface supports
only very simple Hit Filters that match on a single URL path or HTML page
title.
Note You can use the Quick Create interface to quickly create multiple Hit Filters and
the Transaction Filter that uses them and then go back and modify the individual
filters as needed.

4 Click OK.

Transaction Filter Additional Information


This section is displayed every time you view or edit a transaction filter defined in your
system. It contains the following attributes:
• Created—the name of the user who created this object and the creation date.
• Last Update—the name of the user who last updated this object and the update
date.
• Version—The version number for this object. Each time a change is made, the
version number is incremented.

Special Events
The model used to describe the events that define each Transaction Filter is also used to
define three global events. These events are not specific to any particular Transaction
Filter, but they run as part of the Hit Analysis process that occurs for each hit captured
by the system. These events defined outside Transaction Filters stand on their own.
Certain aspects of the Transaction Event model do not apply to the special events.
Specifically, anything that depends on the enclosing Transaction Filter or the active
transaction is not applicable, since these events are not evaluated in the context of a
transaction.
The three events that can be defined are:
• Global Stop Session Event—this event is used to define conditions that must
trigger the end of a session. Without a Global Stop Session Event, the only way to
identify the end of a session is by a session timeout. This event allows activities
Configuring the Hit Analysis Process 107
Special Events

that are known to stop sessions (user logout being the most common example) to
immediately close a session, without having to wait for the session to time out.
• Global Stop Transaction Event—this event works much like the Global Stop
Session Event, but rather than closing the entire user session, it stops any active
transaction(s). Global Stop Transaction Events are rarely meaningful, as it is
unlikely that a common activity applies to all Transaction Filters in the system,
unless it is something such as a logout page (in which case the Global Stop
Session Event is probably a better choice).
• Post Hit Analysis Event—this event is fired as the very last step for each hit
processed by the system. This is the only place where certain data is guaranteed to
be updated prior to processing.
For detailed information about Special Events, see the following sections:
• Special Events Configuration
• Special Events Additional Information

Special Events Configuration


The special events provided with the initial FxV configuration are undefined. Once you
define them, you can further modify them as needed to meet the requirements of your
installation.
To define a Special Event in your system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Special Events section.
The list of events is displayed.
Note No Special Events are defined in the initial FxV configuration.

3 Click the Edit icon beside the event you want to define.
Note Editing is the only operation available for an undefined special event.

The Default <Event Name> page appears.


4 Fill in the information to customize this event, as necessary, then click OK.
The newly defined special event is now marked as IS Defined.
108 Foglight Experience Viewer
User Guide

To configure a Special Event defined in your system:


1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Special Events section.
The special events that have been defined in your system are marked as IS
Defined.
3 Do one of the following:
• To view an existing Special Event, click the View icon.
• To edit an existing Special Event, click the Edit icon.
• To cancel all customizations made to a Special Event, click the Delete icon.
4 Click OK.

Special Events Additional Information


This section is displayed every time you view or edit a special event defined in your
system. It contains the following attributes:
• Created—the name of the user who created this object and the creation date.
• Last Update—the name of the user who last updated this object and the update
date.
• Version—The version number for this object. Each time a change is made, the
version number is incremented.

Custom Fields
Custom Fields allow specialized information to be associated with individual hits,
sessions, and transactions.
For detailed information about Custom Fields, see the following sections:
• “Custom Fields Attributes” on page 109
• “Custom Fields Configuration” on page 113
• “Custom Fields Additional Information” on page 115
Configuring the Hit Analysis Process 109
Custom Fields

Custom Fields Attributes


Custom fields are highly configurable. The following sections describe the attributes
that can be configured for each Custom Field:
• Name
• Description
• Storage (Scope)
• Storage (Type)
• Value Assignment Mode
• Update If Blank

Name
This field defines the name of the Custom Field. Custom Field names must be unique
within a given scope, but the same name may be used in different scopes (that is, you
can have a Hit Custom Field named foo and a Session Custom Field named foo). Each
Transaction Filter defines its own unique scope (that is, two different Transaction Filters
can define Custom Fields named foo).

Note This name may contain only letters, numbers, and spaces. Other characters are not
allowed. Names must be less than 80 characters long, and cannot contain leading or trailing
spaces.

Description
This attribute is optional. If populated, it is used to build tooltips and reports.

Storage (Scope)
This attribute controls how the Custom Field is stored in the capture database and how it
should be used for search results (that is, not shown at all, included with search results,
or included in the first column in the results). Once you define a custom field, you can
not change its scope, but you can change the storage details within that scope.

Important The Storage (Scope) attribute cannot be overridden by filters that update this Custom
Field.
110 Foglight Experience Viewer
User Guide

Select one of the Update Hits options to store the field with all matching hits for easy
searching later. The following table presents the values you can select for hit-scoped
custom fields.

Value (Update Hit) Description

Available only during Hit For transient values that do not need to be stored
Analysis in the database.

Show in Search Results For fields that should appear in search results.

Not in Search Results For fields that should not be in search results, but
can be seen when examining hit details for a
specific hit.

First Column in Search Results For identifying fields (for example, username)
that should appear first in search results (far left
side of the results screen).

Select one of the Update Session options to store the field with the sessions of all
matching hits, for use by Transaction Filters and/or when searching for sessions. The
following table presents the values you can select for session-scoped custom fields.

Value (Update Session) Description1

Available only during Hit For transient values that do not need to be stored
Analysis in the database.

Show in Search Results For fields that should appear in search results.

Not in Search Results For fields that should not be in search results, but
can be seen when examining session details for a
specific session.

First Column in Search Results For identifying fields (for example, username)
that should appear first in search results (far left
side of the results screen).

Show in Search Results (+ Audit For fields that should appear in search results, and
Log) be used for audit logging.
Configuring the Hit Analysis Process 111
Custom Fields

Value (Update Session) Description1

Not in Search Results (+ Audit For fields that should not be in search results, but
Log) can be seen when examining hit details for a
specific hit, and be used for audit logging.

First Column in Search Results For identifying fields (for example, username)
(+ Audit Log) that should appear first in search results (far left
side of the results screen). Also used for audit
logging.
1
The options that end with “(+ Audit Log)” can be used to flag custom fields for audit logging
when sessions are viewed by FxV users. This allows you to track the data seen by individual
FxV users.

Custom Fields with transaction scope can only be created when creating or editing the
Transaction Filter that defines their scope.

Storage (Type)
A Custom Field must be declared as either a string (Store Values as Text) or numeric
(Stores Values as Numbers) field. Explicitly defining a custom field as numeric enables
searching based on numerical comparison operators (for example, is the value of custom
field “shopping cart value” greater than 100?). Numeric sorting is also enabled.
Once you define a custom field, you can not change its storage type.

Important The Storage (Type) attribute cannot be overridden by filters that update this Custom
Field.

Value Assignment Mode


This attribute controls how the Custom Field is updated, which is most useful for fields
added to sessions. The field value can be “reset” for each match, or “set on the first
match” to lock in just the initial value. String fields allow “appending” to an existing
field using a custom separator; numeric fields allow “incrementing”. When appending
strings, a maximum of 2 K is stored in a Custom Field value.

Important This setting can be overridden by filters that update this Custom Field.
112 Foglight Experience Viewer
User Guide

When a custom field already has a value, several different options are available.

Value Description1

Reset value Default mode in which the custom field value is


always set to the value defined by the current
value source [2].

Set value only if not already set Once the custom field value is set, it is not
changed [1].

Append to value Concatenates values as they are encountered (an


optional separator can be inserted between the
values) [1232].

Increment value Increments a numerical custom field value by the


amount defined by the current the value source
[8].

Append value (if unique) Similar to Append to value, but only unique
values are added [123].

Append value (if unique, Similar to Append to value, but values like “foo”
ignoring case) and “Foo” are considered identical [123].
1
The values in brackets represent the final custom field value in the case where three
assignments to the same field occur in the following sequence: 1, 2, 3, 2.

Update If Blank
This attribute determines how blank values should be treated. By default, this attribute
is set to No, and blank values are ignored (that is, a previous custom field value is not
replaced by a blank value). Set to Yes to retain blank values when setting or appending
strings.

Important This attribute can be overridden by filters that update this Custom Field.
Configuring the Hit Analysis Process 113
Custom Fields

Custom Fields Configuration


You can create, edit, view, copy, and delete all Custom Fields defined in your system,
via the FxV browser interface.
Several “default” Custom Fields are provided with the initial FxV configuration. These
Custom Fields can be modified as needed to meet the requirements of your installation.
They also provide examples of what can be accomplished by defining custom fields.
The following Custom Fields are installed by default:
• Entry Page—identifies the first HTML page visited by the user during this
session.
• Initial Referer—identifies the Web page the user was visiting prior to starting this
session.
• Last Page Visited—identifies the last HTML page visited by the user during this
session.
To configure the Custom Fields defined in your system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Custom Fields section.
The Custom Fields already defined in your system are displayed in table format.

Column Name Description

Name Name of the Custom Field, preceded by two icons indicating the
scope and the type of the field:
• Hit Scope
• Session Scope
• Transaction Scope
• Values store as text
• Values stored as numbers
114 Foglight Experience Viewer
User Guide

Column Name Description

References A list of objects that either set the value of the Custom Field or
access its value. Each reference is preceded by two icons:
• The filter or event reads the value of the Custom Field
• The filter or event sets the value of the Custom Field
• Hit Filter
• Transaction Filter
• Special Event
If this Custom Field is stored in the Analysis Repository, the
Analysis Repository icon is also displayed.

3 Do one of the following:


• To view an existing Custom Field, click the View icon.
• To edit an existing Custom Field, click the Edit icon.
• To copy an existing Custom Field, click the Copy icon.
• To remove a Custom Field from the system, click the Delete icon.
Note If a Custom Field is in use, you can delete it from the system only after you
remove all references to it (including any Analysis Repository mappings, if
applicable).

• To create a new Custom Field, click Create Custom Field, then fill in the
information requested.
Note Custom Fields can also be created from within the Hit Filter, Transaction Filter,
and Special Event editors.

4 Click OK.
The Custom Field page that appears when you view or edit a Custom Field contains a
section named Updated By. This is a read-only table that lists all of the objects (Hit
Filters, Transaction Filters, or Special Events) that update the value of that Custom
Field. The primary purpose of this table is to show any overridden values. Since
updating objects can override the “assignment mode”, “appended values separator”, and
“update if blank policy”, this table has columns for each of these three settings. The
values in these columns are blank unless the updating object overrides the value. If it
does, the overridden value is shown in the table.
The only exception to this rule is when a Custom Field has an assignment mode of
“append” and defines a separator character. If a Hit Filter, Transaction Filter, or Special
Configuring the Hit Analysis Process 115
Metrics

Event overrides and removes the separator, the Used By table displays a value of
“{none}” in the Separator column.
Any changes made to a Custom Field are automatically propagated to its “updating
objects”, unless they are overriding the value. Overrides can be cleared by editing either
the Custom Field or the overriding object, and modifying the values so that they match.
For details on referencing and updating Custom Fields from Hit Filters, Transaction
Filters, and Special Events, see the following sections: “Hit Filters: Set Custom Fields”
on page 82, “Transaction Event: Set Custom Fields” on page 99, and “Special Events”
on page 106.

Custom Fields Additional Information


This section is displayed every time you view or edit a custom field defined in your
system. It contains the following attributes:
• Created—the name of the user who created this object and the creation date.
• Last Update—the name of the user who last updated this object and the update
date.
• Version—The version number for this object. Each time a change is made, the
version number is incremented.

Metrics
A Metric is a global numeric value incremented by Hit Filters, Transaction Filters, and
Special Events.
For detailed information about Metrics, see the following sections:
• “Metrics Configuration” on page 115
• “Metrics Additional Information” on page 117

Metrics Configuration
You can create, edit, view, copy, and delete all Metrics defined in your system, via the
FxV browser interface.
A single “default” Metric is provided with the initial FxV configuration:
116 Foglight Experience Viewer
User Guide

• HTTP Response Code $0—counts occurrences of individual HTTP Response


Codes.
This metric may be modified as needed to meet the requirements of your installation. It
provides an example of what can be accomplished by defining metric values.
To configure the Metrics defined in your system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Metrics section.
The Metrics already defined in your system are displayed in table format.

Column Name Description

Name Name of the Metric.


Metric names that contain the substitution characters $0 are
considered “dynamic metrics”. These metric names result in the
creation of multiple metrics. The $0 is replaced at Hit Analysis
time based on the $0 Source defined by the Hit Filters, Transaction
Filters, or Special Events that updated the metric value.
Note The $0 Source is not part of the Metric definition, since it can vary
depending on where the metric is updated. The $0 Source fields are
only displayed by the editors that define the actual metric updates.

References A list of objects that either update the value of the Metric or access
it (in the case of the Analysis Repository). Each reference is
preceded by an icon indicating the type of the updating object:
• Hit Filter
• Transaction Filter
• Special Event
• Analysis Repository
Note If a Metric is not updated by any object in the Hit Analysis model, this
metric is not displayed in the Analysis Repository Configuration
page.

3 Do one of the following:


• To view an existing Metric, click the View icon.
Configuring the Hit Analysis Process 117
Captured Metadata

• To edit an existing Metric, click the Edit icon.


• To copy an existing Metric, click the Copy icon.
• To remove a Metric from the system, click the Delete icon.
Note If a Metric is in use, you can delete it from the system only after you remove all
references to it (including any Analysis Repository mappings, if applicable).

• To create a new Metric, click Create Metric, then fill in the information
requested.
4 Click OK.
For details on referencing and updating Metrics from Hit Filters, Transaction Filters,
and Special Events, see the following sections: “Hit Filters: Increment Metrics” on
page 86, “Transaction Event Increment Metrics” on page 103, and “Special Events” on
page 106.

Metrics Additional Information


This section is displayed every time you view or edit a metric defined in your system. It
contains the following attributes:
• Created—the name of the user who created this object and the creation date.
• Last Update—the name of the user who last updated this object and the update
date.
• Version—The version number for this object. Each time a change is made, the
version number is incremented.

Captured Metadata
Metadata captured by the system can be used when searching and when building Hit
Filters and Transaction Filters.
To display metadata captured by your system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Display the Captured Metadata menu options.
3 To display a specific metadata, click one of the following links:
• View Captured Web Servers
118 Foglight Experience Viewer
User Guide

• View Captured Paths


• View Captured Field Names
• View Captured Request Header Names
• View Captured Response Header Names
• View Captured Cookie Names
• View Captured HTML Titles
• View Captured Browser Types
• View Captured ISPs
• View Captured Cities
• View Captured Regions
• View Captured Countries
Note Data displayed by clicking these links can be exported to CSV files.

FxV users can control the amount of metadata that is included in the Captured Field
Names, Captured Request Header Names, Captured Response Header Names, and
Captured Cookie Names lists by editing the Metadata Sensitivity property in the Server
Configuration page (for details, see “Server Configuration” on page 170).
The Metadata Sensitivity property specifies the minimum reference count required for a
name to be included in the list. The reference count is the number of hits that contain the
field, header, or cookie name in any of the database segments. For example, setting the
Metadata Sensitivity property to “1” results in including all names in the list; setting this
property to “1000” results in including only those names that appear in at least 1000 hits
in a database segment.

Note This property affects only the Captured Field Names, Captured Request Header Names,
Captured Response Header Names, and Captured Cookie Names metadata lists.

Scripts
FxV uses scripts (snippets of custom logic written in the Groovy scripting language) to
allow its users to extend the functionality of the Foglight Experience Viewer system.
Scripts can be used to integrate FxV with other systems, as well as perform more
complex analysis of the data than is allowed by the standard hit analysis filters (such as
evaluating complex match expressions, transforming hit data, applying complex regular
expressions, etc.). A script is executed by a Hit Filter or a Transaction Filter, which can
use the output variables published by the script to update custom fields or metrics.
Configuring the Hit Analysis Process 119
Scripts

For detailed information about Scripts, see the following sections:


• “Scripts Configuration” on page 119
• “Scripts Additional Information” on page 120

Scripts Configuration
To implement scripts into the FxV Hit Analysis model:
1 Determine the script requirements.
2 Select the appropriate Hit Analysis APIs.
3 Select the script’s execution frequency.
4 Determine the script’s output.
5 Write the script.
6 Test the script.
7 Add the script to the Hit Analysis model.
For detailed information about implementing scripts into the FxV Hit Analysis Model,
see the Groovy Scripting API documentation.
For additional information about the Groovy scripting language, see http://
groovy.codehaus.org/Documentation.
To manage Scripts via the FxV browser interface:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Scripts section.
The Scripts already defined in your system are displayed in table format.

Column Name Description

Name Script name.


120 Foglight Experience Viewer
User Guide

Column Name Description

Executed/ The number of times this script has been executed and the number
Errors of times it has failed in the current day (since 12:00 AM).
Note The number of failures is incremented when an uncaught exception
occurs while executing the script, keeping the script from running to
completion.

Used By Lists any Hit Filters or Transaction Filters currently using this
Script.
The Disabled icon following the name indicates that the filter is
currently disabled.

3 Do one of the following:


• To view an existing Script, click the View icon.
• To edit an existing Script, click the Edit icon.
• To copy an existing Script, click the Copy icon.
• To remove a Script from the system, click the Delete icon.
• To create a new Script, click Create Script. In the Script definition page, fill
in the information requested.
• To access the Groovy Scripting API documentation, click Create Script. In
the Script definition page, click the Groovy Scripting API link in the Script
Definition area.
4 Click OK.
Since scripts commonly make use of regular expressions, the Script create/edit page
provides direct access to the FxV Regular Expression Tester. For additional information
about this topic, see “Appendix: Java Regular Expressions in FxV Hit Analysis” on
page 187.

Scripts Additional Information


This section is displayed every time you view or edit a script defined in your system. It
contains the following attributes:
• Created—the name of the user who created this object and the creation date.
• Last Update—the name of the user who last updated this object and the update
date.
Configuring the Hit Analysis Process 121
Sensitive Data Protection

• Version—The version number for this object. Each time a change is made, the
version number is incremented.

Sensitive Data Protection


This section focuses on FxV features that allow sensitive data (for example, credit card
numbers, passwords, etc.) captured by FxV to be hidden from some or all users of the
FxV browser interface.

Note The features described in this section are just a subset of the options available for dealing
with sensitive data.

The following considerations are important when planning to protect sensitive data:
• Full-disk encryption can be configured on the FxV appliance to ensure that
sensitive data cannot be extracted directly from stolen hard drives.
• The FxV browser interface can be configured such that all traffic is sent via https
(SSL).
• All configuration changes of the FxV capture system are logged to ensure that
unauthorized changes can be identified.
• The Foglight Experience Monitor (FxM) can be configured such that certain data
is never passed to FxV.
• Entire hits can be dropped (not stored), by configuring the Storage Restrictions
for a Hit Filter.

Sensitive Hit Details


In many cases, sensitive data is passed into the appliance as a result of the user entering
the data into a form. Such data appears in FxV as request fields. For example, when a
user enters a credit card number into a payment form and submits the form, a hit
containing a request field cc_num=4123456781234567 may be captured. Such data
can be hidden from display be defining a Sensitive Hit Detail.
A Sensitive Hit Detail is defined by the following attributes:
• Name—is the name given to the Sensitive Detail, for identification purposes.
• Description—reserved for describing the sensitive data.
122 Foglight Experience Viewer
User Guide

• Hit Detail Type—can be one of the following options: “Request Field” (the most
common), “Request Header”, “Response Header”, and “Cookie”.
• Matching Logic—specifies how the Hit Detail Name must be interpreted. A value
of “=” indicates that the name must match exactly in order to be identified as
sensitive. Other values options include “Starts With”, “Ends With”, “Contains”,
and “Matches” (which treats the Hit Detail Name as a regular expression). FxV
also provides “ignore-case” versions of each of these options (except for
“Matches”), which allow for case-insensitive comparisons.
• Hit Detail Name—is the name of the sensitive field/header/cookie (for example,
cc_num). Depending on the value of the Matching Logic attribute, this value may
be an exact name, a partial name, or a regular expression.
• Replacement Text—specifies a fixed string to be displayed in place of the
sensitive data. This field is optional.
• Always Sensitive—indicates whether the sensitive data should be considered
sensitive regardless of the privileges of the FxV user. This flag is set to “Yes” by
default. When set to “No”, users with special privileges are able to see the actual
value of the sensitive data, while other users see only the Replacement Text (or
nothing, if no replacement text is specified). In order to see sensitive data flagged
as “not always sensitive” a user must be a member of a User Group that has the
Sensitive Data Display privilege turned on.
Example:
Hit Detail Type = Request Field
Hit Detail Name = cc
Matching Logic = Starts With (IC)
Replacement Text = ****
Always Sensitive = No
In this scenario, the value of any request field starting with the letters cc (or CC or cC or
Cc) is replaced with the string “****” when displayed to a typical user. However, users
who are members of a User Group with the Sensitive Data Display privilege can see the
actual credit card numbers.

Note Regardless of the value of the Always Sensitive flag, the sensitive data is always captured
and stored in the database. The replacement occurs when the data is queried by an FxV
Configuring the Hit Analysis Process 123
Sensitive Data Protection

user. For information on disabling the storage of sensitive data completely, see “Define Hit
Storage Restrictions” on page 87.
Data marked as sensitive is not available for use during Hit Analysis. For example, if the
request field cc_num is identified as sensitive, this setting cannot be circumvented by
creating a Hit Filter that populates a custom field with the value of the cc_num request field.
If this were attempted, the replacement text would be stored in the custom field, not the
original value. This is true regardless of the value of the Always Sensitive flag, since the Hit
Analysis processing is performed outside the context of any specific user.

Sensitive Response Content Expressions


Sensitive Hit Details are ideal in situations where the user is entering the sensitive data
to be hidden. However, some Web sites render sensitive data directly in the content of a
page. For example, after a user creates a new account, the next page might contain text
such as “Thank you for joining. Your username is superdog and your password is
5uP3rd0G”. In this case, the user’s password is part of the HTML response and cannot
be hidden using a Sensitive Hit Detail. Sensitive Response Content Expressions can
handle such situations.
A Sensitive Response Content Expression is a regular expression that gets applied to the
response content and hides or replaces the specified portion of the page.

Note Sensitive Response Content Expressions are only applied to text/html hits.

A Sensitive Response Context Expression is defined by the following attributes:


• Name—is the name given to the Sensitive Response Content Specification, for
identification purposes.
• Description—reserved for describing the sensitive data.
• Regular Expression—a regular expression that identifies the sensitive text on the
page. This expression usually contains one or more sets of “grouping parenthesis”
that identify the portion of the expression to be hidden.
• Sensitive Groups—one or more numbers that identify which of the “groups”
defined by the parenthesis in the expression are to be hidden. If there are no
grouping parenthesis, the value of this attribute is always “0”, which refers to the
entire regular expression. Pairs of parenthesis are numbered “1..n”, starting from
the left.
124 Foglight Experience Viewer
User Guide

For example, the expression “abc(def)ghi(jkl)mno” has two groups. Group #1 has
the value “def”. Group #2 has the value “jkl”. Group #0 always represents the
entire expression, so the value of group #0 is “abcdefghijklmno” in this case.
Note The most common scenario is to define a regular expression with a single pair of
grouping parenthesis and the Sensitive Groups attribute set to “1”.

• Replacement Text—optional field that allows for the specification of a fixed


string to be displayed in place of the sensitive data.
• Always Sensitive—flag that indicates whether the sensitive data should be
considered sensitive regardless of the privileges of the FxV user. It is set to “Yes”
by default. When set to “No”, users with special privileges are able to see the
actual value of the sensitive data, while other users see only the Replacement Text
(or nothing, if no replacement text is specified). In order to see sensitive data
flagged as “not always sensitive” a user must be a member of a User Group that
has the Sensitive Data Display privilege turned on.
Example:
Regular Expression = “and your password is <b>(.*)</b>\.”
Sensitive Groups = 1
Replacement Text = “[HIDDEN BY FXV]”
Always Sensitive = No
In this scenario, consider that the Web site displays the following message (text shown
as HTML code) after a user introduces sensitive information into a form:
Thank you for joining. Your username is <b>superdog</b> and
your password is <b>5uP3rd0G</b>.
The Regular Expression finds the password between the <b> and </b> tags. The
parenthesis identify the specific text that is sensitive (anything between the tags, that is,
the password itself). Since there is one group of parenthesis, the Sensitive Group is “1”.
When displaying this page to a typical user, the content displays as:
Thank you for joining. Your username is superdog and your
password is [HIDDEN BY FXV].

When displaying this page to a user with Sensitive Data Display privileges, the original
response is displayed:
Thank you for joining. Your username is superdog and your
password is 5uP3rd0G.
Configuring the Hit Analysis Process 125
Hit Analysis Configuration Options

Note Similarly to Sensitive Hit Detail data, sensitive content identified by Sensitive Response
Content Expressions is stored in its original form. Replacement occurs at data access time
rather than capture time.
Regardless of the value of the Always Sensitive flag, the sensitive data is always captured
and stored in the database. The replacement occurs when the data is queried by an FxV
user. For information on disabling the storage of sensitive data completely, see “Define Hit
Storage Restrictions” on page 87.

Hit Analysis Configuration Options


This section contains various settings that relate to the Hit Analysis process:
• “Session Configuration” on page 125
• “Archiver Configuration” on page 126
• “Script Configuration” on page 127
• “Other Configuration” on page 127

Session Configuration
This section provides several configuration options for how the system should identify
the end of a session. These settings apply to all Archivers in the system. The following
options are available:
• Maximum Hits Per Session—defines the maximum number of hits that can be
contained within a single session. When this limit is reached, the session is
stopped and a new session (with the same Session ID) is started.
• Maximum Session Duration—defines the maximum time range that a single
session can span before being stopped. When this limit is reached, the session is
stopped and a new session (with the same Session ID) is started.
• Minimum Hits For “Long” Session—defines how many hits must be present
within a session before it is considered “long” for timeout purposes.
• Count Only HTML Hits for “Long” Session—defines if all hits or only HTML
hits should be counted when determining if a session is “long” for timeout
purposes.
126 Foglight Experience Viewer
User Guide

• Timeout for “Short” Sessions—defines the length of idle time before a “short”
session is timed out.
Note A “short” session is a session that contains one hit or a very small number of hits.
“Short” sessions usually “time-out” fairly quickly, which allows them to be identified
as closed more quickly then “long” sessions. This also improves the performance of
the Archiver, since fewer active sessions must be maintained in memory.

• Check Only HTML Hits for “Short” Session Timeout—for “short” sessions,
determines if all hits or only HTML hits should be interpreted as active use by the
client.
• Timeout for “Long” Sessions—defines the length of idle time before a “long”
session is timed out. This value should always be set to match the session timeout
defined by the Web server, to ensure that sessions timed out by the application are
marked as timed-out by FxV at the same time.
• Check Only HTML Hits for “Long” Session Timeout—for “long” sessions,
determines if all hits or only HTML hits should be interpreted as active use by the
client.
To configure the session timeout settings in the system:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Hit Analysis Configuration Options menu.
3 Edit the settings in the Session Configuration area as required, then click Save
Changes.
The settings are now saved and are applied to all Archivers in the system.

Note In addition to these settings, session completion can be further configured by defining a
Global Stop Session Event.

Archiver Configuration
This sections provides several configuration options that allow you to control the way
captured data is segmented by the capture system. These settings apply to all Archivers
in the system:
• Maximum DB Segment Span—data captured by the FxV Archivers is stored in
database segments. The size of each segment is limited to a fixed maximum size,
as well as a maximum time span (whichever comes first). This setting allows that
Configuring the Hit Analysis Process 127
Hit Analysis Configuration Options

time span to be configured. Typically, this value is set to 24 hours. It can be


changed to a smaller number when it is desirable to purge data from the system
after a small period of time (data is removed from the system one segment at a
time, based on the value of the Maximum DB Segment Retention option).
• Maximum DB Segment Retention—data captured by the FxV Archivers is stored
in database “segments”. This setting determines how long each segment is stored.
Note In addition to removing segments based on their age, Archivers also remove old
segments (oldest first) as needed to free up space for new segments. A value of 0
(zero) is interpreted as “no maximum”, in which case segments are removed only as
needed to make space for new data.

For detailed instructions on how to configure Archivers in a system via the FxV browser
interface, see the FxV Installation and Administration Guide.

Script Configuration
The settings in this section are applied to all Scripts defined in the system:
• Maximum Number of Uncaught Failures Per Script—defines the maximum
number of times that a given script can fail (by throwing an exception not handled
by the script) before the script is disabled.
• HTTP Connection Timeout—defines the maximum amount of time (in
milliseconds) that a script requesting an HTTP connection waits before timing
out.
• HTTP Socket Timeout—defines the maximum amount of time (in milliseconds)
that a script using an HTTP connection waits for a response before timing out.

Other Configuration
The settings in this section allow certain aspects of the Hit Analysis process to be
customized:
• Additional Parsed Response Types—allows you to control the response content
types that are parsed. By default, when processing hits, the response content is
only evaluated for hits whose response content type is set to text/html.
Additional response types can be included by specifying in this field a comma-
separated list of content types (for example: text/xml,text/plain).
These content types are utilized at capture time only. The primary use of this
setting is to make the content of certain kinds of hits available to hit filters, for use
128 Foglight Experience Viewer
User Guide

in match conditions or as a source in a custom field or metric update. The


following cases may occur:
• Case 1: If the content type on a hit is text/html, FxV applies sensitive
content rules, makes it available to hit filters, and performs keyword indexing
on it.
• Case 2: If the content type on a hit is not text/html, but is in the list of
content types specified in the Additional Parsed Response Types field, FxV
does not apply sensitive content rules, but makes it available to hit filters, and
performs keyword indexing on it.
• Case 3: If neither one of the conditions specified in Case 1 and Case 2 applies,
FxV does not apply sensitive content rules, does not make it available to hit
filters, and does not perform keyword indexing on it.
Note The content types are case sensitive.

Hit Analysis Configuration Change Log


This section allows you to view the changes made to the Hit Analysis Configuration and
roll back to any of the previous configuration versions. The number of versions shown
is configurable (Show Last 5, 25, 100, or 1000).
To explore the Hit Analysis Configuration versions:
1 On the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Hit Analysis Configuration Change Log section.
The log information is displayed in table format.

Column Name Description

Version Every time a change is made to anything on the Hit Analysis page,
a new version of the Hit Analysis Configuration model is created.
Version numbers are incremented sequentially.

User The name of the FxV user who modified the model.

Update Type The type of the change made: Create, Delete, Import, Rollback, or
Update.
Configuring the Hit Analysis Process 129
Hit Analysis Configuration Change Log

Column Name Description

Object Type The type of object that was created, updated, or deleted (for
example, Hit Filter, Custom Field, etc.).
Note This column is blank if the Update Type is Rollback or Import.

Name The name of the object that was created, updated, or deleted.

Date/Time The time when the change took place.

3 Do one of the following:


• To view the changes made on a specific version, click the View icon in that
row.
The Hit Analysis Configuration Summary page for that version opens, showing
everything that was in a previously-saved version of the Hit Analysis
configuration.
Review the changes, then click OK.
• To import selected changes made on a specific version, click the View icon in
that row.
The Hit Analysis Configuration Summary page for that version opens, showing
everything that was in a previously-saved version of the Hit Analysis
configuration.
a Review the changes, then click Import.
The screen changes to a mode that allows you to select the items that you want
to import.
b Select the check box for each of the items you want to import, or select the
Import check box to import all items in the list.
c Click Import Selected Items.
The selected items are now imported and the Hit Analysis Configuration
Change Log is updated to reflect this operation.
• To roll back to any of the previous configuration versions, click the Rollback
icon.
All changes made in the versions you are rolling back are lost. You need to
confirm that you want to roll back that version, before proceeding with the
130 Foglight Experience Viewer
User Guide

operation. The Hit Analysis Configuration Change Log is updated to reflect the
rollback operation.
Note When you use the Rollback option, the entire Hit Analysis configuration is imported
(that is, you cannot choose individual pieces of the Hit Analysis configuration to be
imported).

• To configure the number of versions shown in the Hit Analysis Configuration


Change Log, select one of the available options from the Show Last list (5, 25,
100, or 1000).
The Hit Analysis Configuration Change Log is updated accordingly.

Hit Analysis Import/ Export Configuration


The Export Configuration and Import Configuration buttons on the Hit Analysis
page allow you to perform the following operations, respectively:
• Export the Hit Analysis Configuration to a portable XML format. The resulting
file may be later imported, to recreate the saved Hit Analysis Configuration.
• Import the Hit Analysis Configuration from an external XML file. This feature is
useful to quickly augment or replace existing Hit Analysis Configuration settings.

Important The only things imported/exported by this FxV feature are the settings found on the Hit
Analysis page. Users, user groups, resource groups, custom search screens, etc. are
not part of this export/import.

To export the Hit Analysis Configuration of an FxV system:


1 In the FxV browser interface, on the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Click Export Configuration.
The File Download dialog box appears.

3 To save the Hit Analysis Configuration file, click Save.


Configuring the Hit Analysis Process 131
Hit Analysis Import/ Export Configuration

4 Select a name and a destination directory for the Hit Analysis Configuration file,
then click Save.
The file is now downloaded in the specified directory.
To import an external Hit Analysis Configuration into your FxV system:
1 In the FxV browser interface, on the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Click Import Configuration.
The Import Configuration page appears.

3 Specify the XML Hit Analysis Configuration file to be imported and the path to its
source directory, then click Load File.
132 Foglight Experience Viewer
User Guide

The configuration is validated, and the Hit Analysis Configuration Summary page
opens, displaying all Hit Analysis objects available for import.

4 By selecting one of the options available in the Import Mode list, you can choose
how you want to handle the objects existing in your system when importing an
old configuration:
• Drop current configuration; replace with selected items—nothing from the
currently-active Hit Analysis Configuration is retained. The only possible
exception are the Configuration Options (if you choose not to import the old
settings, the current ones continue to be used).
• Keep current configuration; add/replace selected items only—the current
configuration remains in place. Selected objects from the old configuration are
added to the current configuration. If a selected object exists in both models,
the current version is replaced.
5 Select the check box for each of the items you want to import, or select the
Import check box to import all items in the list.
6 Click Import Selected Items.
The selected items are now imported into your system, and the Hit Analysis
Configuration Change Log is updated to reflect this operation.
7
Hit Analysis Examples

This chapter provides several examples illustrating common uses of Hit Filters and
Transactions Filters that FxV users can define via the FxV browser interface.

This chapter contains the following sections:


“Simple Data Extraction” Example.............................................................................................134
“Event Occurrence Counting” Example .....................................................................................134
“Session-Level Event Occurrence Counting” Example .............................................................135
“Parsing HTML Content” Example.............................................................................................136
“Parameterized Metrics” Example .............................................................................................137
“Buy Tunnel” Example ...............................................................................................................138
134 Foglight Experience Viewer
User Guide

“Simple Data Extraction” Example


A very simple use of Hit Filters is to extract data from a hit and store it as a Custom
Field, either on the hit itself, or on the session. By copying data to Custom Fields, you
make that data available in search results, therefore, you make that data easier to search
for.
This example shows you how to transfer the credit card type from a request field to a
session custom field:
Name: Credit Card Type Extraction
Match Condition: Request Path = /submitcc.do
Custom Field:
Name: Card Type
Storage: Update Session, Show in Search Results
Value Source: Request Field card_type
Populated: Always
Value Assignment Mode: Append value on each match (if unique); Separator: /
Update If Blank: No
In this example, the value of the card_type field is appended to the Card Type custom
field. This would result in a value such as Visa/MC if a customer were to make two
different payments in the same session (using different card types).

“Event Occurrence Counting” Example


Web site owners are often interested in knowing how frequently certain types of events
are occurring on their Web site. For example, they may be interested in counting the
number of times that a certain error page is displayed. Using a simple Hit Filter with a
Metric update, the frequency of any hit-level occurrence can easily be tracked.
This example shows you how to track the occurrence of a specific hit-level event:
Name: Error Page Counter
Match Condition: Request Path contains error
Hit Analysis Examples 135
“Session-Level Event Occurrence Counting” Example

Metric:
Name: Error Pages
Increment By: Fixed Amount (1)
Incremented: Always

“Session-Level Event Occurrence Counting” Example


A different technique is required to count the number of sessions in which some specific
event occurs. If the event can potentially occur more than once per session, the
technique described in the “Event Occurrence Counting” Example section is not
appropriate, since each matching hit would increment the metric.
This example shows you how to modify the Hit Filter defined in the “Event Occurrence
Counting” Example section to count the number of sessions in which at least one error
page occurred. Starting from that Hit Filter, you must remove the metric update (since it
increments the metric each time an error page occurs):
Name: Error Page Counter
Match Condition: Request Path contains error
You also have to define a simple Transaction Filter that starts a transaction as soon as an
error page occurs:
Name: Sessions With Errors
Event Name/Label: Error/E
Allow Multiples: No
Transaction Entry State: Not Started
Event Condition: Hit matches Hit Filter Error Page Counter
Set Transaction Status: No Change
Start/Stop Actions: Start Transaction
Metric:
Name: Error Sessions
Increment By: Fixed Value (1)
Increment Policy: Always increment when transaction stops
136 Foglight Experience Viewer
User Guide

Storage:
Store Transactions: Never
Store Custom Fields: Never
Store Events: Never
Commit Pending Metric Updates: Always
This type of transaction demonstrates the power of Transaction Filters to do more than
just track a series of steps that make up a logical transaction. Transaction Filters are
useful for many types of analysis that involves more than one hit per session.
Notice that in this example you have configured the Transaction Filter to not store
transaction information. The only purpose of the Transaction Filter is to increment the
Error Sessions metric. There is no need to store the transaction as well.
Also notice that the no “stop” condition is necessary for the transaction, which simply
ends when the session ends. This is the expected behavior, since your goal is to count
only one error page per session. This is specified by setting the Allow Multiples option
to No for the one event in this filter.

Note Setting the Allow Multiples option to a value other than No results in each Hit Filter match
incrementing the metric.

“Parsing HTML Content” Example


In situations where the data necessary to evaluate a condition can only be found in the
response content sent from the Web server, the FxV hit analysis process provides
mechanisms to allow this to be done. There are a few key factors to keep in mind when
defining Hit Filters that reference the response content:
• Parsing response content is expensive. While the hit analysis process is designed
to handle this sort of processing, it should not be done excessively. A hit analysis
configuration with a large number of Hit Filters parsing the response content of
every page could reduce the capture capacity of the system.
• When parsing the response content is necessary, keep the pages being parsed to a
minimum. In many cases, this can be accomplished by adding additional match
conditions to reduce the number of pages that have to be parsed.
Suppose the system being monitored has been known to return an error page with the
following response content:
Hit Analysis Examples 137
“Parameterized Metrics” Example

SQL Exception
Database error 12345 has occurred. Please call support.
A simple approach to tracking these sorts of errors might be defined as follows:
Hit Filter Name: SQL Exceptions
Match Conditions: Response Content Contains “SQL Exception”
While this filter would work, it has the unfortunate effect of causing every hit to be
parsed for the string “SQL Exception”. Whenever possible, an additional condition
should be added to reduce the number of pages being scanned. In many cases, the URL
or Request Path can be used to reduce the set of pages potentially containing the text
being searched for. In this example, you can use the HTTP Response Code.
Hit Filter Name: SQL Exceptions
Match Conditions: Response Code = 500 AND Response Content Contains “SQL
Exception”

Important Match conditions are evaluated in the order that they appear. Always make sure that any
evaluation of the response content appears last in your list of match conditions.

“Parameterized Metrics” Example


In addition to updating simple metrics such as “Exception Count”, Hit Filters and
Transaction Filters can also be used to update parameterized metrics. A parameterized
metric is a metric whose name varies based on some piece of data from the captured
data.
This example presents one of the default Hit Filters included with FxV:
Hit Filter Name: Default HTTP Error Codes
Match Conditions: Response Code = 400 OR Response Code > 401
Error Conditions: Every hit matching the Match Conditions will be marked with the
ERROR status.
Metric:
Name: HTTP Response Code $0
$0 Source: Response Code
Increment By: Fixed Amount (1)
138 Foglight Experience Viewer
User Guide

Incremented: Conditional: when the status recorded by this Hit Filter is Error
The key feature of this Hit Filter is the use of $0 in the name of the metric. This is a
special token that the system replaces with the value of the “$0 Source” in order to
create multiple unique metrics. For example, if a hit is captured with a response code of
404, a metric called Http Response Code 404 would be incremented. Note that the “$0
Source” can be just about anything. This example uses the Response Code here, but it
could as well use the value of a request field, some other hit detail such as Country, or
even some piece of data extracted from the page content.

Note When using parameterized metrics, ensure that “$0 Source” has a reasonably small
number of possible values. Using something like the session ID or some other highly
dynamic piece of data may produce hundreds or thousands of distinct values, resulting in
an unmanageable number of distinct metric names.

“Buy Tunnel” Example


This section documents a typical Transaction Filter and its component Hit Filters used
to track a common “buy tunnel” operation (that is, the process an end user goes through
when purchasing a product online). The intent of this example is to show how Hit
Filters and Transaction Filters work together to capture valuable information about a
customer’s online experience. During this example, note how Custom Fields are used as
a tool to transfer hit-level data from Hit Filters to Transaction Filters (and from the
session to the transaction).
This example includes eight Hit Filters, one Transaction Filter, and the definition of a
Global Stop Session Event. For details, see these topics:
• ““Buy Tunnel”: Hit and Transaction Filters Overview” on page 139
• “Transaction Event: Add to Cart” on page 139
• “Transaction Event: Checkout” on page 140
• “Transaction Event: Shipping” on page 141
• “Transaction Event: Payment” on page 141
• “Transaction Event: Confirmation” on page 143
• “Transaction Event: Cart Abandoned” on page 145
• “Hiding Sensitive Data” on page 147
Hit Analysis Examples 139
“Buy Tunnel” Example

“Buy Tunnel”: Hit and Transaction Filters Overview


Start by setting up the following Hit Filters:
• Login—populates a session custom field containing the user’s email address
when a user logs in to the system.
• Add to Cart—triggered when a product is added to the shopping cart.
• Checkout—triggered when a user goes to check out.
• Shipping Selected—triggered when a user chooses a shipping preference.
• Confirmation—triggered when a user is taken to the confirmation page. It updates
a transient custom field (to be used by the Transaction Filter) containing the final
shopping cart value.
• Transaction Complete—triggered when a user finalizes a transaction.
• View Cart—triggered when the user’s shopping cart is displayed. It updates a
session custom field that contains the total value of the items in the cart.
• Logoff—triggered when a user logs out the system.
The “Buy Tunnel” Transaction Filter relies heavily on many of these Hit Filters. The
remainder of this example walks you through the events that make up the Transaction
Filter, providing more detail about each of the Hit Filters as they are encountered.

Transaction Event: Add to Cart


Start tracking a transaction as soon as a user adds something to the shopping cart,
although you can start earlier (for example, as soon as the user starts browsing the
product catalog).
The basic setup for this event is the following:
Event Name/Label: Add to Cart /A
Allow Multiples: No
Transaction Entry State: Not Started
Every time you define an event that starts a transaction, the entry state for the event
must be Not Started (that is because you cannot start a transaction if it is already
started).
The match condition for this event is the following:
Hit matches Hit Filter Add to Cart (with status Any)
140 Foglight Experience Viewer
User Guide

In order to understand this match condition, you must look at the Add to Cart Hit Filter.
This Hit Filter is very simple. Its sole purpose is to trigger the beginning of a
transaction. Its only “action” is to flag a hit as having matched. The match conditions
for this Hit Filter are:
Request Path Ends with /product_info.php AND Request Field action =
add_product
These match conditions would successfully match a URL such as:
http://www.myonlinestore.com/catalog/
product_info.php?action=add_product&products_id=26
Now that you know what it takes to trigger the beginning of this transaction, you can
look at the actions you need to perform in the first event. Since the point of this event is
to mark the beginning of the transaction, you must first configure the system
accordingly. In the Transaction Filter definition page, set:
Start/Stop Actions: Start Transaction
In addition to starting the transaction, you can also keep count of the total number of
transactions that have been started. You can do this by incrementing a metric:
Name: Entered Tunnel
Increment By: Fixed Value (1)
Increment Policy: Always increment when transaction stops

Transaction Event: Checkout


This event is a simple progress check. By adding steps throughout the transaction, you
can gain insight into the process (for example, at what point in the process users who do
not complete their transactions are leaving the “buy tunnel”). The purpose of this step is
to track users who click the Check Out button.
The setup for this event is the following:
Event Name/Label: Checkout / B
Allow Multiples: No
Transaction Entry State: Active
The match condition for this event is the following:
Hit matches Hit Filter Checkout (with status Any)
Hit Analysis Examples 141
“Buy Tunnel” Example

The Checkout Hit Filter has the following match conditions:


Request Path Ends with /checkout_shipping.php AND Response Code = 200
In this case, some occurrences of this filter return a 302 (redirect) response code. These
hits are not interesting for the purpose of this example, so you must specify a specific
HTTP Response Code, to narrow the match conditions.
The Checkout Transaction Event has no impact on the status or state (Not Started/
Active/ Stopped) of the transaction. The only action defined for this event is another
metric increment, allowing you to track the number of users who continue on to this
step:
Name: Checkout Clicked
Increment By: Fixed Value (1)
Increment Policy: Always increment when transaction stops

Transaction Event: Shipping


This event is similar to the Checkout Transaction Event, but with a different path and a
different metric name (Shipping Selected). For more information, see “Transaction
Event: Checkout” on page 140.

Transaction Event: Payment


This event occurs when the user reaches the Order Confirmation (payment) page. The
match conditions for this event are similar to those in the previous steps (see
“Transaction Event: Checkout” on page 140).
However, in this case, the Hit Filter used to trigger this event is going to extract the final
value of the shopping cart (which is, the amount being paid) into a custom field. The
definition of this Hit Filter custom field is the following:
Name: Final Cart Value
Storage: Update Session, Available only for Transaction Filters; Store Values as
Numbers
Value Source: Response Content >Sub-Total:<.*?\$([0-9,]+\.\d\d)
Populated: Always
Value Assignment Mode: Reset value on each match
Update If Blank: No
142 Foglight Experience Viewer
User Guide

The key part of this custom field definition is the value source. The page we are dealing
with contains a table with the following text:
Sub-Total: $69.99
Flat Rate (Best Way): $5.00
Total: $74.99
Your goal is to extract now the Sub-Total (the cost of the items, excluding shipping). In
order to identify how to extract this value from the response content, you need to look at
the HTML source for the page. This particular table is rendered by the following
HTML:
<table border=”0” cellspacing=”0” cellpadding=”2”>
<tr>
<td align=”right” class=”main”>Sub-Total:</td>
<td align=”right” class=”main”>$69.99</td>
</tr>
<tr>
<td align=”right” class=”main”>Flat Rate (Best Way):</td>
<td align=”right” class=”main”>$5.00</td>
</tr>
<tr>
<td align=”right” class=”main”>Total:</td>
<td align=”right” class=”main”><b>$74.99</b></td>
</tr>
</table>
Since there are multiple amounts on the page, you need to take care to extract the one
correct one. The following expression successfully returns 69.99:
>Sub-Total:<.*?\$([0-9,]+\.\d\d)
This expression means:
“Search for the text ‘>Sub-Total:<’ followed by any number of characters, until
you find a ‘$’ followed by one or more digits/commas (the comma allows for
values such as $1,234.56), followed by a decimal point, followed by two more
digits.”
Hit Analysis Examples 143
“Buy Tunnel” Example

Everything after the dollar sign is wrapped in parenthesis, indicating the portion of the
match to be returned as the value of the custom field.

Important Notice the use of the “?” in this regular expression. Placing the “?” after the “.*” means
“match any character, but do not be greedy”.

If you leave off the “?”, the regular expression becomes:


>Sub-Total:<.*\$([0-9,]+\.\d\d)
and it returns the value 74.99, since the “.*” would match all characters until the last
dollar sign on the page.
These sorts of subtle details demonstrate the importance of using the embedded regular
expression tester to validate your expressions against real data.

Transaction Event: Confirmation


The final step in the “buy tunnel” (for successful transactions) occurs when the user
submits the payment information and is redirected to the “Checkout Success” page. This
event is triggered by a very simple Hit Filter, one that matches on a specific path, and
otherwise does nothing. But in this case, the Transaction Event also performs several
important actions when this Hit Filter matches.
The first action the event must perform is to mark the transaction complete:
Set Transaction Status: No Change
Start/Stop Actions: Stop Transaction
Next, you are going to set two transaction custom fields based on data collected during
the processing of the transaction. One of these custom fields, Email, comes from a Hit
Filter that was not discussed yet. At some point during the user’s session, users were
required to log in with their email address. This step is not tracked as part of the
transaction, but a Hit Filter (Login) is set up to extract the user’s email address and to
add it to the session.
The match conditions for this event are the following:
Request Path Ends with /create_account.php OR Request Path Ends with /
login.php
These conditions cover the case where an existing user logs in to the system, as well as
the case where a new user creates an account. Since your Web application uses the same
144 Foglight Experience Viewer
User Guide

request field for the email address on both pages, you are able to use a single Hit Filter
to handle both cases.
This Hit Filter also defines a custom field:
Name: Email
Storage: Update Session, First Column in Search Results; Store Values as Text
Value Source: Request Field email_address
Populated: Always
Value Assignment Mode: Reset value on each match
Update If Blank: No
This custom field becomes part of the session data for this user and appears as the first
column in any session search results screen.
Getting back to the final Transaction Event (Confirmation), you can use this custom
field value to store the user’s email address with the transaction as well. This way, the
value can also be displayed in transaction search results.
The custom field definition in the event is the following:
Name: Email (The name is the same, but it is a different custom field, since it is in
transaction scope here.)
Storage: Update Transaction, First Column in Search Results
Value Source: Session Field—Email
Value Assignment Mode: Reset value each time this event occurs
Update If Blank: No
This technique of copying a value from a session custom field to a transaction custom
field is very common. It allows hit-level data to be transferred to the transaction (which
has no direct access to hit data). In fact, you can use the same technique for one more
custom field:
Name: Total Sale Amount
Storage: Update Transaction, Show in Search Results
Value Source: Session Field—Final Cart Value
Value Assignment Mode: Reset value each time this event occurs
Update If Blank: No
Hit Analysis Examples 145
“Buy Tunnel” Example

Recall that in the payment step (see “Transaction Event: Payment” on page 141), you
extracted the final cart value from the page content and stored it in a transient custom
field (that is, the value was associated with the session for the specific purpose of
moving it over to the transaction at this point; the value is not stored in the database at
the session level).
Finally, you can now also use this event to update two more metrics. The first metric is
similar to the other counters incremented in previous steps (see “Transaction Event:
Add to Cart” on page 139 and “Transaction Event: Checkout” on page 140):
Name: Completed Transactions
Increment By: Fixed Value (1)
Increment Policy: Always increment when transaction stops
The second metric uses a different “Increment By” strategy, to track the total value of all
completed transactions:
Name: Total Value of Completed Transactions
Increment By: Session Custom Field Final Cart Value
Increment Policy: Always increment when transaction stops

Note You use here the same session custom field used to set the Total Sale Amount transaction
custom field to increment a metric by the value of the item(s) purchased.

Transaction Event: Cart Abandoned


This event is very different from the other events that make up the “Buy Tunnel”
Transaction Filter definition. The purpose of this event is to track transactions that are
not successfully completed, in other words, transactions that match the start event (Add
to Cart), but do not match the stop event (Confirmation).
The first important difference in the definition of this event is the entry state:
Transaction Entry State: Stopped
This means that this event can only take place after the transaction has stopped. In order
for this condition to make sense, you need a corresponding match condition for the
event: Transaction stopped due to global stop condition. This special match condition
fires when one of the following occurs:
• The Global Stop Session Event occurs
146 Foglight Experience Viewer
User Guide

• The Global Stop Transaction Event occurs


• The session times out (based on the session timeout specified in the Session
Configuration section of the Hit Analysis screen)
In this example, you are interested in either the Global Stop Session Event occurrence or
a session timeout. Either of these events would indicate that the user never completed
the purchase initiated. To better understand what the Global Stop Session Event means,
you need to look at another Hit Filter (Logoff). All that this Hit Filter does is to mark
hits when they match its simple match condition:
Request Path Ends with logoff.php
The Default Stop Session Event is an event like any other Transaction Event, except that
it is not contained within a Transaction Filter. In our case, it is defined with a very
simple match condition:
Hit matches Hit Filter Logoff
The only purpose of this Default Stop Session Event is to stop the active session (and
therefore any active transactions) when the Logoff Hit Filter matches. The entry state
(N/A) and default action (Stop Session) are not editable on the Default Stop Session
Event.
Getting back to the Cart Abandoned event, you can now see that its match condition of
Transaction stopped due to global stop condition is triggered as a result of a session
timeout or a user logout.
The only thing left now is to look at the Custom Fields and Metrics set by this event.
Before you look at these values, look at another Hit Filter, called View Cart.
The View Cart Hit Filter is not explicitly referenced by the “Buy Tunnel” Transaction
Filter. However, it does set a session Custom Field that you can use to update a Custom
Field or a Metric in the Cart Abandoned event.
When shoppers add items to their cart, they are taken to the /shopping_cart.php page.
This page shows the current contents of their cart, including the total cost. The View
Cart Hit Filter is triggered by hits matching this request path. Much like the Hit Filter
used by the Payment event, it extracts the current value of the shopping cart using a
regular expression against the response content, and stores it in a session Custom Field.
This field always contains the current value of the user’s shopping cart (whether they
are still in the middle of the transaction, they completed it, or they abandoned it). This is
the Custom Field you can use to update the transaction in the Cart Abandoned event.
The Custom Fields set by the Cart Abandoned event are the following:
Hit Analysis Examples 147
“Buy Tunnel” Example

• Lost Sale Amount—this field is set to the value of a session Custom Field called
Cart Value. The intent is to capture the value of the abandoned shopping cart.
• Email—as you did on successful completion of a transaction, you can transfer the
value of the session Custom Field called Email into a transaction Custom Field,
so that both sessions and transactions can be identified by the user’s email
address.
The Metrics set by the Cart Abandoned event are:
• Abandoned Shopping Carts—a simple counter, incremented by “1” each time the
event occurs.
• Total Value of Abandoned Shopping Carts—incremented by the value of the Cart
Value session custom field.

Hiding Sensitive Data


The final item of note in this example is a Sensitive Hit Detail definition used to prevent
FxV users from viewing credit cart numbers entered by users of the Web site. Setting up
a Sensitive Hit Detail is relatively simple once you know the name of the request field
used to transfer the credit card number to the Web server.
In this example, you can navigate to the page on which the credit card number is entered
and view the HTML source for the page. Within the HTML, you would find:
<td class=”main”>Credit Card Number:</td>
<td width=”10”><img src=”images/pixel_trans.gif” border=”0”
alt=”” width=”10” height=”1”></td>
<td class=”main”><input type=”text” name=”cc_number_nh-
dns”></td>
This HTML tells you that the name of the credit card request field is cc_number_nh-
dns, so the Sensitive Hit Detail should be defined as follows:
Hit Detail Type: Request Field
Matching Logic: =
Hit Detail Name: cc_number_nh-dns
Replacement Text: *****HIDDEN*****
Always Sensitive: Yes
148 Foglight Experience Viewer
User Guide

Once this Sensitive Hit Detail is in place, anyone viewing captured sessions (including
sessions captured before the Sensitive Hit Detail was in place) would only see
*****HIDDEN***** in place of the credit card number.
8
Configuring the Analysis
Repository

This chapter presents the operations an FxV user can perform to configure a secondary
database for long-term storage of session and transaction data captured by FxV. This
database is configured via the FxV user interface. The stored data can be accessed via
direct SQL queries or through the Foglight.

This chapter contains the following sections:


Overview....................................................................................................................................150
Architecture ...............................................................................................................................151
Configuration .............................................................................................................................154
Data Transfer.............................................................................................................................160
Access .......................................................................................................................................161
150 Foglight Experience Viewer
User Guide

Overview
The FxV Analysis Repository is a secondary database that offers you long-term storage
of session and transaction data captured by FxV. This add-on feature eliminates the
primary capture database’s restrictions, that is, session and transaction search screens
returning a small number of sessions/transactions, limited FxV capture capacity, and
inability to write report-style queries against the FxV data.
The Analysis Repository provides the following key features:
• Longer term storage of session and transaction data.
• Configurable database structures.
• Simple MySQL database format, accessible:
• From within Foglight, via a Foglight data source (as part of the Cartridge for
Foglight Experience Viewer)
• Via standard SQL query tools, such as Quest’s Toad for MySQL
• Via third-party reporting tools
The Analysis Repository is designed to run on any FxV appliance. For single-appliance
installations with minimal traffic (or for POC installations), the Analysis Repository can
co-exist with the FxV Server and Archiver components, sharing space with the primary
capture database.
Sites with multiple Archivers require a Storage appliance in order to consolidate the
session/transaction data into a single database. The Analysis Repository makes use of
the FxV appliance’s local disk, so the Storage appliance does not necessarily need to
have an attached SAN device. Users with a multi-Archiver installation may enable the
Analysis Repository without adding a Storage appliance to their system. However, in
this configuration each Archiver appliance has its own Analysis Repository containing
only the sessions/transactions captured by that Archiver. Therefore, in order to obtain a
complete view of the Analysis Repository, users would have to perform identical
queries against multiple databases and merge the results.

Important The addition of a Storage appliance is strongly recommended for FxV systems with more
than one Archiver.

The Analysis Repository is disabled by default. Leaving this feature disabled does not
impact the performance of existing FxV systems. Users that plan to enable this feature
must first ensure that their system meets the installation and configuration requirements
Configuring the Analysis Repository 151
Architecture

(for details, see the FxV Installation and Administration Guide), then configure the
database structure (for details, see “Configuration” on page 154).

Architecture
The structure of the Analysis Repository is designed to be very simple. It provides two
types of tables:
• A single session table—contains one row for each captured session, indexed by
the sessionID, startTime, and lastHitTime.
• One transaction table for each Transaction Filter—the tables contain one row
for each captured transaction. The tables are indexed by the sessionID, startTime,
and lastHitTime.
The columns in these tables can contain session/transaction details (location
information, hit counts, etc.), custom field values, metric update values, and Transaction
Event counts. The specific columns defined for each table are user-configurable.

The Session Table


The session table stores information about the sessions captured by FxV. Each row in
this table corresponds to a single session. The columns in this database are user-
configurable with the exception of the following standard columns:
• sessionID—the session identifier for each session.
• startTime—the time at which each session started.
• lastHitTime—the time at which each session ended (this is the timestamp of the
last hit in the session, not the time at which the session timed out).
The user-configurable columns in the session table correspond to any of the following
types of data:
• Session Details—pre-defined details related to the session:
Note The timing values of type DOUBLE are expressed in seconds (and fractions of a
second).

• Browser Type (TEXT)


• City (TEXT)
• Client IP (TEXT)
• Client Time (DOUBLE)
152 Foglight Experience Viewer
User Guide

• Country (TEXT)
• End to End Time (DOUBLE)
• Errors (INTEGER)
• HTML Errors (INTEGER)
• HTML Hits (INTEGER)
• HTML Warnings (INTEGER)
• ISP (TEXT)
• Processing Time (DOUBLE)
• Region (TEXT)
• Stop Reason (TEXT)
• Stored Hits (INTEGER)
• Subnet (TEXT)
• Time of First Error (DATETIME)
• Time of Last Error (DATETIME)
• Total Hits (INTEGER)
• Total Time (DOUBLE)
• Transaction Count (INTEGER)
• Transaction Errors (INTEGER)
• Transaction Warnings (INTEGER)
• Username (TEXT)
• Warnings (INTEGER)
• Hit Filter Matches (INTEGER)—one column for each Hit Filter in the system.
The values indicate the number of hits the Hit Filter matched during the session.
• Custom Fields (TEXT or DOUBLE, depending on the Custom Field type)—one
column for each session-scoped Custom Field.
• Metric Deltas (INTEGER)—one column for each metric. The values indicate
changes to the metric values made within the scope of the session.
Users can choose which of these columns are included in the database structure, on a
column-by-column basis. The column names are user-configurable; each type of
column has a fixed prefix (for example, SDT_ for Session Details, HFM_ for Hit Filter
Matches, etc.), but the core of the column name is specified by the user or populated
with the default names.
Configuring the Analysis Repository 153
Architecture

The Transaction Tables


In addition to the session table, the Analysis Repository may optionally contain an
additional table for each Transaction Filter defined in the system. The rows in these
tables correspond to transactions as defined by the Transaction Filter.
Each transaction table contains the following standard columns:
• sessionID—the session identifier for each transaction.
• startTime—the time at which each transaction started.
• lastHitTime—the time at which each transaction ended (this is the timestamp of
the last hit in the transaction, not the time at which the transaction timed out).
The user-configurable columns in the transaction tables correspond to any of the
following types of data:
• Transaction Details—pre-defined details related to the transaction:
Note The timing values of type DOUBLE are expressed in seconds (and fractions of a
second).

• Client Time (DOUBLE)


• End to End Time (DOUBLE)
• Errors (INTEGER)
• HTML Errors (INTEGER)
• HTML Hits (INTEGER)
• HTML Warnings (INTEGER)
• Processing Time (DOUBLE)
• Status (TEXT)
• Stop Reason (TEXT)
• Stored Hits (INTEGER)
• Time of First Error (DATETIME)
• Time of Last Error (DATETIME)
• Total Hits (INTEGER)
• Total Time (DOUBLE)
• Warnings (INTEGER)
• Transaction Events (INTEGER)—one column for each Transaction Event
defined by the Transaction Filter. The values indicate the number of times the
event occurred within the scope of the transaction.
154 Foglight Experience Viewer
User Guide

• Custom Fields (INTEGER or TEXT, depending on the Custom Field type)—one


column for each transaction-scoped Custom Field.
• Metric Deltas (INTEGER)—one column for each metric. The values indicate
changes to the metric values made within the scope of the transaction (only
changes made by the Transaction Filter are included).
Users have complete control over which Transaction Filters have corresponding
transaction-specific tables in the Analysis Repository and over the types of columns
each of these transaction-specific tables contains. However, the Transaction Details
columns are defined only once, and are applied to all transaction-specific tables. For
example, if you choose to include the Client Time transaction detail, it is added as a
column to every transaction table you define.
Users can choose which of these columns are included in the database structure, on a
column-by-column basis. The column names are user-configurable; each type of
column has a fixed prefix (for example, TDT_ for Transaction Details, TEV_ for
Transaction Events, etc.), but the core of the column name is specified by the user or
populated with the default names.
The names of the transaction-specific tables are user-configurable; they contain a
common prefix (txn_) and a core name specified by the user.

Configuration
Users can configure the Analysis Repository structure via the FxV browser interface.
The configuration process contains two major phases:
1 “Analysis Repository Initial Setup” on page 154
2 “Analysis Repository Detailed Configuration” on page 159

Analysis Repository Initial Setup


The initial setup phase consists of allocating a fixed-size space for the Analysis
Repository in the FxV appliance, that is, defining the maximum number of elements
that the database can contain. This phase is crucial in configuring the Analysis
Repository and must be performed correctly. Once you have allocated the space for this
Configuring the Analysis Repository 155
Configuration

database, any changes to the column counts result in all Analysis Repository data
being lost.

Note Changing the database size and/or the table retention limits does not result in losing the
previously captured data.

To initialize the Analysis Repository:


1 In the FxV browser interface, on the menu bar, click Configure > Hit Analysis.
The Hit Analysis page appears.
2 Expand the Analysis Repository menu options.
Note By default, the Analysis Repository is disabled.

3 Enable the Analysis Repository by setting the Maximum Analysis Repository Size
to a value other than zero.
Important The Maximum Analysis Repository Size defines the maximum size for the
Analysis Repository. Depending on the current size of the repository and the
current growth rate (how quickly the size of the repository is increasing),
decreasing the maximum size may result in the immediate termination of any
queries actively running against the Analysis Repository.

4 Define the maximum number of hours of data that an individual table in the
Analysis Repository can retain, by setting the Maximum Table Retention field.
Distinct tables are created for each segment in the primary capture database, so
this value must be greater than the maximum segment span, as defined in the Hit
Analysis Configuration Options page (see “Archiver Configuration” on
page 126).
Note The default value of zero sets no time limit for the Analysis Repository; in this case,
data is deleted only when the size of the Analysis Repository nears its limit.

5 Define the maximum amount of time (in minutes) that an individual query against
the Analysis Repository is allowed to run, by setting the Maximum User Query
Duration field. Queries running longer than this limit are terminated.
6 Specify the time at which the Analysis Repository maintenance is performed each
day, by setting the Maintenance Time field. At this time, any queries running
against the Analysis Repository are terminated in order to allow routine database
maintenance activities to be performed.
7 Define the maximum number of columns of each data type that can be configured
to store data in the session table, as well as the maximum number of these
columns that can be indexed for improved search performance, by typing in the
156 Foglight Experience Viewer
User Guide

values in the provided fields. For details about this data type, see “Session Table
Data Types” on page 156.
Tip Optimize the table size while also allowing for future growth.

8 Define the maximum number of columns of each data type that can be configured
to store data in each transaction table, as well as the maximum number of these
columns that can be indexed for improved search performance, by typing in the
values in the provided fields. For details about this data type, see “Transaction
Table Data Type” on page 158.
Tip Optimize the table size while also allowing for future growth. All transaction tables will
have the same limits.

9 Click Save Changes.


The Analysis Repository is now enabled and its initial setup phase completed.

Session Table Data Types


A key point when defining the maximum number of columns of each data type is that
the numbers should be set to allow for future database growth. The session table has a
limit of 1,000 total columns. Each transaction table can also have up to 1,000 columns.
Unused columns take up space, so while it is recommended to allocate more columns
than are currently needed, it is also recommended to maintain a low count of unused
columns.
Example
Consider the number of TEXT columns in the session table. A maximum of nine
pre-defined session details can be stored in TEXT columns, if they are turned on.
In addition, one column is needed for each non-numeric session custom field.
Consider that users want to capture all nine session details and 10 non-numeric
custom fields. At a minimum, they need to configure 19 TEXT columns for the
session table. However, users are strongly encouraged to set the value much
higher than 19, to allow for future database growth. Each additional non-numeric
custom field (beyond the original 10) that they want to include will require a new
TEXT column. A value such as 30 might be appropriate in this case.
Configuring the Analysis Repository 157
Configuration

The following table illustrates the type of columns that can be defined in the session
table.

Column Type Column Description

TEXT Defines the maximum number of database columns that can be


configured to store textual data in the session table, as well as the
maximum number of these columns that can be indexed for
improved search performance.
Data stored in TEXT columns includes nine session details
(Browser Type, City, Client IP, Country, ISP, Region, Stop Reason,
Subnet, and Username) and session custom fields configured to
store values as text.

INTEGER Defines the maximum number of database columns that can be


configured to store numeric (integer) data in the session table, as
well as the maximum number of these columns that can be indexed
for improved search performance.
Data stored in INTEGER columns includes nine session details
(Errors, HTML Errors, HTML Hits, HTML Warnings, Stored Hits,
Total Hits, Transaction Count, Transaction Errors, Transaction
Warnings, and Warnings), and metric deltas.

DOUBLE Defines the maximum number of database columns that can be


configured to store decimal (double) data in the session table, as
well as the maximum number of these columns that can be indexed
for improved search performance.
Data stored in DOUBLE columns includes four session details
(Client Time, End to End Time, Processing Time, and Total Time)
and session custom fields configured to store values as numbers.

DATETIME Defines the maximum number of database columns that can be


configured to store timestamp data in the session table, as well as
the maximum number of these columns that can be indexed for
improved search performance.
Data stored in DATETIME columns includes two session details
(Time of First Error and Time of Last Error).
Note No user-configurable data is stored in DATETIME columns. The only
reason to allocate more than two columns of this type would be to
prepare for potential product changes in the future (for example, new
session details).
158 Foglight Experience Viewer
User Guide

In addition to defining the number of columns of each data type, users can also define
the maximum number of these columns that can be indexed. Indexed columns should be
used for data that is expected to commonly appear in search criteria. The session table
has a limit of 60 indices.
Example
If users intend to do a lot of queries based on location, the SDT_CITY, SDT_REGION,
and SDT_COUNTRY columns would be good candidates for indices.

Note Configuring the number of indexed columns of each type also implicitly sets the maximum
number of non-indexed columns of each type. For example, if you allocate 50 INTEGER
columns and 10 of them indexed, the maximum number of non-indexed INTEGER columns
allowed is 40.

Transaction Table Data Type


A key point when defining the column/index counts for transaction tables is that the
counts apply to each table, not all tables in aggregate.
Example
If you define 10 TEXT columns, you can have 10 TEXT columns in each Transaction-
specific table, and they can be different columns.
The following table illustrates the type of columns that can be defined in each
transaction table.

Column Type Column Description

TEXT Defines the maximum number of database columns that can be


configured to store textual data in each transaction-specific table,
as well as the maximum number of these columns that can be
indexed for improved search performance.
Data stored in TEXT columns includes two transaction details
(Status and Stop Reason) and transaction custom fields configured
to store values as text.
Configuring the Analysis Repository 159
Configuration

Column Type Column Description

INTEGER Defines the maximum number of database columns that can be


configured to store numeric (integer) data in each transaction-
specific table, as well as the maximum number of these columns
that can be indexed for improved search performance.
Data stored in INTEGER columns includes seven transaction
details (Errors, HTML Errors, HTML Hits, HTML Warnings,
Stored Hits, Total Hits, and Warnings), transaction event counts,
and metric deltas.

DOUBLE Defines the maximum number of database columns that can be


configured to store decimal (double) data in each transaction-
specific table, as well as the maximum number of these columns
that can be indexed for improved search performance.
Data stored in DOUBLE columns includes four transaction details
(Client Time, End to End Time, Processing Time, and Total Time)
and transaction custom fields configured to store values as
numbers.

DATETIME Defines the maximum number of database columns that can be


configured to store timestamp data in each transaction-specific
table, as well as the maximum number of these columns that can be
indexed for improved search performance.
Data stored in DATETIME columns includes two transaction
details (Time of First Error and Time of Last Error).
Note No user-configurable data is stored in DATETIME columns. The only
reason to allocate more than two columns of this type would be to
prepare for potential product changes in the future (for example, new
transaction details).

Analysis Repository Detailed Configuration


The repository’s detailed configuration phase consists of specifying the elements to be
captured in this database. It assumes that the initial configuration phase is already
completed (for details, see “Analysis Repository Initial Setup” on page 154).
To define the Analysis Repository structure:
1 In the Analysis Repository section of the Hit Analysis page, click Configure
Database Schema.
160 Foglight Experience Viewer
User Guide

The Analysis Repository Configuration page appears, listing all the possible
columns and tables (for Transaction Filters) that can be added to the database.
Initially, all of the column names are blank, which means that no data is stored,
other than the three pre-defined standard columns (see “Architecture” on
page 151). Also, all transaction table names are initially blank, which means that
no transaction filter-specific tables are created.
2 In the provided Column Name fields, type in names for all columns you want to
include in the session and transaction-specific tables.

Alternatively, click the Set to Default icons beside any of these fields to
populate them with their default names.
Note Click the Clear icon beside each of these fields to clear the column name.
Tip The Set to Default and Clear icons at the top of each section allow you to
auto-populate or clear an entire section at once.

3 For improved search performance, configure certain columns to be indexed, by


selecting Indexed from the list associated with them.
4 Click OK.
The Analysis Repository structure is now defined.
Note Columns can be added, removed, or renamed at any time, but the changes only
apply to data loaded into the Analysis Repository after the changes have been
made. For example, if you have three days of data in your Analysis Repository and
realize that you want to add two columns, you can do that. The three days of data
continues to be in the database, but their rows are not updated to populate the new
columns. These columns contain null values for all previously-loaded data.

The configuration of the Analysis Repository is part of the Hit Analysis Configuration,
which can be exported as XML and imported back into the system. This contains all
settings related to the Analysis Repository, including the individual column mappings.
For more information, see “Hit Analysis Import/ Export Configuration” on page 130.

Data Transfer
The Analysis Repository is not populated in real time like the primary capture database.
Instead, database segments are transferred to the Analysis Repository when they close.
The duration of each segment is defined on the Archivers Configuration section in the
Hit Analysis Configuration Options page (see “Archiver Configuration” on page 126).
It can be set to as little as one hour, in which case completed sessions would appear in
the Analysis Repository no later than one hour after completion.
Configuring the Analysis Repository 161
Access

Access
The Analysis Repository is a MySQL database, accessible:
• From within Foglight, via a Foglight data source (as part of the Cartridge for
Foglight Experience Viewer). For more information, see “Accessing the
Analysis Repository from Foglight” on page 161.
• Via standard SQL query tools, such as Quest’s Toad for MySQL. For more
information, see “Accessing the Analysis Repository from Toad for MySQL”
on page 161.
• Via third-party reporting tools.
Access to the Analysis Repository is read-only. You cannot delete rows from the
Analysis Repository tables. Data in the Analysis Repository is removed, as needed,
based on the configured Maximum Analysis Repository Size and Maximum Table
Retention. When data is removed from the Analysis Repository, it is removed according
to database segment boundaries.

Accessing the Analysis Repository from Foglight


Data in the Analysis Repository is accessible from within Foglight, via a Foglight data
source (as part of the Cartridge for Foglight Experience Viewer). For details, see the
Cartridge for Foglight Experience Viewer documentation.

Accessing the Analysis Repository from Toad for MySQL


Data in the Analysis Repository is also accessible via standard SQL query tools (such as
Quest’s Toad for MySQL), by using the settings specified in the following table.

Setting Value/ Description

Host The FxV appliance’s host/IP address.


For multiple-appliance installations, the Analysis Repository is
stored on the Storage appliance.

Port 3306

Account Name fxv


162 Foglight Experience Viewer
User Guide

Setting Value/ Description

Password fxv
This is the default value. To change the password, see
“Appliance Maintenance” on page 168.

Database Name fxv

Table Names session and txn_* (as defined when configuring the database)
Note The session table can be joined with the txn_ tables, but this
process is quite complex. Sessions are identified by their
sessionID, startTime, and lastHitTime (this is because you can
have multiple sessions with the same session ID). To get the
transactions for a specific session, you have to use a WHERE
clause. For example:
WHERE session.sessionID =
txn_xxx.sessionID AND session.startTime <=
txn_xxx.startTime AND session.lastHitTime
>= txn_xxx.lastHitTime

Quest’s Toad for MySQL is freeware. You can download it from Quest’s Web site (http:/
/www.quest.com/toad-for-mysql) and install it on your local machine. For more
information about accessing the FxV Analysis Repository via Toad for MySQL, see the
Quest’s Toad for MySQL documentation set.
To access an Analysis Repository using Toad for MySQL:
1 Open Toad for MySQL.
Note This procedure assumes that you have already downloaded and installed Toad for
MySQL on your local machine.

2 Create a connection to the database by clicking File > New > Connection.
3 In the Create New Connection dialog box define the fields as follows:
• Connection type—select TCP from the list.
• Host—use the IP address of the FxV appliance.
• User—type in fxv.
• Password—type in fxv.
• Database—type in fxv.
• Port—select 3306 from the list.
• Name—type in a name for the database to display (for example, FxV
Analysis Repository).
Configuring the Analysis Repository 163
Access

When the connection is established, the name of the database appears in the
Connection Manager pane.
4 Display the information stored in the FxV Analysis Repository by selecting its
name in the Connection Manager pane, and then clicking Tools > Database
Explorer.
The FxV Analysis Repository tab appears in the Database Explorer pane.
5 Select the Views tab.
The list of session and transaction tables available in this Analysis Repository
appears.

6 Explore the tables by clicking their names in the left pane.


7 View the data stored in each of these tables by clicking the Data tab in the right
pane.
164 Foglight Experience Viewer
User Guide
9
Performing Advanced FxV
Administration

This chapter presents the operations that a user with full administrative privileges can
perform in order to configure and administer the system via the FxV browser interface.

This chapter contains the following sections:


Archivers....................................................................................................................................166
Collectors...................................................................................................................................167
Superuser Tasks........................................................................................................................168
166 Foglight Experience Viewer
User Guide

Archivers
Archivers capture hits collected by Collectors, then apply Hit Filters and Transaction
Filters, and update the capture database for later search and replay. All FxV appliances
have a pre-installed default Archiver.
To view the Archivers already configured in your system:
• In the FxV browser interface, on the menu bar, click Configure > Archivers.
The Archivers page appears, showing all Archivers in the system. You can view,
edit, or remove the existing components, or add new ones. In order to add an
Archiver, the Archiver must be running and must be accessible from the Server.

Depending on your system’s configuration, an administrator may be required to do the


following operations with Archivers:
• Remove the default Archiver from FxV appliances configured as Server, Server/
Storage, or Storage.
• Register a remote Archiver/Storage component with the FxV Server (which
informs the Server where to perform searches).
Note Remote Archiver/Storage refers to those components configured on appliances
other than the Server appliance. One or multiple Archiver and Storage components
can be registered remotely with a Server.

• Register a local Archiver/Storage component with the FxV Server.


Note Each Storage component must be registered with the local Server before it can
begin processing data.

• Test the ability of the FxV Server to communicate with all currently configured
Archivers (by clicking Test Connections).
For detailed instructions on how to configure Archivers in a system via the FxV browser
interface, see the FxV Installation and Administration Guide.
Performing Advanced FxV Administration 167
Collectors

Collectors
The Collector components run on Foglight Experience Monitor appliances, but the
Collector Group configurations (that govern those Collector components) are centrally
defined in the Foglight Experience Viewer Server.
Collector Groups allow load-balancing configuration to be centrally controlled across
all Foglight Experience Monitor and Foglight Experience Viewer appliances. Each
Collector polls the Foglight Experience Viewer Server for load-balancing and security
configuration needed for data capture (including the correct private route for monitored
traffic). Collector Groups are also used to roll-up health metrics, to alert on capture
errors or other data quality problems.
To view the Collector Groups configured in your system:
• In the FxV browser interface, on the menu bar, click Configure > Collectors.
The Collectors page appears, showing the following sections:
• Attached Collectors—lists all the Collectors that have made contact with the
Server.
• Collector Groups—lists all the Collector Groups defined in the system.

For detailed instructions on how to configure Collector Groups in a system, see the FxV
Installation and Administration Guide.
168 Foglight Experience Viewer
User Guide

Superuser Tasks
This section presents the list of “superuser tasks” that an administrator can perform via
the FxV browser interface. For details about these topics, see the following sections:
• “Appliance Maintenance” on page 168
• “Metrics” on page 172

Appliance Maintenance
An administrator can use the buttons available in the Appliance Maintenance section to
perform the following operations, as part of the FxV appliance maintenance:
• Manage SSH—turns on/off the SSH (secure shell) service for all FxV hosts.
Note For detailed instructions on how to enable/disable the SSH access via the FxV
browser interface, see the FxV Installation and Administration Guide.

• Manage Foglight Integration—configures the transmission of metric data to a


remote Foglight 5 Server. For more information, see “Manage Foglight
Integration” on page 169.
• Server Configuration—sets various configuration parameters for your system.
For more information, see “Server Configuration” on page 170.
• Generate Support Bundle—creates a ZIP file that can be sent to the FxV support
team for analysis.
Note For detailed instructions on how to generate a support bundle, see the FxV
Installation and Administration Guide.

• Upgrade Appliance(s)—applies an upgrade to all FxV hosts. Before using this


feature, you must obtain a .runx file necessary for the upgrade. The upgrade menu
gives you the option of applying the upgrade immediately or at a future date/time.
Note For detailed instructions on how to upgrade an older FxV installation to the latest
FxV version, see the FxV Upgrade Field Guide.

• Kill All Analysis Repository User Queries—kills all currently active Analysis
Repository queries.
Note Due to the open nature of the FxV Analysis Repository, it is possible to perform
queries that may take an extremely long time to complete.
Performing Advanced FxV Administration 169
Superuser Tasks

• Reset Analysis Repository Password—resets the password for the fxv MySQL
user account for the Analysis Repository.

Note The two Analysis Repository tasks described in this section are only available if the Analysis
Repository has been enabled.

Manage Foglight Integration


In order to view user-defined metric data (as defined by Hit Filters and Transaction
Filters), the Foglight Experience Viewer Server must be configured to send metrics to a
remote Foglight Management Server. Data collected is pushed to the Foglight
Management Server, where monitored services can be viewed in various formats and
level of details.
The Manage Foglight Integration feature allows end-user metrics to be collected and
analyzed alongside metrics from the backing application, database, and network
services.
To configure the communication with a remote Foglight 5 Server for metric integration:
1 In the FxV browser interface, on the menu bar, click Configure > Superuser
Tasks.
The Superuser Tasks page appears.
2 Click Manage Foglight Integration.
The Foglight Integration page appears.

3 Fill in the following fields:


• Foglight Server IP Address—specifies the IP address of the remote Foglight 5
Server where metrics should be collected. The FxV Server contacts the
170 Foglight Experience Viewer
User Guide

Foglight Server using this IP address, so this address must resolve from the
FxV Server appliance.
If you want to send metrics from FxV to Foglight via SSL, you must specify
the HTTPS/SSL port number for Foglight in the Foglight Server IP Address
field (for example, 10.10.11.1:8443). By default, the HTTPS/SSL port
number is 8443.
To disable Foglight 5 integration, leave the Foglight Server IP Address field
blank.
• Appliance Group Identifier—specifies a unique identifier for the appliance
group managed by this FxV Server.
Note This field needs to be modified only when multiple FxV Servers feed data into a
single Foglight 5 Server.

• FxV Hostname/IP Override—specifies the Hostname or IP address of the


Foglight Experience Viewer (FxV) Server that should be sent to the Foglight 5
Server. This is the Hostname/IP that Foglight 5 must use to create links back to
FxV. Leave this field blank to determine the address automatically.
4 Click OK.
The Foglight 5 integration settings are now configured.

Server Configuration
This feature allows you to set advanced configuration parameters that impact the entire
FxV system.
To set the advanced configuration parameters:
1 In the FxV browser interface, on the menu bar, click Configure > Superuser
Tasks.
The Superuser Tasks page appears.
2 Click Server Configuration.
Performing Advanced FxV Administration 171
Superuser Tasks

The Server Configuration page appears.

3 Fill in the available fields as needed, then click OK.


Note If you want all browser traffic to be secure (over HTTPS), change the SSL
Redirection Enabled setting to Yes. The SSL Redirection Hostname may be left
blank.

The advanced configuration parameters are now set.

Important These settings rarely need to be changed. Exercise extreme caution when making any
changes, as the impact to the system may be significant.
172 Foglight Experience Viewer
User Guide

Metrics
A metric is an instance of time-series data used to indicate and track health or
performance factors, and to trigger actions when thresholds are breached. Examples of
metrics include the value of abandoned shopping carts per day in dollars, or the total
size of the database in TB.
During hit processing and analysis, the FxV Archiver performs several actions,
including updating the metrics for each hit. The metric updates are recorded in the
database for later searching.
The FxV Server maintains all metrics related to user sessions. It also pushes metric data
up to a remote Foglight Management Server, if configured to do so. For more
information, see “Manage Foglight Integration” on page 169.
The FxV browser interface allows the administrator to perform the following operations
with metrics:
• View Metrics—view system health metric data collected for the Foglight
Experience Viewer.
• Run Metric Based Diagnostics—analyze, capture, and replay health metrics over
the past 24 hours (user-defined metrics can only be viewed in the Foglight
Management Server). The resulting report lists any problems found and
recommends solutions for each of these problems.
• Reset Metric History—delete all metric history.
Important Metric history cannot be recovered once deleted.

To view the FxV metrics:


1 In the FxV browser interface, on the menu bar, click Configure > Superuser
Tasks.
The Superuser Tasks page appears.
2 Click View Metrics.
The Metrics page appears, showing the collected system health metrics organized
in the following four groups:
• Errors/Warnings
• Archivers
• Collectors
Performing Advanced FxV Administration 173
Superuser Tasks

• Server
Note For a description of metrics that can be viewed in the FxV browser interface, see
“Appendix: FxV Metrics” on page 199.

3 Review the metrics collected for the FxV appliance by navigating through the
available tabs. Use the Hide or Show icons to hide or show (respectively) an
entire category.
Up to seven icons may appear to the right of any given metric value:
• Move your mouse pointer over this icon to view the timestamp that the
Metric’s value was last updated.
• Move your mouse pointer over this icon to quickly view a graph of the
metric’s history over the last 60 minutes.
• Click this icon to view the Metric History. This screen displays a large
graph of the metric’s history and provides options for displaying alternate time
ranges.
• Click this icon to open the Alarm Log for the metric. This screen shows all
alarms (that is, any time the metric’s status changed) for the metric.
Note This icon is only displayed for metrics that have alarm conditions defined.

• Click this icon to view the alarm conditions defined for the metric.
Note This icon is only displayed for metrics that have alarm conditions defined.

• Click this icon to delete the metric.


Note This option is only displayed when the system determines the metric is no longer
being updated.
174 Foglight Experience Viewer
User Guide

To run a metric-based diagnostic on the FxV metrics collected during the last 24 hours:
1 In the FxV browser interface, on the menu bar, click Configure > Superuser
Tasks.
The Superuser Tasks page appears.
2 Click Run Metric Based Diagnostics.
The Diagnostics page appears. The resulting report lists any problems found and
recommends solutions for each of these problems.
Note Only certain predefined health metrics are checked by these diagnostics. Custom
metrics defined by hit or transaction filters are not included.
10
Managing User Account Settings

This chapter presents the settings that the FxV browser interface provides to FxV
administrators for managing the FxV users and the permissions they have when using
the FxV resources.
The following diagram illustrates the relationship between the concepts presented in
this chapter.
Figure 1

This chapter contains the following sections:


Users and User Groups.............................................................................................................176
Resource Groups ......................................................................................................................183
Preference Groups ....................................................................................................................184
176 Foglight Experience Viewer
User Guide

Users and User Groups


Each person who logs into the FxV system requires a distinct User account. Each User
belongs to one or more User Groups. For detailed information about Users, see the
following sections:
• “User Preference Settings” on page 176
• “Pre-Defined User Accounts” on page 176
• “Managing User Accounts” on page 177
User Groups define permissions for FxV users logging into the FxV browser interface
and accessing system resources. For detailed information about User Groups, see the
following sections:
• “User Group Settings” on page 178
• “Pre-Defined User Groups” on page 181
• “Managing User Groups” on page 182

User Preference Settings


The default user preference settings for a User account are determined by the Preference
Group to which the user belongs. The Preference Group also defines which settings can
be overridden by the user.
Each User can belong to only one Preference Group. The user preference settings
consist of Session Explorer Settings (which allow you to change the behavior of the
Replay browser interface) and Search Settings (which allow you to change the behavior
of the Search pages).
For more information about preference settings, see “Setting User Preferences” on
page 71 and “Preference Groups” on page 184.

Pre-Defined User Accounts


Two Users are pre-defined and delivered by default with new FxV systems:
• admin—defined as part of the administrators user group, for which all
administrative privileges are enabled (except for “Foglight Auto-Login” and
“Sensitive Data Display”). When logged in as this user, you can manage user
accounts, user groups, preference groups, and resource groups.
Managing User Account Settings 177
Users and User Groups

• guest—defined as part of the guests user group, for which all administrative
privileges are disabled (except for “Hit Content Replay”). When logged in as this
user, you can only use the resources offered by the FxV browser interface for hit,
session, transaction, and custom searches.

Note From the system’s perspective, these default users are not special in any way. They can be
modified or deleted from the system like any other user.

Managing User Accounts


To manage a User account:
1 Log into the FxV browser interface.
Note To manage User accounts, you must log into FxV as a user who is in a User Group
configured with the “Users” capability.

2 On the menu bar, click Configure > Users.


The Users page appears, displaying the list of existing users.
3 Do one of the following:
• To view an existing user account, click the View icon.
• To edit an existing user account, click the Edit icon.
Editing includes defining the user’s account settings (Last Name, First Name,
Email, Preference Group, and User Groups), changing user’s password (click
Change Password, then fill in the information requested), and managing
resources owned by the user (see “To manage the resources owned by a User:”
on page 178).
• To remove a user account from the system, click the Delete icon.
• To disable/enable a user account, click the Disable / Enable icon.
Note The Disabled icon appears beside a User name when it is disabled.

• To create a new user account, click Create User, then fill in the information
requested.
Note A “personal” User Group and a “personal” Resource Group are automatically
created for each new User in the system. Each personal group corresponds to a
single user and cannot be deleted.
178 Foglight Experience Viewer
User Guide

• To disable all user accounts defined in the system, click Disable All Users,
then confirm that you want to disable all users in the system, except yours.
Note The disabled users are automatically logged out of the system and are treated as
though they do not exist. The Disabled icon papers beside the User names
when they are disabled.

• To enable all user accounts defined in the system, click Enable All Users,
then confirm that you want to enable all users currently disabled in your
system.
Note Depending on the state of the user and your privileges, some of these icons may not
appear for certain Users. Specifically, users who own resources cannot be deleted
from the system. To remove such a user, first edit it, click View Resources, and
either delete or re-assign ownership of any resources owned by this user.

To manage the resources owned by a User:


1 Log into the FxV browser interface.
Note To manage User accounts, you must log into FxV as a user who is in a User Group
configured with the “Users” capability.

2 On the menu bar, click Configure > Users.


The Users page appears, displaying the list of existing users.

3 Click the Edit icon beside the User for which you want to manage resources.
The User page appears, displaying the fields available for editing.
4 Click View Resources.
The User Resources page appears, displaying the list of resources owned by the
selected User, or a warning message in the case the User does not own a particular
type of resource.
5 Do one of the followings:
• To re-assign the ownership of a resource, click the Change Owner icon.
• To change the Resource Groups to which a resource belongs to, click the
Share icon.
• To delete a resource from the list of owned resources, click the Delete icon.
Note Depending on the state of the resource and your privileges, some of these icons
may not appear for certain resources.

User Group Settings


For each User Group, the following settings must be configured:
Managing User Account Settings 179
Users and User Groups

• Administrative Privileges—determines the administrative permissions available


to the users of the system. Most of these privileges correspond to the options in
the Configure menu on the FxV menu bar. They are grouped in the following
categories:
• User and Resource Group Management—allows users in this User Group to
manage User accounts, User Groups, and Resource Groups, as specified in the
following table.

Setting Name Setting Description

Resources Manages individual resources (that is, deletes


resources or re-assigns ownership of resources).

User & Resource Manages User Groups and Resource Groups.


Groups

Users Creates, edits, and deletes User accounts.

• System Configuration—allows users in this User Group to administer the FxV


system (that is, running system diagnostics, resetting the capture database,
upgrading the appliance, etc.), as specified in the following table.

Setting Name Setting Description

Archivers Configures the data capture appliances by


managing the Archivers in your system.

Collector Groups Configures the data capture appliances by defining


the Collector Groups in your system.

Edit Hit Analysis Allows the users in this User Group to edit the Hit
Analysis settings.

Sensitive Data Configures the sensitive data settings (Sensitive


Hit Details and Sensitive response Content
Expressions).
180 Foglight Experience Viewer
User Guide

Setting Name Setting Description

Superuser Performs FxV superuser tasks, which include


managing SSH, configuring the Server, generating
support bundle, upgrading the appliance,
importing/exporting configurations, viewing
metrics, etc.

View Hit Analysis Allows the users in this User Group to view the Hit
Analysis settings.

• Other Privileges—allows users in this User Group to manage other


miscellaneous privileges, as specified in the following table.

Setting Name Setting Description

Content Replay Zip Exports user sessions as ZIP files.


Export

Foglight Auto-Login Enables users in this User Group to automatically


log in to the FxV appliance when drilling down
from Foglight, without the need for re-
authentication.

Hit Content Replay Displays the Content Replay view when exploring
a session of interest.

Sensitive Data Display Makes visible to users sensitive data captured by


the system, flagged as “not always sensitive”. For
more information, see “Sensitive Data Protection”
on page 121.
Note Enabling the “Sensitive Data Display” option
negates the protection provided by any Sensitive
Data definitions in the system not marked as
“Always Sensitive”. This feature must be used
only for very specific users, who are trusted to
view such data. These users must be added to a
User Group created specifically for this purpose.

• Members—defines the list of users who are part of a User Group.


Managing User Account Settings 181
Users and User Groups

• Resource Groups—defines the Resource Groups to which a User Group is linked.


For detailed information about resources and Resource Groups, see “Resource
Groups” on page 183.
Each User Group can be linked to any number of Resource Groups. These links
determine which resources the users who belong to the User Group can access.
For each Resource Group linked to this User Group, you can specify the
privileges that members of this User Group have with respect to the resources in
that Resource Group, as follows:
• Create—users allowed to create new resources in the corresponding Resource
Group.
• Edit Owned—users allowed to edit resources in the corresponding Resource
Group if they are the owner of the resource.
• Remove Owned—users allowed to delete resources in the corresponding
Resource Group if they are the owner of the resource.
• Edit Not Owned—users allowed to edit resources in the corresponding
Resource Group even if they are not the owner of the resource.
• Remove Not Owned—users allowed to delete resources in the corresponding
Resource Group even if they are not the owner of the resource.
• View—users allowed to view resources in the corresponding Resource Group.
Granting “view access” allows users to view resource definitions without
being able to edit the resources.
• Use—users allowed to use resources in the corresponding Resource Group.
Granting use access allows users to use resources that they cannot otherwise
access.

Pre-Defined User Groups


Four User Groups are pre-defined and delivered by default with new FxV systems:
• administrators—group for which all administrative privileges are enabled (except
for “Foglight Auto-Login” and “Sensitive Data Display”). The admin user is pre-
defined as part of this group.
• guests—group for which all administrative privileges are disabled (except for
“Hit Content Replay”). The guest user is pre-defined as part of this group.
• Personal UG:admin—”personal” group automatically created when the admin
user is defined. This group can be deleted only by deleting the admin user.
182 Foglight Experience Viewer
User Guide

• Personal UG:guest—”personal” group automatically created when the guest user


is defined. This group can be deleted only by deleting the guest user.

Note From the system’s perspective, these default user groups are not special in any way. They
can be modified or deleted from the system like any other user group.

Managing User Groups


To view, edit, delete, or create a new User Group:
1 Log into the FxV browser interface.
Note To manage User Groups, you must log into FxV as a user who is in a User Group
configured with the “User & Resource Groups” capability.

2 On the menu bar, click Configure > User Groups.


The User Groups page appears, displaying the list of existing User Groups.
3 Do one of the following:
• To view additional information about a User Group, click the View icon.
• To edit a User Group, click the Edit icon.
The editing page allows you to add or remove users from the User Group, but
also to add, edit, and remove links from the User Group to the Resource
Group.
• To remove a User Group from the system, click the Delete icon.
• To create a new User Group, click Create User Group, then fill in the
information required.
a Fill in the Group Name (must be unique).
b Enable the privileges necessary for this User Group by selecting the
appropriate check boxes in the Administrative Privileges. For details about
administrative privileges, see “User Group Settings” on page 178.
c From the Non-members list, using the >> and << buttons, move into the
Members list the Users who should be part of this User Group.
Note The Non-members list originally contains all User accounts already defined for
this system.
Managing User Account Settings 183
Resource Groups

d Add Resource Groups that should be linked to this User Group, and specify
the privileges users should have with respect to the resources in these groups.
For details about resource privileges, see “User Group Settings” on page 178.
Note Depending on the state of the User Group and your privileges, some of these icons
may not appear for certain User Groups.

Resource Groups
A Resource Group is a collection of resources that can be associated with one or more
User Groups, in order to customize the users’ access to those resources.
The following types of resources can be created in the system: Custom Search Screens
and Saved Searches (for more information about these topics, see “Creating and Editing
Custom Search Screens” on page 67 and “Creating and Editing Saved Searches” on
page 66).
Each resource is owned by a single User, but it can be associated with one or more
Resource Groups. For different User Groups, you can grant/restrict the access privileges
to these resources. For more information about these topics, see “Users and User
Groups” on page 176.
Three Resource Groups are pre-defined and delivered by default with new FxV
systems:
• Default—group that associates all resources with the two pre-defined User
Groups (administrators and guest).
• Personal RG:admin—“personal” group automatically created when the admin
user is defined. This group is deleted only when the admin user is deleted.
• Personal RG:guest—“personal” group automatically created when the guest user
is defined. This group is deleted only when the guest user is deleted.

Note From the system’s perspective, these default resource groups are not special in any way.
They can be modified or deleted from the system like any other resource group.

To view, edit, delete, or create a new Resource Group:


1 Log into the FxV browser interface.
Note To manage Resource Groups, you must log into FxV as a user who is in a User
Group configured with the “User & Resource Groups” capability.

2 On the menu bar, click Configure > Resource Groups.


184 Foglight Experience Viewer
User Guide

The Resource Groups page appears, displaying the list of existing resource
groups.
3 Do one of the following:
• To view additional information about a Resource Group, click the View
icon.
• To edit the information of a Resource Group, click the Edit icon.
The editing page allows you to define the type of resources that can be placed
in this Resource Group (by selecting the corresponding Allowed Resource
Types check boxes), to view the list of resources associated to this Resource
Group (by clicking View Resources), and to control the mapping between the
User Group and Resource Groups (that is, specify the privileges users should
have with respect to the resources in the Resource Groups linked to any
number of User Groups).
• To remove a Resource Group from the system, click the Delete icon.
Note Each “personal” Resource Group is linked with the user’s personal User Group
and cannot be deleted.

• To create a new Resource Group, click Create Resource Group, then fill in
the information requested.
Note Depending on the state of the Resource Group and your privileges, some of these
icons may not appear for certain Resource Groups.

Preference Groups
A Preference Group is a group of FxV Users with common default user preference
settings. Each User belongs to exactly one Preference Group.
Within a group, you can:
• Manage the list of Users assigned to the Preference Group.
• Define which values can be changed by the Users in the group, by selecting/
clearing the lock check box beside a particular setting in the Session Explorer
Settings or the Search Settings lists. Settings that are locked cannot be overridden
by individual Users. Settings that are unlocked serve as defaults, and may be
changed by individual Users. Locking a setting overrides immediately any user-
specified values.
Managing User Account Settings 185
Preference Groups

For a detailed description of Session Explorer Settings and Search Settings


options, see “Session Explorer Settings” on page 72 and “Search Settings” on
page 75.
Only one Preference Group (Default Preferences) is pre-defined and delivered by
default with new FxV systems.
To view, edit, delete, or create a new Preference Group:
1 Log into the FxV browser interface.
Note To manage Preference Groups, you must log into FxV as a user who is in a User
Group configured with the “User & Resource Groups” capability.

2 On the menu bar, click Configure > Preference Groups.


The Preference Groups page appears, displaying the list of existing Preference
Groups.
3 Do one of the following:
• To view additional information about a Preference Group, click the
View icon.
• To edit a Preference Group, click the Edit icon. Editing includes moving
users into this group.
• To remove a Preference Group from the system, click the Delete icon.
Note Preference Groups that have users associated with them cannot be deleted.

• To create a new Preference Group, click Create Preference Group, then fill
in the information required.

Important Users cannot be explicitly removed from a Preference Group.

To remove users from a Preference Group:


• Add the users to a different Preference Group.
The users are then automatically removed from the Preference Group in which
they were previously members.
186 Foglight Experience Viewer
User Guide
A
Appendix: Java Regular
Expressions in FxV Hit Analysis

Regular expressions allow you to identify and extract data during several steps of the
FxV Hit Analysis process. The details of how they are used varies slightly depending on
the circumstance.
This appendix examines these differences thoroughly. For details, see these topics:
Regular Expressions Overview .................................................................................................188
Regular Expressions in FxV ......................................................................................................192
FxV Regular Expression Tester .................................................................................................198
188 Foglight Experience Viewer
User Guide

Regular Expressions Overview


Before looking at the specifics, it is important to understand the basic concepts that
apply to all regular expression usage in FxV. The purpose of a regular expression is to
define a pattern that can match any number of similar strings. For example, consider a
Web page that contains the following text, which may be slightly different, depending
on the active user:
Username: joe
Username: john
Username: sarah
A regular expression can be used to extract the unique portion of the text from each
page. In this case, the expression might look like this:
Username: (\w+)
Here, the next “Username: ” is matched exactly, followed by one or more letters or
digits (the “\w” stands for “letter or digit” and the “+” represents “one or more”). The
parenthesis around “\w+” indicate that the text matched by that part of the expression is
the text to be returned.
The following table lists many of the most common characters used when writing
regular expressions. Note that this table presents two types of examples:
• Example 1—the intent is to show that for the boolean question “does this
expression match this input”, the answer would be “yes”:
expression matches sample-text
• Example 2—the intent is to show that for the request “return the portion of the
input text that matches this expression”, the answer would be the result:
expression returns result for input sample-text

Characters Description Examples

Anything other than Non-special characters that Dog matches Dog


[\^$.|?*+() match the exact text specified.

\ (backslash) followed Matches the special character, 5\+3=8 matches 5+3=8


by any one of ignoring its special meaning.
[\^$.|?*+()
Appendix: Java Regular Expressions in FxV Hit Analysis 189
Regular Expressions Overview

Characters Description Examples

\d Matches any digit (0-9). \d\d matches 12


abc\d matches abc5

\w Matches any letter or digit. \w\w\w matches 123


\w\w\w matches abc
\w\w\w matches a2c

\s Matches any whitespace a\sZ matches a Z


character (spaces, tabs, new
lines, etc.).

\D, \W, \S Negated versions of \d, \w, \D\D matches AB


and \s. That is, they match \W\W matches @#
non-digits, non-word- \S\S\S matches a1#
characters, and non-
whitespace.

. Matches any single character. a.c matches abc


a.c matches a2c
a.c matches a c

^ Matches at the start of the ^foo matches foobar


string being evaluated (start of ^foo does not match
line). barfoo

$ Matches at the end of the bar$ matches foobar


string being evaluated (end of bar$ does not match
line). barfoo

| Matches what is on the left or a|b|c matches a


right of the “pipe” character. a|b|c matches b
a|b|c does not match d

? Makes the preceding item ab? matches a


optional. This optional item is ab? matches ab
included in the match if it
exists.
190 Foglight Experience Viewer
User Guide

Characters Description Examples

* Matches zero or more of the a.*a returns abaca for


preceding item. As much data input bbabacabb
as possible is included in the a.*a matches aa
match. For more information,
see “Greedy vs. Reluctant
Expressions” on page 191.

*? Matches zero or more of the a.*?a returns aba for


preceding item. As little data input bbabacabb
as possible is included in the a.*?a matches aa
match. For more information,
see “Greedy vs. Reluctant
Expressions” on page 191.

+ Matches one or more of the a.+a returns abaca for


preceding item. As much data input bbabacabb
as possible is included in the a.+a does not match aa
match. For more information,
see “Greedy vs. Reluctant
Expressions” on page 191.

+? Matches one or more of the a.+?a returns aba for


preceding item. As little data input bbabacabb
as possible is included in the a.+?a does not match
match. For more information, aa
see “Greedy vs. Reluctant
Expressions” on page 191.

{n} Matches exactly n of the z{3} matches zzz


preceding item.

{n,m} Matches between n and m of z{2,4} matches zz


the preceding item. As much z{2,4} matches zzz
data as possible is included in z{2,4} matches zzzz
the match.For more z{2,4} returns zzz for
information, see “Greedy vs. input zzz
Reluctant Expressions” on
page 191.
Appendix: Java Regular Expressions in FxV Hit Analysis 191
Regular Expressions Overview

Characters Description Examples

{n,m}? Matches between n and m of z{2,4}? returns zz for


the preceding item. As little input zzz
data as possible is included in
the match. For more
information, see “Greedy vs.
Reluctant Expressions” on
page 191.

Greedy vs. Reluctant Expressions


The distinction between “*?” and “*” (and “+?” and “+”) is subtle, but important. The
following example illustrates the main difference between these concepts. Consider the
following HTML page:
<HTML><BODY>Your username is <B>wilma</B> and your password
is <B>bambam</B></BODY></HTML>
An initial attempt to extract the username from this page might look like this:
username is <B>(.*)</B>
This would seem to say “Look for the text ‘username is’, and then extract the value
between the <B> and </B> tags.”. This is exactly what the expression does. The
problem is that “.*” is a “greedy” operator, meaning that it matches as much as it can. In
this case, the result is:
wilma</B> and your password is <B>bambam
What happened is that the expression found “username is <B>”, and then looked for the
last occurrence of “</B>”. Everything in between is returned as the result.
In order to extract just the username, you need to use a “reluctant” expression:
username is <B>(.*?)</B>
The result you get now is the one you were actually looking for:
wilma
192 Foglight Experience Viewer
User Guide

Regular Expressions in FxV


Regular expressions can be used in FxV in several places. Depending on the purpose,
the type of expression needed varies slightly. The following sections outline these
differences:
• “Looking for “Matches” (Yes or No)” on page 192
• “Extracting a Single Value” on page 193
• “Identifying Sensitive Response Content” on page 195

Looking for “Matches” (Yes or No)


Regular expressions can be used to answer questions of the form:
Does some piece of data match some expression.
In this case, the job of the regular expression is to return a boolean result: yes (the data
matched the expression), or no (it did not).
There are two places where this type of expressions are used within Hit Analysis: Hit
Filter match conditions and Sensitive Hit Details.

Using “Matches” in Hit Filter Match Conditions


When defining a Hit Filter match condition, several operators are available for
selection. One of these operators is “Matches”.
When you select “Matches”, the text specified after the operator is expected to be a
regular expression. For example:
Request Field phoneNum Matches \d{3}-\d{3}-\d{4}
This expression considers a hit to have matched if it contains a request field called
phoneNum with a value such as 303-555-1212 (three digits, a dash, three digits, a dash,
four digits). It does not match values such as 3035551212 or (303)555-1212.

Using “Matches” in Sensitive Hit Detail Specifications


When defining a Sensitive Hit Detail, several “Matching Logic” operators are available
for selection. One of these operators is “Matches”.
When you select “Matches”, the text specified as the Hit Detail Name is expected to be
a regular expression. For example:
Appendix: Java Regular Expressions in FxV Hit Analysis 193
Regular Expressions in FxV

Request Field matches ^cc\d+$


This expression considers a request field to be sensitive if the name of the field starts
with the letters “cc”, and ends with one or more digits (with nothing else between the
“cc” and the digits). Fields such as cc1 and cc500 are considered sensitive while fields
such as xcc1, cc500z, and ccx500 are not.

Extracting a Single Value


It is often useful to extract some unique piece of data from a data source that contains
additional, unwanted text. For example, you might want to extract the numeric value
123.45 from the text “Total Cost: $123.45”.
This type of operation can be performed anywhere in the Hit Analysis UI where the
characters “/ /” appear. This includes Custom Field value assignment, Metric value
assignment, and Hit Filter match conditions.

Note This operation is different than using the “Matches” operator in a Hit Filter match condition.

When extracting a value in this manner, the regular expression must contain a set of
parenthesis that indicates the portion of the expression to be returned. Expressions
defined without any parenthesis do not work.

Note Only one set of parenthesis may be used in this context. Specifying an expression such as
(\d+)xyz(\d+) does not return data from both sets of parenthesis. Only the data matching the
first set of parenthesis (“Group 1”) is returned.

Assigning Custom Field or Metric Values


When defining the value source for a Custom Field or Metric, click the icon “/ /” to
specify a regular expression that should extract some portion of the value source rather
than the entire value.
For example, suppose a Request Field called “total” contains values such as 123.45USD
and 5.00USD, and you want to extract only the numeric portion of these values into a
numeric Custom Field. After selecting the “total” field as the value source, the
following expression would extract the desired data:
(\d*\.\d\d)
194 Foglight Experience Viewer
User Guide

This expression means “match zero or more digits, a period, and two more digits.” The
entire expression is wrapped in parenthesis, indicating that the entire value must be
returned.
The following expression can also be used for extracting the same desired data:
(\d*\.\d\d)USD
As long as all values are followed by the letters “USD”, this expression would produce
the same results as the expression in the first example.
Similarly, the expression:
\d*\.\d\d(\S{3})
could be used to extract the three-letter currency code into a “currency” custom field.
This expression can be read as “match zero or more digits followed by a period, two
more digits, and three more non-whitespace characters. Return only the three non-
whitespace characters.”.
The expression:
(\S{3})
would not produce the desired results, as it would return only the 123 for the input data
123.45USD.

Extracting a Portion of a Hit Detail in a Hit Filter Match Condition


The same technique can be used to extract a portion of a hit detail when defining a Hit
Filter match condition. Only the extracted portion of the data is used when evaluating
the selected operator. For example, a match condition could be defined like this:
Client IP / ^(\d+)\.\d+\.\d+\.\d+$ / = 192
The text between the slashes is entered by clicking the icon “/ /”, and filling in the text
box that appears. In this case, the expression is looking for an IP address (one or more
digits, a period, one or more digits, a period, one or more digits, a period, one or more
digits). Since only the first series of digits is surrounded by parenthesis, that portion of
the address is used to evaluate the rest of the match condition. In this example, hits
would match if the first number in the Client IP address is 192.
Appendix: Java Regular Expressions in FxV Hit Analysis 195
Regular Expressions in FxV

Identifying Sensitive Response Content


Another way to use regular expressions in the Hit Analysis process is as part of a
Sensitive Response Content Expression. In this case, the regular expression is used to
identify the portion(s) of the response content considered “sensitive”.
There are a few aspects of this type of expression that are unique to this particular use.

Multiple Matches
A Sensitive Content regular expression is matched repeatedly until the end of the page
being processed is reached. For example, consider the following HTML page:
<HTML><BODY>Your credit card number is 4123123412341234 and
my credit card number is <B>4111222233334444</b></BODY></
HTML>
The following regular expression would identify both credit card numbers as sensitive:
\d{16}
This expression means “Look for 16 consecutive digits. Once you find a match, start
looking again.”. An expression like this does not require any wrapping parenthesis since
everything that matches the expression is considered sensitive.
However, it is not always possible to define an expression like this that matches
everything that is sensitive, without also matching non-sensitive data. For example,
consider the following HTML page:
<HTML><BODY>Account:123-4567<BR>Phone:303-555-1212</BODY></
HTML>
and the expression:
\d{3}-\d{4}
This would match both 123-4567 and 555-1212. If the goal is to mark the account
number as sensitive, but not the phone number, this expression is not sufficient. Instead,
additional context should be provided and parenthesis should be used to identify the
sensitive portion of the expression:
Account:(\d{3}-\d{4})
This expression is explicitly looking for the text “Account:” followed by an account
number matching the expected format. Since the phone number is not preceded by
“Account:”, it is not a match. The parenthesis indicate that only the account number
itself (not the text “Account:”) should be considered sensitive.
196 Foglight Experience Viewer
User Guide

Group Numbers
Data matched by Sensitive Response Content Expressions is returned in one or more
“groups”. Expressions that do not contain parenthesis have only one group: Group 0.
This group contains everything returned each time the expression matches. For
example:
Expression: \d\d
Content: 12 34 foobar 56
Group 0 (match 1): 12
Group 0 (match 2): 34
Group 0 (match 3): 56
When one or more set of parenthesis are included in the expression, each set of
parenthesis identifies a new “group”. Group 0 represents the entire expression. For
example:
Expression: (\d)(\w)
Content: 1a 3b 5c
Group 0 (match 1): 1a
Group 1 (match 1): 1
Group 2 (match 1): a
Group 0 (match 2): 3b
Group 1 (match 2): 3
Group 2 (match 2): b
Group 0 (match 3): 5c
Group 1 (match 3): 5
Group 2 (match 3): c
Any time you use parenthesis in a Sensitive Response Content expression, it is
important to ensure that the appropriate group(s) are marked as sensitive. The most
common case is to define a single group (Group 1) and mark it as sensitive. For
example:
Regular Expression: Password=<b>(\w+)</b>
Content: Password=<b>trinity</b>
Sensitive Groups: 1
Appendix: Java Regular Expressions in FxV Hit Analysis 197
Regular Expressions in FxV

In this case, only the password itself (the text between the <b> and </b> tags) would be
considered sensitive. However, if Group 0 were specified rather than Group 1,
everything matching the expression (Password=<b>trinity</b>) would be considered
sensitive.

Multiline Matching
There are subtle details related to evaluating regular expressions against text that spans
multiple lines. Consider the following HTML page:
<HTML>
<BODY>
<B>
Hello1
World2
Goodbye3
</B>
</BODY>
</HTML>
The character “$” matches the end of a line, not the end of the document. The
expression:
World(\d)$
returns 2 for Group 1.
The wildcard operator (*) does not stop at the end of a line. The expression:
<B>(.*)</B>
returns for Group 1:
Hello1
World2
Goodbye3

Note The behavior described in this section corresponds to the Java regular expression flags
DOTALL and MULTILINE. This note is addressed to readers familiar with Java regular
expressions (or those using a Java regular expression tester other than the one provided in
the FxV browser interface).
198 Foglight Experience Viewer
User Guide

FxV Regular Expression Tester


The FxV browser interface has a built-in expression tester, which can be accessed by
clicking the icon (.?) besides the Regular Expression text boxes, wherever they appear
in the Hit Analysis interface. The regular expression tester runs in three slightly
different modes, depending on the context:
• When opened from a “matches” context, it returns either “true” or “false”, to
indicate whether or not the given expression matches the specified test input.
• When opened from an “extract value” context, it requires the presence of
matching parenthesis, and returns the data that would be extracted by these
parentheses.
• When opened from the Sensitive Response Content Expression screen, it returns
a summary of all matches and all groups (including Group 0). In this context, to
ensure that the desired data is being properly extracted, it is recommended to
copy the entire HTML source from a sample page, and use it as the sample search
text.
B
Appendix: FxV Metrics

Foglight Experience Viewer monitors two types of metrics:


• System metrics—can be viewed in FxV browser interface and in Foglight.
• User-defined metrics—can only be viewed in Foglight.
This appendix provides the complete list of FxV system metrics and contains the
following topics:
Archiver Metrics.........................................................................................................................200
Collector Metrics........................................................................................................................234
Server Metrics ...........................................................................................................................236
Error/Warning Metrics................................................................................................................241
200 Foglight Experience Viewer
User Guide

Archiver Metrics
You can select an Archiver to display its individual metrics, or “Totals” to display total
metrics across all Archivers in the system.

Note Certain metrics are displayed only for individual Archivers.

The Archiver metrics include the following categories:


• “Hit Summary Metrics” on page 200
• “Analysis Repository Metrics” on page 208
• “Archiver Capacity Metrics” on page 210
• “Archiver Health Metrics” on page 215
• “Archiver Appliance Metrics” on page 224
• “Archiver Search Usage Metrics” on page 228
• “Script Metrics” on page 233

Hit Summary Metrics


This Metric Category provides statistics about all hits captured and analyzed by the
system. These Metrics are used primarily to troubleshoot capture and data-quality
problems.

Note These metrics do not reflect the health of your Web applications, only the capture/replay
system itself. Collector metrics show health of the capture system from the Foglight
Experience Monitor viewpoint.

The following metrics are provided in this category:

Metric Name Displayed in


Totals

% Of Hits Discarded No

% Of Hits Dropped By Hit Filter No

% of Hits With Errors Yes


Appendix: FxV Metrics 201
Archiver Metrics

Metric Name Displayed in


Totals

% of Hits With Warnings Yes

Exceptions - Capture Error No

Exceptions - Chunked Transfer Error No

Exceptions - Connection Reset No

Exceptions - Content Dropped By Archiver No

Exceptions - Content Dropped By Collector No

Exceptions - Content Dropped By Filter No

Exceptions - Corrupted Response No

Exceptions - Incomplete Content No

Exceptions - Missing Property No

Exceptions - No Response No

Exceptions - No Session ID No

Exceptions - Timeout No

Exceptions - Truncated Response No

Exceptions - Unknown Exception No

Hits Discarded Yes

Hits Dropped By Hit Filter Yes

Hits Failed Yes

Hits Processed Yes

Hits With Corrupt Cookie Names No


202 Foglight Experience Viewer
User Guide

Metric Name Displayed in


Totals

Hits With Corrupt Request Field Names No

Hits With Corrupt Request Header Names No

Hits With Corrupt Response Header Names No

Hits With Errors Yes

Hits With Warnings Yes

% Of Hits Discarded
Reports percentage of hits sent to a Capture Appliance but discarded before processing
could complete. Each Archiver has a limited amount of memory to buffer hits before
processing and, when this buffer fills up, incoming hits are discarded.
% Of Hits Discarded = Hits Discarded / Hits Processed

% Of Hits Dropped By Hit Filter


Reports percentage of hits intentionally dropped by Hit Filters in order to conserve disk
space. If close to 100%, it may be that a Hit Filter has inadvertently been created to drop
all incoming hits.
% Of Hits Dropped By Hit Filter = Hits Dropped By Hit Filter / Hits Processed

% of Hits With Errors


Reports percentage of hits marked as errors by a Hit Filter. All hits are considered OK
until an error is detected.
% of Hits With Errors = Hits With Errors / Hits Processed

% of Hits With Warnings


Reports percentage of hits marked as warnings by a Hit Filter. All hits are considered
OK until an error is detected.
% of Hits With Warnings = Hits With Warnings / Hits Processed
Appendix: FxV Metrics 203
Archiver Metrics

Exceptions - Capture Error


Reports number of hits that were not captured properly. This indicates hits that were
incomplete or corrupted, but no other error condition was present, such as a connection
reset or timeout. This metric is automatically updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Chunked Transfer Error


Reports number of hits using chunked encoding that could not be de-chunked due to
invalid formatting. This metric is automatically updated prior to hit analysis. These hits
were processed normally, however the content may not display properly in session
playback.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Connection Reset


Reports number of hits that were terminated by a connection reset by either the client or
the server. Due to the abnormal termination of the hit, not all hit details, including
content, may have been captured. This metric is automatically updated prior to hit
analysis.

Note To find these, do a hit search on the Exception property for this exception type.

Exceptions - Content Dropped By Archiver


Reports number of non-text hits whose response content was dropped due to the fact
that the captured content did not match the Content-Length header. This metric is
automatically updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.
204 Foglight Experience Viewer
User Guide

Exceptions - Content Dropped By Collector


Reports number of non-text hits whose response content was dropped due to the fact
that the captured content exceeded the maximum response size defined for this
Collector. This metric is automatically updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Content Dropped By Filter


Reports number of hits whose content was dropped by a hit filter rule. This metric is
automatically updated upon completion of hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Corrupted Response


Reports number of hits that had a corrupt response. A corrupt response is usually due to
either response headers that could not be parsed or compressed content that could not be
inflated. This metric is automatically updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Incomplete Content


Reports number of text hits whose captured content did not match the Content-Length
header. This metric is automatically updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Missing Property


Reports number of hits that were missing one or more required properties. The required
properties are: URL, Method, and ClientIP. This metric is automatically updated prior to
hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.
Appendix: FxV Metrics 205
Archiver Metrics

Exceptions - No Response
Reports number of hits for which no response was captured. This is often caused by a
connection reset or timeout. Due to the abnormal termination of the hit, not all hit
details, including content, may have been captured. This metric is automatically updated
prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - No Session ID
Reports number of hits that were not sessionized. This metric is automatically updated
prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Timeout
Reports number of hits that timed out. A timeout is determined by the Monitor when the
server does not respond within the TCP timeout period. This metric is automatically
updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Exceptions - Truncated Response


Reports number of text hits whose response content was truncated due to the fact that
the captured content exceeded the maximum response size defined for this Collector.
This metric is automatically updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.
206 Foglight Experience Viewer
User Guide

Exceptions - Unknown Exception


Reports number of hits that had an unknown exception. This metric is automatically
updated prior to hit analysis.

Note To find these metrics, do a hit search on the Exception property for this exception type.

Hits Discarded
Reports number of hits sent to a Capture Appliance but discarded before processing
could complete. Each Archiver has a limited amount of memory to buffer hits before
processing, and when this buffer fills up incoming hits are thrown away.

Note Discarded hits are not counted as Hits Processed. Discards can also occur at the Collector
before hits are sent.

Hits Dropped By Hit Filter


Reports number of hits intentionally dropped by Hit Filters in order to conserve disk
space. It is not uncommon to drop hits in static site areas, or hits for non-HTML
resources (like images or style sheets).

Hits Failed
Reports the number of hits that could not be processed due to an Archiver runtime
failure. The logs on the Capture Appliance should have further detail on these failures.

Note Discarded hits are not counted as Hits Processed.

Hits Processed
Reports number of hits captured by a Collector, sent to a Capture Appliance, analyzed,
and processed successfully.
Not all of these hits were necessarily written to the capture database for later search and
replay: some hits may have been dropped by a Hit Filter, or discarded while under load,
or in rare cases may have failed to process successfully.

Note A steady increase in this value indicates steady incoming traffic from attached Collectors.
Appendix: FxV Metrics 207
Archiver Metrics

Hits With Corrupt Cookie Names


Reports number of hits that had corrupt cookie names. This metric is automatically
updated prior to hit analysis. These hits were processed normally, however the invalid
details were not written to the capture database.

Note To find these metrics, do a hit search on the Corrupt Hit Details custom field using a value of
“*” and the field value will indicate the detail type affected.

Hits With Corrupt Request Field Names


Reports number of hits that had corrupt request field names. This metric is
automatically updated prior to hit analysis. These hits were processed normally,
however the invalid details were not written to the capture database.

Note To find these metrics, do a hit search on the Corrupt Hit Details custom field using a value of
“*” and the field value will indicate the detail type affected.

Hits With Corrupt Request Header Names


Reports number of hits that had corrupt request header names. This metric is
automatically updated prior to hit analysis. These hits were processed normally,
however the invalid details were not written to the capture database.

Note To find these metrics, do a hit search on the Corrupt Hit Details custom field using a value of
“*” and the field value will indicate the detail type affected.

Hits With Corrupt Response Header Names


Reports number of hits that had corrupt response header names. This metric is
automatically updated prior to hit analysis. These hits were processed normally,
however the invalid details were not written to the capture database.

Note To find these metrics, do a hit search on the Corrupt Hit Details custom field using a value of
“*” and the field value will indicate the detail type affected.
208 Foglight Experience Viewer
User Guide

Hits With Errors


Reports number of hits marked as errors by a Hit Filter. All hits are considered OK until
an error is detected.

Hits With Warnings


Reports number of hits marked as warnings by a Hit Filter. All hits are considered OK
until an error or warning is detected.

Analysis Repository Metrics


This Metric Category contains all Metrics for the Analysis Repository.
The following metrics are provided in this category:

Metric Name Displayed in


Totals

Age of Latest Data No

Age of Oldest Data No

Analysis Repository Current Size Yes

Analysis Repository Maximum Size No

Maintenance Cycles Yes

Queries Canceled Yes

Segments - Last Load Time No

Segments - Loaded Yes

Segments - Waiting To Load Yes

Total Sessions Yes

Total Transaction Tables Yes

Total Transactions Yes


Appendix: FxV Metrics 209
Archiver Metrics

Age of Latest Data


Reports the age (in hours) of the most recent data in the Analysis Repository. Capture
segments are loaded after they close. This metric is normally between 0 and the number
of hours that a capture segment stays open.

Age of Oldest Data


Reports the age (in hours) of the oldest data in the Analysis Repository.

Analysis Repository Current Size


Reports the current size of the Analysis Repository. This metric increases throughout
the day as each capture segment is loaded. Once the Analysis Repository fills up, data is
removed during the daily Maintenance Cycle.

Analysis Repository Maximum Size


Reports the maximum size of the Analysis Repository as specified in the Analysis
Repository Configuration.

Maintenance Cycles
Reports the number of Maintenance Cycles that have been performed. Maintenance is
performed each day at the time specified in the Analysis Repository Configuration. This
metric is reset to 0 after each restart, and is primarily used to indicate when the
Maintenance Cycle was performed.

Queries Canceled
Reports the number of user queries that were canceled. A query is canceled if it exceeds
the maximum query time as specified in the Analysis Repository Configuration, or if it
is active at the start of a Maintenance Cycle.

Segments - Last Load Time


Reports the number of seconds used to load the last capture segment into the Analysis
Repository.

Segments - Loaded
Reports the number of capture segments loaded into the Analysis Repository. This is a
daily total and is reset to 0 at midnight each day.
210 Foglight Experience Viewer
User Guide

Segments - Waiting To Load


Reports the number of capture segments waiting to be loaded into the Analysis
Repository. It is normal to see a small number of capture segments waiting to load from
time to time. However, if this metric stays above 0 for a long time, this indicates that the
capture segments are closing faster than they can be loaded. This may result in some
capture segments not being loaded into the Analysis Repository.

Total Sessions
Reports the total number of sessions in the Analysis Repository.

Total Transaction Tables


Reports the total number of transaction tables in the Analysis Repository. There is one
transaction table for each Transaction Filter selected in the Analysis Repository
Configuration.

Total Transactions
Reports the total number of all transactions in the Analysis Repository. This total
includes transactions for all Transaction Filters that have been selected in the Analysis
Repository Configuration.

Archiver Capacity Metrics


This Metrics Category provides sizing information based on hits written to the capture
database. These Metrics are primarily used for capacity planning, some have alarm
conditions defined by default.

Note These metrics do not reflect the health of your Web applications, only the capture/replay
system itself.

The following metrics are provided in this category:

Metric Name Displayed in


Totals

% Of Hits With Content Keyword Indexing No

% Of HTML Hits No
Appendix: FxV Metrics 211
Archiver Metrics

Metric Name Displayed in


Totals

Available Database Aspects No

Available Database Hits Yes

Available Database Segments Yes

Available Database Sessions Yes

Average Content Size No

Average HTML Content Size No

Capture Database Size Yes

Capture Rate Yes

Content Uniqueness No

Database Growth Rate Yes

HTML Content Uniqueness No

Open Segment Aspects No

Raw Capture Rate Yes

Storage Database Size No

% Of Hits With Content Keyword Indexing


Reports percentage of hits in the open database segment that were written to the content
keyword index, as directed by a Hit Filter.

Note Indexing a large percentage of hits can negatively impact system performance and increase
disk use.
212 Foglight Experience Viewer
User Guide

% Of HTML Hits
Reports percentage of hits in the open database segment that were recognized as HTML
hits.
An HTML hit is defined as a hit with a Content-Type response header starting with text/
html, containing a proper HTML tag in the response content, and having a response
code of 200.

Available Database Aspects


Reports number of aspects across all database segments.
Aspects are created dynamically as needed to track fields, cookies, headers, and other
searchable database objects.

Note A large number of aspects can degrade search performance and may indicate a data-
quality problem.

Available Database Hits


Reports number of hits in all segments of the capture database. This is the total count of
hits available for search and replay.

Note Hits are removed from the database as segments expire.

Available Database Segments


Reports number of database segments currently used in the capture database. Each
segment contains a time span of captured data.

Note More segments are required for more replay history, but a large number of segments can
degrade search performance. Segments are deleted as they expire.

Available Database Sessions


Reports number of sessions in all segments of the capture database. This is the total
count of sessions available for search and replay.

Note Sessions are removed from the database as segments expire.


Appendix: FxV Metrics 213
Archiver Metrics

Average Content Size


Reports average size of unique response content for non-HTML hits, calculated for the
open database segment.
A Non-HTML hit is defined as a hit with a Content-Type response header other than
text/html* or without a proper HTML tag in the response content. These include images
and style sheets.

Average HTML Content Size


Reports average size of unique response content for HTML hits, calculated for the open
database segment.
An HTML hit is defined as a hit with a Content-Type response header starting with text/
html, containing a proper HTML tag in the response content, and having a response
code of 200. These include HTML page hits from static sites and dynamic Web
applications.

Capture Database Size


Reports total size of all database segments currently stored in the Capture Appliance
databases. If there are no Storage Appliances, this is the total volume of data available
for searching and replay. If a storage tier is present, this measures just the short-term
capture portion of the total database.

Note The capture database size grows as data is collected, and shrinks as old segments expire
and are dropped over time.

Capture Rate
Reports hits per second received from all attached Collectors over the last two minutes.
This measures the number of hits written to the capture database for later searching.

Content Uniqueness
Reports percentage of non-HTML hits in the open database segment that have unique
values. The higher the uniqueness value, the more disk space is used.
A Non-HTML hit is defined as a hit with a Content-Type response header other than
text/html* or without a proper HTML tag in the response content.
214 Foglight Experience Viewer
User Guide

Database Growth Rate


Reports change in the size of the open database segment database over the last two
minutes. This measures the volume of data being written to the Capture Appliance
disks.

HTML Content Uniqueness


Reports percentage of HTML hits in the open database segment that have unique values.
The higher the uniqueness value, the more disk space is used.
An HTML hit is defined as a hit with a Content-Type response header starting with text/
html, containing a proper HTML tag in the response content, and having a response
code of 200.

Open Segment Aspects


Reports number of aspects in the open database segment.
Aspects are created dynamically as needed to track fields, cookies, headers, and other
searchable database objects.

Note A large number of aspects can degrade search performance and may indicate a data-
quality problem.

Raw Capture Rate


Reports raw hit traffic received from all attached Collectors over the last two minutes.
This measures the total network bandwidth consumed between the Collectors and the
Archiver.

Storage Database Size


Reports total size of all database segments currently maintained by Storage Appliances.
If there are no Storage Appliances, this value will be zero. Otherwise, this measures the
total size of the long-term storage database.

Note The storage database grows over time as data is collected and eventually saturates at its
maximum capacity.
Appendix: FxV Metrics 215
Archiver Metrics

Archiver Health Metrics


This Metric Category provides information about Archiver internal health. These
Metrics are primarily used to troubleshoot capture problems, some have alarm
conditions defined by default.

Note These metrics do not reflect the health of your Web applications, only the capture/replay
system itself.

The following metrics are provided in this category:

Metric Name Displayed in


Totals

Batch Queue Count No

Batch Queue Limit No

Cached Items No

Clock Skew from Server No

Config Available No

Hit Analysis - Custom Fields Active No

Hit Analysis - Failures Yes

Hit Analysis - Rules Fired No

Hit Analysis - Session Hit Filters Active No

Hit Analysis - Session Timeouts No

Hit Analysis - Sessions Active Yes

Hit Analysis - Sessions Completed No

Hit Analysis - Transaction Hit Filters Active No

Hit Analysis - Transactions Active Yes


216 Foglight Experience Viewer
User Guide

Metric Name Displayed in


Totals

Hit Analysis - Transactions Completed No

JVM Free Memory No

JVM Total Memory No

JVM Used Memory No

Refresh Metric Time No

Segment Transfer Attempts No

Segment Transfer Completions No

Segments Closing No

Segments Copying No

Segments Dropped Before Transfer Yes

Segments Waiting to Copy No

Segments Waiting to Transfer No

Shared Memory Size No

Shared Memory Used No

TCPBatches JVM Free Memory No

TCPBatches JVM Total Memory No

TCPBatches JVM Used Memory No

Write Buffers Failed Yes


Appendix: FxV Metrics 217
Archiver Metrics

Batch Queue Count


Reports the number of raw hits waiting to be processed. Each Archiver has a limited
number of hits that it can buffer before processing.

Note The Archiver queue may often be at the limit. The TCPBatches process buffers incoming
hits in shared memory and feeds these raw hits to the Archiver for processing.

Batch Queue Limit


Reports total number of raw hits that can be stored prior to processing. Each Archiver
has this limited number of hits that it can buffer.

Note This metric value does not change while the Archiver is running. It is needed for legacy
reasons.

Cached Items
Reports number of unique data items currently held in the Archiver memory cache. A
very large value may indicate a data quality problem or extreme load.

Clock Skew from Server


Reports an estimated number of milliseconds that the clock of the Capture Appliance
running this Archiver and the Foglight Experience Viewer Server are skewed.

Config Available
Reports whether configuration polling back to the Server was successful on the last
attempt. When an Archiver is added to a Server, it begins to contact the Server
periodically for configuration settings.

Important When an Archiver is restarted, it must be able to contact the Server successfully once
before it is able to capture incoming hits.
218 Foglight Experience Viewer
User Guide

Hit Analysis - Custom Fields Active


Reports number of custom field facts defined by all active Sessions and Transactions.
One fact is created for each custom field value.

Note Larger values indicate increased memory usage.

Hit Analysis - Failures


Reports number of times an internal failure has occurred in the rules engine. Any
failures are most likely triggered by a recent change in the Hit Analysis Configuration.
If failures have been reported, export your current configuration and send it to the
Foglight Experience Viewer support team.

Hit Analysis - Rules Fired


Reports number of times a Hit Filter match condition or Transaction Filter event
condition was true.

Note Larger values indicate increased CPU time spent on hit analysis. Creating hit filters or
transaction filters tends to increase this value.

Hit Analysis - Session Hit Filters Active


Reports number of hit filter facts defined by all active Sessions. One fact is created for
every session for every hit filter defined.

Note Larger values indicate greater memory usage. Creating hit filters tends to increase this
value.

Hit Analysis - Session Timeouts


Reports number of completed sessions that stopped due to a timeout rule rather than the
more specific Default Session Stop Condition.
Appendix: FxV Metrics 219
Archiver Metrics

Hit Analysis - Sessions Active


Reports number of session facts currently active in the Archiver rules engine used for
hit analysis. This value increases as new session IDs are captured, and decreases as
sessions stop. This indirectly measures the number of concurrent client browsers.

Note Larger values indicate increased memory usage and higher concurrency. A large number of
sessions may also indicate timeouts that are set too short.

Hit Analysis - Sessions Completed


Reports number of sessions that stopped and were written to the capture database.

Hit Analysis - Transaction Hit Filters Active


Reports number of hit filter facts defined by all active Transactions. One fact is created
for every transaction for every dependent hit filter type matched.

Note Larger values indicate increased memory usage.

Hit Analysis - Transactions Active


Reports number of transaction facts currently active in the Archiver rules engine used
for hit analysis. This value increases as start events are met for defined Transaction
Filters, and decreases as transactions stop.

Note Larger values indicate increased memory usage and CPU time spent on hit analysis.
Creating transaction filters tends to increase this value.

Hit Analysis - Transactions Completed


Reports number of transactions (across all Transaction Filters) that stopped and were
written to the capture database.
220 Foglight Experience Viewer
User Guide

JVM Free Memory


Reports unused memory within the Java Virtual Machine (JVM) running the Archiver.
This value should oscillate as normal “garbage collection” occurs. A consistently low
amount of free memory may indicate a problem.

Note The amount of free memory can change very rapidly as the JVM automatically resizes the
heap.

JVM Total Memory


Reports total heap memory allocated to the Java Virtual Machine (JVM) running the
Archiver. The JVM automatically resizes the size of the heap in order to keep “garbage
collection” times to a minimum.

Note This value can increase suddenly when a spike in capture traffic occurs. It decreases
gradually over periods of relative inactivity.

JVM Used Memory


Reports amount of memory currently in use within the Java Virtual Machine (JVM)
running the Archiver. This value should oscillate as normal “garbage collection” occurs.
A consistently high amount of used memory may indicate a problem.

Note The amount of used memory can change very rapidly as the JVM automatically resizes the
heap.

Refresh Metric Time


Reports time required to recalculate dynamic metrics, including: Available Database
Aspects/Hits/Segments/Sessions, Capture Rate, Database Size, and all metrics in the
Archiver Appliance category.

Note These dynamic metrics values are updated in the background every two minutes. A large
sustained value indicates the appliance is under heavy load.
Appendix: FxV Metrics 221
Archiver Metrics

Segment Transfer Attempts


Reports number of times this Archiver tried to transfer a database segment to/from a
remote Storage Appliance. Since transferring segments involves a bidirectional
handshake, attempts are tracked on both the Capture Appliance and Storage Appliance.

Note This value is zero if segment transfer is not enabled. Otherwise, the value should increase
linearly as attempts are made.

Segment Transfer Completions


Reports number of times this Archiver successfully transferred a segment to/from a
remote Storage Appliance. Since transferring segments involves a bidirectional
handshake, attempts are tracked on both the Capture Appliance and Storage Appliance.
This value is zero if segment transfer is not enabled. Otherwise the value should equal
Segment Transfer Attempts, if no retries or errors occurred.

Segments Closing
Reports number of segments currently being closed by the Archiver.

Note This value should be between zero and one when the Archiver is healthy.

Segments Copying
Reports number of segments currently being copied to a remote Storage Appliance.
Copying the segment is the most expensive and longest phase of the segment transfer
process.

Note This value should be between zero and two when the Archiver is healthy.

Segments Dropped Before Transfer


Reports number of times a database segment was deleted from the local disk before it
could be transferred to a remote Storage Appliance.

Note If segment transfer is enabled, this value should be zero, unless errors have occurred.
Otherwise, this value increases linearly.
222 Foglight Experience Viewer
User Guide

Segments Waiting to Copy


Reports number of database segments that have been closed and flushed, but where the
copy to the remote Storage Appliance has not yet started.

Note If segment transfer is enabled, this value should be small, unless errors have occurred.
Otherwise, this value increases linearly.

Segments Waiting to Transfer


Reports number of database segments that have been copied to the remote Storage
Appliance, but where the initialization and transfer has not yet completed.

Note If segment transfer is enabled, this value should be small, unless errors have occurred.
Otherwise, this value increases linearly.

Shared Memory Size


Reports total size of shared memory which is used to store raw hits prior to being sent to
the Archiver for processing. Each TCPBatches has a limited number of hits that it can
buffer.

Note This metric value does not change while the Archiver is running.

Shared Memory Used


Reports the amount of shared memory consumed by raw hits waiting to be processed.
Each TCPBatches has a limited amount of memory to buffer hits before being sent to
the Archiver for processing, and when this buffer fills up TCPBatches stops reading
batches from the network which may cause hits to be discarded.

Note TCPBatches should normally use very little shared memory, but short periodic bursts are
normal. A sustained value above 25% of the shared memory size indicates the Capture
Appliance is having difficulty keeping up with incoming traffic.
Appendix: FxV Metrics 223
Archiver Metrics

TCPBatches JVM Free Memory


Reports unused memory within the Java Virtual Machine (JVM) running TCPBatches.
This value should oscillate as normal “garbage collection” occurs. A consistently low
amount of free memory may indicate a problem.

Note The amount of free memory can change very rapidly as the JVM automatically resizes the
heap.

TCPBatches JVM Total Memory


Reports total heap memory allocated to the Java Virtual Machine (JVM) running
TCPBatches. The JVM automatically resizes the size of the heap in order to keep
“garbage collection” times to a minimum.

Note This value can increase suddenly when a spike in capture traffic occurs. It decreases
gradually over periods of relative inactivity.

TCPBatches JVM Used Memory


Reports amount of memory currently in use within the Java Virtual Machine (JVM)
running TCPBatches. This value should oscillate as normal “garbage collection” occurs.
A consistently high amount of used memory may indicate a problem.

Note The amount of used memory can change very rapidly as the JVM automatically resizes the
heap.

Write Buffers Failed


Reports the number of times a write buffer was lost due to an internal error. The support
bundle includes all the error details.
224 Foglight Experience Viewer
User Guide

Archiver Appliance Metrics


This Metrics Category provides information about the health of the Foglight Experience
Viewer hardware running the Archiver (and the Server if one is configured). These
Metrics are used to troubleshoot hardware and software problems.

Note These metrics do not reflect the health of your Web applications, only the capture/replay
system itself.

The following metrics are provided in this category:

Metric Name Displayed in


Totals

Buffer Memory Used No

Cache Memory Used No

CPU I/O Wait Percent No

CPU Idle Percent No

CPU System Percent No

CPU User Percent No

Database Cache Blocks Available No

Database Cache Blocks Used No

Database Dirty Cache Blocks No

Database Read Cache Hit Rate No

Database Write Cache Hit Rate No

Filesystem Quest Used Percent No

Filesystem Root Used Percent No

Filesystem Tmp Used Percent No


Appendix: FxV Metrics 225
Archiver Metrics

Metric Name Displayed in


Totals

Filesystem Var Used Percent No

Free Memory No

Processes Blocked No

Processes Running No

RAID Array Degraded No

Swap Memory Used No

Buffer Memory Used


Reports amount of memory used by the appliance operating system to buffer disk
writes.

Note This value is typically low even in high-volume installations. Archivers rely upon write
buffering at the RAID controller, so there should be almost no write buffering in the
operating system.

Cache Memory Used


Reports amount of memory used by the appliance operating system for disk read
caching.

Note The operating system uses free memory as a disk cache to improve I/O performance. A
significant amount of memory (50% or more) may be allocated to disk caching, or this value
may be as low as one-eighth of total memory.
226 Foglight Experience Viewer
User Guide

CPU I/O Wait Percent


Reports percentage of CPU time spent waiting for disk operations, averaged over a five-
second period when the Metric was last updated.

Note This value can spike up to 70%+ for very short periods of time (one observation), but should
usually be 10% or less. A consistent high value indicates a drive failure or serious software
problem.

CPU Idle Percent


Reports percentage of CPU time spent idle, averaged over a five-second period when
the Metric was last updated.

Note A consistent low value indicates the appliance is heavily loaded.

CPU System Percent


Reports percentage of CPU time spent running system processes, averaged over a five-
second period when the Metric was last updated.

Note This value should normally be 10% or less.

CPU User Percent


Reports percentage of CPU time spent running user processes, averaged over a five-
second period when the Metric was last updated. These user processes include the
Archiver and Server JVM processes.

Note A consistent high value indicates the appliance is heavily loaded.

Database Cache Blocks Available


Reports the number of block currently available in the database key buffer cache.

Database Cache Blocks Used


Reports the maximum number of blocks that were allocated at one time from the
database key buffer cache.
Appendix: FxV Metrics 227
Archiver Metrics

Database Dirty Cache Blocks


Reports the number of modified blocks in the database key buffer cache that have not
been flushed to disk.

Database Read Cache Hit Rate


Reports the read hit rate in the database key buffer cache.

Database Write Cache Hit Rate


Reports the write hit rate in the database key buffer cache.

Filesystem Quest Used Percent


Reports percentage of space used on the “/quest” filesystem.

Filesystem Root Used Percent


Reports percentage of space used on the “/” filesystem.

Filesystem Tmp Used Percent


Reports percentage of space used on the “/tmp” filesystem.

Filesystem Var Used Percent


Reports percentage of space used on the “/var” filesystem.

Free Memory
Reports amount of appliance operating system memory that is currently not used for any
purpose.

Note It is normal to have almost no free memory, since the operating system tends to use any
available memory either for disk read caching or write buffering.
228 Foglight Experience Viewer
User Guide

Processes Blocked
Reports total number of processes that are waiting to run, averaged over a five-second
period when the metric was last updated.

Note This value may be consistently zero for lightly loaded systems. Under heavy load, this value
oscillates normally as I/O operations are synchronized. A consistent high value may
indicate a threading problem.

Processes Running
Reports total number of processes currently running, averaged over a five-second period
when the Metric was last updated.

Note This value may be small for lightly loaded systems. Under heavy load, this value oscillates
normally as I/O operations are synchronized.

RAID Array Degraded


Reports true if the performance of the appliance RAID array is currently degraded. This
could be due to a disk failure, or because the RAID array is currently initializing or
repairing in the background.

Swap Memory Used


Reports amount of temporary swap memory used by the appliance operating system.

Note This value should always be zero, Archivers are configured to never swap.

Archiver Search Usage Metrics


This Metrics Category provides information about searches and other remote calls
between the Server and the Archivers. These metrics indicate how heavily the FxV
browser interface is being used.

Note These metrics do not reflect the health of your Web applications, only the capture/replay
system itself.
Appendix: FxV Metrics 229
Archiver Metrics

The following metrics are provided in this category:

Metric Name Displayed in


Totals

Browser Types Calls No

Cities Calls No

Collector IDs Calls No

Content Calls No

Cookies Calls No

Countries Calls No

Fields Calls No

Hit Detail Calls No

Hit Detail Calls (empty) No

Hit Detail Calls (slow) No

HTML Titles Calls No

ISPs Calls No

Local Configuration Calls No

Metrics Calls No

Oldest Hits Calls No

Paths Calls No

Regions Calls No

Request Headers Calls No

Response Headers Calls No


230 Foglight Experience Viewer
User Guide

Metric Name Displayed in


Totals

Search Hits Calls No

Search Hits Calls (empty) No

Search Hits Calls (slow) No

Search Sessions Calls No

Search Sessions Calls (empty) No

Search Sessions Calls (slow) No

Search Transactions Calls No

Search Transactions Calls (empty) No

Search Transactions Calls (slow) No

Web Servers Calls No

Browser Types Calls


Reports number of requests for all Browser Types. This metadata is used in the FxV
browser interface for searching and building filters.

Cities Calls
Reports number of requests for all Cities. This metadata is used in the FxV browser
interface for searching and building filters.

Collector IDs Calls


Reports number of requests for all Collector IDs. This metadata is used in the FxV
browser interface for searching and building filters.

Content Calls
Reports number of searches for hit content based on specific hit or session IDs, used for
replay or export. Raw content returned by the Archiver is rewritten by the Server as
needed for display purposes.
Appendix: FxV Metrics 231
Archiver Metrics

Cookies Calls
Reports number of requests for all Cookie names. This metadata is used in the FxV
browser interface for searching and building filters.

Countries Calls
Reports number of requests for all Countries. This metadata is used in the FxV browser
interface for searching and building filters.

Fields Calls
Reports number of requests for all Field names. This metadata is used in the FxV
browser interface for searching and building filters.

Hit Detail Calls


Reports number of searches for hit details based on specific hit or session IDs, used for
replay or export.

Hit Detail Calls (empty)


Reports number of searches for hit details that returned zero results.

Hit Detail Calls (slow)


Reports number of searches for hit details that took longer than 15 seconds to complete.

HTML Titles Calls


Reports number of requests for all HTML titles available. This metadata is used in the
FxV browser interface for searching and building filters.

ISPs Calls
Reports number of requests for all ISPs. This metadata is used in the FxV browser
interface for searching and building filters.

Local Configuration Calls


Reports number of requests for local Archiver configuration (hardware-specific
settings).
232 Foglight Experience Viewer
User Guide

Metrics Calls
Reports number of requests for raw metric values. The Server updates Metric History
every two minutes, so a steady number of calls is normal.

Oldest Hits Calls


Reports number of requests for the “oldest hit”. This metadata used in the FxV browser
interface when searching.

Paths Calls
Reports number of requests for all Paths. This metadata is used in the FxV browser
interface for searching and building filters.

Regions Calls
Reports number of requests for all Regions. This metadata is used in the FxV browser
interface for searching and building filters.

Request Headers Calls


Reports number of requests for all Request Headers. This metadata is used in the FxV
browser interface for searching and building filters.

Response Headers Calls


Reports number of requests for all Response Headers. This metadata is used in the FxV
browser interface for searching and building filters.

Search Hits Calls


Reports number of searches for captured hits.

Search Hits Calls (empty)


Reports number of hit searches that returned zero results.

Search Hits Calls (slow)


Reports number of hit searches that took longer than 10 seconds to complete.
Appendix: FxV Metrics 233
Archiver Metrics

Search Sessions Calls


Reports number of searches for captured sessions.

Search Sessions Calls (empty)


Reports number of session searches that returned zero results.

Search Sessions Calls (slow)


Reports number of session searches that took longer than 10 seconds to complete.

Search Transactions Calls


Reports number of searches for captured transactions.

Search Transactions Calls (empty)


Reports number of transactions searches that returned zero results.

Search Transactions Calls (slow)


Reports number of transactions searches that took longer than 10 seconds to complete.

Web Servers Calls


Reports number of requests for all Web Servers. This metadata is used in the FxV
browser interface for searching and building filters.

Script Metrics
This Metric Category contains all Metrics concerning the Hit Analysis Scripts. These
are dynamic metrics that are collected for each script defined.

Note They are displayed for individual Archivers, as well as for “Totals”.

The only metrics in this category are:


• <ScriptName> Errors—this metric reports how many times the script has failed
(that is, exited with an error).
• <ScriptName> Execution Count—this metric reports how many times the script
has executed.
234 Foglight Experience Viewer
User Guide

Both metrics are reset to 0 at midnight and represent daily totals.

Collector Metrics
You can select a Collector to display its individual metrics, or “Totals” to display total
metrics across all Collectors in the system.

Note Certain metrics are displayed only for individual Collectors.

The Collector metrics are currently grouped into one category, Collector Health.

Collector Health
This Metric Category provides information about the health of any attached Collectors.
These Metrics are primarily used to troubleshoot problems in the Foglight Experience
Monitor, some have alarm conditions defined by default.

Note These metrics do not reflect the health of your Web applications, only the capture/replay
system itself.

The following metrics are provided in this category:

Metric Name Displayed in


Totals

Clock Skew from Server No

Hits Captured Total Yes

Hits Discarded Total Yes

Hits Discarded - Dropped Packet Yes

Hits Discarded - Malformed URL Yes

Hits Discarded - Shared Memory Full Yes

Hits Discarded - Upload Queue Full Yes


Appendix: FxV Metrics 235
Collector Metrics

Metric Name Displayed in


Totals

Hits In Progress No

Upload Queue - % Utilization No

Upload Queue - Hits Waiting to Upload No

Clock Skew from Server


Reports the estimated number of milliseconds that the clock of the appliance running
this Collector and the Foglight Experience Viewer Server are skewed.

Hits Captured Total


Reports the total number of hits the Collector has sent to the Archiver, since the
Collector was last restarted.

Hits Discarded Total


Reports the total number of hits that were discarded at the Collector before being sent to
an Archiver. There are a number of reasons why a hit may be discarded.

Hits Discarded - Dropped Packet


Reports the number of hits that were discarded at the Collector because a packet was
dropped within the hit or SSL connection. This is typically because the Collector is
overloaded.

Hits Discarded - Malformed URL


Reports the number of hits discarded at the Collector because of an unparseable or
nonstandard URL.

Hits Discarded - Shared Memory Full


Reports the number of hits discarded at the Collector because the shared memory
segment was full. Each Collector has a shared memory buffer used to transfer data,
when shared memory fills up then incoming hits are discarded.
236 Foglight Experience Viewer
User Guide

Hits Discarded - Upload Queue Full


Reports the number of hits discarded at the Collector because the upload queue was full.
Each Collector has a queue to buffer hits prior to being sent. When this queue fills up
then hits are discarded.

Hits In Progress
Reports the number of hits currently being captured by the Collector but not yet sent to
an Archiver.

Upload Queue - % Utilization


Reports the percentage of the upload queue (between the Collector and the Archivers)
that is currently in use. Each Collector has a queue to buffer hits prior to being sent,
when this queue fills up then hits are discarded.

Upload Queue - Hits Waiting to Upload


Reports the number of hits currently in the upload queue between the Collector and the
Archivers. The Hits Captured metric is not incremented until the hit is removed from the
upload queue.

Server Metrics
The Server metrics include the following categories:
• “Server Health” on page 237
• “Server Sessions” on page 239
• “Server Replay Cache” on page 239
• “Server Requests to Archivers” on page 240
Appendix: FxV Metrics 237
Server Metrics

Server Health
This Metric Category provides information about the internal health of the Server.
These Metrics are primarily used to troubleshoot Server problems, some have alarm
conditions defined by default.

Note These metrics do not reflect the health of the Web servers running your Web applications,
only the Foglight Experience Viewer Server itself.

Database Size
Reports size of Server database, used to store Metric History and system configuration.

Note Captured hits, sessions, and transactions are not stored in the Server database, which is
relatively small.

JVM Free Memory


Reports unused memory within the Java Virtual Machine (JVM) running the Server.
This value should oscillate as normal “garbage collection” occurs. A consistently low
amount of free memory may indicate a problem.

Note The amount of free memory can change very rapidly as the JVM automatically resizes the
heap.

JVM Total Memory


Reports total heap memory allocated to the Java Virtual Machine (JVM) running the
Server. The JVM automatically resizes the size of the heap in order to keep “garbage
collection” times to a minimum.

Note This value can increase suddenly when a spike in capture traffic occurs. It decreases
gradually over periods of relative inactivity.
238 Foglight Experience Viewer
User Guide

JVM Used Memory


Reports amount of memory currently in use within the Java Virtual Machine (JVM)
running the Server. This value should oscillate as normal “garbage collection” occurs. A
consistently high amount of used memory may indicate a problem.

Note The amount of used memory can change very rapidly as the JVM automatically resizes the
heap.

Last Event History Trimming Elapsed Time


Reports time taken to trim old Alarm History data from the Server database. This action
is performed in the background every 15 minutes.

Last Metric Update Elapsed Time


Reports time taken to update the current values of all Metrics tracked by the Server. This
action is performed in the background every two minutes.

Last Metric Update Value Count


Reports total number of Metrics currently being tracked by the Server. These metrics are
updated in the background every two minutes.

Last Value History Trimming Elapsed Time


Reports time taken to trim old Metric History data from the Server database. This action
is performed in the background every 15 minutes.

Metadata Cache Size


Reports number of items currently stored in the Server metadata cache. This metadata
from the capture database is used to build lists of field, cookie, and header names for use
in the FxV browser interface.

Open Database Connections


Reports number of database connections currently open by the Server. A consistently
high number of connections (especially when few users are logged in) may indicate a
problem.
Appendix: FxV Metrics 239
Server Metrics

Server Sessions
This Metric Category provides information about the Users currently logged into the
Foglight Experience Viewer Server. These Metrics are primarily used to troubleshoot
Foglight Experience Viewer Server problems.

Note These metrics do not reflect the activity on the Web servers running your Web applications,
only the Foglight Experience Viewer Server itself.

Active Admin Users


Reports number of Users with administrative privileges currently logged into the Server.

Active Users
Reports number of Users currently logged into the Server, including administrators.

Server Replay Cache


This Metric Category provides information about the sequences (sessions or
transactions) cached in memory by the Server. This cache improves performance by
avoiding calls to the Archivers for captured data. These Metrics are primarily used to
troubleshoot Server problems.

Hits in Replay Cache


Reports number of hits currently stored in the Replay Cache. This value normally
increases and decreases as requests are made.

Missed Requests to the Replay Cache


Reports number of times the Server did not find a requested sequence (session or
transaction) in the Replay Cache. The Server then had to call the Archivers for captured
data.

Reloaded Requests to the Replay Cache


Reports number of times a requested sequence (session or transaction) was found in the
Replay Cache, but the sequence had been cleared to reduce memory usage. The Server
then had to call the Archivers for captured data.
240 Foglight Experience Viewer
User Guide

Requests for Sequences


Reports total number of sequences (sessions or transactions) that have been requested
from the Server.

Requests for Sequences Requiring Content Parsing


Reports number of sequences (sessions or transactions) requested that required the
content to be parsed. This is typically because sequence contained at least one frameset.
These sequences take longer to rewrite than sequences without frames.

Requests to the Replay Cache


Reports number of times the Server tried to find a sequence (session or transaction) in
the Replay Cache.

Sequences in Replay Cache


Reports number of sequences (sessions or transactions) currently stored in the Replay
Cache. This value normally increases and decreases as requests are made.

Server Requests to Archivers


This Metrics Category provides information about the calls made by the Server to the
Archivers for captured data and raw metric values. These Metrics are primarily used to
troubleshoot Server or Archiver problems (especially search problems), some have
alarm conditions defined by default.

Metrics Available From Archiver


Reports if the Server was able to collect raw metric values from the specified Archiver.
This is done in the background every two minutes as the Server updates Metric History.
If No (that is, if the Server is not able to collect raw metric values), this indicates a
serious problem with the Archiver.

Searches for Hit Details


Reports the number of searches to the Archivers for hit details used for replay.
Appendix: FxV Metrics 241
Error/Warning Metrics

Searches for Hit Details (Slow)


Reports the number of searches to the Archivers for hit details (used during replay) that
took more than 15 seconds to complete.

Searches for Hits


Reports the number of searches to the Archivers for captured hits.

Searches for Hits (Slow)


Reports the number of searches to the Archivers for captured hits that took more than
10 seconds to complete.

Searches for Sessions


Reports the number of searches sent to the Archivers for captured sessions.

Searches for Sessions (Slow)


Reports the number of searches to the Archivers for captured sessions that took more
than 10 seconds to complete.

Searches for Transactions


Reports the number of searches to the Archivers for captured transactions.

Searches for Transactions (Slow)


Reports the number of searches to the Archivers for captured transactions that took
more than 10 seconds to complete.

Error/Warning Metrics
The Error/Warning Metrics list contains all metrics that are currently in an “error” or
“warning” state, as defined by their thresholds. There is no default “Error/Warning” set
of metrics.

Only metrics which have thresholds defined (identified by the little View icon at the
end of the metric row) are subject to appearing on this page.
242 Foglight Experience Viewer
User Guide

Errors
This Metric Category contains all Metrics that currently have an Error status. These
metrics are used to indicate serious problems that require immediate attention.

Note This summarizes all other metric categories.

Warnings
This Metric Category contains all Metrics that currently have a Warning status. These
metrics are used to indicate potential problems that bear further investigation.

Note This summarizes all other metric categories.


C
Appendix: Glossary

This appendix provides the complete Foglight Experience Viewer terminology.

This appendix contains the following sections:


Component Architecture Terms .................................................................................................244
Data Model Terms .....................................................................................................................250
Product Feature Terms ..............................................................................................................253
Networking and Database Terms ..............................................................................................256
244 Foglight Experience Viewer
User Guide

Component Architecture Terms


These terms are used when describing the following:
• Major hardware/software components and connections shown in the Foglight
Named End User Configurations
• Component Architecture of the Foglight Experience Viewer
• Visual interface of the Foglight Experience Viewer

Term Definition

Agent A software component of the Foglight Experience Monitor that consumes raw
HTTP packets, determines the session ID for each hit, and passes raw hit data
to its local Collector to be transmitted to a Foglight Experience Viewer.

Analysis Secondary database that offers long-term storage of session and transaction
Repository data captured by FxV.

appliance Generic term for a collection of appliances (Foglight Experience Monitors and
group Foglight Experience Viewers), installed side-by-side or federated across
different datacenters, where the intent is to unify appliances together for
searching and reporting purposes.
Every appliance group has one (and only one) Server component running. Data
from multiple appliance groups can be combined in Foglight, but the appliance
groups do not directly know about each other, and represent completely
independent systems.
Appendix: Glossary 245
Component Architecture Terms

Term Definition

Archiver A software component of the Foglight Experience Viewer that accepts hits from
a local Tcpbatches component (through a shared memory interface), and writes
user sessions into the database for later searching and reporting.
The Archiver performs real-time analysis with Hit Filters and Transaction
Filters that profile and alert upon end user behaviors and problems. Multiple
Archivers can be configured in parallel (on separate Foglight Experience
Viewer hosts) to scale up for large capture loads.
The Archiver maintains its subset of the capture database on a local disk, and
manages the transfer of database segments to the storage tier (if present). The
Archiver responds to requests by the Server for search results or detailed
session data. Each Archiver has a complete view of each individual user session
it receives. If one Archiver of several fails, then its user sessions are simply
unavailable until the Archiver is repaired, but the rest of the capture database
can be used.

browser The main visual interface to the Foglight Experience Viewer, a local Web
interface application that runs as part of the Server software component. The browser
interface is used to configure the appliance and to perform searches, and for
user session reporting and visualizations (including visual replay). The browser
interface also supports workflows where customers drill down from the
Foglight or Foglight Experience Monitor interfaces.

capture General term for the searchable repository formed by all available Archiver and
database Storage components (across all available Foglight Experience Viewers). The
capture database is a “distributed database” that is intelligently partitioned and
managed across multiple appliance hosts.

capture tier Generic term for one or more Foglight Experience Viewers used to capture hits
and store user sessions (a collection of Archiver components). The capture tier
is wholly dedicated to the task of capturing data when a storage tier is present.

Collector A software component of the Foglight Experience Monitor that accumulates


hits from its local Agent component (where raw HTTP packets are consumed),
and then sends hits across a network socket to the Tcpbatches listener on the
Foglight Experience Viewer. Within the Foglight Experience Monitor, data is
transferred from the Agent to the Collector through a local shared memory
interface.
246 Foglight Experience Viewer
User Guide

Term Definition

Collector Generic term for one or more Collector components (running on multiple
Group Foglight Experience Monitors). Collector Groups, which are defined in the
Foglight Experience Viewer browser interface, are primarily used to define
configuration parameters—each Collector polls the Foglight Experience
Viewer Server for load-balancing and security configuration needed for data
capture (including the correct private route for monitored traffic). Collector
Groups are also used to roll-up health metrics and to alert on capture errors or
other data quality problems.

console See “browser interface” on page 245.

crossover A physical cable that is often used to transfer hits from the FxM to FxV, or
cable database segments between FxV appliances as part of a storage tier
configuration. A crossover cable is generally considered more secure than
sending sensitive data through a private switch.
Note A standard (straight) network cable can be used in place of a crossover cable,
since the network interfaces on both FxV and FxM appliances support auto polarity
correction.

database See “capture database” on page 245.

DB See “capture database” on page 245.

end user The visitor who makes use of the Web server, whose hits and pages are
monitored by the Foglight Experience Monitor, and whose user session is
captured and stored by the Foglight Experience Viewer.

Foglight Related appliance-based product that provides the Foglight Experience Viewer
Experience with data, also licensed and sold with Foglight for user performance and user
Monitor session analysis. The Foglight Experience Viewer cannot be sold without the
Foglight Experience Monitor.

Foglight See “Agent” on page 244.


Experience
Monitor Agent
Appendix: Glossary 247
Component Architecture Terms

Term Definition

Foglight This is the main visual interface to the Foglight Experience Monitor, a Web
Experience application that runs as part of its Apache software component.
Monitor
browser
interface

Foglight An appliance-based product (hardware and software) licensed and sold with
Experience Foglight as part of its user session analysis capability. This term refers
Viewer somewhat interchangeably to the total package, the specific Dell or MBX
hardware, and/or the general collection of software components running on the
hardware. The appliance is imaged at the factory, and includes a Linux
operating system (Suse Novell Enterprise Linux 10) with all other supporting
components pre-loaded.

Foglight See “Archiver” on page 245.


Experience
Viewer
Archiver

Foglight See “browser interface” on page 245.


Experience
Viewer
browser
interface

Foglight See “Server” on page 248.


Experience
Viewer Server

Foglight See “setup menu” on page 248.


Experience
Viewer setup
menu

Foglight See “Storage” on page 249.


Experience
Viewer Storage

Foglight user A customer who has an account for Foglight and the Foglight Experience
Viewer, who wants to analyze and report on user sessions.
248 Foglight Experience Viewer
User Guide

Term Definition

private switch A switch or hub that is used for interconnections between the end user
appliances and the Foglight host. This does not necessarily mean that there is a
single switch device dedicated to Foglight. It is common to create a private
subnet on one or more shared switches to implement this concept in the field.

Rsync A software component that runs on the Foglight Experience Viewer that is used
to transfer database segments from the capture tier to the storage tier. Rsync is
not proprietary to Quest but is a standard preconfigured Linux service. Rsync
operations are managed by the Archiver and Storage components, as part of the
segment transfer process (configured from the setup menu).

SAN Acronym for storage area network. See “storage area network” on page 249.

Server A software component of the Foglight Experience Viewer that hosts the
browser interface and provides the central point of control and configuration.
The Server is used to define users, User Groups, Hit Filters, Transaction Filters,
and all other Foglight Experience Viewer configuration data. The Archiver,
Storage, and Collector components poll the Server for configuration
periodically. When a user performs a search, it is the Server that contacts all
available Archiver and Storage components, and then returns a merged view of
the search results. The Server maintains all metrics related to user sessions and
pushes metric data up to Foglight.

setup menu A text-menu program on the Foglight Experience Viewer that is used to manage
all appliance-local settings, including network cards, time synchronization,
firewall, disk encryption, and component services. This is accessed by logging
in as the setup user locally, or by using the su setup command (switch user)
from the support shell.

SSH A software component that optionally runs on the Foglight Experience Viewer
to allow command-line remote access for administrative or management tasks.
SSH is not proprietary to Quest but is a standard Linux service. An SSH client
(such as Putty) is needed to connect to the SSH service. For security reasons,
SSH access is not allowed by default, but can be turned on through the setup
menu.
Appendix: Glossary 249
Component Architecture Terms

Term Definition

Storage A software component of the Foglight Experience Viewer that accepts database
segments from another appliance for searching and long-term maintenance.
The Storage component has a search API that is identical to the Archiver, but
Storage cannot be used to capture hits like the Archiver. Ordinarily the portion
of the capture database managed by the Storage component is physically
located on a SAN, rather than local disk.

storage area A collection of storage disks that is made available to the Foglight Experience
network Viewer by physically installing a HBA card (QLogic preferred) and attaching a
fiber connection to the storage array. Dell/EMC is Quest’s preferred vendor for
SAN equipment. SAN installations are not limited by the number of drives
installed on the Foglight Experience Viewer (450 GB/host) but can scale out to
24 TB/host or more.

storage tier Generic term for one or more Foglight Experience Viewers used for long-term
storage and other advanced features (a collection of Storage components). The
storage tier provides dedicated resources for searching and reporting that do not
compete with real-time activities on the capture tier. Without a storage tier,
searches and other use of the browser interface is always prioritized below the
real-time capture of user sessions (which can be very demanding). A POC
configuration (such as Antero) does not include a storage tier. Customers with a
large number of concurrent users performing searches, requiring very fast
searching in general (such as in call centers), or needing SAN integration
should definitely have a storage tier. The Server should run on the storage tier
whenever possible to free up resources for capture.

Tcpbatches A software component of the Foglight Experience Viewer that listens for hits
sent by the Collector component of the Foglight Experience Monitor.
Tcpbatches opens and listens on a TCP socket (port 7623 on the Foglight
Experience Viewer) and when hit data is received, it forwards the hit data to its
local Archiver through a shared memory interface.

Web server An installation of Apache, IIS, iPlanet, or other popular software used to host a
customer Web site.
250 Foglight Experience Viewer
User Guide

Data Model Terms


The data model terms are used to describe the followings:
• Data captured by the Foglight Experience Viewer
• Data models materialized by the Foglight Experience Viewer for searching and
visualization features
• Visual interface of the Foglight Experience Viewer

Term Definition

aspect Aspects are the most basic data type in the database, used to encode hit, session,
and transaction data elements. Aspects can represent numbers, de-duplicated
strings, and other data types such as event or filter matches. Each aspect (such
as hit request field X, or session custom field Y) consumes 2-6 tables within the
database. This term is used only in a monitoring and health context, metric
names in particular. A large number of aspects is one indication of data
pollution, meaning that the collection of aspects grows without bounds.

database See “capture database” on page 245.

database The database is divided into segments by time period. Each database segment
segment can contain up to 24 hours or 1.5M hits of information. Each Archiver has only
one of these segments open for writing at a time; this segment is called open
segment. The remainder of the closed segments are accessed only when
searching. This term is used primarily in monitoring and health context for
attribute names, and when describing the transfer of data between the capture
tier and storage tier.

captured hit A hit that was successfully captured into the database, and is online for
searching and reporting.

discarded hit A hit that was lost prior to transmission to an Archiver, or lost by an Archiver
that was overloaded.

event See “transaction event” on page 253.

failed hit A hit that could not be captured because an internal error occurred on the
appliance.
Appendix: Glossary 251
Data Model Terms

Term Definition

hit A hit is an HTTP request from an end user to a Web server, and the resulting
HTTP response. A hit has a URL, response code, headers, and other data
defined in the HTTP protocol. No two hits are identical. Every hit includes a
Session ID, assigned by the Agent that gathered the hit data on the Foglight
Experience Monitor. The Session ID is typically based on cookies set by the
Web application. A “Web page” typically consists of several hits: an HTML hit
that defines the page, and a series of non-HTML hits for page resources (like
images and style sheets).

hit custom Numeric and string fields generated by Hit Filters and stored with the hits for
fields easy searching later. Clearly distinguished from HTTP request fields.

hit details The collection of all data elements (and their values) that make up a hit:
Captured HTTP length, Client IP, Cookies, End-To-End Time, Client Time,
Processing Time, Server IP, FxM RefererID, FxM UrlID, Exception, Monitor
ID, Request Browser, Request Field, Request Header, Request Method,
Request Path, Request Referer, Request Server, Request URL, Request Type,
Response Code, Response Contains Frameset, Response Contains HTML tag,
Response Content, Response Content Type, Response Header, Response
HTML Title, Hit Identifier, Batch ID, Session ID, Custom Fields.

hit dropped by A hit that was intentionally dropped by a Hit Filter that indicated the hit should
Hit Filter not be stored. Metrics will still be updated for a dropped hit.

hit status The final status (OK, Error, Warning) of a hit after all Hit Filters were applied.
A hit has a status of OK unless any problems are noted.

hit type Whether the hit contains HTML or non-HTML content.

marked hit A hit found to match a particular Hit Filter.

metric A metric is an instance of time-series data used to indicate and track health or
performance factors, and to trigger actions when thresholds are breached.
Examples of metrics include the value of abandoned shopping carts per day in
dollars, or the total size of the database in TB.

metric update An individual change to a metric that is recorded in the database for later
searching. Searching metric updates is an easy way to find user sessions that
impacted a particular business indicator.
252 Foglight Experience Viewer
User Guide

Term Definition

session A session is a set of hits (all having the same Session ID at the Agent that
gathered the hit data) that represents the interaction of the end user with the
Web server. A session is uniquely identified by its Session ID, Start Time (the
earliest hit timestamp in the session), and Last Hit Time (the latest hit
timestamp). A session begins when the first hit is seen for a given Session ID,
and ends after a time-out period or other stop conditions. An active session is
one that has started, but not yet stopped or timed out.

session Numeric and string fields generated by Hit Filters or Transaction Filters and
custom fields stored with the session for easy searching later.

session details The collection of all data elements (and their values) that make up a session:
Session ID, Start Time, Last Hit Time, Time of First Error, Time of Last Error,
Total Hits, HTML Hits, Total Errors, Total Warnings, HTML Errors, HTML
Warnings, Total Transactions, Transaction Errors, Transaction Warnings, End-
To-End Time, Client Time, Processing Time, City, Region, Country, ISP,
Subnet, BrowserType, Username, Session Custom Fields.

sessionless hit A hit where a Session ID was not specified by the Agent, usually indicating that
there was no sessionizing rule defined by the Foglight Experience Monitor that
led to a meaningful result. The details of these hits can be inspected, but there is
no user session for reporting or visual replay.

transaction A transaction is a portion of a session that matched a particular set of events


defined by a Transaction Filter. Like a session, a transaction is uniquely
identified by its Session ID, Start Time, Last Hit Time, and its transaction
name. An example of a transaction might be “Buy a Book”, where the end user
is tracked through an entire e-commerce interaction. Each transaction has a
status based on its filter definition and a list of matched events.

transaction Numeric and string fields generated by Transaction Filters and stored with the
custom fields transactions for easy searching later.

transaction Similar to session details (see “session details” on page 252), but for
details transactions. Includes transaction events and transaction custom fields.
Appendix: Glossary 253
Product Feature Terms

Term Definition

transaction Each transaction has a list of events that describe activity in that particular
event instance. An event is a checkpoint that occurred, did not occur, or occurred
multiple times within the transaction. Event definitions are maintained as part
of a Transaction Filter.

user session See “session” on page 252.

Product Feature Terms


These terms are used for describing the following:
• Specific marketable features provided by the Foglight Experience Viewer
• Configurable elements of the Foglight Experience Viewer

Term Definition

Custom Search A Custom Search Screen is a simplified search screen intended for less
Screen technical users or for frequently used searches. Each Custom Search Screen is
based on a Saved Hit Search, a Saved Session Search, or a Saved Transaction
Search (see “Saved Search” on page 254).

Hit Analysis Refers to the process by which an Archiver analyzes each hit according to rules
configured on the Hit Analysis screen. This includes the processing and
evaluation of Hit Filters, Transaction Filters, Special Transaction Events,
Scripts, Sensitive Hit Details, Sensitive Response Content Expressions, Session
Configuration settings, and Archiver Configuration settings. It results in
updates to hit, session, and transaction data including the setting of Custom
Fields and Metrics as well as updates to other hit, session, and transaction
details.
254 Foglight Experience Viewer
User Guide

Term Definition

Hit Filter A collection of conditions and actions used to analyze hits as they are captured.
Hit Filters can be used to detect and alert on any per-hit conditions, to mark
interesting hits for later searching, or to manage hit/session storage.
Hit Filters are used to define events within Transaction Filters. Hit Filters can
use powerful regular expressions or Scripts to extract data from any hit details.
Hit Filters give a helpful and relevant context to the captured data to make
searching and reporting easier.

Hit Search Find hits based on a collection of search criteria including custom fields, metric
updates, and Hit Filter matches.

Metadata Information maintained in the database to make searching or filter creation


easier, such as lists of the field names or Web servers that have been captured.

Preference A group of users with common default user preference settings. Within a
Group Preference Group, you can define which values can and cannot be changed by
the users in the group. A user belongs to only one Preference Group.

Resource A general term for an object configured in the system and owned by a single
user of the system. Resources are assigned to Resource Groups which allow
access to them to be restricted based on User Groups. Resources include:
Custom Search Screens and Saved Searches.

Resource A collection of Resources that can be associated with one or more User Groups
Group in order to grant users access to those resources.

Saved Search Used to save and quickly retrieve frequently used search criteria. Saved
Searches can also be used to define search phases for Custom Search Screens.

Script A snippet of custom logic written in the Groovy scripting language. A Script is
executed by a Hit Filter or Transaction Filter, which can use the output
variables published by the Script to update custom fields or metrics. Scripts can
perform advanced tasks, including: evaluating complex match expressions,
transforming hit data, applying complex regular expressions, or integrating
with external systems.
Appendix: Glossary 255
Product Feature Terms

Term Definition

Sensitive Hit A specification of one or more request fields, request headers, response headers
Detail or cookies identified as “sensitive” (for example, fields used to submit credit
cards or passwords). A Sensitive Hit Detail is not displayed to users in search
results or replay. Instead, the data is replaced with alternate configurable text or
left blank.

Sensitive A specification of a regular expression used to parse response content and


Response remove “sensitive” data (for example, credit card numbers rendered within the
Content page body by the Web server). These expressions are evaluated prior to pages
Expression being replayed, in order to avoid displaying sensitive data to the user.

Session Merging multiple sessions together during a visual replay (not at capture time).
Combining Used as a last resort when there are sessionizing problems.

Session Find sessions based on a collection of search criteria including custom fields,
Search metric updates, Hit Filter matches, errors found, or transactions completed.

Transaction A collection of conditions and actions used to analyze sessions as they are
Filter captured, and to identify the distinct transactions within those sessions.
Transaction Filters can be used to detect and alert on any activity within a
session, and to calculate custom metrics across aggregated sessions or
transactions. A Transaction Filter can detect multiple events that provide
information about site problems and client activity within a related set of pages.
Transaction Filters give a helpful and relevant context to the captured data to
make searching and reporting easier.

Transaction Find transactions based on a collection of search criteria built from the filter
Search definition, including custom fields, events, and metric updates if they are
defined.

User Group Defines permissions for users logging into the browser interface. Every user
belongs to at least one User Group.
256 Foglight Experience Viewer
User Guide

Networking and Database Terms


These terms are used to define various networking and database concepts.

Term Definition

DNS Domain Name System. Is the way that Internet domain names are located and
translated into Internet Protocol addresses. A domain name is a meaningful and
easy-to-remember “handle” for an Internet address.
Because maintaining a central list of domain name/IP address correspondences
would be impractical, the lists of domain names and IP addresses are
distributed throughout the Internet in a hierarchy of authority. There is probably
a DNS server within close geographic proximity to your access provider that
maps the domain names in your Internet requests or forwards them to other
servers in the Internet.

Ethernet The most widely-installed local area network (LAN) technology. Specified in a
standard, IEEE 802.3, Ethernet was originally developed by Xerox and then
developed further by Xerox, DEC, and Intel.

HBA Host Bus Adapter. Is a circuit board and/or integrated circuit adapter that
provides input/output (I/O) processing and physical connectivity between a
server and a storage device. Because the HBA relieves the host microprocessor
of both data storage and retrieval tasks, it can improve the server's performance
time. An HBA and its associated disk subsystems are sometimes referred to as a
disk channel.

HTTP Hypertext Transfer Protocol. Is the set of rules for transferring files (text,
graphic images, sound, video, and other multimedia files) on the World Wide
Web. As soon as a Web user opens their Web browser, the user is indirectly
making use of HTTP. HTTP is an application protocol that runs on top of the
TCP/IP suite of protocols (the foundation protocols for the Internet).

NIC Network Interface Card. Computer circuit board or card that is installed in a
computer so that it can be connected to a network.

NTP Network Time Protocol. Is a protocol that is used to synchronize computer


clock times in a network of computers. In common with similar protocols, NTP
uses Coordinated Universal Time (UTC) to synchronize computer clock times
to a millisecond, and sometimes to a fraction of a millisecond.
Appendix: Glossary 257
Networking and Database Terms

Term Definition

SCP Secure Copy Protocol. Secure copy is a means of securely transferring


computer files between a local and a remote host or between two remote hosts,
using the Secure Shell (SSH) protocol.

SSH Secure Shell, sometimes known as Secure Socket Shell, is a Unix-based


command interface and protocol for securely getting access to a remote
computer. It is widely used by network administrators to control Web and other
kinds of servers remotely. SSH is actually a suite of three utilities - slogin, ssh,
and scp - that are secure versions of the earlier UNIX utilities, rlogin, rsh, and
rcp. SSH commands are encrypted and secured in several ways. Both ends of
the client/server connection are authenticated using a digital certificate, and
passwords are protected by being encrypted.

TCP/IP Transmission Control Protocol/Internet Protocol. Is the basic communication


language or protocol of the Internet. It can also be used as a communications
protocol in a private network (either an intranet or an extranet). When you are
set up with direct access to the Internet, your computer is provided with a copy
of the TCP/IP program just as every other computer that you may send
messages to or get information from also has a copy of TCP/IP.

URL Uniform Resource Locator (previously Universal Resource Locator). Is the


unique address for a file that is accessible on the Internet. A common way to get
to a Web site is to enter the URL of its home page file in your Web browser's
address line. However, any file within that Web site can also be specified with a
URL. Such a file might be any Web (HTML) page other than the home page, an
image file, or a program such as a common gateway interface application or
Java applet. The URL contains the name of the protocol to be used to access the
file resource, a domain name that identifies a specific computer on the Internet,
and a pathname, a hierarchical description that specifies the location of a file in
that computer.

WAN Wide Area Network. Is a geographically dispersed telecommunications


network. The term distinguishes a broader telecommunication structure from a
local area network (LAN). A wide area network may be privately owned or
rented, but the term usually connotes the inclusion of public (shared user)
networks.
258 Foglight Experience Viewer
User Guide

Term Definition

XML Extensible Markup Language. Is a flexible way to create common information


formats and share both the format and the data on the World Wide Web,
intranets, and elsewhere. XML can be used by any individual or group of
individuals or companies that wants to share information in a consistent way.
Index

A architecture 18
about Foglight 10 browser interface 28
about Foglight Experience Viewer 10 data model 22
about Quest Software 15 metrics 199
account information, setting preferences 72 overview 17
advanced administration terminology 243
appliance maintenance 168 Archiver configuration 125
Archivers 166 Archivers, administering 166
Collectors 167
Foglight integration 169 B
metrics 172 browser interface
overview 165 automatic login via Foglight 29
Server configuration 170 logging in 29
superuser tasks 168 screen elements 30
Analysis Repository tips 34
accessing
from Foglight 161
C
from Toad for MySQL 161
architecture 151 captured metadata, overview 117
configuration 154 Collectors, administering 167
data access 161 Combined Sessions view 63
data transfer 160 component architecture terms 244
detailed configuration 159 configuration change log 128
initial setup 154 configuration import/ export 130
overview 150 configuring
session table 151 Analysis Repository 149
transaction tables 153 custom fields 113
analyzing, session 21 hit analysis process 79
appliance Hit Filters 90
advanced administration 165 metrics 115
260 Foglight Experience Viewer
User Guide

Sever 170 FxV 13


special events 107 suite 12
storage settings 87
Transaction Filters 105 E
contacting Quest 15
editing
Content Replay view 57
Custom Search Screens 67
creating
Saved Searches 66
Custom Search Screens 67
search settings 75
Saved Searches 66
Session Explorer settings 72
custom fields
export configuration 130
additional information 115
attributes 109
configuration 113 F
overview 23, 108 Foglight integration 169
setting values 82
Custom Search Screens H
creating 67
hit analysis
defining 65
captured metadata 117
editing 67
configuration change log 128
performing a custom search 49
configuration options 125
configuring process 79
D custom fields 108
data model examples 133
custom fields 23 Buy Tunnel 138
hits 24 Event Occurrence Counting 134
metrics 24 Parametrized Metrics 137
overview 22 Parsing HTML Content 136
sessions 26 Session-Level Event Occurrence Counting 135
terms 250 Simple Data Extraction 134
transactions 27 export configuration 130
defining Hit Filters 80
Custom Search Screens 65 import configuration 130
hit storage restrictions 87 metrics 115
session storage restrictions 88 performing 20
detailed configuration, Analysis Repository 159 scripts 118
Details view 53 sensitive data protection 121
documentation special events 106
cartridge 13 Transaction Filters 92
core 13 Hit Filters
feedback 14 actions 81
Index 261

additional information 92 grouping 81


configuration 90 menu bar
configure storage settings 87 Content Replay 59
execute scripts 82 metrics
increment metrics 86 additional information 117
match conditions 80 Archiver 200
match conditions, defining 81 Collector 234
match conditions, grouping 81 complete list 199
overview 19, 80 configuration 115
set custom field values 82 Error/Warnings 241
Hit Inspector view 55 incrementing 86
hit search, overview 43 overview 24, 115
hit status, updating 89 running a metric-based diagnostic 172
hit storage Server 236
defining restrictions 87 viewing 172
hits
overview 24 N
processing 18
networking terms 256
https 29

P
I
performing
import configuration 130
hit analysis 20
initial setup, Analysis Repository 154
searches 20
processing, hits 18
J product feature terms 253
Java regular expressions
in FxV Hit Analysis 187 R
overview 188
regular expressions
testing in FxV 198
assigning custom field or metric values 193
using in FxV 192
extracting a portion of a hit detail 194
extracting a single value 193
M greedy vs. reluctant 191
managing group numbers 196
preference groups 184 identifying Sensitive Response Content 195
resource groups 183 looking for “matches” 192
user account settings 175 multiple matches 195
user and user groups 176 mutiline matching 197
match conditions using “matches” in Hit Filter match conditions 192
defining 81
262 Foglight Experience Viewer
User Guide

using “matches” in Sensitive Hit Detail session table


specifications 192 data types 156
overview 151
S session, analyzing 21
sessions, overview 26
Saved Searches
setting
creating 66
account information preferences 72
editing 66
custom fields values 82
overview 48
user preferences 71
scripts
Special Events
additional information 120
additional information 108
executing 82
configuration 107
overview 118
special events, overview 106
search settings, editing 75
SSL 29
searches
storage
hit search 43
configuring settings 87
performing 20, 35
mark hit 88
result limits 48
update hit status 89
saved searches 48
superuser tasks
session search 38
appliance maintenance 168
time constraints 37
Foglight integration 169
transaction search 40
overview 168
sensitive data protection
Server configuration 170
overview 121
support 16
sensitive hit details 121
sensitive response content expressions 123
sensitive hit details 121 T
sensitive response content expressions 123 technical support 16
session configuration 125 terms
Session Explorer component architecture 244
Combined Sessions view 63 data model 250
Content Replay view 57 networking 256
Details view 53 product feature 253
Hit Inspector view 55 text conventions 14
Timeline view 54 time constraints, searches 37
toolbar 52 Timeline view 54
Transactions view 61 toolbar
Session Explorer settings, editing 72 Session Explorer 52
session search, overview 38 transaction event actions 97
session storage changing transaction/session state 98
defining restrictions 88 executing scripts 98
Index 263

incrementing metrics 103 managing 182


setting custom field values 99 pre-defined 181
setting transaction status 97 user preferences
transaction event match conditions 94 account information 72
always true 96 search settings 75
grouping 97 Session Explorer settings 72
hit count by Hit Filter 95 setting 71
Hit Filter match 94 user sessions, analyzing 51
Hit Filter ordering 95 users
post transaction stopped by global stop condition 96 group settings 178
post transaction stopped by transaction stop managing user accounts 177
condition 97 pre-defined user accounts 176
script output 95 preference settings 176
session custom field 95
session/transaction duration 96 V
session/transaction hit counts 95
views
transaction custom field 95
Combined Sessions 63
Transaction Filters
Content Replay 57
additional information 106
Details 53
configuration 105
Hit Inspector 55
event match conditions 94
Timeline 54
overview 19, 92
Transactions 61
storage configuration 104
transaction event actions 97
transaction events 93
transaction search, overview 40
transaction tables
data types 158
overview 153
Transactions view 61
transactions, overview 27

U
updating, hit status 89
user account
managing resource groups 183
managing settings 175
managing user and user groups 176
preference groups 184
user groups
264 Foglight Experience Viewer
User Guide

Vous aimerez peut-être aussi