Vous êtes sur la page 1sur 28

By Tom Fiske and Simon Bragg

ARC STRATEGIES
JANUARY 2007


Emerging Practices and Strategies for Using
Advanced Analytics in Manufacturing
Executive Overview .................................................................... 3
Eliminating the Data Rich/Information Poor Dilemma....................... 4
Compelling Business Drivers for Advanced Analytics ........................ 5
Adopt Pervasive Use of Analytics .................................................. 6
Choose the Right Tool for the Job ................................................14
How Users Derive Value from Advanced Analytics ..........................17
Some User Experiences with Advanced Analytics............................19
Recommendations .....................................................................26

THOUGHT LEADERS FOR MANUFACTURING & SUPPLY CHAIN
ARC Strategies January 2007
2 Copyright ARC Advisory Group ARCweb.com

On-line Optimization
Closed-Loop,
Open-Loop Advisory,
APC
Quality Control
Decision Support
Off-line Optimization
Performance Monitoring,
Analysis, Insights,
Improvements
MES/
CPM
Multivariate
Analysis
Neural Nets,
Decision
Trees
Multivariate
Visualization
ERP & Supply Chain Applications ERP & Supply Chain Applications
LIMS LIMS
Integration Layer
SCADA
HMI
SCADA
HMI
Historian
DCS
PLC
Integration Layer
Virtual Sensors
Multivariate
Statistical
Process
Control
Multivariate
Modeling
Asset Management
Equipment Monitoring,
Condition Monitoring,
Fault Detection
Process, Performance and Equipment Monitoring & Control

Advanced Analytics Applications in Manufacturing



Prod Mgt. LIMS/QIS ERP
Historians DCS/PLC
Data Aggregation
SCADA/HMI SCADA/HMI Prod Mgt. Historians DCS/PLC ERP/SCM LIMS/QIS
Plant Plant
Management Enterprise Enterprise
Supply
Chain
QA/QC
Sales &
Finance
Operations Engineering
Role Based Dashboard or Workspace
A
d
v
a
n
c
e
d

A
n
a
l
y
t
i
c
s

O
p
t
i
m
a
l

D
e
c
i
s
i
o
n
s


Analytics Provides Analysis and Intelligence to Manufacturing Data
and Performance-Based Decision Support
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 3
Staying competitive means transforming all
the data in the plant into information and
making that information available in the
proper context to all personnel involved in
operations. Manufacturers must adopt a
comprehensive strategy to exploit Analytics
to gain a competitive advantage.
Executive Overview
For process manufacturing enterprises, the value-added in its manufactur-
ing operations is the primary determinant of business performance. These
companies use asset intensive operations to make their products. Extract-
ing the greatest value from these assets is difficult in the global economy as
market opportunities come and go quickly due to rapidly changing cus-
tomer requirements and extreme global competition. Globalization is
forcing companies to improve their productiv-
ity and quality by achieving greater flexibility,
agility, safety, and flawless manufacturing exe-
cution.
In a global marketplace, manufacturers are un-
der incredible pressure to improve their
profitability or they will not be able to compete
in the global economy. Historically, this meant that companies reduced
their staffs to improve their cost position and competitiveness. Companies
today continue to lose their most valuable asset people to retirement.
The exodus of workers severely erodes a companys knowledge base and
compromises their productivity if not duly compensated by other means.
Competitive pressure is forcing enterprises to deploy and use their physical
and human assets more effectively. The greater scrutiny of manufacturing
operations leaves the smaller workforce with less time to collect data and
make insightful analysis, and correspondingly good business decisions
about operations. With less time for analysis, people must not only work
harder, but also work smarter. Employees must have access to critical on-
demand information to make better decisions to improve the bottom line.
The key to success is not to generate and distribute more data, but to im-
prove understanding of the data and distribute timely information that
companies already have in numerous disparate applications scattered
throughout their organization. It also means using embedded intelligent
applications that interpret the data and take action. Advanced Analytics is
a key element in the distribution and interpretation of information. Staying
competitive means transforming all the data in the plant into information
and making that information available in the proper context to all person-
nel involved in operations. Manufacturers must adopt a comprehensive
strategy to exploit Analytics to gain a competitive advantage.
ARC Strategies January 2007
4 Copyright ARC Advisory Group ARCweb.com
Advanced Analytics provides a major
opportunity for manufacturers to improve
their operations and competitiveness by
transforming data rich systems into
valuable contextual information that forms
the backbone of deeper understanding
and better decision making.
Eliminating the Data Rich/Information
Poor Dilemma
Manufacturers must consistently meet or exceed performance targets and
strive for continuous improvement to thrive. Data is collected to provide a
measure of how well a company is doing compared to its key performance
indicators (KPIs). Data is also collected on just about anything else that
might be useful in identifying problems and highlighting opportunities for
improvement. Proliferation of IT and automation
systems has furthered encouraged the buildup of
data on all aspects of the process, equipment, and
operations.
The growing popularity of Analytics software re-
flects this buildup of data. The ability to graphically
present information on dashboards and drill-downs
and to slice and dice this information across various dimensions has become
an essential capability of plant floor and business applications. Statistical Qual-
ity Control (SQC) charts are also becoming common for monitoring of
performance throughout the manufacturing enterprise.
While current Analytics solutions are powerful, they lack certain capabili-
ties that manufacturers need to achieve optimum performance. Most
Analytics packages today only support univariate analysis. They allow
users to examine individual variable in every possible way, but they offer
little more than OLAP-style filtering to identify relationships between vari-
ables. Manufacturers are finding that the multivariate relationships are
the ones confounding their efforts to drive better performance.
Whether trying to optimize a production process or any other complex ac-
tivity, relationships among process variables have a significant impact upon
the best operating practices. Recipes that work in one plant do not work in
another because of differences in environment, equipment, and other vari-
ables. Demand for product varies across regions because of differences in
consumer taste and competitor actions. Not understanding these relation-
ships causes manufacturers to make decisions that lead to sub-optimal
performance. In todays world, this is no longer acceptable. Companies
that ignore or average out influencing factors can find themselves losing
market position to more innovative competitors who exploit their data to
develop deeper process knowledge.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 5
Companies with extensive banks of
data on their operations have the
opportunity to develop competitive
advantage and instill a culture of
continuous improvement, but only if
they can exploit this data to develop
new, more innovative processes.
On-line, conventional Analytics solutions may help manufacturers recog-
nize that a problem has occurred, but they offer no insight as to what is
needed to restore and improve performance. Root cause analysis, based
solely upon ad hoc comparisons of individual variable traces is likewise a
cumbersome approach to driving improvement. Even if relationships are
detected, there is still nothing in conventional Analytics to guide develop-
ment of acceptable operating procedures. Solutions that support true
multivariate analysis of data are required and can bring substantial benefit.
Advanced Analytics, such as multivariate analysis, can be used to turn
mounds of data into information to address manufacturing performance.
ARC uses the term Advanced to describe a set of sophisticated visualiza-
tion, data analysis, and modeling tools that extend the capabilities of
conventional Analytics. How can end users apply these tools and associ-
ated best practices to optimize performance in manufacturing operations?
Compelling Business Drivers for
Advanced Analytics
Manufacturers in many industries make similar products, using similar
equipment, with materials sourced from similar suppliers, in plants staffed
by people with similar educational backgrounds. Superior manufacturing
and business processes are among the last remaining points of differentia-
tion.
As a collection of sophisticated tools and techniques,
Advanced Analytics can help manufacturers gain
additional, deeper insight into their operations.
These solutions can help companies identify rela-
tionships and acceptable, multivariate operating
envelopes. In many cases, this information is use-
ful for developing multivariate, causal models that
support prediction of performance and develop-
ment of best practices for new scenarios. This predictive modeling capabil-
ity is invaluable for such things as increasing production throughput and
optimizing product mix around the most profitable products. In addition,
such capabilities are applicable for use as off-line solutions or embedded
on-line applications within plant automation systems.
ARC Strategies January 2007
6 Copyright ARC Advisory Group ARCweb.com
Manufacturers have made considerable
progress over the last decade using Ana-
lytics to improve their operational
management. Overall performance
goals, like profitability, have been re-
fined into constituent KPIs that help to
focus attention on the real constraints
and problem areas. Advanced Analytics
can take this to the next level, by high-
lighting the root causes for KPI
variations. With this knowledge in
hand, organizations can better focus
their actions towards specific products,
customers and resources and signifi-
cantly reduce all variations. Moreover,
operations that are more consistent open
the door for more rapid improvement.
Advanced Analytics are more sophisticated than conventional Analytics
and use of these practices in your organization requires a change in organ-
izational culture. Operating personnel need training in the use of highly
sophisticated visualization and monitoring tools. Power users will also be
required to exploit the full power of these capabilities. The potential bene-
fits are compelling in todays global marketplace. New competitors are not
reluctant to take advantage of advanced analytics and hesitation on your
part will only empower your competition to take the lead.
Advanced analytics address a broad and diverse range of business proc-
esses and manufacturing operations. Although the focus of this report is on
manufacturing, Advanced Analytics are finding use in every aspect of a
manufacturing organization: from finance and HR to marketing, R&D, and
plant floor operations.
Adopt Pervasive Use of Analytics
Manufacturing offers a wealth of opportunities to exploit the power of Ad-
vanced Analytics. Manufacturing operations use complex processes and
practices along with devices, equipment, process units, and plants. Efforts
to standardize practices and drive continuous improvement are frequently
Function Description
Finance Understand what drives financial
performance.
HR Identify the most appropriate employee
for each task, given their personality
and the job requirements
Marketing Identify the characteristics of the most
profitable customers.
Pricing Identify the optimal price for each
segment, based on customer type,
competitor actions, and company
capabilities
R&D Identify profitable combinations of
product and service features
Manufacturing Identify and correct production
anomalies, improve control, and make
continuous improvements
Advanced Analytics Enable Continuous Refinement
of Business and Manufacturing Practices
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 7
Automation systems already
collect, process, and store
thousands of measurements.

Nevertheless, operators and
engineers need better tools to
assess, consolidate, reconcile,
and generally make sense of
what all this data is telling them.
frustrated by the need for exceptions to address different products, custom-
ers, markets, geographic areas, etc. Understanding the real impact of these
factors enables organizations to gain better control of their operations and
predict the impact of new developments.
Many companies have already recognized the value of Advanced Analytics
in the control of their production operations. Process and automation engi-
neers frequently use these tools to develop more advanced operator
interfaces and control strategies. The technical background of many man-
agers in this arena also makes production a fertile area for expanding the
use of Advanced Analytics to other production activities.
Operating and technical support personnel are tasked
with gaining deeper insight into the behavior of produc-
tion processes and providing means for monitoring,
controlling, optimizing, innovating, and improving opera-
tions. Their automation systems already collect, process,
and store thousands of measurements that can be used as
the source for advanced visualization and performance
calculations. What they need are better tools to assess,
consolidate, reconcile, and generally make sense of what
all this data is telling them. They understand that there is valuable infor-
mation within their data that awaits detection. Moreover, transforming this
data into meaningful, multivariate relationships will allow operators, engi-
neers, and supervisors to reduce operating costs and improve product
quality.
The approaching departure of retiring workers provides further impetus
for adopting Advanced Analytics in the production area. Loss of this ex-
perience will severely erode the companys knowledge base of what they
do to address the exceptions. It will also reduce the organizations ability
to identify, preempt, and effectively manage production problems. Com-
panies can use Advanced Analytics to help them avoid this critical situation
by explicitly capturing knowledge and building models that incorporate
important process relationships.
Some companies are already achieving significant benefits and gaining a
competitive advantage by deploying Advanced Analytics throughout their
entire organization. Other companies must do the same or risk falling be-
hind their competition. Leading users are exploiting Advanced Analytics
in both On-line and Off-line modes to drive benefits across the production
ARC Strategies January 2007
8 Copyright ARC Advisory Group ARCweb.com
spectrum. These companies are using Advanced Analytics pervasively
throughout all levels of their production environment in a variety of ways
including:
Embedded applications that perform device or equipment level analy-
sis and diagnostics
Visualization of data for process analysis, continuous improvements,
and decision support
Critical Condition Detection and Prevention
KPI monitoring
Predictive modeling
Reporting and analysis
Ad-hoc and On-Demand Analysis and Decision Support
Inferential property estimation
Following are some examples of how users are exploiting the power of Ad-
vanced Analytics.
Multivariate Statistical Quality Control
Statistical Process Control (SPC), or univariate SPC, has been successfully
applied in the discrete parts industry for decades. SPC is rooted in the use
of statistical variation of measurements to determine how the process is
performing and to detect any unusual events or faults that may occur.
In the process industries, there are typically many variables that together
determine the final product results. In most situations, these variables are
not independent of each other. Since the process is driven by the underly-
ing phenomena, the relationship between variables must be considered.
Multivariate Statistical Process Control (MSPC) provides a mechanism for
gaining the powerful benefits of conventional SPC in these more complex
environments. MSPC is applicable to any manufacturing environment,
process, hybrid or discrete, where multiple, dependent variables must be
simultaneously monitored and controlled.
A key feature of MSPC is its ability to model a process and quality data set
with a smaller, simplified set of variables. Using available process data, a
control region can be created representing the optimal run conditions. In
addition, the decomposition of an MSPC model is useful for providing
process insight and determining the cause of a process excursion.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 9
Typical benefits of models
developed using Advanced
Analytics include production
increases of a few percent,
reduced emissions compliance
costs of up to 50 percent, and
improved product consistency.
Causal and Predictive Modeling
One of the most powerful uses of Advanced Analytics is in the develop-
ment of causal models. Such models are being used extensively by Process
manufacturers and are at the heart of some of their most
successful process control and monitoring activities. Other
industries are adopting some of these developments in
their own quest to improve their operations. The follow-
ing categories provide a convenient structure for
discussing the focus of some of the most popular applica-
tions for Causal and Predictive Modeling:
Virtual Sensors
Off-line optimization
On-Line Optimization & Advanced Process Control (APC)
Asset Management
Decision Support
Virtual Sensors
Variously called soft-sensors or inferential measurements, Virtual Sensors
are causal models that use available or easy-to-access process variables into
an estimate of an unavailable, or hard-to-measure, target variable. Various
Advanced Analytic techniques are used in building virtual sensors from
historical data.
Quality Control
One important application for virtual sensors is in the estimation of quality
parameters that are otherwise not available on-line, e.g., in the food or
pharmaceutical industry, where fermentation occurs under sterile condi-
tions. Without direct measurements of quality during production, plants
can produce excessive off-spec product while waiting for off-line analysis
results. To minimize this, manufacturers are often forced to run such proc-
esses conservatively, wasting valuable production time and capability.
By using Advanced Analytic tools to explore their historical data, manufac-
turers can identify combinations of process conditions, such as
temperature, pressure, and material specs, which produce acceptable qual-
ity. This information can then be used to develop a virtual sensor for
product quality. Instead of waiting 20 minutes for a lab result, or 6 weeks
for a thermal resistance measurement of an insulation product, operations
incorporating a virtual sensor for these parameters will have immediate
ARC Strategies January 2007
10 Copyright ARC Advisory Group ARCweb.com
feedback, albeit approximately, what quality the current process set-points
are generating.
Advanced Analytics can also improve the efficiency of the quality control
function by reducing the frequency of human quality checks and for early
release of intermediates. In addition, virtual sensors should also be consid-
ered for augmenting hardware analyzers or as possible alternatives.
Off-line Optimization
Improved fidelity of Process Models is one of the most valuable benefits
manufacturers can get from the use of Advanced Analytics. These models
can be used off-line to evaluate facility designs, to de-bottleneck existing
facilities, to develop good operating practices for new products, etc. Mod-
els developed using Advanced Analytics can also be used On-Line for
operator guidance and process control.
Advanced Analytics applied to historical data provides a basis for building
Off-Line models that reflect the importance and the impact of different
process parameters. Once identified, these relationships can be used to pre-
dict performance for a given set of conditions and to develop optimal
practices in terms of quality, energy consumption, throughput, etc. Typical
benefits of models developed using Advanced Analytics include produc-
tion increases of a few percent, reduced emissions compliance costs of up to
50 percent, and improved product consistency.
FDAs Process Analytical Technology (PAT) Initiative
FDA compliance can be challenging for manufacturers. Advanced Ana-
lytics provides a means to reduce the cost of compliance by facilitating
analysis of practices relative to the design space or operating region
where the process has been validated. Doing this requires a deep under-
standing of the effect that each process variation and possible value of
input variables has on the quality of the product.
Identifying the Best Practices for Identical/Similar Lines
Often manufacturers produce the same product on similar or identical
lines. Intel for instance, has a strategy of copy exactly. Most likely, how-
ever, lines will produce different yields or qualities. The lines can be
compared, and the most successful operating procedures and set points can
be determined.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 11
Recipe Optimization
In the food industry, it is important to produce a given product with a con-
sistent taste, look, and aroma, while achieving specific nutritional
requirements. If the source of the ingredients or manufacturing environ-
ment changes, then the quantities of ingredients need to change. There can
be hundreds of ingredient options and experts all have rules of thumb as to
how the ingredients affect the taste, look, and aroma of the final product.
Advanced Analytics can help companies evaluate and capture these rules
and optimize costs for available ingredients.
Reduced Grade Transition Times
Models that incorporate all important variables and process dynamics can
help manufacturers minimize both transition time and off-spec product
during transitions. Advanced Analytics can be helpful in improving the
quality of models for these complex situations.
Performance Management and Continuous Improvement Programs
KPIs are critical for effective performance management. However, many
KPIs today are derived using simplistic methods, such as the averages of
data across many different situations, and have control limits based solely
upon intuition and wishful thinking. Such KPIs often provide false alarms
and miss important trends and events. They also have limited
capability in identifying problems or prediction outcomes.
Using analytics to build KPIs resolves many issues associated
with univariate KPIs. KPIs constructed using analytic meth-
ods inherently combine multiple factors. These model-based
KPIs are useful for detecting trends, troubleshooting prob-
lems, and predicting outcomes. Since they are a combination
of many factors, they can also be used for root cause analysis.
These types of KPIs can be used by all types of plant person-
nel, at all levels, and with all types of process data.
Advanced Analytics supports continuous improvement programs such as
six-sigma in a number of ways. Advanced Analytics provides the statistical
capability and process intelligence to support the rigorous DMAIC model
often used in six sigma to deliver breakthroughs in performance. The
DMAIC model describes the sequential steps that should be taken in assess-
ing and improving performance. It has five components: Define, Measure,
Analyze, Improve, and Control. The first four helps companies to identify
M
E
A
S
U
R
E
ANALYZE
DEFINE DEFINE
I
M
P
R
O
V
E
I
M
P
R
O
V
E
CONTROL
Elements of the DMAIC
Six Sigma Model
ARC Strategies January 2007
12 Copyright ARC Advisory Group ARCweb.com
problems and implement solutions. Control, which is the last part of
DMAIC, makes sure that the process retains gains that have been made.
Within the Six-Sigma context, Advanced Analytics helps companies quan-
tify performance expectations so that products, processes, equipment, and
plants can be monitored against these expectations. The system can quickly
identify events that have a detrimental impact on the process and automati-
cally alert personnel as to what happened and assist in determining why.
On-line Optimization and Advanced Process Control
Leading manufacturers often move Off-line models to On-line once the
model has been proven. By doing this, they can maintain optimal perform-
ance despite changes in equipment performance and raw materials. On-
line models can also reduce the number of spurious alarms. The quality of
such models has gained recognition in recent years, and even the FDA
states that with proper understanding of the process, companies can take
corrective actions during process upsets.
Another benefit of On-Line process models is that production qualities and
quantities can be dynamically predicted. This can provide early warning
that tanks will overflow or that other constraints will be violated. If these
warnings are integrated with production scheduling, there can be sufficient
time to allow rescheduling. This can be particularly useful during a
changeover, where abnormal quantities of off-spec material can be pro-
duced.
Asset Management
Advanced Analytics models can also be used to predict the health of pro-
duction equipment or a process. In particular, changes in the correlation
among multiple variables can be used to predict the likely onset of a failure.
Such information can be used to give operators sufficient time to ameliorate
the potential problem and notify maintenance of the need to investigate the
situation during the next planned shutdown. On-Line models developed
for this purpose allow organizations to broaden their replacement of waste-
ful preventive maintenance practices with more effective condition based
asset management strategies. The analysis can also provide deeper under-
standing of the conditions that merit maintenance, helping the company to
reduce failures while still avoiding wasteful work.
The paper industry provides some good examples of the use of Advanced
Analytics to develop on-line models for asset management. Prediction of
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 13
The characteristics of a machine
or process that is about to fail can
be identified using Advanced
Analytics, giving operators time to
ameliorate the potential problem.
paper breakages and failures of rotating machinery are popular applica-
tions in this industry. One paper plant using Advanced Analytics reported
that they were able to reduce paper breaks by 40 percent.
Decision Support and Knowledge Management
Market demands and customer requirements are creating a need for greater
agility and flawless execution, yet the increasing number of retiring work-
ers and the associated loss of knowledge is putting companies at risk. At
the same time, companies are pushing more decisions down to the produc-
tion floor where they have the greatest benefit on profitability. To combat
the loss of knowledge and make the remaining worker more production,
companies must adopt pervasive use of collaborate knowledge manage-
ment tools, including Advanced Analytics.
Effective use of Advanced Analytics for manufacturing decision-support
offers significant performance improvements for most manufacturers. The
use of real-time and On-Demand decision support tools using current
plant data is helping production workers to optimize asset effectiveness by
reducing cost and raw material usage, increasing yields, improving reliabil-
ity, and increasing agility. Offline or on-demand operations decision
support tools allow users to adjust and improve plant operations on a daily
basis to accommodate changing market conditions. Real-time operations
decision support tools (open loop) calculate optimized process conditions
and provide the results in an advisory fashion to the operator.
Tools that only report historical data are not equipped to guide pressing
decisions. Since Advanced Analytics decision support tools are model-
based and have predictive capability, users can perform what-if analysis to
see how their actions affect plant performance. This capability is important
when there are multiple objectives with complex interacting processes that
create unintended or unanticipated consequences to a
decision.
The use of Advanced Analytics for strategic and opera-
tional support can significantly improve the consistency
and quality of decision making throughout the entire or-
ganization by enabling outcomes bases upon quantifiably criteria rather
than instinct. Advanced Analytics also accelerate the decision making
process, permitting production personnel to focus on other issues that are
more important. In the end, companies that best utilize Advanced Ana-
ARC Strategies January 2007
14 Copyright ARC Advisory Group ARCweb.com
Using Multivariate Visualization
techniques, operators and
engineers, who are not statisticians,
can rapidly see which combinations
of process parameters produce good
product, and which do not.
lytics to make better decisions will have achieved the greatest competitive
advantage.
Choose the Right Tool for the Job
Advanced Analytics includes a broad range of multivariate technologies.
The most basic are approaches to multivariate visualization that focus sim-
ply on presenting information. Other multivariate technologies utilize
multi-dimensional metrics and multivariate statistics to assign points to
specific categories and/or provide best estimate models for predicting
dependent variables from a set of related independent variables. Under-
standing what practices and technologies are available along with the
limitations and applicability of each is important to deriving value. Listed
below are a few examples of popular techniques that manufacturers use.
Multivariate Visualization Methods
Multivariate Visualization goes far beyond the simple pie and bar charts
beloved by the conventional Analytics suppliers. Good multivariate visu-
alization tools enable users to cluster data points, spot correlations, and
identify principle components without using statistic analysis tools. These
tools exploit the inherent abilities of most humans and enable almost eve-
ryone in an organization, to contribute to a continuous improvement
initiative without extensive training.
N-dimensional geometry, supported by a few suppli-
ers, is particularly appropriate for visualizing the
effects of hundreds of process measurements from
many batches, on overall quality and efficiency. Each
input and output parameter is plotted on one of many
parallel vertical lines. When all the batch parameters
are plotted, that batchs fingerprint appears as a zigzag
line. If multiple batches are run, there are multiple zigzag lines forming an
operating envelope.
In principle, batches produced with identical settings should produce
product of the same quality. A difference may indicate that the plant is de-
grading. These tools can be used to support control by identifying the
appropriate values for the set points given a target specification.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 15
Multivariate Statistical Process Control (MSPC)
Typically, Statistical Process Control (SPC) methodologies are applied to
systems or processes in which only one variable is measured and tracked
on control charts. These charts are designed and calibrated such that if a
value goes outside of established control limits, there is a high probability
that the process is out of control, while variations within the limits are
trusted as being normal process variations.
This univariate monitoring scheme is adequate when there
is only one variable to monitor and control. When there is
more than one variable to be simultaneously monitored, in-
dividual charts become cumbersome, and more importantly
misleading. Confidence that all factors are simultaneously
in control drops rapidly and critical process excursions
can be missed.
Multivariate Statistical Process Control (MSPC) overcomes
this disadvantage by monitoring several variables simulta-
neously. Using multivariate quality control methods, engineers, and manu-
facturers who monitor complex processes can therefore monitor the
stability of their process. The idea is straightforward when one considers
two variables. Rather than having two independent control bands, the safe
control region becomes a two-dimensional ellipse whose axis and shape
reflect any correlation between the two variables. Uncorrelated variables
would have a circular envelope. This shifts to more of an elliptical shape as
correlation increases. In all cases, the shape encompasses those combina-
tions of points for which there is high confidence that the process is under
control.
As MSPC is applied to more simultaneous variables, it becomes impossible
to present the n-dimensional envelope graphically. In such cases, simple
charts are used to track derived statistical functions and control limits on
these charts reflect the safe region for the combined function.
Implementing an MSPC strategy requires specialized software that can ex-
tract the needed information from historical data off-line and on-line
software to track and present actual process performance. Once imple-
mented, MSPC can be a powerful tool for control and continuous
improvement.
Variable A
V
a
r
i
a
b
l
e

B
MSPC
Enables large numbers
of interactive Process
variables to be
monitored
simultaneously
Time Time
T
i
m
e
T
i
m
e
ARC Strategies January 2007
16 Copyright ARC Advisory Group ARCweb.com
Multivariate Statistics and Analysis Tools
Multivariate statistics is a set of analysis techniques, which involves the
analysis of more than one statistical variable at a time. There are many dif-
ferent models, each with its own type of analysis. Regression analysis is
one of the most popular methods of modeling processes. The main advan-
tage of regression methods is its ability to extrapolate beyond the data used
to build the model. However, the empirical approach requires selecting a
model structure that does not provide a clear relationship to the underlying
physical process. In addition, the method cannot handle input variables
that are collinear, nor can it deal with noise in the data. An extension to
multivariate regression called partial least squares overcomes these short-
comings.
The partial least squares (PLS) method is often used to build inferential
models. This technique is restricted to linear operating ranges and de-
signed to handle data with collinear variables and noise. PLS is extensible
to nonlinear regions by using other technologies such as fuzzy logic or neu-
ral networks. Regression models generally require less data than other
techniques, however, choosing the appropriate model structure, variables,
and determining parameter values is challenging. PLS models can predict
product quality during or at the end of a run, thus providing an early warn-
ing of a process upset.
Principal component analysis (PCA) attempts to determine a smaller set of
variables that summarizes the most important part of a larger data set while
simultaneously filters out noise. PCA and its variations are often used to
determine equipment and process faults.
Neural Networks
Neural networks represent a powerful tool for developing nonlinear rela-
tion-ships between input and output variables. Building models with
neural networks can provide insight and greater un-
derstanding of the process. Neural networks, with
their ability to derive meaning from complicated or
imprecise data, can be used to extract patterns and de-
tect trends that are too complex to be noticed by either
humans or other computer techniques. Neural net-
work models can be incorporated into optimization
schemes and advanced process control strategies.
Input layer Hidden layer Output layer
True
True
Input layer Hidden layer Output layer
True
True
Neural Networks
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 17
Some companies derive enormous
value from Advanced Analytics,
while others do not. The difference
generally does not lie in the abilities
of the staff, but in the way in which
the initiative is managed.
Handling Real Data
Although Advanced Analytics are powerful, efforts to use them can be con-
founded by reality. Some Advanced Analytics suppliers report that up to
80 percent of the effort in identifying a solution is spent in getting and pre-
processing the data. The actual analysis is relatively quick. Thus, the abil-
ity of the tool to shorten the time to identify a solution is important. Quite
often, the following attribute differentiate software packages: data integra-
tion, preprocessing of data, clustering by operating mode, variable length
batches, and model diagnostics.
How Users Derive Value from Advanced
Analytics
Some companies derive enormous value from Advanced Analytics, while
others do not. The difference generally does not lie in the abilities of the
staff, but in the way in which the initiative is managed. There are cases of
an engineer who purchases a single license, who identifies a problem,
solves it, and persuades the organization to implement the appropriate
changes. However, such an engineer who can access the data, understand
the meaning of the data, has the analytical skills, and the influence in the
organization is rare. Moreover, after this success, that engineer is likely to
be assigned to other tasks. The problem with this is that improvements,
like on-line models, require constant maintenance, since
the characteristics of the situation change over time.
Most likely, new problems will emerge, which will not
be solved, and performance returns to previous levels.
Five characteristics of companies that derive the great-
est value from Advanced Analytics include:
Cross-Functional Teams
Applications that deliver the greatest value generally span multiple func-
tions. Most teams comprise an IT expert who can access the data, engineers
who understand the process and the data, and sometimes a statistician who
builds and validates the model. In addition, other skills may be required,
depending on the nature of the problem.
ARC Strategies January 2007
18 Copyright ARC Advisory Group ARCweb.com
There is some debate about the local versus centralized analyst model. If
there is local resentment towards head-office, then such organizational
models tend to fail. The value of a centralized model is that it leverages the
analytical skills across many plants to solve problems at the local level. Of-
ten, the central analysts monitor the data, and occasionally advise local
individuals to do something different. This works, so long as there are not
too many false-positive nuisance alarms.
Enterprise Wide
Successful companies adopt a standard tool across their enterprise. Users
share their experience and develop common procedures. When selecting a
tool, many place just as much emphasis on the packages ability to visualize
complex data and ease of use as it does on how well it supports advanced
statistical methods.
Training
Most people have studied and forgotten the simple univariate statistics that
they learned in college. Hoping that people can learn Advanced Analytics
on the job is not practical. Some investment in training must be expected.
For most packages, power users with technical degrees generally require
about two weeks training, and results users around half a day.
Commitment to Model Maintenance
Situations change over time. Plants get new equipment, new material sup-
pliers are selected, and distribution channels change. This requires periodic
maintenance of all models to ensure that the data and analysis reflects cur-
rent, rather than historical, operating conditions.
Management Commitment
To develop these four characteristics, management must demand rigorous
data analysis as part of any justification for change. In addition, manage-
ment must create the budgets and set aside peoples time to enable the
creation of cross-functional teams, allow people to take training courses,
and devote the time to maintain models. If a single department leads an
Advanced Analytics initiative, such as quality control, IT, R&D, or Opera-
tions, it will likely fail. A multi-functional team is needed and management
commitment is required to ensure that these factions work together.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 19
Some User Experiences with Advanced
Analytics
Following are some case studies of manufacturers that are using Advanced
Analytics to drive improvements in their businesses. Some have already
demonstrated significant benefits. Others provide valuable information
regarding the lessons they have learned in their efforts to embrace these
emerging practices and technologies in their operations.
Dofasco Steel
Dofasco Steel in Canada was one of the pioneers in the use of on-line MSPC
to monitor process performance. Back in the early 1990s, Dofasco identified
several potential applications for MSPC within their production facilities.
They saw a benefit of working with MSPC technology in that it provided a
good balance between adequate model fidelity, process insight, ease of de-
velopment, and robustness in the presence of noisy or missing data. At the
time, however, the MSPC technology was immature, so they decided to de-
velop and implement their own solutions.
One successful application was in their continuous
casting operations. Initially their focus was on
monitoring the operation to detect the onset of a
caster strand breakout and to prevent it from hap-
pening. Caster breakouts are catastrophic failures
in the steel casting process. They occur when the
shell of a strand does not form properly, causing
molten steel to spill, damaging equipment, and
causing production delays for all downstream op-
erations. Since the introduction of MSPC, Dofasco
has rapidly progressed from offline data analysis
to on-line monitoring and control.
Since the on-line implementation in 1997, Dofasco has enjoyed its highest
productivity levels and has seen a sharp decline in the number of break-
outs. Prior to the implementation of the real-time process monitoring
system the continuous casting process experienced about ten breakouts a
year, which is about average for the industry average. Now they are down
to about 2 or 3 breakouts a year. The cost of a typical breakout can range
from $500K to several million dollars due to lost production, missed deliv-
Solidifying Shell
Molten Steel
Ladle
Slab
Liquid Pool
Bottom Mold
Strand
Solidifying Shell
Molten Steel
Ladle
Slab
Liquid Pool
Bottom Mold
Strand
Continuous Casting Process
ARC Strategies January 2007
20 Copyright ARC Advisory Group ARCweb.com
eries, safety issues, and equipment damage. With the confidence of fewer
breakouts the average casting speed has increased.
The on-line prediction capability has also been applied at Dofasco to de-
termine the optimal amount of reagent needed to remove sulfur accurately
from pig iron in its Torpedo Car Desulphurization Station. The harsh envi-
ronment of the facility presented challenges in providing reliable, high
quality process data. Because of this, previous control methodologies
achieved limited success. The new MSPC model has been on-line since
1995 and has required one retuning. The amount of sulfur reagent has been
reduced by 8.5 percent, and the iron was increased by 0.25 percent.
NOVA Chemicals
NOVA Chemicals is a global manufacturer of plastics and chemicals with
revenues of about $5.6 billion. The company produces several billion
pounds of ethylene, polyethylene, styrene, and polystyrene each year. The
market in which they compete is cyclical in nature, with profits inextricably
linked to the delicate balance of supply and demand. Companies typically
compete on price. Manufacturing efficiency and effectiveness often deter-
mines the winners and losers in this market.
Path to Manufacturing Excellence
NOVA Chemicals understands its markets well and understands what it
takes to be successful. Part of the companys strategy is to be the low cost
provider, build upon their sustainable competitive advantage, and invest
only for high returns. The company has a history of implementing manu-
facturing excellence initiatives around focus areas that encompass process
automation, maintenance and reliability, engineering, quality, process con-
trol, and operations. Its manufacturing excellence program has the goal to
maximize total margin contribution of each product while ensuring that
they operate in a safe and environmentally responsible manner. To accom-
plish this goal, the company set out to create and maintain an infrastructure
as well as business processes that supports excellence in all aspects of its
manufacturing facilities.
NOVA Chemicals Allies with Suppliers to Develop RPM Solution
In its effort toward driving manufacturing excellence, NOVA Chemicals
sought to enable real-time data visualization throughout the company to
improve operational decision-making. The company worked with two of
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 21
Once the operators were acquainted with
the new system and had enough
confidence to trust the results, they
began to push the plant closer to the
theoretical limits and keeps it there, thus
generating significant value and benefit.
its technology providers, SAP and Pavilion Technologies, to develop a pilot
application at its Joffre site in Alberta, Canada.
NOVA Chemicals Gets Real with the Production Rate Metric
Leveraging information to achieve manufacturing excellence at the poly-
ethylene plant located in Joffre was not always easy. Collecting and
analyzing the data from different information sources took an inordinate
amount of time and only provided a historic perspective of what happened.
Although not ideal, this information was used as a basis for troubleshooting
and improving capacity utilization. As a result, NOVA Chemicals initiated
a pilot project to deliver real-time performance information to operators,
engineers, plant management and executives to drive operational decision-
making by providing a window into what is currently happening on the
plant floor and by providing insight as to what could happen on the plant
floor.
The pilot project uses Pavilions Model Predictive
Intelligence (MPI) technology to provide real-time
visibility, predictive analytics for what-if sce-
nario evaluation and theoretical versus plant
capability performance insight. MPI technology
complements the NOVA Chemicals implementa-
tion of Advanced Process Control (APC) by
allowing the same models used in advanced process control to be leveraged
for real-time performance management. Integration with financial data in
their ERP system via SAP xMII application and the NetWeaver integration
platform provides additional visualization and functional capability, in-
cluding the ability to see contribution margin by product.
Operations Crank up Plant Utilization
The pilot application provides NOVA Chemicals operators, engineers,
management, and executive team with a production rate capability metric
that uses real-time data to predict the optimal utilization for the plant and
compare it to the current performance. To augment the production rate
metric, the system provides insight into the primary constraints that are
inhibiting the plant from achieving optimal performance utilization. The
combination of accurate performance data and an ability to identify and
push constraints, provides operators with the information necessary to
make better, faster, and more informed decisions to continuously maximize
production.
ARC Strategies January 2007
22 Copyright ARC Advisory Group ARCweb.com
Once the operators became acclimated to the metric and had confidence to
trust the results, they began to leverage the tool to push the actual plant
performance closer to the theoretical limits. To support the decision-
making process, the system allows users to drill down into the causes of
variability to determine which factors are contributing to bottlenecks and
other problems. The system also provides the ability to create simulations
to determine the implications of potential operating changes intended to
generate incremental capacity.
A Strategic Tool for Better Business Decisions
Using SAP xMII, NOVA Chemicals is able to disseminate information
through SAPs Enterprise Portal to anyone in its organization that needs its.
More importantly, the integration of real-time plant floor data with finan-
cial information from the ERP systems provides a powerful tool for both
tactical and strategic decision-making.
NOVA Chemicals uses this information in a number of ways. The com-
pany continuously tracks the revenue contribution margin for every
product, order, and customer. This information is put in the context of lost
capacity utilization and delivered via the manufacturing portal to give
management support for important business and financial decisions con-
cerning its products and customers. The ability of the system to examine
the effects of constraints and bottlenecks also provides plant management
with valuable insight and guidance into plant capital investment decisions
that reduce lost capacity utilization.
Significant Benefits Encourages Enterprise-wide Adoption
Although NOVA has not explicitly published or stated ROI for the project,
they do claim they have achieved benefits. For instance, the insight pro-
vided by MPI allows NOVA Chemicals to make operational adjustments
that increase capacity. With the success of the initial project and benefits
realized, NOVA Chemicals plans to roll-out the solution across its enter-
prise. The company plans to bolster its capability to facilitate decision-
making with other metrics that include quality management and produc-
tion run consistency to name a few.
Lubrizol
Lubrizol is a global producer of advanced chemicals and specialty materials
for the transportation, consumer, and industrial markets. The company is
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 23
organized into two businesses: Lubrizol Additives and Noveon, which
produce lubricant additives and specialty chemicals, respectively.
Lubrizol Additives is a leading user of Advanced Analytics, and the com-
panys journey to its current position is instructive. Lubrizols culture has
always emphasized databased decision-making, in R&D and throughout
their operations, which up to and through 1990s had generated many pock-
ets of excellence. Nevertheless, senior management support was
peripheral, and each project required a separate but rigorous justification,
which slowed progress. Although each project initially demonstrated good
results, many times they were unsustainable due to lack of corporate-wide
support and an integrating infrastructure.
Around 2002, Lubrizol formalized what it knew: that a software package
alone does not enable successful adoption of Advanced Analytics. Success-
fully embedding analytics within an organization requires an
infrastructure, which includes an appropriate organizational structure, cul-
ture, hardware, systems, people, and processes.
Project Organization and Structure
To further address Lubrizols manufacturing systems, in 2002, they decided
to heighten attention to progress by creating cross-functional Operations
Management System (OMS) Teams consisting of key personnel from vari-
ous groups including logistics, purchasing, statistics, IT, controls,
executives, and suppliers to name several. An OMS steering team was also
defined and above this, an OMS executive team was defined which in-
cludes the VP operations.
The OMS goals are driven by clear business objectives and a written char-
ter, vision and mission statement. The mission of the OMS is to reduce cost
and improve quality, efficiency, and customer satisfaction by providing a
mechanism for capturing, storing, and exchanging relevant data; providing
tools to access and analyze the data; efficiently using process automation
systems to analyze, control, and optimize their manufacturing processes.
The vision was that analytics should cover all facets of their operations.
Manufacturing applications include asset management applications, proc-
ess optimization solutions, abnormal situation prevention, adaptive process
control, and quality control. However, other facets include, for example
demand forecasting, inventory modeling and management, and using de-
sign of experiments not only in R&D but also for supplier selection.
ARC Strategies January 2007
24 Copyright ARC Advisory Group ARCweb.com
Strategic
Needs
Shared
information
management
Automation,
analysis, and
information
systems
Data
Enterprise
Resource
Planning Systems
Business Processes
Automation and Information Architecture
Operational
Excellence
Manufacturing
Execution
Systems
Process Analysis
& Control Systems
(Equipment & Instrumentation)
Valves, instrumentation,
meters, equipment,
sensors, controllers, etc.
Asset management, diagnostics,
monitoring, performance, control, etc.
Integration between systems, data
accessibility, on-line & off-line, etc.
Full data accessibility, data analytics
and modeling, troubleshooting,
optimization, etc.
Full vertical & horizontal integration,
leverage business and operational
processes, etc.

Operations Management System Focuses on Business Objectives
To measure progress, the steering team defined the following six KPIs, each
with its own scorecard and stretch goals for each year:
1. Quantified Business Results: e.g., performing pre-project justification
versus project postmortem. Lubrizol finds that it takes about 8 months
to create an appropriate analysis methodology. They also found that
this was time well spent.
2. Customer Satisfaction: as reported by internal operators, section super-
visors, and strategic suppliers are getting the right information.
3. Meeting Commitments: such as milestones, project delivery, etc.
4. Generation of New Business Revenue
5. Migration Status: Lubrizol closely monitors the status of moving legacy
systems to strategic suppliers and the status of upgrading legacy in-
strumentation.
6. Dynamic Market Share: i.e., percentage of spend devoted to strategic
suppliers like Emerson Process Management.
Initially, the team asked, where are we now, and where do we want to
be, which led to a vision for three levels of analytics:
1. KPIs: The team sought to automate the generation of routing and re-
petitive reports.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 25
2. Off-line analysis: Lubrizols vision was to simplify offline analysis by
making it easier for process engineers to assemble the data sets and per-
form their own add hoc data analysis.
3. On-line analysis: Lubrizols intends to use many offline models in an
online fashion. This means that every minute or so a system is queried
about its current status and new data is input into the model. Impor-
tantly, results were to be deployable, not only to other systems, e.g.,
the DCS, but also to people. Within the control system, both high and
low level alarms were to be built around this control strategy, because
the predicted quality being out of bounds is as important as a high
temperature alarm.
The OMS team knew that its members would always change roles and posi-
tions. Therefore, each OMS team member has a back-up person, who
knows that someday he will be joining the team. To avoid the Im from
head office, Im here to help problem, team members were drawn from
operations in plants across the globe. In addition, the OMS teams sought
relationships with similar groups in other companies to trade best practices
in embedding analytics within the organization.
Project Phases
The integration aspects of the project split into two phases: the first focuses
on vertical integration between ERP and plant floor systems while the sec-
ond focuses on horizontal integration between applications and analysis
tools. During the first phase, which is essentially complete, the OMS built
an integrated automation and information infrastructure by, in part, stan-
dardizing on an automation platform, software, and other information
technology. This included identifying
and adopting standard analytic tools
from SAS, Statgraphics, and Umetrics.
In addition, the OMS determined how
to aggregate, consolidate, and access
the necessary data for analysis. Lubri-
zol realizes that to obtain the greatest
benefits from Advanced Analytics it
must integrate all its systems so that all
the necessary data for analysis and re-
porting is readily available.
Lubrizols Integration Strategy
.net Web services
SAP Process Order
& Recipe
Consumption
Data
Firewall
Resource
Optimization and
Planning Application
Batch Exec &
Campaign Mgr
Historian &
Recipe Exchange
PRO+
Operator Interface
Recipe Transfer
via XML
Consumption from
Batch Historian
event file via XML
Control Network
LZ Domain
SAP Analysis server(s)
Analysts
Embedded
analytics
Device level analysis /
diagnostics
Device level analysis /
diagnostics
Embedded
analysis &
diagnostic apps.
Embedded
analysis &
diagnostic apps.
Business &
Process Analytics
Business &
Process Analytics
.net Web services
SAP Process Order
& Recipe
Consumption
Data
Firewall
Resource
Optimization and
Planning Application
Batch Exec &
Campaign Mgr
Historian &
Recipe Exchange
PRO+
Operator Interface
Recipe Transfer
via XML
Consumption from
Batch Historian
event file via XML
Control Network
LZ Domain
SAP Analysis server(s)
Analysts
Embedded
analytics
Device level analysis /
diagnostics
Device level analysis /
diagnostics
Embedded
analysis &
diagnostic apps.
Embedded
analysis &
diagnostic apps.
Business &
Process Analytics
Business &
Process Analytics
Data Transfer
via XML
Pro+
.net
Web services
Batch Exec.
Consumption
ERP
Recipe +
Schedule
Historian
Operator interface
Data Transfer
Analysis servers
Analysts
Embedded
Analysis
XML
ARC Strategies January 2007
26 Copyright ARC Advisory Group ARCweb.com
The second phase involves creating an infrastructure that uses Advanced
Analytics for a broad range of applications whether it is for routine re-
porting, offline analysis and improvements or online monitoring and
optimization.
Despite users adamant requests, the OMS teams intentionally delayed roll-
ing out the tools until the data was clean, easily accessible, and the results
deployable, e.g., capable of being fed back into a unit. A premature rollout
would mean that not only would it take users longer to do multivariate
analysis, but that the analytics culture would not take root. In two or
three years times, Lubrizol believes that the whole company will improve
its performance at a far faster rate, when the right infrastructure is in place.
To summarize, Lubrizol believes that best practices require a holistic ap-
proach, support from top management, an integrated infrastructure,
appropriate analytics tools, and the right culture. In addition, the OMS
teams must include the right mix of capabilities during development and
deployment. Such capabilities include an understanding of what to do and
how to do it, the companys business objectives, appropriate technologies,
and backed by management to influence the use of analytics.
Recommendations
ARC offers the following recommendations for those involved in operating
or improving the performance of manufacturing performance:
Develop a strategic plan for the adoption and pervasive use of Ad-
vanced Analytics. Your plan should include a benefit analysis with
well-defined and measurable metrics to ensure business objectives are
met. Recognize the need to develop and maintain new applications
Create a cross functional team consisting of representatives from all
stakeholders. Include executive staff on the team. Without commit-
ment from top management, the project is not like to meet expectations.
Training is an important ingredient to reap the full benefits of analytics.
Support a culture that promotes the use of analytics. It is important to
empower employees not only with information but also with responsi-
bility and accountability to effect change and improve performance.
ARC Strategies January 2007
Copyright ARC Advisory Group ARCweb.com 27
Analysts: Tom Fiske, Simon Bragg
Editors: Sid Snitkin, Larry OBrien
Distribution: MAS-H and MAS-P Clients
Acronym Reference: For a complete list of industry acronyms, refer to our
web page at www.arcweb.com/C13/IndustryTerms/
APC Advanced Process Control
CPM Collaborative Production
Management
DCS Distributed Control System
DOE Design of Experiments
ERP Enterprise Resource Planning
FDA Food & Drug Administration
HMI Human Machine Interface
HR Human Resources
IT Information Technology
KPI Key Performance Indicator
LIMS Laboratory Information Manage-
ment System
MES Manufacturing Execution System
MPI Model Predictive Intelligence
MSPC Multivariate Statistical Process
Control
OLAP On-Line Analytical Processing
OMS Operations Management System
PCA Principal Component Analysis
PLC Programmable Logic Controller
PLS Partial Least Squares
QIS Quality Information System
R&D Research & Development
SCADA Supervisory Control and Data
Acquisition
SPC Statistical Process Control
SQC Statistical Quality Control


Founded in 1986, ARC Advisory Group has grown to become the Thought
Leader in Manufacturing and Supply Chain solutions. For even your most com-
plex business issues, our analysts have the expert industry knowledge and
firsthand experience to help you find the best answer. We focus on simple,
yet critical goals: improving your return on assets, operational performance,
total cost of ownership, project time-to-benefit, and shareholder value.
ARC Strategies is published monthly by ARC. All information in this report is
proprietary to and copyrighted by ARC. No part of it may be reproduced with-
out prior permission from ARC.
You can take advantage of ARC's extensive ongoing research plus experience
of our staff members through our Advisory Services. ARCs Advisory Services
are specifically designed for executives responsible for developing strategies
and directions for their organizations. For membership information, please
call, fax, or write to:
ARC Advisory Group, Three Allied Drive, Suite 212, Dedham, MA 02026 USA
Tel: 781-471-1000, Fax: 781-471-1100, Email: info@arcweb.com
Visit our web pages at www.arcweb.com






3 ALLIED DRIVE DEDHAM MA 02026 USA 781-471-1000
BOSTON, MA | WASHINGTON, D.C. | PITTSBURGH, PA | PHOENIX, AZ | SAN FRANCISCO, CA
CAMBRIDGE, U.K. | DSSELDORF, GERMANY | MUNICH, GERMANY | HAMBURG, GERMANY | PARIS, FRANCE | TOKYO, JAPAN | BANGALORE, INDIA | SHANGHAI, CHINA

Vous aimerez peut-être aussi