Vous êtes sur la page 1sur 10

AQS Continuous Improvement Evaluation

AQS Continuous Improvement Evaluation Tool

This matrix tool is designed to facilitate the evaluation of supplier continuous improvement systems relative to the expectations of
“Boeing Quality Management System Requirements for Suppliers: Addendum 1”. Addendum 1 capability is indicated for those supplier
systems that meet minimum expectations as defined by the “shaded boxes” of the matrix. For purposes of Addendum 1 assessment, no
numeric score will be calculated.

Supplier use of this matrix for self-assessment purposes is strongly encouraged. Companies desiring to develop an advanced continuous
improvement system may find this matrix to be a valuable guide, particularly when combined with “AQS Guidelines” and D1-9000-1,
“AQS Tools,” both available on the Boeing “Doing Business” website.

Preferred Supplier Certification

This same matrix is used for AQS Continuous Improvement evaluation when the supplier is participating in the Boeing Preferred Supplier
Certification program. Boeing personnel will review the supplier’s continuous improvement system relative to the expectations of this tool,
and score the matrix appropriately. The supplier’s system must meet the minimum expectations (shaded boxes) for Bronze level
qualification in the Preferred Supplier Certification program. Silver and Gold level expectations are substantially higher, with
correspondingly higher minimum scores. Please note: PSC evaluations are the only instance in which scores are assigned to this matrix.

Note: Line and section numbers are provided on the AQS CI Evaluation Tool to establish a direct linkage to the AQS Continuous Improvement
Evaluation – Detailed Tool.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 1 of 10
AQS Continuous Improvement Evaluation
A. Continuous Improvement System
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
A.1 Continuous Improvement System Development ---- The organization shall develop and document a continuous improvement System.
Procedures exist but do not CI system developed and Documentation is consistent and CI documentation is updated Procedures are coordinated
cover all CI requirements. documented. integrated. and revised on a regular basis. enterprise-wide for consistency
1 CI system procedures Manufacturing functions use CI and standardization, so that all
Ref. Addendum 1, 4.1.1 procedures. departments practice CI.
Quality System is used for Improvement activities are Improvement activities are part Improvement activities are Improvement activities are
nonconformance detection only. isolated. of organizational priorities. guided by strategic performance consistently reviewed and
2 Guidance of CI activities Inspection based. measures. guided by top Management and
applied to business processes.
Little or no management Localized management Management reviews the CI Management integrates CI Management uses CI
3 Management involvement involvement in structured CI implements and reviews requirements and allocates activities by asking questions of information to make business
in developing a CI system activities. improvement activities. adequate resources. employees, holding reviews and decisions. CI System is used to
monitoring for progress. help run the business.
Corrective and preventive action Corrective and preventive action Corrective and preventive action Emphasis is on preventive Processes are well-documented
4 CI system integrated with does not include CI uses CI methodologies. routinely uses CI action. using best practices. Processes
quality system corrective methodologies. methodologies. are designed to prevent
and preventative actions problems and reduce variability.

Training is undocumented and Documented curriculum exists. Support personnel, including Training is delivered and Training needs continuously re-
unstructured. Process improvement training responsible management, are documented JIT as evaluated, refresher training
5 Development and delivery delivered to team members. trained in process improvement improvement team needs provided, and delivery
of training methods. dictate. methodologies updated.

Personnel have little or no Personnel have basic CI Team Members have advanced CI "specialists" focused on Past CI performance and
background in performing CI knowledge and experience. CI knowledge and experience. mentoring new coaches and experience is used in employee
6 Knowledge and Most CI team members have CI team members routinely developing and applying development. A robust CI
experience with CI basic CI skills. demonstrate CI skills. advanced techniques. CI team culture exists throughout the
members experienced with company.
basic and advanced tools.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 2 of 10
AQS Continuous Improvement Evaluation
A. Continuous Improvement System
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
A.2 Performance Measures ---- The supplier shall develop performance measures, such as measures of waste, quality, cycle time, and Customer
Little or no measures are tied to Supplier has developed PMs are linked to operations, The relationship between PMs PMs are developed at the top
performance. performance measures (PMs) processes, or area and are and business objectives has level, flowed down to each area,
1 Development of visible to shop personnel. been quantified and is and supported by working level
performance measures periodically reviewed. performance measures to avoid
Ref. Addendum 1, 4.1.2
Measures are rarely used to Measures are used to identify Cost of poor quality is used to Management uses multiple PMs The company assesses cost
identify problem areas. problem areas and improvement prioritize some improvement to improve the overall system reduction impact of variation
2 Types of performance opportunities. efforts. Some measures of and checks for improvement reduction/control activities.
effectiveness and efficiency are value.
measures used.

Performance data is received Customer supplied data is Measures of customer Product teams actively pursue Product teams routinely analyze
from the customer. received and analyzed for satisfaction are used to identify detailed customer feedback all measures of customer
trends. and prioritize improvement (survey, plant visit, etc.) to satisfaction, implements system
3 Measures of customer opportunities. Product support identify and prioritize improvements, and checks for
measures (e.g. warranty, repair improvement opportunities. effectiveness.
satisfaction and overhaul) are also tracked. Product support measures are
communicated to mfg. and/or
design for improvement.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 3 of 10
AQS Continuous Improvement Evaluation
A. Continuous Improvement System
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
A.3 Management Review ---- The supplier shall conduct periodic Management reviews of these improvement activities, paying close attention to the
effect on the performance measures, and modifying these activities as necessary.
Little or no management Supplier holds periodic reviews Management supports CI Management consistently Management consistently leads
involvement in CI activities. by management to gage effect activities with resources, and makes informed choices CI activities and makes timely
of activities on performance identifies and prioritizes new regarding the continuation or decisions regarding resource
1 Management review and measures and modify them as improvement opportunities. closure of CI activities. commitment and prioritization of
support of CI activities needed. Management participates as a activities. Management
team member. understands and uses process
improvement tools in their
Ref. Addendum 1, 4.1.4 reviews.
No goals for process Supplier links performance Performance measure goals are Supporting performance Performance measures show
2 Impact of CI activities on improvement have been measures to improvement set, and progress is made. measures are improving. steady long-term improvements,
PMs established. projects. and customer satisfaction is
Ref. Addendum 1, 4.1.4 improving.
Inconsistent methods for Consistent methods exist for CI projects are prioritized by CI project priorities are CI resources are optimized
prioritization of CI efforts selection of CI projects. their impact to the performance coordinated across through consistent prioritization
3 Mgmt. prioritization of CI measures. departments. of all company improvement
activities projects.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 4 of 10
AQS Continuous Improvement Evaluation
B. Improvement Activity
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
B.1 Analysis of Products and Processes ---- The supplier shall analyze its products and processes to identify improvement actions. Due
consideration shall be given to Key Characteristics and the full range of process improvement tools in meeting this requirement.
No data collected, or data Process and product data Product problems are analyzed Process and product analysis Improvements are leveraged
collected but not analyzed. collected and analyzed for and linked to processes needing activities occur in a systematic across the enterprise.
1 Product and process
improvement opportunities. improvement. Key processes fashion from top to bottom. Key
analysis have been identified. processes are further reviewed
Ref. Addendum 1, 4.1.3 and analyzed.
Supplier has not considered the When analysis indicates, KCs Supplier examines for part- Advanced analytical methods Variation is managed through
2 Need for variation need for variation reduction. have been identified. family or process Key are used to identify Key Process process control.
reduction and KCs Characteristics. Parameters.
Ref. Addendum 1, 4.1.3
Process documentation consists Process Flows are developed Cell team members have an Cell and support organizations’ All team members fully
only of technical specs. provided for most cells and key understanding of the process process inputs, outputs, understand the process flow
by the customer. processes by a work center flow and know how their activities and improvements are along with both internal and
team that includes operators. activities relate. Flow charts are completely documented. external customer requirements
3 Process knowledge and maintained and updated as and can explain them.
documentation process changes are
implemented. Support
organization processes are
Little or no Variation Plan in place to manage Key Key Characteristic activities are Key Characteristics are Advanced variability reduction
control/reduction activities. Characteristics prioritized. periodically reviewed for tools are used to avoid variation
4 KC management suitability. in product and/or process
Product and process knowledge Product families, features and Product value streams are Product value streams are Value stream analysis includes
is disjointed. associated manufacturing documented at a high level analyzed at a detailed level. suppliers and customers.
5 Product value stream processes have been identified. internal to the supplier. Material Future states are created. Multiple mapping tools are
knowledge and information flow are well utilized to analyze the value
understood and documented. stream.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 5 of 10
AQS Continuous Improvement Evaluation
B. Improvement Activity
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
B.2 Implementation of Improvement Actions/Projects ---- The organization shall implement improvement actions to positively impact the
performance measures.
Improvement activities have had Improvement activities/projects Improvement projects are well Management outside the project Improvements are leveraged
1 Implementation of little or no impact on implemented that positively defined and managed by the area is aware of projects and across the enterprise.
improvement actions/ performance measures impact the performance team towards completion. results.
projects measures.
Ref. Addendum 1, 4.1.3
Supplier uses a fire-fighting Proactive use of CI problem Problem solving efforts lead to Problem solving efforts lead to Problem solving efforts support
approach to problem solving. solving tools for identification of controlling sources of variation. process redesign. better product design.
2 Problem solving activities sources of variation.

Little or no structure exists for Improvement projects are A process exists for obtaining An implementation schedule is The process for executing
working improvement projects assigned to a project leader. resources and allocating time to created for large projects. A projects is understood by all and
work on large projects. Small system is in place to document continuously improved. Project
3 Activity/project improvements/actions are the activities and results of the
actions are completed on time
management and follow up assigned to responsible parties project and ensures the or elevated for review. Project
and tracked to completion. completion of outstanding results are communicated
actions. among other groups as lessons
CI problem solving tools are not Basic tools of problem solving Process improvement tools and Advanced graphical techniques Advanced tools (e.g. DOE,
used for identification of sources are being used appropriately in statistical knowledge are (e.g. scatter plot, box-whisker, FMEA, regression, variation
of variation. some areas (e.g. flow charts, routinely used in projects (e.g. tier charts . . .) are used to simulation . . .) are commonly
4 AQS tool usage Pareto analysis, fish bone, run histograms, capability, Gage support analysis. used when needed.
chart). R&R, estimates of variation. . .).

CI activities and projects are not Basic project records are More detailed records include Project information is in a A comprehensive database of
documented. retained such as why chosen, data, tools, solutions, and standard format and is projects exists.
5 CI activity and project description, and results. control methods (cover PDCA). accessible by others.

Basics concepts of Lean Lean activities are well Cells are implemented to Set-up reduction activities are Cells are operating in pull
Manufacturing (e.g. JIT, 5S, established. optimize flow. Processes are conducted. The cell is balanced production. Suppliers are well
value stream mapping . . .) are standardized and well and operators are cross-trained. integrated into the lean
6 Lean/JIT mfg. Tool usage not used. documented. enterprise.

Isolated CI projects occur. Improvement activities/projects Improvement activities and Projects are occurring in all Improvement activities and
common in manufacturing, actions commonly conducted in parts of the business such as projects are consistently a high
7 Breadth of project assembly and test. support organizations such as finance, purchasing, people, etc. priority throughout the
implementation manufacturing engineering, Suppliers and customers enterprise.
quality, tooling, design and involved when needed.
include suppliers as needed.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 6 of 10
AQS Continuous Improvement Evaluation
C. Variation Management
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
C.1 Variation Management (General requirements) ---- The analysis of products and processes, implementation of improvement projects, or the
existence of customer defined Key Characteristics may indicate that variation management activities are required. Variation control methods other
than SPC may be used, but measurable evidence must demonstrate that the controls are effective. For Boeing defined Key Characteristics, the
requirement to control and reduce variation may be waived in situations where it is impossible or prohibitively expensive to meet the control and
capability requirements of AS9103 Section 5. These exceptions must be documented by the organization and approved by Boeing.
Little or no documentation on Documentation exists for all Variation management Variation management Documentation and
identified Key Characteristics. identified Key Characteristics documentation exists for core documentation contains lessons understanding of processes
including mfg. process elements manufacturing processes in learned. leads to further improvement.
that influence variation in KCs addition to KCs.
1 Documentation of KC or as well as their control
process information techniques and measurement

Ref. AS9103, 5.2

Little or no variation Variation management activities Demonstrable improvement Part families and/or process Demonstrable improvement in
management activity observed performed on identified KCs exists from part level variation variation management activities part family and/or process
on Key Characteristics. until control and capability management activities. implemented. variation management activities.
2 Variation management established, unless agreed to by
activities the customer.

Ref. AS9103, 5.1 & 5.6

Little or no measurable evidence Measurable evidence Organization performs regular Monitoring methodology Comprehensive database of
of control. demonstrates that the selected validation of control method implemented that ensures control techniques and methods
variation control method is effectiveness. Monitoring process performance through is consistently used to establish
3 Verification of control effective. Appropriate methodology drives control of process settings. effective and efficient process
method and process monitoring methodology improvement activities. controls.
monitoring implemented to ensure
continued performance.

Ref. AS9103, 5.1& 5.4

Little or no standardized Variation control methods such Key Processes are standardized Demonstrated improvement is Process settings are well
processes exist. as tooling, control of process through the use of variation evident from use of variation controlled to ensure consistent
settings, standard processes control methods. control methods. processes.
and mistake proofing may be (e.g. Mistake proofing is used to
4 Variation control methods used to ensure process stability ensure non-conforming product
and capability. is not passed on.)

Ref. AS9103, 5.4

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 7 of 10
AQS Continuous Improvement Evaluation
C. Variation Management
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
C.2 Variation Control Methods ---- Statistical process control charts can be an effective means of controlling variation and obtaining process knowledge. This
process knowledge will help establish mistake proof processes, develop up stream process controls, contribute to process and product redesign, and eventually
alleviate the reliance on control charts. The requirements in this section are conditional on the selection of SPC as the variation control method.
Control charts do not represent Measurements used for control Summaries of control chart data Lessons learned from data Control chart data is analyzed at
normal production output. charts represent the normal are readily available to all being used on similar a more detailed level (i.e. by
1 Data collection and usage production output. employees. products/processes. shift, machine, etc).

Ref. AS9103, 5.3a

Control Chart chosen is Control Chart chosen is Control charts periodically Process shifts and trends are Advanced methods of control
2 Control chart selection inappropriate for application. appropriate for application. reviewed for appropriateness. investigated (e.g. Western are used (e.g. Target charts,
and usage Electric rules) Multivariate charts, EWMA).
Ref. AS9103, 5.3a
Errors in control limit calculation. Control limits computed Control limits ‘frozen’ with Control limits are reviewed Control Limits are recalculated
appropriately and reflect the appropriate data and the data periodically for needed changes. for processes that exhibit
3 Calculation of control current process. used in control limits is readily statistically significant change.
limits identifiable.
Ref. AS9103, 5.3a
Little or no Control Chart usage. When similar KCs from different Reduction of control chart Process control is used Process control yields
products are combined on the frequency is evident as a result extensively in lieu of charts for improvements in other, non-Key
same control chart, the of Process Improvement. each Key Characteristic. Characteristics.
4 Control charting characteristics shall have similar
processes variability and be traceable to
the specific part or product.

Ref. AS9103, 5.3.c

Out-of-control conditions are Out-of-control conditions Corrective action consistently Processes are stable and Processes are designed to
identified, but no action taken. identified, and investigated. addresses the root cause of the predictable. minimize or eliminate out-of-
5 Process control and Action is taken to remove or out of control condition. control conditions.
correction minimize the effect of these
causes. Ref. AS9103, 5.1
Potential sources of variation Sources of variation are Sources of variation are Advanced Statistical Techniques Advanced Statistical Techniques
are not consistently identified for identified and controlled for identified and controlled for Key are sometimes used to identify are routinely used to identify and
6 Identification of variation incapable Key Characteristics. incapable Key Characteristics. Characteristics. and correlate sources of correlate sources of variation
sources variation with Key with Key Characteristics.
Ref. AS9103, 5.1 & 5.2

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 8 of 10
AQS Continuous Improvement Evaluation
C. Variation Management
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
C.3 Variation Management Activities ---- The identification, control, and optimization of key processes impacts the performance measures. When a
process is not in control, the out-of-control condition must be investigated and corrective action shall be taken. A process is considered capable if its
Cpk exceeds 1.33. If the process is not capable, the organization will identify and control sources of variation and take corrective action. (D1-9000-1
contains information that is useful in diagnosing sources of variation as well as providing guidance in techniques to reduce measurement error and
process variation.)
Data collected but capability Process capability shall be Capability measures calculated Reporting of process capability Management uses process
measure not established. established for Key for Process Key Characteristics. to Management on a regular capability database for process
1 Establishing capability
Characteristics basis. improvement efforts.
Ref. AS9103, 5.1 & 5.3.a
Little or no action plans for For Cpks (or equivalent Action plans for capability Majority of processes have Cpks Capability values show stability
incapable processes. measures) that are less than improvement show progress (or equivalent measures) of over time.
2 Process capability and 1.33, corrective action plans are over time. >1.33.
improvement in place for improvement.

Ref. AS9103, 5.1 & 5.3b

Inconsistent records of Improvements are documented. Lesson learned database is Process capability information is Savings from improvements are
improvement activities. shared so as to improve similar used in choosing the calculated and reviewed. Plans
3 Use of capability data parts / processes. appropriate manufacturing link capability improvement to
process/equipment (i.e. shop cost-benefit ratio.
loading, capacity loading, etc.).
Little or No gage variation Gage variation studies are Gage variation studies are Gage variation studies are Gage variation studies are
studies are performed. performed for incapable Key routinely performed for Key routinely performed prior to any performed as a regular part of
Characteristics when analysis Characteristics, and corrective data collection. the calibration/certification
4 Gage variation studies indicates. action taken as needed. process.

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 9 of 10
AQS Continuous Improvement Evaluation
D. Variation Management in Design
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
D. Variation Management in Design ---- Suppliers often find that quality problems can best be confronted during the engineering design phase.
Many manufacturing products and business processes are not capable of meeting minimum requirements even when they are operating at maximum
effectiveness and efficiency. The fundamental cause is often found in the design of the product itself.
Organization has defined a The organization shall manage Communication process Communication process affects Communication process causes
communication process, but it is the interfaces between different ensures customer and the producibility of the product and process
1 Customer/Organization ineffective and assignment of groups involved in design and organization involvement customers design. Measurable improvements including
communication throughout responsibility is unclear. development to ensure effective throughout the product and cost / quality improvements are producibility, life cycle, and
the design process communication and clear process design evolution. achieved. serviceability as appropriate.
assignment of responsibility.
Ref. AS9100, 7.3.1
No process exists for identifying Design and Development Process for identifying essential Some features identified as Nearly all features identified in
essential design and Inputs: Inputs relating to product design and development variation critical are designed the design process as variation
2 Identification and development requirements. requirements shall be requirements includes a out by a design/build team and critical, are designed out by a
determined and records design/build team review to the remaining features are design/build team.
elimination of variation
maintained (see 4.2.4). These identify variation critical identified as Key
critical attributes during inputs shall include other attributes. Characteristics.
design. requirements essential for
design and development.
Ref. AS9100, 7.3.2
Engineering data incomplete at All pertinent data required to Reference points coordinated Reference points coordinated Coordination of reference points
drawing release with respect to allow the product to be between design, manufacturing, between design, manufacturing results in improved quality and
3 Reference points manufacturing and inspection identified, manufactured, and inspection/test at a and inspection/test within and customer satisfaction due to
data. inspected, used and maintained minimum of customer interface between most stages of reduced variation.
(e.g.,datums, electrical
shall be defined by the points. fabrication or assembly.
ground) established to organization; for example:
minimize variation drawings, part lists,
Ref. AS9100, 7.3.3
The organization does not The organization shall review The organization uses basic The organization has a The organization has a detailed
evaluate its ability to meet the requirements related to the manufacturing yield information database of core process database of process capabilities.
defined product requirements product prior to the during contract review and capabilities. The information is The database includes supplier
4 Manufacturing process prior to contract acceptance. organization’s commitment to design development. used to influence contracts with capabilities for frequently
capabilities influence supply a product to the customers and create robust subcontracted manufacturing
contracting and design customer. The review shall designs. processes. The information is
ensure that the organization has used to influence contracts with
the ability to meet the defined suppliers and customers to
requirements. create robust designs.
Ref. AS9100, 7.2.2 c
Documented manufacturing All pertinent data required to Manufacturing process Development of manufacturing Manufacturing information is
instructions required changes to allow the product to be information developed by a process information during the routinely included early in the
add detail during build of initial identified, manufactured, cross-functional design/build design phase is influenced by design process to influence
product. inspected, used and maintained team. “standard work” practices. design.
5 Development of shall be defined by the
organization; for example:
standardized manufacturing information on material,
processes processes, type of
manufacturing and assembly of
the product necessary to ensure
the conformity of the product.
Ref. AS9100, 7.3.3

Copyright © 2002 The Boeing Company. All rights reserved Fourth Edition 09/05/2002 Page 10 of 10