TO BE (Certified)
OR NOT TO BE,
That is the Question
S oftware testing is at last
emerging as a significant
discipline within I.T. Always
the poor relation to analysis and design,
the processes of verification and valida-
tion are now seen to be fundamental to
the development process as a whole.
They should be applied from the begin-
ning of the development life cycle, and
be a core element of risk management
for I.T. projects. Emerging technologies
and business applications of I.T. are
breaking new ground. E-commerce, for
example, places I.T. at the heart of the
business process and failure of com-
puter systems becomes a major busi-
ness risk. This is equally if less drama-
tically true of all the other forms of enter-
prise application integration. Reliable
computer systems that add real value
to the business are now essential to
success, and failure of any kind is be-
coming increasingly unacceptable.
Software testing is the key to managing
these business risks.
This growing sense of the importance
of software testing has its disad-
vantages, however. A growing interest
in testing will attract new blood into the
discipline, but how do we ensure that
these new recruits are competent and
professional about their work? Most
software testers have learned their skills
through experience with a limited
amount of ad hoc training and there
have been few standards to guide them;
university courses on software quality
assurance and testing are still relatively
March 2000
By Brian Hambling
uncommon; testing practice has had to
evolve to keep pace with software devel-
opment practice and the literature
struggles to keep pace with emerging
testing techniques and methods. What
software testing clearly needs is a
properly defined discipline supported
by professional standards and a
process of ensuring that testers are
aware of the professional standards
expected of them.
The Need for
Certification
How do we develop and grow good
testing professionals? In the past, few
have sought software testing as a
career; they may have come to it as a
second choice, evolved into it from a
career in software development, or been
pushed into it by a short term crisis.
Most software testing professionals
learned the hard way, and it has typically
taken somewhere between 5 and 10
years to build a sound testing career.
This process of organic development
will no longer work, because:
♦ Demand is too high for good
testers to trickle through in sufficient
numbers;
♦ The testing challenge is growing
too rapidly for testers to learn all that
they need to know by experience alone;
♦ Testers need a professional culture
Journal of Software Testing Professionals
that will not happen by chance, because
testers are spread thinly across a vast
territory;
♦ Testers need a development path
that is not dependent solely on the good
will of any one employer.
Development of software testing pro-
fessionals must be managed so that
opportunities are created early in a new
professional’s career and so that a
coherent and continuous development
path exists even for those who cannot
or who do not wish to remain with a
single employer. Thus there is a need
for some kind of development mechan-
ism that is at least partially independent
of employers, easily and universally
available, broad in technical scope, and
underwritten by other software testing
professionals through a professional
body.
Certification Schemes
for Software Testing
Professionals
Knowledge-Based
Aproaches to
Certification
Knowledge-based certification is based
on a required level of knowledge rather
than on a specific program of education.
Certificates are awarded to candidates
who can show that they have acquired
http://www.softdim.com/iist 23
the necessary level of knowledge in a
number of specified areas.
The Quality Assurance Institute (QAI)
QAI has two important certification
programs in the area of software quality
assurance – Certified Software Test
Engineer (CSTE) and Certified Quality
Analyst (CQA). The CQA qualification
will not be considered in this article,
though it may be of relevance to those
wishing to broaden a career in testing
to encompass the wider quality arena.
Certified Software Test Engineer
(CSTE)
In QAI’s words: “The CSTE is expected
not only to possess the skills required
to pass the CSTE examination, but also
to be a change agent……You cannot
rely on your current knowledge to meet
tomorrow’s job demands.”
The knowledge expected of a test pro-
fessional is detailed in a Common Body
of Knowledge (CBOK), consisting of 5
categories, encompassing 16 knowledge
domains, which are further subdivided
into key topical areas.
The categories and domains are:
♦ General skills: Communications,
Professional Development, Quality
Principles and Concepts, Processes for
Software Development, Operation and
Maintenance.
♦ Test Skills/Approaches: Testing examination is made up of the following
Principles and Concepts, Verification and
Validation Methods, Test Management,
Environment and Standards.
♦ Test Planning: Risk Analysis, Test
Tactics, Planning Process.
♦ Executing the Test Plan: Test De- ♦ General Knowledge, Conduct and
sign, Performing Tests, Defect Tracking
and Management.
♦ Test Result Analysis, Reporting
and Improvement: Quantitative Meas-
urement, Test Reporting, Improving the
Testing.
QAI believes that certification candi-
24 http://www.softdim.com/iist
dates should keep themselves updated
in line with its own stated aims: “The
exam expects that familiarity with current
industry topics by way of experience,
periodicals and textbooks is continually
being maintained…. The examination
instrument is continually reviewed and
revised to implement these factors into
the testing process.” Given that the
CSTE examination is a relatively new en-
hancement to the program, there is as
yet little evidence of this regular up-
dating of the examination; previously
candidates were required only to submit
evidence of work experience. The move
to an examination is a welcome trend,
indicating that QAI has recognized the
need for some common baseline of
knowledge that is tested.
American Society for Quality (ASQ)
ASQ’s certification program encom-
passes 7 different certificates, one of
which is for software quality engineers
– the Certified Software Quality Engineer
(CSQE). The CSQE requirements include
education and experience, proof of
professionalism and a multiple-choice
examination. Up to 8 years of relevant
experience is required, 3 of which must
be in a decision-making position - the
actual level of experience required is
determined by the candidate’s level of
education. Proof of professionalism
requires membership of a professional
body or the endorsement of at least 2
members of such a body.
The Body of Knowledge for the CSQE
categories (the breakdown for the
relevant testing category only is shown
to indicate the depth of coverage within
the certificate as a whole) :
Ethics
♦ Software Quality Management
♦ Software Processes
♦ Software Project Management
♦ Software Metrics, Measurement
and Analytical Methods
♦ Software Inspection, Testing,
Verification and Validation: Inspection,
Journal of Software Testing Professionals
Testing, Verification and Validation.
♦ Software Audits
♦ Software Configuration Manage-
ment
Since software testing is mentioned in
only one of the eight categories listed,
the CSQE is not an appropriate cer-
tificate for software testing profes-
sionals, at least not while they wish to
focus their career solely or predomin-
antly on testing.
These basic requirements are supple-
mented by a list of minimum expecta-
tions for Certified Software Quality
Engineers.
Key Characteristics of the Knowledge-
Based Approach to Certification
The CBOK requirement for CSTE is well
defined and comprehensive, requiring
a good understanding of the theory and
practice of testing. The basis of the cer-
tificate is the achievement of a level of
knowledge sufficient to pass the CSTE
examination, which QAI claims will be
continually updated. After certification,
successful candidates must provide
evidence of continuing professional
education to retain their certificate.
There is, however, no syllabus for the
examination. As a result, the extent of
study needed from the required read-
ing is unclear. The resources available
to the CSTE Board may be such that it
is more aware of new technologies and
advanced testing practices than a po-
tential candidate. This makes bench-
marking difficult, and it would, therefore,
not be straightforward for a potential
employer to assess the value of the
qualification, unless they were already
familiar with the CSTE. There is no
indication of the level at which a CSTE
can be expected to operate, which again
makes assessment of the value of the
certificate difficult. There is no link
between the CSTE and experience to
enable an employer to use the CSTE as
a stepping-stone in a career devel-
opment program.
March 2000
The QAI CQA and the ASQ CSQE
schemes are aimed at the quality profes-
sional rather than the testing specialist.
The CSQE requirements for experience
and proof of professionalism clearly
position this certification program at the
more senior end of the spectrum. Those
new to software testing would be
excluded from such a program, but those
wishing to extend their experience and
expertise later in their career might find
this a suitable certificate to aim for.
The knowledge-based certification
programs require applicants to acquire
knowledge sufficient to pass an examin-
ation that is continuously updated to
reflect changes in the discipline. The
method of knowledge acquisition is not
defined.
Education-Based
Certification Programs
An education-based certification is one
based on the successful completion of
one or more courses, each with an exam-
ination to test knowledge acquired.
International Institute for Software
Testing (IIST)
The IIST has been created to focus
attention on the problem of encouraging
new software testers from other spe-
cialisms and to ensure a sound baseline
of knowledge and experience. There is
currently just a single certification
program - the Certified Software Testing
Professional (CSTP).
Certified Software Testing Professional
CSTP exists to encourage development
of a broad range of testing profes-
sionals, including managers and aud-
itors as well as testing practitioners. It
promotes a broad, life cycle view of soft-
ware testing.
CSTP requires both formal training and
job experience. The job experience re-
quirement is a minimum of one year of
relevant experience. The formal training
requirement consists of ten days of
March 2000
formal instruction in the body of
knowledge approved by the IIST advisory
board. This formal training can be
acquired by taking IIST courses, courses
approved by IIST and running at PSQT
conferences, or (for up to 2 days of the
required training only) courses run by an
alternative training provider and within
the required curriculum.
The required curriculum for certification
as a CSTP is:
♦ Principles of Software Testing and
Test Case Design Techniques: Levels of
Testing, Code-Based Testing, Require-
ment-Based Testing, Test Design Speci-
fication.
♦ Test Management: Planning, Sched-
uling, Resources, Risk Management.
♦ Testing Process: Defining the Test
Process, Measuring the Testing Process,
Improving the Testing Process, Testing
Mainframe Applications, Testing Client/
Server Applications, Testing Internet and
Web Applications, Testing Object-
Oriented Applications, Testing Embedded
Systems.
♦ Test Execution: Test Scripting, damental Test Process, Psychology of
Reporting, Environment, Defect Tracking.
♦ Test Automation: Tool Evaluation
and Selection, Scripting.
♦ Requirement Definition and Refine- ♦ Testing Throughout the Life-
ment: Writing Testable Requirements,
Requirement Validation, Requirement
Traceability.
♦ Static Testing (Inspections, Reviews,
and Walkthroughs): Defining the
Process, Defining Standards, Measuring
the Process, Improving the Process,
Defect Data Analysis.
The Information Systems Examination
Board (ISEB) of the British Computer
Society (BCS)
The ISEB provides industry-recognized
qualifications that measure competence,
ability and performance in many areas of
I.T. Qualifications are available in areas
such project management, telecom-
munications, service management, data
Journal of Software Testing Professionals
protection, systems analysis and
design, information security and soft-
ware testing.
There is currently only one qualifica-
tion in software testing - the Found-
ation Certificate, though a Practitioner
Certificate and a Practitioner Diploma
are planned. Demand for the Found-
ation Certificate courses has been
high; at the time of writing over 600
people have so far achieved the
Foundation Certificate. The Practi-
tioner Certificate syllabus is in pre-
paration for courses to begin early in
2001. No specific plans have yet been
published for the Diploma. Courses
are delivered by accredited course
providers, of which there are currently
seven, working from an approved
syllabus.
ISEB Foundation Certificate in
Software Testing
The Foundation Certificate course
lasts for 3 days and covers:
♦ Principles of Testing: Termin-
ology, Why Testing is Necessary, Fun-
Testing, Re-Testing and Regression
Testing, Expected Results and Prior-
itization.
Cycle: Models for Testing, Economics
of Testing, High Level Test Planning,
Acceptance Testing, Integration
Testing in the Large, Functional and
Non-Functional System Testing, In-
tegration Testing in the Small, Com-
ponent Testing, Maintenance Testing.
♦ Dynamic Testing Techniques:
Black and White Box Techniques,
Error Guessing.
♦ Static testing: Reviews and the
Test Process, Types of Review, Static
Analysis.
♦ Test Management: Organization
and Configuration Management, Test
Estimation, Monitoring and Control,
Incident Management, Standards for
Testing.
http://www.softdim.com/iist 25
♦ Tool Support for Testing: Types of
CAST Tool (Computer-Aided Software
Testing), Tool Selection and Imple-
mentation.
Key Characteristics of the Education-
Based Approach to Certification
The CSTP requires formal training
through approved courses. Ten days
of training in 1 or 2 day modules must
be completed, each of which is separ-
ately examined by a short written
examination; a score of 80% is necessary
in each of the written examinations for
award of the certificate. Courses are run
mainly by the IIST, though some
courses are prepared and taught by
specialists not employed permanently
by the IIST. Examinations are set by the
course deliverers and moderated by the
IIST. The experience requirement is
sufficiently undemanding to allow those
new to testing to qualify reasonably
quickly and therefore encourages an
early move to certification for those new
to testing.
The ISEB Foundation Certificate
syllabus is published via a web site,
where accredited trainers are also
identified. Approved training courses
are reviewed against the published
syllabus and the one-hour examination,
in multiple-choice format, is set and
administered by the ISEB independently
of the trainers.
The education-based approach to
certification uses approved courses and
course providers to ensure that the
required knowledge is imparted by
appropriately experienced professionals
and to an approved syllabus. One key
advantage of this approach is that the
fundamentals can be properly estab-
lished, using trusted members of the
profession to impart an agreed set of
basic principles, before applicants move
on to more advanced subjects. Both
IIST and ISEB build on basic courses
with more advanced training, providing
a well defined route to senior prac-
titioner status.
26 http://www.softdim.com/iist
Conclusion
The software testing discipline has
reached a level of maturity where
certification is both necessary and
desirable; necessary to underpin the
required levels of professionalism in the
industry, and desirable to provide soft-
ware testing professionals with a
qualification that reflects their know-
ledge, skill and experience. The number
of certificates so far issued by the certi-
fication bodies indicates a strong de-
mand, though it is likely that only a tiny
The challenge
now is to
encourage
industry to
recognize and
support the
certification
programs
proportion of those practicing as
software testers have yet been certified.
Alternative certification approaches
provide potential certificate holders with
mechanisms to suit their individual
needs and preferences.
The knowledge-based approaches
favor those who cannot find time or
opportunity to attend training events
and who are capable of sustained
private study before taking the
examination. The absence of a standard
syllabus adds to the difficulty of
preparing for the certificate examination,
and the approach tends to favor the
more experienced professional seeking
recognition. The knowledge-based
Journal of Software Testing Professionals
approach also carries with it the risk that
applicants will acquire knowledge in a
way that inhibits further development.
The education-based approach, on the
other hand, provides for very broad
introductory training with relatively
small-scale examinations and tends to
attract the beginner to testing and those
who wish to consolidate on early
experience. The CSTP courses cover a
broad range of topics, some general and
some specific to particular application
types, while the ISEB Foundation
Certificate is a single 3-day course with
a 40-question multiple-choice examin-
ation. ISEB plans to introduce a more
advanced course in early 2001, requiring
approximately 8 days of training at a
more advanced level for software testing
practitioners. There is clearly scope for
the CSTP and ISEB courses to be used
together, for example ISEB Foundation
as a component of the CSTP require-
ments, or CSTP as a precursor to the
ISEB Practitioner course. The education-
based approach is well suited to cor-
porate schemes, where testing profes-
sionals can acquire a common language
and culture via a certification scheme
applied across an organization. It also
ensures that certificate holders have a
genuinely common underlying lan-
guage and discipline, and therefore
propogates good testing practices in a
consistent way.
The rapid development of certification
programs, coupled with initiatives such
as Special Interest Groups and career
progression models (such as the
Industry Structure Model of the BCS)
provides a flexible but sound under-
pinning for the software testing
discipline. The challenge now is to
encourage industry to recognize and
support the certification programs to
ensure their continued growth and
influence in building a true software
testing profession.
¾Brian Hambling can be reached at
ImagoQA, 52-54 High Holborn, London
WC1V 6RL, UK
+44 207 421 8109
or Bhambling@imagoqa.com.
March 2000