Issues in Knowledge Acquisition

Some of the most important issues in knowledge acquisition are as follows:

Most knowledge is in the heads of experts Experts have vast amounts of knowledge Experts have a lot of tacit knowledge o They don't know all that they know and use o Tacit knowledge is hard (impossible) to describe Experts are very busy and valuable people Each expert doesn't know everything Knowledge has a "shelf life"

Requirements for KA Techniques

Because of these issues, techniques are required which:

Take experts off the job for short time periods Allow non-experts to understand the knowledge Focus on the essential knowledge Can capture tacit knowledge Allow knowledge to be collated from different experts Allow knowledge to be validated and maintained

KA Techniques
Many techniques have been developed to help elicit knowledge from an expert. These are referred to as knowledge elicitation or knowledge acquisition (KA) techniques. The term "KA techniques" is commonly used. The following list gives a brief introduction to the types of techniques used for acquiring, analysing and modelling knowledge: Protocol-generation techniques include various types of interviews (unstructured, semi-structured and structured), reporting techniques (such as self-report and shadowing) and observational techniques Protocol analysis techniques are used with transcripts of interviews or other textbased information to identify various types of knowledge, such as goals, decisions, relationships and attributes. This acts as a bridge between the use of protocol-based techniques and knowledge modelling techniques. Hierarchy-generation techniques, such as laddering, are used to build taxonomies or other hierarchical structures such as goal trees and decision networks. Matrix-based techniques involve the construction of grids indicating such things as problems encountered against possible solutions. Important types include the

use of frames for representing the properties of concepts and the repertory grid technique used to elicit, rate, analyse and categorise the properties of concepts. Sorting techniques are used for capturing the way people compare and order concepts, and can lead to the revelation of knowledge about classes, properties and priorities. Limited-information and constrained-processing tasks are techniques that either limit the time and/or information available to the expert when performing tasks. For instance, the twenty-questions technique provides an efficient way of accessing the key information in a domain in a prioritised order. Diagram-based techniques include the generation and use of concept maps, state transition networks, event diagrams and process maps. The use of these is particularly important in capturing the "what, how, when, who and why" of tasks and events. The aim of these techniques is to produce a protocol, i.e. a record of behaviour, whether in audio, video or electronic media. Audio recording is the usual method, which is then transcribed to produce a transcript.

P to o e e tio T c n u s ro c l-G n ra n e h iq e

Interviews
Various types of interviews can be used to produce a transcript. Unstructured interviews have a rough agenda but no pre-defined structure, so that the expert and knowledge engineer are free to explore the domain. This is an inefficient way of gathering detailed knowledge, but can prove useful as an initial interview when little is known of the domain. It also acts as an ice-breaker to establish a rapport between the expert and knowledge engineer. A semi-structured interview combines a highly structured agenda with the flexibility to ask subsequent questions. The questions for a semi-structured interview are ideally constructed some time before the interview and are sent to the expert so he/she can start to prepare responses. For an interview lasting 1 hour, around 10-15 questions might be asked. This allows time in between the set questions for the knowledge engineer to ask supplementary questions to clarify points and ask for more detail where necessary. This is often the preferred style of interview as it helps to focus the expert on the key questions and helps avoid them giving unnecessary information. Another form of interview is the structured interview. This allows no flexibility on the part of the knowledge engineer whose questions are all pre-established. As such, structured interviews often involve filling-in a matrix or other diagrammatic notation.

Commentary
Another family of techniques that produce protocols are think aloud problem-solving or commentary. These techniques generate protocols by having the expert provide a

running commentary on a typical task used in the domain. The basic technique here is the self-report, in which the expert provides a running commentary of their thought processes as they solve a problem. Experimental evidence has shown that self-reports can access cognitive processes that cannot be fully recalled without bias and distortion if explained after the task has been completed. A problem with the self-report technique is that of cognitive overload, i.e. the mental effort required by the expert to provide the commentary interrupts and affects their performance of the task. This is especially true in dynamic domains where time is critical. One way around this is to use an off-line reporting technique. Here the expert is shown a protocol of their task behaviour, typically a video, and asked to provide a running commentary on what they were thinking and doing. An advantage of this is that the video can be paused or run at slow speed to allow time for full explanation. Variants of these reporting techniques involve a second expert commenting on another experts performance.

Teach Back
In the teach back technique, the knowledge engineer describes part of the knowledge that has been acquired during previous sessions or from other sources. The expert comments on what the knowledge engineer is describing to reveal misunderstandings.

Observation
Observational techniques are another way of generating protocols. Simply observing and making notes as the expert performs their daily activities can be useful, although a time-consuming process. Videotaping their task performance can be useful especially if combined with retrospective reporting techniques. On the whole, though, simple observation techniques are rarely used, as they are an inefficient means of capturing the required knowledge.

P to o A a s T c n u s ro c l n ly is e h iq e
Protocol Analysis involves the identification of basic knowledge objects within a protocol, usually a transcript. For most projects, this makes use of categories of fundamental knowledge such as concepts, attributes, values, tasks and relationships. So, for example, an interview transcript would be analysed by highlighting all the concepts that are relevant to the project. This would be repeated for all the relevant attributes, values, tasks and relationships. In some cases, more detailed categories will be used for the identification depending on the requirements of the project. For instance, if the transcript concerns the task of diagnosis, then such categories as symptoms, hypotheses and diagnostic techniques would be used for the analysis. Such categories may be taken from generic ontologies and problem-solving models.

The Protocol Tool in PCPACK can be used to analyse a transcript or other piece of text.

L d e gT c n u s a d rin e h iq e
Laddering techniques involve the creation, reviewing and modification of hierarchical knowledge, often in the form of ladders (i.e. tree diagrams). Here the expert and knowledge engineer both refer to a ladder presented on paper or a computer screen, and add, delete, rename or re-classify nodes as appropriate. Laddering can also involve a set of predefined probe questions, such as "Could you tell me some sub-types of X?", "Could you tell me how you can tell that something is X?" and "Why would you prefer X to Y?". A leading proponent of this is Dr Gordon Rugg.

D g m a e Tc n u s ia ra -B s d e h iq e
These techniques include the generation and use of network diagrams, such as concept maps, state transition networks and process maps (see types of knowledge models). As with laddering, the knowledge engineer elicits knowledge from the expert by mutual reference to a diagram on paper or computer screen. The use of concept maps has been strongly advocated as a comprehensive technique for eliciting many types of knowledge. Use of network diagrams has become a mainstream technique when acquiring knowledge to develop object-oriented software. For example, the industry standard UML (Unified Modelling Language) makes use of concept maps (combined with frames) for object knowledge, state transition networks for dynamic modelling, and process maps for functional modelling. As with laddering, the presentation of knowledge in a network format makes validation very efficient. The ease with which people understand and relate to networks has been demonstrated with experimental evidence showing that people understand and apply knowledge more easily and readily if a concept map notation is used rather than predicate logic. The Diagram Tool in PCPACK allows the creation of most types of diagram.

M trix a e T c n u s a -B s d e h iq e
These techniques involve the construction and filling-in of a 2-dimensional matrix (grid, table). Useful examples are:

Concepts v Properties (attributes and values) Problems v Solutions Hypotheses v Diagnostic techniques Tasks v Resources

The elements within the matrix can contain:

Symbols (ticks, crosses, question marks Colours Numbers Text

The use of frames (see knowledge models) can also be adopted, although this would typically be used for validating previously acquired knowledge rather than for eliciting knowledge from scratch. Timelines (see knowledge models) can also be used to acquire time-based knowledge. The Matrix Tool in PCPACK allows the creation of most types of matrix.

S rtin T c n u s o g e h iq e
Sorting techniques are a well-known method for capturing the way experts compare and order concepts, and can lead to the revelation of knowledge about classes, properties and priorities. The simplest form is card sorting. Here the expert is given a number of cards each displaying the name of a concept. The expert has the task of repeatedly sorting the cards into piles such that the cards in each pile have something in common. For example, an expert in astronomy might sort cards showing the names of planets into those that are very large, those that of medium size and those that are relatively small. By naming each pile, the expert gives information on the attributes and values they use to denote the properties of concepts. Variants of this involve sorting objects or photographs rather than cards in domains where simple textual descriptors are not easy to use. A technique often used in conjunction with sorting techniques is triadic elicitation (aka 'Three Card Trick'). This technique prompts the expert to generate new attributes. This involves asking the expert what is similar and different about three randomly chosen concepts, i.e. in what way are two of them similar and different from the other. This is a way of eliciting attributes that are not immediately and easily articulated by the expert.

R p rto G T c n u e e ry rid e h iq e
The repertory grid technique is used in many fields for eliciting and analysing knowledge and for self-help and counselling purposes. The technique is essentially matrix-based although it is more complex than simply filling-in a matrix of elements. When used in knowledge engineering, the technique usually involves the following four main stages (as described below). The Rep Grid Tool in PCPACK v2 provides full support for each of these stages.

Stage 1
In stage 1 the concepts (called elements) are selected for the grid. For the technique to be successful and not take too much time to operate, the number chosen should be no less than about 7 and no more than about 15. A set of about the same number of attributes (called constructs) is also required. These should be such that the values can be rated on a continuous scale. The attributes can be taken from knowledge previously elicited or generated during the session using triadic elicitation.

Stage 2
Stage 2 involves the rating of each concept against each attribute. A numerical scale is often used, say 1 - 9. For instance, if the concepts are planets in the solar system, each might be rated on its distance from the sun (1 meaning close to the sun, 9 meaning far away), and so on through the other attributes.

Stage 3
In stage 3, the ratings are applied to a statistical calculation called cluster analysis to create a focus grid. These calculations ensure that concepts with similar scores are grouped together in the focus grid. Similarly, attributes that have similar scores across the concepts are grouped together in the focus grid. An example of a focus grid for planets of the solar system is shown in the figure below.

The structures to the bottom and to the right of the gird shown above are dendrograms that indicate the strength of correlations. For instance, the lower dendrogram shows Neptune and Uranus as being very similar planets, and the right-hand dendrogram indicates a correlation between size and lack of density.

Stage 4
In stage 4, the knowledge engineer walks the expert through the focus grid gaining feedback and prompting for knowledge concerning the groupings and correlations shown. If appropriate, extra concepts or attributes are added and then rated to provide a larger and more representative grid. In this way the technique can be used to uncover hidden correlations and causal connections.

L ite -In rm tio a dC n tra e im d fo a n n o s in d P c s in T s s ro e s g a k

Limited-information and constrained-processing tasks are techniques which either limit the time and/or information available to the expert when performing tasks that would normally require a lot of time and information to perform. This provides a quick and efficient way of establishing the key tasks and information used.

An interesting variant of this is the twenty-questions technique. Here the aim is for the expert is to guess something that the knowledge engineer is thinking about (as in the parlour game of animal, vegetable and mineral). The expert is allowed to ask questions of the knowledge engineer who is only allowed to respond yes or no. As the expert asks each question, the knowledge engineer notes this down. The questions asked and the order in which they are asked give important knowledge such as key properties or categories in a prioritised order