Scandinavian Journal of Psychology, 2007, 48, 8796

DOI: 10.1111/j.1467-9450.2007.00535.x

Blackwell Publishing Ltd

Cognition and Neurosciences

Effect of skill level on recall of visually presented patterns of musical notes

VIRPI KALAKOSKI
University of Helsinki, Finland

Kalakoski, V. (2007). Effect of skill level on recall of visually presented patterns of musical notes. Scandinavian Journal of Psychology, 48, 8796. Expertise effects in music were studied in a new task: the construction of mental representations from separate fragments. Groups of expert musicians and non-musicians were asked to recall note patterns presented visually note by note. Skill-level, musical well-formedness of the note patterns and presentation mode were varied. The musicians recalled note patterns better than the non-musicians, even though the presentation was visual and successive. Furthermore, only musicians performance was affected by musical well-formedness of the note patterns when visual gestalt properties, verbal rehearsability, and familiarity of the stimuli were controlled. Musicians were also able to use letter names referring to notes as efciently as visual notes, which indicates that the better recall of musicians cannot be explained by perceptual visual chunking. These results and the effect of skill level on the distribution of recall errors indicate that the ability to chunk incoming information into meaningful units does not require that complete familiar patterns are accessible to encoding processes, yet previous knowledge stored in long-term memory affects representation construction in working memory. The present method offers a new reliable tool, and its implications to the research on construction of representations and musical imagery are discussed. Key words: Expertise, music, memory, musical imagery. Virpi Kalakoski, Department of Psychology, P. O. Box 9, 00014 University of Helsinki, Finland. Tel: 358-9-191 29409; fax: 358-9-191 29443 ; e-mail: virpi.kalakoski@helsinki.

INTRODUCTION Experts exceptional cognitive abilities have been well demonstrated in various tasks, and on multiple levels of cognition and cognitive performance (for a review see Ericsson & Kintsch, 1995; Ericsson, Patel & Kintsch, 2000). These include perception, memory and thinking, encoding and classication of information, and construction of mental representations. Although it is assumed that pre-learned knowledge in long-term memory (LTM) underlies expertise effects, there is disagreement about the specic mechanisms (Ericsson et al., 2000; Simon & Gobet, 2000; Vicente, 2000; Vicente & Wang, 1998). The generally accepted view is that the core of experts exceptional cognitive performance is the ability to rapidly encode pieces of stimulus information into meaningful cognitive units, i.e., chunks (Charness, 1988; Chase & Ericsson, 1982; Chase & Simon, 1973; Ericsson & Kintsch, 1995; Gobet & Simon, 1996; Richman, Staszewski & Simon, 1995). Chunks are commonly dened as a collection of elements having strong associations with one another, but weak associations with elements within other chunks (Gobet, Lane, Croker et al., 2001, p. 236). Since chunking is the core of expert performance, it is informative to study what factors affect the process in which elements are encoded into units, that is, representation construction. The primary aim of the present research is to develop a method for studying the encoding of pieces of information into meaningful units in more detail

and to provide further information about expertise effects in the process of constructing mental representations of musical stimuli.

Expertise effects in music As in several other task domains, the typical effects of skill level and meaningfulness of stimulus material can be found also in music. For example, skilled musicians can perceive and memorize musical melodies and rhythm patterns better than non-skilled subjects, and furthermore, musicians are able to recall musically well-organized structures better than random structures (Charness, 1988; Clifton, 1986; Halpern & Bower, 1982; Meinz & Salthouse, 1998; Roberts, 1986; Sloboda, 1976, 1985). There is also more direct evidence based on eye xations indicating that experts in music use larger units than novices to compare two visually presented note patterns, and that experts also process these units with fewer xations and in less viewing time (Waters, Underwood and Findlay, 1997). Although the nature of mental representations is a main research target in music cognition, the studies have seldom focused on tasks that tease out the process of representation construction. This process was studied here with a new task in which pieces of information need to be combined to construct a unitary representation from fragments. Visual notes were presented serially so that the pattern could not be perceived as a whole, and the fragments of information had

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations. Published by Blackwell Publishing Ltd., 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. ISSN 0036-5564.

88

V. Kalakoski

Scand J Psychol 48 (2007)

to be actively maintained in working memory until there was a larger body of information, which could be connected with pre-learned knowledge or chunked together. Thus, the capacity to maintain such information in working memory before a familiar pattern was discernible was critical for successful chunk construction, distinguishing the task from more typical expert tasks, in which instant mapping of LTM patterns to presented stimuli is possible. Consequently, this kind of method captures the cooperation between working memory and LTM that underlies expert memory performance and is required when fragments of information are united into a meaningful chunk or multiple chunks. The successive presentation of visual notes makes it possible to study the process of representation construction for three reasons. First, the subjects are not given any auditory information; instead, musical patterns have to be constructed from the visually presented notes a mapping only available to experts of music. Secondly, the note patterns are not presented visually in their entirety, but as single notes one by one. Thus, construction of the representation has to be accomplished without the help of an immediately available visual or auditory gestalt. Finally, the patterns are real melodies, consisting of more than ten notes, which exceed the capacity of working memory (Cowan, 2001). Therefore, the present task is not a pure working memory task but requires the use of pre-learned knowledge and skills for the memorization of the presented items. Previous research applying an analogous method concentrated on blindfolded chess (Saariluoma, 1989, 1991; Saariluoma & Kalakoski, 1997, 1998). These studies showed that the construction of meaningful units is affected by skill level and mediated by mental imagery, at least in the visual domain (see also Hatta, Hirose, Ikeda & Fukuhara, 1989; Hishitani, 1989; Kalakoski & Saariluoma, 2001). Theoretically, it is an important question also in the domain of music, whether effective chunking of incoming information into meaningful units can be done even if complete patterns are not immediately accessible to encoding processes but can only be assembled in working memory.

Variables affecting the construction of musical representations In the present study the rst question was whether pre-learned knowledge of music supports representation construction even when the stimuli do not consist of any auditory information. In all experiments, the effect of expertise on representation construction was studied by comparing the performance of musicians and non-musicians in a serial recall task. The goal was to determine whether typical expertise effects, i.e., better memory by musical experts, could be found when participants were not able to see the entire visual note patterns but the representation had to be constructed in the mind. At a more general level, the problem of whether mapping of whole patterns from LTM is needed to produce encoding advantages in expert memory was addressed.

All participating musicians were experts in classical music and experienced in associating visual notation with auditory melodies, for instance, singing or playing melody from notes and notating melodies after hearing them once or a few times. In contrast to musical experts, non-musicians do not receive memory support from pre-learned auditory or music-related knowledge. They are conned to using only the visual modality in the integration of visually presented individual notes into a whole. It was assumed that if an effect of expertise were found, it would reect the role of pre-learned musical knowledge and skills in the representation construction task. It would also show that instant mapping to LTM patterns at encoding was not necessary for expertise effects in recall. The second question concerned the meaningfulness of the stimuli. Several studies have shown that experts exceptional performance can be seen with meaningful material, and the advantage disappears or decreases when random stimuli are used (Charness, 1988). In other words, meaningfulness was assumed to affect musicians but not non-musicians performance. However, the level of meaningfulness of stimuli is difcult to dene exactly (Vicente & Wang, 1998). Although tonal Western music, like language, is a highly structured system, a thorough formal description of its grammar has not been achieved (Tillmann, Bharucha & Bigand, 2000). The regularities of tonal music are acquired not only in music education, but also through implicit learning in everyday life. Therefore, knowledge on complex regularities of melodic, metrical, and rhythmic structures is partly explicit for musicians and tacit for both musicians and nonmusicians (Tillmann, Bharucha & Bigand, 2000). However, only musicians are able to use this knowledge with visually presented notes. In the present study, the problem of dening meaningfulness was handled by comparing stimuli that differed signicantly from each other in musical well-formedness, for instance, musical stimuli and their random counterparts. Stimuli that differed in their coherence when their musical goodness was estimated were used. The method of subjectively estimating stimulus goodness has been used in addition to music research (e.g. Povel & Jansen, 2001) also in working memory and language studies (e.g. Gathercole, 1995; Gathercole, Willis, Emslie & Baddeley, 1991). The subjective assessments of the goodness of stimuli may reect several aspects of prelearned knowledge. What is important in the present context is that they differentiate between good and poor structures in the domain. It was assumed that similarly as in language, also in the domain of music the rated goodness of stimuli is related to pre-learned musical knowledge and skills and that it affects memory performance (Gathercole, 1995). Two pilot experiments and Experiment 1 studied the stimulus properties that underlie representation construction. It was hypothesized that when auditorily and musically differing stimuli are presented visually, only musicians recall performance would be affected by stimulus goodness. This would indicate a contribution of musical imagery. Research

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

Scand J Psychol 48 (2007)

Skill and recall of notes 89

on musical imagery has shown that several musical attributes can be evoked in the absence of any auditory stimulus, in other words, through auditory imagery in short-term working memory (Baddeley & Logie, 1992; Halpern, 1988; Hubbard & Stoeckig, 1988; Zatorre, Halpern, Perry, Meyer & Evans, 1996). These attributes include timbre, pitch and tempo (Halpern, 1988; Keller, Cowan & Saults, 1995). Recent research has also shown that the silent reading of visual musical notation evokes auditory imagery (Brodsky, Henik, Rubinstein & Zorman, 2003; Schrmann, Raij, Fujiki & Hari, 2002). If the auditory goodness of the stimuli affected the musicians memory performance in the present study, it would suggest that the visual and successive presentation method is suitable for studying cognitive processes underlying experts musical imagery. To sum up, the effects of the musical skill level of the participants and the goodness of melodic patterns on recall of visually presented note sequences were studied. The second experiment compared letter and note presentation to address the issue of whether the type of visual stimuli that are presented affect performance. In addition, skill differences in the distribution of recall errors were expected to reveal if there were qualitative differences in how musicians and nonmusicians represent mentally the note patterns.

Fig. 1. (a) An example of a complete musically well-formed note pattern; (b) a randomized version of a musical note pattern.

Pilot experiments
The pilot experiments explored the properties of the stimulus material and the suitability of the piecemeal stimulus presentation method for studying representation construction out of fragments. Since only musicians are able to use musical and auditory encoding with visually presented notes, it was assumed that only they would be able to recall the musically well-formed melodies picked from music theory books better than the random patterns (Fig. 1). An unexpected nding was that both groups of participants, the eight musicians and the eight non-musicians, performed better with musically well-formed than with musically random patterns. The result suggests that musical goodness was confounded with other factors. An assumption that musically well-formed patterns also had visually more easily perceived gestalts than random ones was tested. The results showed that the musical patterns were rated as visually better than their random counter parts. Furthermore, although the general deviation of note pitches was the same in musically good and random melodies, the interval sizes between successive notes differed, which may have affected verbal rehearsability of patterns. The differences in the spatial proximity of consecutive notes may also have affected local chunking of items. The recall levels in the rst pilot experiment were quite low. Therefore, in the second pilot experiment, stimuli consisting of childrens songs were presented. These are musically simple and well-formed and were assumed to facilitate construction of mental representations. Twelve musicians and twelve non-musicians participated in the second pilot experiment. The differences in the visual gestalt properties and verbal rehearsability between the musical and random patterns, as well as the visual proximity of consecutive notes, were controlled by presenting the musical patterns in a reversed sequence. These mirror patterns were visually equally well-organized as the original patterns, and the intervals between successive notes were the same for both stimulus types (Fig. 2). Thus, only the musical and auditory gestalt was changed in the mirror melodies. The musical patterns were estimated to sound better than their mirror counterparts, whereas they were estimated to be visually as good as the mirror patterns. With these stimuli, besides the signicant skill effect, there also was a signicant interaction, showing that only musicians could recall the musical patterns better than their mirror counterparts.

GENERAL METHOD
In every experiment, the participants were presented visually with melodic patterns consisting of individual notes. A single note appeared at a time on a short stave in the centre of a 21-inch PC screen. The notation was very schematic, only ve lines and lled and open dots for quarter and half notes, respectively, were presented. No sharps or ats were shown. Every dot was visible for 2 seconds in the same horizontal location on the stave. In the rst pilot experiment, however, the stave was screen-wide and the individual notes appeared one after the other in successively adjacent locations. Each note was followed by a pause of one second. Once the whole note pattern had been presented, the participants were asked to recall it from the beginning in the correct serial order by marking all the dots they could remember on an empty stave on a recall sheet. The time limit was 1.5 minutes.

Scoring
To make sure that credit was also given for partially correct responses, but not for individual notes that may have been guesses, the participants recall of the note patterns was scored using a chunk scoring system. This system has been devised to be sensitive to partially correct recall in note reading tasks, and to be highly correlated (r > 0.90) with scoring notes in correct serial position (Clifton, 1986). In the chunk scoring system, points are given only for three or more correct notes in succession. One point was given for three successive notes, and then one point for each additional note, for instance, three correct notes in succession would result in one point, four successively correct notes would result in two points, and three successively correct notes together with three successively correct notes in another section of the series would provide 1 + 1 (= 2) points. Thus, perfect recall of the presented 1315 notes would result in 1113 points. Chunk scoring was used as the dependent variable throughout this study.

EXPERIMENT 1 The pilot experiments showed that musicians are able to use pre-learned knowledge to construct a more robust mental

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

90

V. Kalakoski

Scand J Psychol 48 (2007) make this judgment by considering whether the melody sounded odd and strange or whether it sounded more like a typical and possible example of piece of music. From the 36 estimated tunes, 18 with the largest mean difference in musical goodness between a melody and its mirror counterpart were chosen for the experiment. The melodies were simplied to consist of 1020 quarter and half notes. These melodies (M = 3.80, SD = 0.36) and their mirror counterparts (M = 2.30, SD = 0.33) differed signicantly from each other in musical goodness, t(17) = 15.65, p < 0.001. The participants of the memory experiment made similar ratings when tested after the visual note memory task. Skill level did not affect these assessments. Procedure. Eighteen musical patterns and their 18 mirror counterparts were divided into two blocks, each consisting of 9 musical and 9 mirror transposed patterns. The number of notes and the difference in auditory goodness between melodies and their mirror counterparts were equivalent in the two blocks of stimuli. Each participant was presented with one of the two blocks of 18 patterns. The nine mirror transformed patterns in one block were the counterparts of the musical patterns used in the other block, and vice versa. The general procedure of the present study was used in the presentation of the stimuli. The participants were instructed to try to hear the melodies in their minds while storing them. After the memory task, the participants were auditorily presented with the melodies and asked to estimate their musical goodness.

Fig. 2. (a) An example of a complete musically well-formed note pattern; (b) a mirror version of a musical note pattern.

representation of visually presented notes making up a real melody as compared to a mirror image of a melody in which visual complexity and verbal rehearsability were controlled. A subsequent question was whether results of the second pilot experiment were based on direct recognition of the familiar tunes. If this had been the case, the expertise effect should vanish or diminish if entirely unfamiliar tunes were used as stimuli. However, if an expertise effect could still be found, it would suggest that pre-learned knowledge is used in constructing representations out of fragments of novel patterns when instant mapping of the whole pattern is not possible.

Results The effects of skill level (musicians vs. non-musicians) and stimulus type (musical patterns vs. mirror patterns) on chunkscored recall was tested using a 2-way ANOVA (Fig. 3). Effect sizes are reported as partial eta-squared values 2, which are suitable for the analysis of variance and reect the proportion of the total variance accounted for by each dependent variable (Cohen, 1988). The results showed that the musicians were able to recall the note patterns better than the nonmusicians, F(1, 14)= 20.18, p < 0.001, MSE = 118.41, 2 = 0.59, and the musical patterns were recalled better than the mirror patterns, F(1, 14) = 11.56, p < 0.01, MSE = 4.84, 2 = 0.45. The Skill-level Stimulus type interaction was also signicant, F(1, 14) = 6.20, p < 0.05, MSE = 2.59, 2 = 0.31. The musicians recalled the musical patterns better than their mirror counterparts, t(7) = 3.16, p < 0.05, whereas in the control group no such difference was found, t(7) = 1.26, n.s.

Method
Participants. The expert group consisted of eight skilled musicians (age 2034 years, six females and two males) who had actively played one or several musical instruments for 1528 years. Four of them were professional musicians or music educators, and the others were studying towards a professional degree in music. All of them had completed several music theory and performance modules included in the formal education, and they all reported an ability to connect visual notes with auditory tunes. The non-musician group consisted of eight students of psychology (age 2032 years, six females, and two males). Half of them had played some musical instrument as a child for less than 3 years. None of them had practiced any musical instrument for several years or were uent readers of musical notation. None of the musicians or the non-musicians had participated in the pilot experiments. Stimulus material. Simple melodies were picked from foreign musical songbooks for elementary level music education. Only melodies that a professional musician estimated to be musically better than their mirror counterparts were chosen for the stimulus selection test. In the selection test, two series of melodies and their mirror counterparts were presented to 24 non-musicians in two different random orders. Each subject heard half of the melodies and their mirror counterparts. The tunes and their mirror counterparts were played from a computer. The tempo was about two quarter notes per second, and between the tunes there was a pause of 8 seconds during which the participants subjectively rated how good they found the presented melody. They marked their rating on a response sheet by using a ve-point scale, in which extremes indicated a very bad melody or a very good melody. The participants were instructed to

Fig. 3. Recall (mean chunk score and standard error of mean) as a function of skill level and stimulus type; musical patterns vs. mirror patterns.

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

Scand J Psychol 48 (2007)

Skill and recall of notes 91 based on other factors than the visual symbols making up the stimuli, presentation mode will not affect performance. In contrast, if the musicians memory performance is based on perceptual visual chunking, they will perform better in the note presentation condition than in the letter presentation condition, in which musical visual cues are not available. Second, the distribution of recall errors was studied. Previous studies (Halpern & Bower, 1982) have shown that musicians make musically meaningful errors, like displacement of notes horizontally or vertically or permutation of the order of notes, in their recall although the total number of errors or the totals of error subtypes have not been reported in these studies. Unfortunately, these kinds of errors are not only musically meaningful but they are also visually meaningful transpositions. Thus, non-musicians could make similar errors as musicians, even though they represented the stimuli differently. Fortunately, it is possible to determine the modality of errors by studying the interaction between skill level and presentation mode on the distribution of errors. In the visual note presentation condition, it is possible that the groups of musicians and non-musicians do not differ, because the musically meaningful transposition errors that only musicians are able to make are confounded with visually meaningful errors that do not require musical knowledge. For example, in the note condition, displacement of notes one step higher or lower on the stave, for instance, the visual notes e-g-a instead of d-f-g, is a natural musical error for a musician, but also a natural visual error for a non-musician. However, in the letter condition, it is very unlikely that a non-musician would erroneously recall a letter string e-g-a instead of d-f-g. In contrast, if musicians represent the note names as musical patterns when letter presentation is used, this kind of vertical transposition error would be a natural musical error for a musician. Thus, it is hypothesized that in the letter condition, only musicians make more vertical transposition errors, such as e-g-a instead of d-f-g. In contrast, it is likely that non-musicians make more octave number errors than the musicians in the letter condition. For example, a musician knows that the notes h1 and c2 are next to each other on a stave and in pitch, whereas the notes h2 and c1 are far apart. Thus, changing the octave number affects musical meaning. Consequently, it is likely that non-musicians confuse the octaves more often than the musicians, because for them there is no semantic information involved. To sum up, it was hypothesized that the skill effect on the distribution of errors would be evident in the letter presentation condition, but not necessarily in the note presentation condition.

Discussion A signicant effect of skill level on recalling visually presented notes was seen in the rst experiment. In terms of the hypothesis, the better performance of the experts indicates that pre-learned knowledge plays a role in the construction of the representations in this task (see also Charness, 1988; Chase & Simon, 1973; de Groot, 1966; Saariluoma, 1991). The difference in the recall of the musical patterns and their mirror counterparts for the musicians group but not for the non-musicians group indicates that the musicians superior recall of musical patterns was not related to such factors as visual gestalt properties, verbal rehearsability or familiarity of the stimuli. Thus, musicians were able to construct more robust mental representations of novel musically meaningful note patterns than from their auditorily and musically less meaningful counterparts, even though the presentation was visual and piecemeal so that instant mapping of single notes onto LTM patterns was impossible. Instead, a sequence of notes had to be kept active in working memory before access to pattern knowledge in LTM could add encoding support. The skill effect and the performance levels were comparable with those reported in other studies in music cognition. The results of previous studies using different presentation methods and varying levels of randomness in musical stimuli show mean recall levels of about 12.5 notes for non-musicians, and about 26 notes for musically experienced participants (Halpern and Bower, 1982; Meinz and Salthouse, 1998; Roberts, 1986; Sloboda, 1976). In the current study the mean chunk recall levels refer to correct recall of 3 4 or 3 + 3 successive notes in the control group, and 4 8 successive notes or several groups of three or more successive notes in the expert group. Since the method of the present research produces results comparable to those of previous studies in music cognition, it is reliable for studying expertise effects in music.

EXPERIMENT 2 The second experiment concerned the role of presentation mode. It was studied whether the visual symbols that are used play a signicant role in the process of representation construction. The note presentation method used in the previous experiments was compared to another method familiar to musicians, in which letters referring to note names are used to present musical stimuli. In the letter system, a later position in the alphabet corresponds to a higher pitch. This system is less analogical than the note presentation system, in which the higher the note is located on the stave, the higher is the pitch of the sound it refers to. Thus, the relation between letter names and pitches is less direct than the relation between visual notes and corresponding pitches. Furthermore, musicians do not use note names normally in their daily musical activities, unlike typical visual notation. It was assumed that if mental representation construction is

Method
Participants. The expert group consisted of 12 students majoring in music (age 2029 years, 7 females, and 5 males) that had actively practiced one or more musical instruments for 1022 years. They

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

92

V. Kalakoski

Scand J Psychol 48 (2007) Table 1. Recall (chunk scores) as a function of presentation mode (visual notes vs. letter names) and skill level (musicians vs. nonmusicians) Visual notes Skill level Musicians Non-musicians M 7.54 4.22 SD 2.96 1.87 Letter names M 8.12 4.67 SD 3.24 2.47

were studying towards a degree of professional musician or music educator in one of the music education institutes. They had completed several modules and advanced courses in performance, music theory and solfeggio, and two of them already had a degree of music educator. They all reported an ability to connect visual notes with auditory tunes. None of them had participated in the previous experiments. The control group consisted of 12 students of the introductory course in psychology (age 1925 years, 8 females, and 4 males). Some of them had played a musical instrument for less than 5 years as a child, but none of them had had music training for several years. Some of the control participants had knowledge about musical notes, but none of them was able to uently read musical notation. They had not participated in the previous experiments. Stimulus material. Twenty-four note patterns consisting of 16 notes were picked from a songbook used in the elementary and secondary school. The book consisted of childrens songs and traditional melodies. All the melodies were transposed to a scale, in which they could be presented without any sharps or ats. Only lled dots representing quarter notes were used. If the original melody included half notes, they were replaced by two quarter notes in order to preserve the musical rhythm of the melodies. The original idea was to study also the effect of stimulus type (music vs. mirror) as in the previous experiments. Thus, for each note pattern, contrasting pairs, each like a mirror image of the original pattern, were constructed. Unfortunately, it was found only afterwards that one block of stimuli seemed to be more difcult than the others. Consequently, stimulus block affected recall via interaction with stimulus type (musical vs. mirror) and presentation condition (note vs. letter). Because of the problems with the stimulus blocks, the music and mirror conditions were combined, as the effect of stimulus type was not the main interest of this experiment. Procedure. The note patterns were presented in two conditions. In the note condition, 12 patterns were presented on a staff as in the general procedure. In the letter condition, 12 patterns were presented using written note names, so that the note name was shown on the screen for two seconds, followed by a one-second pause before the following note name. Note names consisted of the letters c, d, e, f, g, a, h, which are the symbols used for note names in the Finnish language, and the digits 1 or 2, indicating the octave (e.g., c1, e1, g2). The subjects were instructed to try to hear the melodies in their minds while storing them. The patterns and their presentation order, and the order of conditions, were counterbalanced across the subjects. Scoring. The chunk scoring system was used as in the previous experiment. Furthermore, the number of errors in each experimental condition was counted using the following classication. Displaced notes were classied as belonging to transposition categories if a series consisting of two or more notes: (a) was displaced one, two or three steps above or below the original notes (e.g., e-g-a instead of d-f-g); (b) was displaced in the left-right orientation in the previous or following serial positions; (c) the single notes were in incorrect successive order (e.g., e-g-a instead of a-e-g); or (d) the single notes were displaced an octave apart. Category (e) was reserved for the notes that could not be classied according to the previous rules.

were able to recall the presented sequences better than the non-musicians, F(1, 22) = 11.15 , p < 0.01, MSE = 12.51, 2 = 0.34, but the Presentation mode did not affect recall, F(1, 22) = 1.83, n.s., MSE = 1.93, 2 = 0.08. The interaction between Skill-level and Presentation mode was not signicant either, F(1, 22) = 0.10 , 2 = 0.002. The number of errors was calculated for note and letter conditions. Error percentages (the number of errors divided by the number of all recalled items) in the musician group were 29.71% (SD = 16.95) and 25.65% (SD = 16.56) for note and letter presentation conditions, respectively. In the control group the error percentages were 38.30% (SD = 14.31) and 33.97% (SD = 15.96) for note and letter presentation conditions, respectively. The errors were classied into ve categories. The observed frequencies, expected values, and post hoc cell contributions for the error categories are presented in Table 2. A chi-square test was used to study the relationship between skill level and error category separately for the note and letter conditions. The results showed that in the note condition there was no difference in the error distributions between musicians and the non-musicians, 2(4) = 3.77, n.s. However, in the letter condition, skill level affected the distribution of errors, 2(4) = 38.27, p < 0.001. The post hoc cell contributions at the level of p < 0.05 showed that the musicians made more errors concerning 13 step transpositions, and the non-musicians had less such errors than expected. Furthermore, as seen in Table 2, the musicians made less octave errors, and the non-musicians made them more than expected.

Discussion In this experiment, a signicant effect of skill level was found again, whereas the difference between the letter and note presentation conditions was not signicant. Thus, it seems that musically experienced participants are able to use letter names for notes as efciently as note symbols on a stave. This indicates that the better recall of musicians than the non-musicians cannot be explained by perceptual visual chunking, because in the letter presentation condition the musical visual cues are not available. Similar results have been obtained in chess. Chess players performance in memory tasks with blindfolded encoding was found to be about the

Results The effects of skill level and presentation mode (note vs. letter) on the chunk-scored recall was tested using a 2-way ANOVA (Table 1). The results showed that the musicians

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

Scand J Psychol 48 (2007)

Skill and recall of notes 93

Table 2. Observed frequencies of errors, expected values, and post hoc cell contributions for musicians and control participants by presentation mode: (a) visual notes, and (b) letter names a) Visual notes Skill level Control b) Letter names Skill level Control

Error categories a) 13 steps Observed freq. (Expected values) Post hoc cell contr. b) Position Observed freq. (Expected values) Post hoc cell contr. c) Order Observed freq. (Expected values) Post hoc cell contr. d) Octave Observed freq. (Expected values) Post hoc cell contr. e) Miscellaneous Observed freq. (Expected values) Post hoc cell contr. Totals

Musicians

Total

Musicians

Total

160 (155.19) 0.62 141 (129.80) 1.53 32 (31.61) 0.10 0 (0.96) 0.06 304 (320.44) 1.83 638

164 (168.81) 0.62 130 (141.20) 1.53 34 (34.89) 0.10 0 (1.04) 0.06 365 (348.56) 1.83 694

324

123 (97.26) 4.05 128 (138.23) 1.42 44 (42.46) 0.35 12 (31.10) 4.96 241 (238.95) 0.25 548

74 (99.74) 4.05 152 (141.77) 1.42 42 (43.54) 0.35 51 (31.90) 4.96 243 (245.05) 0.25 562

197

271

280

66

86

63

669

484

1332

1110

Notes: The error categories were (a) notes displaced 13 steps above or below the original notes; (b) notes displaced in the left-right orientation in different serial positions; (c) notes in incorrect successive order; (d) notes with a correct name but a wrong octave; and (e) notes which could not be classied according to the previous rules.

same with visual and verbal presentation, or even if the chess pieces were replaced by black dots that moved according to the rules of the game (Saariluoma & Kalakoski, 1997). Apparently experts are able to rapidly transform any representation format to one that allows the use of pre-learned knowledge. The results concerning the effect of skill level on the distribution of errors indicate that there is a qualitative difference between the two skill groups in the way of representing the presented stimuli. In the letter-name condition, skill level affected the distribution of errors as hypothesized. Musicians made more vertical transposition errors than the control group, whereas the control group made more scale number errors than the musicians. In the visual note pattern condition, the distribution of errors in the two skill groups did not differ, which suggests that the musically meaningful transposition errors may be confounded with visually meaningful errors, as was hypothesized.

GENERAL DISCUSSION The results of the present study indicate that pre-learned knowledge and skills support the construction of meaningful

musical structures even when stimulus presentation is visual and successive. To our knowledge, this phenomenon has not been demonstrated earlier in the music cognition literature. The rst experiment showed that experts superior ability to construct representations is not based on verbal rehearsal or direct recognition of good visual patterns and familiar tunes: only the level of musical meaningfulness of stimuli affected performance, and it affected only the musicians performance. Since this expertise effect was also found when unfamiliar but musically meaningful material was used, the process of utilizing pre-learned knowledge was not based on direct recognition of familiar melodies. The second experiment further showed that the musicians better performance was not based on perceptual visual chunking, but representations could be constructed as efciently from note names as from visual notes. Furthermore, the nature of errors suggested that the representations of musicians and non-musicians differed qualitatively. The results showed that in music, as in other domains, experts are able to use pre-learned task-specic knowledge and skills in construction of representations in order to connect the individual pieces together to a meaningful pattern, and thus overcome the capacity limitations of short-term working

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

94

V. Kalakoski

Scand J Psychol 48 (2007)

memory (Charness, 1988; Chase & Ericsson, 1982; de Groot, 1966; Ericsson & Kintsch, 1995). Theoretically, it is an important nding that an expertise effect can be found in a representation construction task, when perceptual factors are controlled. This nding indicates that the ability to chunk incoming information into meaningful units does not require that complete familiar patterns are accessible to encoding processes (Saariluoma, 1991). Presumably, the musicians were able to construct an internal auditory representation from the visually presented notes, as was requested in the instructions. The musicians subjective experience suggests that they used auditory imagery. Although the evidence is only indirect, several pieces of information concerning the musicians performance indicate that the representation constructed from visually presented notes was based on auditory imagery. First, when other factors were controlled in the rst experiments, the auditory goodness of the stimuli affected the musicians performance. Second, the effect of skill level on the distribution of errors suggests that the musicians used auditory representations, which resulted in musically meaningful errors, whether the presentation format was based on notes or letters. Other studies also suggest that auditory imagery can be evoked with visual presentation of a complex note pattern (Brodsky et al., 2003), and it is reasonable to assume that auditory images were also used in the present task. Although imagery representations often seem to be sensespecic, it is also possible to use several modalities at a time, as is the case with the perceptual world (Intons-Peterson, 1992). It is likely that also other modalities than the auditory one can be used when a representation is constructed from a visual notation. The rst pilot experiment suggests that visual and other non-musical factors can enhance construction of representations from musical notes. This is in line with the previous research on briey presented musical patterns and locations in a matrix that have shown an effect of visual complexity on recall (Halpern and Bower, 1982; Saariluoma, 1992). There is also experimental evidence suggesting that representation construction in music may involve motoric and kinesthetic-like processes (Brodsky et al., 2003; Mikumo, 1994). In the light of the present research and previous ndings, an important issue for the future research is to study in more detail the processes by which separate fragments of information are maintained in the short-term working memory system and united into a meaningful chunk. The present method seems to be suitable for studying this process, although there are also limitations in it. One is that the method does not necessarily demonstrate the genuine excellence the musicians possess. Although musicians have practiced associating visual notation and auditory melodies, the method of presenting briey, or in a piecemeal fashion, visual notes is not a normal task for musicians. The task may also be too limited to provide insights into processing of complex musical pieces. Neither is the method in its

current form suitable for studying musical performance. Therefore, more ecologically valid presentation methods are needed if the main interest of the research is related to other aspects than the mechanisms underlying representation construction in music. Nevertheless, there are several reasons for using the present method to study the process of representation construction. Studying how experts are able to apply their knowledge and skills in a novel task provides a way to capture the chunking mechanisms before they become fully automatic. Working memory capacity limitations must set the upper bounds to what can be included in a chunk when presentation is sequential because elements of a future chunk must all be available at the same time. It would therefore be interesting to rst nd the working memory limits for meaningless material and then manipulate the size of available LTM chunks that can be used to support encoding. This could be done, for instance, by embedding musically well-formed patterns of different length within random tone sequences. For example, this kind of research is needed to solve the controversies concerning what the mechanisms are that underlie how pre-learned knowledge and skills in long-term memory are used in the process of chunking incoming information (Ericsson et al., 2000; Simon & Gobet, 2000; Vicente, 2000; Vicente & Wang, 1998). Furthermore, the multi-modal character of representations is a little-examined phenomenon that can be captured with this method. The question is relevant especially in music, where expert musicians deal with visual note patterns, auditory sounds and motor representations of how to produce the melody with a musical instrument. A subsequent research question is how multi-modal information is maintained and rehearsed in working memory, and how information from perceptual processes, several working memory subcomponents, and long-term memory are integrated into a unitary representation (e.g., Baddeley, 2000). The nature of interaction between long-term and short-term memory has recently become a timely topic not only in expert memory research, but also in the research on short-term memory and immediate serial recall (Botvinick, 2005). Moreover, the successive visual presentation method seems to be a reliable way to study the detailed mechanisms underlying construction of musical representations. The method produced typical expertise effects, and the performance levels were comparable to those of previous studies in music cognition (Charness, 1988; Clifton, 1986; Halpern & Bower, 1982; Meinz & Salthouse, 1998; Roberts, 1986; Sloboda, 1976, 1985). Furthermore, the visual successive presentation method is also applicable to other task domains, for instance, chess and taxi driving (Kalakoski & Saariluoma, 2001; Saariluoma & Kalakoski, 1997, 1998). Therefore, it may offer a way to address more general issues of skilled performance, like whether theories of expertise effects really apply to a wider range of topics (Ericsson et al., 2000; Simon & Gobet, 2000; Vicente, 2000).

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

Scand J Psychol 48 (2007)

Skill and recall of notes 95

Ericsson, K. A. & Kintsch, W. (2000). Shortcomings of generic retrieval structures with slots of the type that Gobet (1993) proposed and modelled. British Journal of Psychology, 91, 571590. Ericsson, K. A., Patel, V. & Kintsch, W. (2000). How experts adaptation to representative task demands account for the expertise effect in memory recall: Comments on Vicente and Wang (1998). Psychological Review, 107, 578592. Gathercole, S. E. (1995). Is nonword repetition a test of phonological memory or long-term knowledge? It all depends on the nonwords. Memory & Cognition, 23, 8394. Gathercole, S. E., Willis, C. S., Emslie, H. & Baddeley, A. D. (1991). The inuences of number of syllables and word-likeness on childrens repetition of nonwords. Applied Psycholinguistics, 12, 349367. Gobet, F. (2000). Some shortcomings of long-term working memory. British Journal of Psychology, 91, 551570. Gobet, F., Lane, P. C. R., Croker, S., Cheng, P. C.-H., Jones, G., Oliver, I. & Pine, J. M. (2001). Chunking mechanisms in human learning. Trends in Cognitive Sciences, 5, 236243. Gobet, F. & Simon, H. A. (1996). Templates in chess memory: A mechanism for recalling several boards. Cognitive Psychology, 31, 1 40. de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton. de Groot, A. D. (1966). Perception and memory versus thought: Some old ideas and recent ndings. In B. Kleinmuntz (Ed.), Problem solving. New York: Wiley. Halpern, A. R. (1988). Mental scanning in auditory imagery for songs. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 434 443. Halpern, A. R. & Bower, G. H. (1982). Musical expertise and melodic structure in memory for musical notation. American Journal of Psychology, 95, 3150. Hatta, T., Hirose, T., Ikeda, K. & Fukuhara, H. (1989). Digit memory of soroban experts: Evidence of utilization of mental imagery. Applied Cognitive Psychology, 3, 2333. Hishitani, S. (1989). The usefulness of studies of imagery experts in imagery research. Journal of Mental Imagery, 13, 119134. Hubbard, T. L. & Stoeckig, K. (1988). Musical imagery: Generation of tones and chords. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 656 667. Intons-Peterson, M. J. (1992). Components of auditory imagery. In D. Reisberg (Ed.), Auditory imagery (pp. 4571). Hillsdale, NJ: Lawrence Erlbaum Associates. Kalakoski, V. & Saariluoma, P. (2001). Taxi drivers exceptional memory of street names. Memory & Cognition, 29, 634 638. Keller, T. A., Cowan, N. & Saults, J. S. (1995). Can auditory memory for tone pitch be rehearsed? Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 635645. Meinz, E. J. & Salthouse, T. A. (1998). The effects of age and experience on memory for visually presented music. Journal of Gerontology: Psychological Sciences, 53B, P60P69. Mikumo, M. (1994). Motor encoding strategy for pitches of melodies. Music Perception, 12, 175197. Povel, D.-J. & Jansen, E. (2001). Perceptual mechanisms in music processing. Music Perception, 19, 169198. Richman, H. B., Staszewski, J. J. & Simon, H. A. (1995). Simulation of expert memory with EPAM IV. Psychological Review, 102, 305330. Roberts, L. A. (1986). Modality and sufx effects in memory for melodic and harmonic musical materials. Cognitive Psychology, 18, 123157. Saariluoma, P. (1989). Chess players recall of auditorily presented chess positions. European Journal of Cognitive Psychology, 1, 309320. Saariluoma, P. (1991). Aspects of skilled imagery in blindfold chess. Acta Psychologica, 77, 6589.

To sum up, the results gathered with the current method specify the variables that affect the process of representation construction in music. With this method, it is possible to carefully control the contents and the timing of the stimulus presentation and representation construction in order to learn more about the nature of processes underlying expertise effects. Since the results are comparable to those of previous research, the method seems reliable. Furthermore, the method catches the auditory nature of representation construction in music and offers a way to study more carefully the cognitive process underlying mental imagery. In future studies, it may offer a new tool to investigate the core of chunking mechanisms and musical imagery, and to solve some of the theoretical controversies involved in this research.
This research was supported by Finnish Konkordia Association. I am grateful to Elisabet Service, Kimmo Alho, and the anonymous reviewers for their valuable comments on the earlier versions of this article.

REFERENCES
Baddeley, A. (1986). Working memory. Oxford: Clarendon Press. Baddeley, A. (2000). The episodic buffer: a new component of working memory? Trends in Cognitive Sciences, 4, 417 423. Baddeley, A. & Logie, R. (1992). Auditory imagery and working memory. In D. Reisberg (Ed.), Auditory imagery (pp. 179197). Hillsdale, NJ: Laurence Erlbaum Associates. Botvinick, M. M. (2005). Effects of domain-specic knowledge on memory for serial order. Cognition, 97, 135151. Brodsky, W., Henik, A., Rubinstein, B.-S. & Zorman, M. (2003). Auditory imagery from musical notation in expert musicians. Perception & Psychophysics, 65(4), 602612. Charness, N. (1976). Memory for chess positions: Resistance to interference. Journal of Experimental Psychology: Human Learning and Memory, 2, 641653. Charness, N. (1988). Expertise in chess, music, and physics: A cognitive perspective. In L. K. Obler & D. Fein (Eds.), The exceptional brain. Neuropsychology of talent and special abilities. New York: Guilford Press. Chase, W. G. & Ericsson, K. A. (1982). Skill and working memory. The Psychology of Learning and Motivation, 16, 158. Chase, W. G. & Simon, H. A. (1973). The minds eye in chess. In W. G. Chase (Ed.), Visual information processing (pp. 215281). New York: Academic Press. Clifton, J. V. (1986). Cognitive components in music reading and sight reading performance. Unpublished Doctoral dissertation, University of Waterloo, Waterloo, Ontario. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral & Brain Sciences, 24, 87185. Deakin, J. M. & Allard, F. (1991). Skilled memory in expert gure skaters. Memory & Cognition, 19, 7986. Deutsch, D. (1980). The processing of structured and unstructured tonal sequences. Perception & Psychophysics, 28, 381389. Dowling, W. J. (1972). Recognition of melodic transformations: Inversion, retrograde, and retrograde inversion. Perception & Psychophysics, 12, 417 421. Ericsson, K. A. & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211245.

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.

96

V. Kalakoski

Scand J Psychol 48 (2007) Tillmann, B., Bharucha, J. J. & Bigand, E. (2000). Implicit learning of tonality: A self-organizing approach. Psychological Review, 107, 885913. Waters, A. J., Underwood, G. & Findlay, J. M. (1997). Studying expertise in music reading: Use of a pattern-matching paradigm. Perception & Psychophysics, 59, 477488. Vicente, K. J. (2000). Revisiting the constraint attunement hypothesis: A reply to Ericsson, Patel & Kintsch (2000) and Simon & Gobet (2000). Psychological Review, 107, 601608. Vicente, K. J. & Wang, J. H. (1998). An ecological theory of expertise effects in memory recall. Psychological Review, 105, 33 57. Zatorre, R. J., Halpern, A. R., Perry, D. W., Meyer, E. & Evans, A. C. (1996). Hearing in the minds ear: A PET investigation of musical imagery and perception. Journal of Cognitive Neuroscience, 8, 2946. Received 9 June 2005, accepted 16 January 2006

Saariluoma, P. (1992). Do visual images have gestalt properties? The Quarterly Journal of Experimental Psychology, 45A, 399420. Saariluoma, P. & Kalakoski, V. (1997). Skilled imagery and longterm working memory. American Journal of Psychology, 110, 177201. Saariluoma, P. & Kalakoski, V. (1998). Apperception and imagery in blindfold chess. Memory, 6, 6790. Schrmann, M., Raij, T., Fujiki, N. & Hari, R. (2002). Minds ear in a musician: Where and when in the brain. NeuroImage, 16, 434440. Simon, H. A. & Gobet, F. (2000). Expertise effects in memory recall: Comments on Vicente and Wang 1998. Psychological Review, 107, 593600. Sloboda, J. A. (1976). Visual perception of musical notation: Registering pitch symbols in memory. Quarterly Journal of Experimental Psychology, 28, 116. Sloboda, J. A. (1985). The musical mind: The cognitive psychology of music. London: Oxford University Press.

2007 The Author. Journal compilation 2007 The Scandinavian Psychological Associations.