Vous êtes sur la page 1sur 69


Contextual Resource Evaluation Environment

CREE Literature Review

CREE Deliverable S1D1

Caroline Ingram and Chris Awre

August 2005

CREE Deliverable S1D1

CREE Literature Review

“Several thousand studies have appeared and, clearly, it is impossible to review all of this
literature.” (Wilson 1994)


Research and teaching in HE and FE is necessarily driven by the demand for the latest
information, but in a multi-disciplinary world, how academic staff, who are increasingly pressed
for time, find relevant resources is rarely considered. Skills in information research are
fundamental, yet the recent proliferation of search tools, portals, gateways, and archives, can be
almost as bewildering as the terminology used to describe them. The means of delivery of
information are, at best, a secondary consideration. Compounding the difficulties in getting
efficient access to information is the developing entrenchment of generic tools - “I’ll Google it”
- and a degree of inertia in the user community to use new specific tools. Familiarity with any tool
is useful in terms of getting a job done quickly, but that tool may not lead the user to the most
appropriate resource, or copy. A more targeted approach would be to use a subject-specific
discovery tool, but the difficulty lies in persuading users to make the switch.

An oft-stated goal of organisations such as JISC and the NSF surrounding electronic resources is
to let users “transparently use and share distributed resources”1. Middleware is software that
connects two or more otherwise separate applications across the Internet or local area networks.
More specifically, the term refers to an evolving layer of services that resides between the network
and more traditional applications for managing security, access and information exchange2. Over
the past few years the way we exploit and share distributed resources has been revolutionised by
developments in middleware, such as peer-to-peer (P2P) networks (Zhang et al. 2004).

The Government’s recent Education Consultation Document 3 states that, “e-Learning has the
potential to revolutionise the way we teach … and to bring high quality, accessible learning to
everyone – so that every learner can achieve his or her full potential.”

Aims and Objectives

This CREE project literature review will review the functional and technical aspects of
information discovery, delivery and use in Higher Education in two separate sections. The
objectives of the literature review are:

• To try to define the scope of discovery, retrieval, access and delivery mechanisms for
information used in research, learning and teaching
• To focus on how users deal with information they find online
• To draw together recent existing research about what users want (architecturally and
technologically) and what technologies they use
• To summarise current and possible future technologies suitable for use in discovering,
locating, retrieving and using online resources

In discussing the functional aspects of information discovery, delivery and use it has, been
necessary and useful to refer and describe some of the technologies that can be used to enable this
functionality. The technical review provided a further opportunity to discuss the technologies
themselves and how they have and are being used to facilitate the desired functionality. It is,
though, intended that functional and technical sections can be read separately as required.

NSF middleware initiative: www.nsf-middleware.org
Department for Education and Skills, Towards a Unified e-Learning Strategy July 2003
http://www.dfes.gov.uk/elearningstrategy/strategy.stm [12/08/03]

CREE Deliverable S1D1


Introduction 2
Contents 3
Functional Literature Review 4
1. Methodology 4
2. Discovering information 5
3. Locating information 9
4. Requesting information 10
5. Delivering/Accessing information 11
6. Presenting information 13
7. Interacting with information 16
8. Moving information 18
9. Using information 21
10. Managing information 23
11. Conclusions 26
12. References 27
Functional Literature Review Update 33
1. Searching behaviour (discover) 33
2. Searching library resources (locate) 35
3. Library vs. online searching (locate and request) 35
4. The Digital Library Portal (delivering and accessing information) 37
5. LibQUAL+ (presentation of information) 37
6. Use of online information (use and management of information) 39
7. The future potential of searching and browsing? 40
8. References 41
Technical Literature Review 42
1. Methodology 42
2. Z39.50 42
3. SRW/U 45
4. OAI-PMH 47
5. Web search engines 49
6. RSS 51
7. OpenURL 52
8. Metasearching 54
9. Usability 55
10. Portal frameworks 57
11. JSR 168 and WSRP 58
12. Conclusions 60
13. References 60

CREE Deliverable S1D1

Functional Literature Review

Dr Caroline Ingram

1. Methodology

The literature review will be carried out twice, once towards the start of the project to enable
development of criteria to assess and test user requirements for portal embedded and non-portal
embedded search and resource-push interfaces; and a second time after the user-requirements
phase. This approach closely follows the IDEAL (Initiating, Diagnosing, Evaluating, Acting,
Learning) model adapted from the Carnegie-Mellon Software Engineering Institute Capability
Maturity Model (CMM)4.

The functions suggested in this review are taken from the MODELS Information Architecture
(MIA) Functional Model 5 with additional suggestions to take into account the environment in
which CREE will work. It is intended that this will allow a perspective to be gained both from the
user and from the material or information involved.

Fig. 1: Information Environment Functional Model

The JISC Information Environment (Powell and Lyon 2002) provides resources for learning,
teaching and research to UK higher and further education, and thus services which provide the
functions in the model above must be consistent with its architecture. The Information
Environment enables resource discovery through the various portals in its ‘presentation layer’ (see
Fig 2), including the discipline specific UK Resource Discovery Network (RDN) hubs 6, the RDN
also being one of the participating gateways in the European Renardus service7.

Content providers in the ‘provision layer’ are expected to disclose their metadata for searching,
harvesting and for alerting services. This means that all resources within the Information
Environment should have an Internet search interface for end-users and at least some of the
following for machine-to-machine resource discovery: a Z39.50 (Bath Profile cross-domain
compliant) search interface; an OAI (Open Archives Initiative)8 interface for metadata harvesting;
and an RDF Site Summary (RSS) (Powell 2002) alert channel capability. In addition resources

RDN, Resource Discovery Network: http://www.rdn.ac.uk/
Renardus: http://www.renardus.org/
Open Archives Initiative: http://www.openarchives.org/

CREE Deliverable S1D1

may support OpenURL (Van de Sompel and Beit-Arie 2001) for article discovery and location,
where appropriate.

Throughout the review the institutions and features referenced are included as examples of
developments to date rather than as a definitive set. The case studies set out below give outline
descriptions of services (mostly developed by JISC), which could be used by the community for
the function being described. There are many similar services in existence worldwide, some of
these are referenced below. Chowdhury and Chowdhury (2001) have also recently published a
summary of online services offering information and searching functions. Where possible freely
available documentation has been referenced enabling readers to follow up citations where
relevant. Footnotes contain links to relevant websites (all verified in October 2004).

2. Discovering information

This section addresses how users discover or search for information. This includes reference to
where users discover information, why they are discovering information, what tools they use, and
why they use these tools in particular.

2.1. Discovery on the Internet (or web searching)

Much research on user behaviour in respect of information retrieval (IR) systems has been carried
out (Dervin and Nilan 1986; Johnstone et al. 2004). Although research on retrieval from the World
Wide Web (e.g. Johnson et al. 2003) is not as advanced, there are some studies (e.g. Proper and
Bruza 1999) which use the phrase ‘information discovery’ to apply solely to the identification and
retrieval of relevant content from electronic sources.

Surveys of web usage give some sense of what the average web searcher is doing and point to
differences between web searches and queries with traditional IR systems. Observations of the
average web searcher (Spink et al. 1998; Ellis et al. 1998) point out that ineffective use of the web
may be caused by lack of understanding of how a search engine interprets a query. Few users are
aware of when a search service defaults to AND or OR and expect a search engine to
automatically discriminate between single terms and phrases. Also, devices such as “relevance”
work well if the user ranks ten or more items, when in reality users will only rank one or two items
for feedback (Croft, 1995). Koll (1993) found that users provide few clues as to what they want,
approaching a search with an attitude of 'I'll know it when I see it', which it is implied would be the
case for web searching also (Spink et al. 1998).

Larsen (1997) thought that Internet search systems would evolve to meet the behaviour of the
average web searcher. There has been a shift towards the introduction of search features that
appear to respond to the ways in which users actually search these systems, e.g. in search
assistance and navigation, but the notion that improved interaction is key in improving results
maybe attractive in principle but not necessarily true in reality. Often "users don't want to interact
with a search engine much beyond keying in a few words and letting it set out results" (Andrews
1996). Most users do not use advanced search features (Johnson et al. 2001), or enter complex
queries, or want to interact with search systems. As a consequence many search engines now try to
automate query formulation, shifting the burden from the user to the system, and meeting Larsen’s
(1997) expectation.

Spink et al. (1998) conducted a study in which 357 Excite users responded to an interactive survey
asking about their search topics, intended query terms, search frequency for information on their
topic and demographic data. Most respondents searched on a single topic. Many of the terms were
clearly meant as a phrase but there was no indication that the users had thought to add quotation
marks, which Excite requires to indicate a phrase search, otherwise terms are linked by the
Boolean operator OR. Similarly, few queries included explicit Boolean or other operators.

CREE Deliverable S1D1

Search engines are constantly improving their web search facilities9. Jansen et al. (2000) analysed
transaction logs containing queries and from this argued: "while Internet search engines are based
on information retrieval principles, Internet searching is very different from IR searching as
traditionally practised and researched in online databases, CD-ROM and OPACs". They found that
web users are not comfortable with Boolean and other advanced means of searching, and do not
frequently browse results beyond the first page. Other studies also show that most web searchers
do not view more that the first 10 results (Hoelscher 1998; Silverstein et al. 1999). In addition,
Jansen et al. (2000) found that few users went on to modify their original query and view
subsequent results. That the behaviour of web searchers follows the principle of least effort (Zipf
1949) has also been recorded by Marchionini (1992) who stated, “humans will seek the path of
least cognitive resistance" in relation to pre-web interfaces for information searching, and, with
reference to the Internet, Griffiths (1996) "increasing the cognitive burden placed on the user …
can affect successful retrieval of information. Where an application required fewer actions from
the user, greater success was achieved as there was less possibility for a user to make an error".

2.2. Student discovery strategies

A number of studies have been conducted into student use of electronic resources. In their
observational study of student Internet use, Cmor and Lippold (2001) concluded that: “students
would like to use the Web for everything”. They also put forward a number of other observations:

• Students will spend hours searching or just a few minutes

• Searching skills vary and students will often assess themselves as being more skilled than
they actually are
• Students will give discussion list comments the same academic weight as peer reviewed
journal articles.

Navarro-Prieto et al. (1999) sought to develop an empirically based model of web searching in
which 23 students were recruited from the School of Cognitive and Computer Science, University
of Sussex. Ten of these participants were Computer Science students and thirteen were Psychology
students. Their findings highlight a number of interesting points:

• While the Computer Science students are more likely to be able to describe how search
engines develop their databases, neither of the two groups have a clear idea of how search
engines use the queries to search for information;
• Most participants considered their levels of satisfaction with the results of their search to
be 'good' or 'OK'
• Most participants cannot remember their searches, and tended to forget those search
engines and queries that did not give any successful results.

From their research Navarro-Prieto et al. (1999) were able to identify three different general
patterns of searching:

1) Top-down strategy, where participants searched in a general area and then narrowed down
their search from the links provided until they found what they were looking for
2) Bottom-up strategy, where participants looked for a specific keyword provided in their
instructions and then scrolled through the results until they found the desired information.
This strategy was often used by experienced searchers
3) Mixed strategies, where participants used both of the above in parallel, a strategy only
used by experienced participants.


CREE Deliverable S1D1

Twidale et al. (1995) conducted a study that considered the role of collaborative learning during
information searching which informed the development of the Ariadne10 Collaborative Browsing
Project. Referring to relevant literature they identified common problems as: retrieving zero hits,
retrieving hundreds of hits, frequent errors, little strategy variation and locating few of the relevant
records. More general observations revealed a number of collaborative interactions between
students working in groups and comparing results.

David Nicholas and colleagues at City University, London, have conducted extensive studies on
users’ information behaviour, and have noted that in today’s Internet and digital library
environments users are ‘powerful’, ‘short on attention’, ‘promiscuous’, ‘untrusting’, and above all
are interested in the speed of delivery (Nicholas et al. 2003; Nicholas, et al. 2003).

Griffiths and Brophy (2002) record two results which raise very interesting and important issues in
terms of student use of Internet resources:

• Students prefer to locate information or resources via a search engine above all other
options, and Google is the search engine of choice
• 70% of participants in the study felt that their search was successful, but only half of these
thought that it was easy to locate information.

Griffiths and Brophy (2002) also reported that students either have little awareness of alternative
ways of finding online information to the search engine route or have tried other methods and still
go back to Google. Further to this, even when students are able to locate information it is not
always easy (even when using Google), and with a third of participants failing to find information,
user awareness, training and education needs to be improved.

However, there is still a tension between the use of online and print materials, as stated by Sandra
Grove, at St Georges Medical School, in a case study on information use: “Electronic information
is excellent for searching, but it doesn’t substitute for browsing – our students still want to borrow
materials and books are portable” (Simmons 1999).

Even amongst non-student users in a recent (Young 2004) survey of 243 users of the ALTIS
hospitality, leisure, sport and tourism RDN hub 11, scoring the question “I mainly use Google to
search the web”, using a value from 1 to 5 depending on how the user felt about a statement,
where a value of 1 indicated that they strongly agreed and value of 5 indicated that they strongly
disagreed, gave the following results:

Google Usage




1 2 3 4 5 More


Fig 2: “I mainly use Google to search the web”, ALTIS Survey results (Young 2004)

Further, Smith and Oliver (2004) observed that there seems to be a relationship between search
behaviour and academic maturity. Students in their earlier years don't persist with anything beyond


CREE Deliverable S1D1

Google, as they do not perceive the need. If the IE is truly to be embedded and integrated into
learning and teaching further work needs to be done to equip students with the awareness and
skills to use electronic resources other than Google.

2.3. Use of services for discovery

JISC content services consist of subject gateways, bibliographic databases, data, film and media,
and electronic journals providing access to information and resources from a wide variety of
content providers. JISC also provides network services (JANET12, JISCmail13), development
services (CETIS14, UKOLN15) as well as services that provide expert, specialist help in some
specific area or discipline. However, it is also common to refer to other electronic brokers,
catalogues, and tools as services:

Fig 3: JISC Information Environment (Powell 2003)

Almost all the types services and activities mentioned in Figure 3 offer some kind of user-interface
direct to the end-user and can therefore be thought of as presentation layer services or activities to
a certain extent. However, it is usually possible to select the core function of a chosen service,
project or software application and to position it accordingly. In this literature review services
have been placed as case studies into a particular functions from the MODELS Information
Architecture (MIA) Functional Model 16, however, some may fulfil multiple roles in the
architecture presented in Figure 3 (i.e. an aggregator service may also offer a presentation layer to
the user).

Powell (2003) mentions that he does not present the architecture with the intention of saying that
the only route to content is through a presentation layer 'portal', i.e. moving from front to back
through the diagram. Rather, it is expected that services will interoperate with each other in a
variety of ways and end-users are unlikely to perceive a neatly regimented hierarchy of services.


Xgrain and the EDINA GetRef service

The Xgrain project looked at ways to develop a cross-searching mechanism for Abstracts and
Indexing (A&I) databases and Library OPACs in the JISC Information Environment. These


CREE Deliverable S1D1

provide key discovery facilities for references to journal articles and other information objects that
can be accessed from the desktop. However, there is evidence that they are under-used 17,
especially in learning and teaching. Anecdotal evidence attributes this to a lack of
interconnectivity between the services and a perception that they may be too complex for use by

Xgrain (cross-grain) is a broker offering 'shallow' cross-searching of A&I databases across the
Information Environment (using the Z39.50 protocol among others) to address these underlying
problems. It also offers the facility to be transferred into other user interfaces for in-depth
searching and to connect automatically to request and delivery mechanisms.

A fully functional cross-searching tool, GetRef, has been developed using XGrain that enables the
user to present a simple search string to multiple services and receive the matching results from
each. EDINA plans to launch the GetRef service to the UK FE and HE communities.

GetRef has the following features:

• May be used directly by the end-user, or may be accessed by a Library portal via a Z39.50
• Provides OpenURL links, which may be resolved by any institutional resolver service,
such as SFX or LinkFinderPlus.
• EDINA will configure targets (bibliographic databases) and each institution's local OPAC
on an institution-by-institution basis, removing the need for library staff to spend time on
setup and configuration.
• Local branding to retain the look-and-feel common to an institution's other local services.
• Provides a variety of online learning materials, which cover cross-searching concepts and
gives guidance to users on conducting research activities.

A recent study carried out at York University in Canada (Fernandez 2002) found that online
current awareness providers (sending alerts to academics) are hard put to keep up with the
proliferation of web-based services that keep scientists updated. The currency of certain services
(e.g. PubMed and Scifinder Scholar) and their ready availability of linked full text and tables of
contents have made them popular among academic staff. A recent article by De Stricker (2002)
reinforces some of these conclusions. Holistic services, such as ISI Web of Knowledge, may also
meet some user needs (Quint 2001) in terms of discovering the latest information in their field, but
are unlikely to replace other discovery tools (Fernandez 2002).

3. Locating information

Following information discovery, this section looks at how the user locates the information
(assuming the discovery process doesn’t do this already), and what next step or steps the user can
or does take in locating the information.

Information literacy is often described as the ability to locate, manage and use information
effectively for a range of purposes (Bruce 1997). As such it is an important ‘generic skill', which
allows people to engage in effective decision-making, problem solving and research.

Smith and Oliver (2004) studied students at University College London doing an information
skills course at UCL. They asked students: a) to describe how they would go about producing the
bibliography; and b) to explain why they would do it this way. For discovering and locating
resources for a bibliography, only one source of electronic information was cited three times: the
UCL library, followed by two mentions of Google. Two students were less specific, mentioning
the Internet or websites in general. Only two students cited specific academic tools, which were
SciFinder, Science Direct and Athens. The most popular explanations for why they used these

internal EDINA report

CREE Deliverable S1D1

resources were awareness and ease of use. Only three students made a value judgement on the
resources they cited, one who noted the accuracy of the UCL library, and three Chemistry students
who appreciated the quality of information contained in the scientific databases.

Smith and Oliver (2004) surmise that user behaviour is very closely tied to specific need and the
nearer information resources are to the user's immediate concern, the more likely they are to be
located and used further. Kirkwood (2003) also concluded that decisions about the extent to which
students use particular media components are based at least as much upon the perceived benefits
to be gained from them, as from any intrinsic characteristic or quality of the particular materials or


ZBLSA and the GetCopy Service

The ZBLSA Project developed a linking tool that provides portals with the means to locate
services pertaining to journals. The tool developed is now used by EDINA under the brand name
'GetCopy'. It connects discovery of a reference to a journal article with services providing the most
appropriate full-text copy in printed or electronic form.

The main client communities of GetCopy are the A&I database services that operate at the JISC
National Data Centres and the RDN subject portals. They will use GetCopy to locate appropriate
copies of journal articles whose existence has been discovered by the end-user.

For libraries the main features link the end user from a reference to full text, locate an appropriate
copy of a journal or journal article, operate on existing permissions (i.e. not be involved in
authorisation or authentication) and eliminate the requirement for a library to host and maintain
complex table of subscription information to locate appropriate copies.

For publishers, GetCopy has been designed to be lightweight and business-neutral by operating on
existing permissions; this eliminates the need to be involved in the authorisation or authentication
required for document delivery transactions. GetCopy simply determines the location of
appropriate copies, and directs the end-user accordingly. It is intended that GetCopy will be a
primary mechanism for directing end-users from the UK FE/HE sector to publisher web sites.

The JUSTEIS report (Armstrong et al. 2002) conclusions imply that greater availability and easier
access to specialised services do not result in greater usage. In particular they found that students
used subject gateways very infrequently, and that e-journals were plagued by licensing and
bundling problems, though JISC deals were felt to help in this respect. The research community
was using services such as Web of Science18, but such services seem less applicable to other user

Referring back to case studies outlined in Simmons (1999), the information professionals
interviewed for these studies were commonly finding that users of online discovery systems (such
as library catalogues) were often needing further assistance in locating resources.

4. Requesting information

The next part of the process is to understand how the user requests the information once it has
been located. This section also investigates whether this is an online process that is just a click or
two away or a more detailed process involving a mixture of online and paper processes

Now ISI Web of Knowledge Service for UK Education, the new platform by which users can access all ISI products:

CREE Deliverable S1D1

Finding and locating references is one issue, getting the primary information referenced is often
completely different. Reference databases contain all the bibliographic information that is needed
to identify the request for an article in an unambiguous way. The aim would be to enable an end-
user sitting at a remote terminal or a terminal in the Library, to identify and select a bibliographic
reference from an electronic source (for example from a networked CD-ROM or online
bibliographic database) and then, with no or minimal re-keying, identify whether that item is held
in their own or a local library. If not, the system should transmit a request automatically to a
remote document supplier (such as British Library Document Supply Centre), with the opportunity
for the user to state the preferred delivery format and deadline. If the document concerned was a
journal article (as is the case with most inter-library loan requests), that article could then be
transmitted electronically direct to the user's own terminal or to a local printer.



The overall aim of the Docusend project was to create an accessible, easy to use one stop
integrated journal article delivery service. Docusend aims to bring together a wide variety of
document delivery and related services in ways that appear transparent and seamless to the user.

The specific objectives of the project were to:

• Create an accessible, easy to use one-stop integrated journal article delivery service
• Provide seamless access to journal resources from resource discovery through to delivery
• Develop the capability to act as a national gateway for document delivery requests
• Build a consortium of content suppliers including resource rich libraries and publishers
• Deliver journal articles in electronic format wherever possible
• Test and, if feasible, implement direct end-user requesting and end-user delivery

It will be critical to Docusend’s future success that it integrates with the full range of discovery
and location services being implemented in UK higher and further education. Since location is at
the heart of Docusend’s functionality, it is in this area that particular attention needs to be paid
both to technical interoperability and to business processes. Users will benefit from the use of
Docusend, which will take a lot of the routine work out of document delivery, initially for ‘docdel’
library staff, but subsequently for library users as well. The service will enable users (staff or end-
users) to find the documents they need within the collections of participating libraries, to request
them and to have them supplied electronically wherever possible, or in hard copy.

Thus, using a system such as that described above a user should be able to identify an article
useful to their research and receive it electronically, without needing to leave their own desk, and
with the minimum of transcription. The management information needed by the Library should all
be incorporated, but this would be invisible to the end-user, who would only be interested in
getting hold of the documents as quickly as possible. Where documents are returnable items (e.g.
books borrowed from another library or the DSC), the requesting mechanism would work in just
the same way, but the end-user could receive an electronic message that the book was available to
be collected, rather than an electronic version of it.

5. Delivering/accessing information

Next we consider how the user accesses the information or get it delivered. Delivery can be
online, where access is a more appropriate term, or through an inter-library loan process.

Chowdhury (2004) summarised that the user is the focal point of all information retrieval systems
because the sole objective of any information storage and retrieval system is to transfer
information from the source to the user. This is particularly true in the case of publishers
supplying access to published works.

CREE Deliverable S1D1

OpenURL19 is a protocol for interoperability between an information resource and a service

component. OpenURL works on the idea that links should lead users to an appropriate copy of a
resource. The OpenURL standard enables a user who has retrieved an article citation, for example,
to obtain immediate access to the "most appropriate" copy of that object through the
implementation of extended linking services, but in reality it carries out more than one step in the
process. The selection of the best copy is based on user and organisational preferences regarding
the location of the copy, its cost, and agreements with information suppliers, and similar
considerations. This selection occurs without the knowledge of the user; it is made possible by the
transport of metadata with the OpenURL link from the source citation to a "resolver" (the link
server), which stores the preference information and the links to the appropriate material. The
query sent contains metadata about the resource, rather than simply identifying the location of the

The two case studies in this section describe successful services in use in the academic community
in the UK, which provide access to resources and information online. Some services (as previously
mentioned) offer functionality in more than one section of the MODELS Information Architecture
(MIA) Functional Model 20.



ZETOC is a service designed for the academic community, paralleling the British Library’s
Inside21 service, which is targeted more towards commercial users. The primary ZETOC product
is an email alerting service based on tables of contents from over 20,000 journals and 16,000
conference proceedings. The resulting database is not only used to provide alerts to registered
users but is also available as a searchable target. All articles listed in ZETOC are available from
the British Library Document Supply Centre (DSC).

ZETOC includes the provision of links to full text by enabling references to be passed to
institutional OpenURL resolvers (where implemented), so that the individual user is directed to a
copy via the institution’s subscription to the journal. For institutions without a resolver service, the
reference can be passed to COPAC or LitLink. It is also possible to obtain a copyright cleared
copy from DSC, although evidence suggests that few in the academic community take this route
since there is an additional cost.

Where a copy cannot be located, mechanisms have been devised to enable a request to be
formulated for inter-library loan from the British Library via the institutional library service. The
local institution can customise the inter-library loan page. Z39.50 clients such as EndNote and
Bookwhere are also supported, enabling users to download and import references into their chosen
reference management package.
An evaluation report (November 2002) based on a survey of over 600 users from a range of
institutions suggests that ZETOC is widely used in UK higher education and reported
overwhelmingly positive views from users:

“ZETOC is a successful service. The majority of the respondents praise the way it offers
them a service they have not had before which enables them to keep on top of current
developments in their fields. It is a service that they find relatively straightforward to use
with very few usability problems.”


CREE Deliverable S1D1

There are a number of areas in which the service could be further enhanced. The use of ZETOC as
an OpenURL enabled search target is in development, which may attract further use. Users also
need greater coverage, which could be explored with the British Library.

Access to electronic journals is provided either by publishers themselves, or by aggregators. Most

e-journal publishers provide access to their journals from their websites (Lee and Morris 2000).
Usually if a library subscribes to a print version then access to its electronic version is available
free or for a small additional fee. Aggregators can be subscription agents, database providers,
index and abstracting publishers or primary publishers (McKay 1999) and they organise e-journal
access and administer passwords, table of content services, usage statistics and archiving (Lee and
Morris 2000). Aggregators also provide the advantage of providing access to a number of e-
journals through one search interface.



Since its launch in May 1998, according to its website, Ingenta has developed and grown to
become the leading Web exchange of academic and professional content online.

Ingenta supplies access to 6,000+ full-text online publications and a further 27,000+ publications.
For accessing Ingenta services registration is not always required, and anyone can use Ingenta and,
where available, access summaries of the articles. Registering allows access to full text according
to the personal or institutional subscription, and personal profiles can be set up

Ingenta provides publishers of academic and professional content with the means to leverage the
Web as a profitable distribution and marketing channel. The recent acquisition of Catchword
means Ingenta now gives publishers access to the most comprehensive set of online publishing
solutions available in the market, as well as serving the various online information needs of users.

Ingenta provides libraries and researchers with access to the most comprehensive collection of
academic and professional content available online. More than 8,000 academic, research and
corporate libraries, institutions and consortia, from around the world, currently rely on Ingenta for
managed access to academic and professional content.

EBSCO online22, an aggregator that provides access to full text articles from scholarly journals, is
based in the USA. As with Ingenta, EBSCO’s Electronic Journals Service (EJS) handles
electronic journal access and management needs. EJS Enhanced offers extensive features that help
with e-journal management tasks such as: tracking the registration status of e-journals,
authentication assistance to facilitate both on-campus and remote access to e-journal content,
automatic management of e-journal URLs and much more. EJS Enhanced serves as a consolidated
gateway that allows users to search and link to (OpenURL supported) more than 10,000 journals
and 5 million articles.

6. Presenting information

How the information is presented to the user, and whether this is related to any further functions
that can then be carried out, is addressed. We take into account whether there is an accessible
presentation alternative, and how the presentation of information influences what a user is able to
do next.

Presentation layer services provide the end-user with a "personalised, single point of access to a
range of heterogeneous network services, local and remote, structured and unstructured".

CREE Deliverable S1D1

Presentation within a portal is very important and it will be a key element within the CREE

Almost all access to JISC IE services is currently through the end-user's Web browser (Powell
2003), though there are some exceptions to this. Desktop reference managers, like EndNote and
ReferenceManager, and other desktop tools such as FeedReader and the Google toolbar, provide
the end-user's desktop with much of the functionality one might expect to get from portals and
other services. In the future, increased access to services through mobile technology such as
phones23 and PDAs (Kukulska-Hulme 2002) is anticipated.


RDN Subject Portals

The JISC defines a portal as "a network service that brings together content from diverse
distributed resources using technologies such as cross searching, harvesting, and alerting, and
collates this into an amalgamated form for presentation to the user”.

The RDN Subject Portals Project24 has been developing portal functionality for five of the subject
hubs of the Resource Discovery Network (RDN), BIOME, the bio-medical sciences hub; EEVL,
the engineering, maths and computer sciences hub; HUMBUL, the humanities hub; PSIGate, the
physical sciences hub and SOSIG, the social sciences hub. A subject portal, for the purposes of
this project, is a tailored view of the web within a particular subject area, with access to high-
quality information resources made easier for the user through aggregated cross searching;
streamlined account management; user profiling; and the provision of additional services.

The subject portals developed by the individual hubs are still demonstrators. SPP beta portals are
now available at the SOSIG, PSIgate, GEsource, Artifact, EEVL and BIOME hubs, and
HUMBUL will be made publicly available soon. SOSIG has received very positive feedback from
user testing. It was not planned to make them part of the hub/RDN service at this stage, however,
this is being explored with JISC. The other element of the project was to make the portlet software
available under an OSS licence.

Subject gateways are essentially online libraries containing links and pages on a variety of
categorised topics. They are good resources for users trying to do serious online research on a
specific area of interest (Bradley 1999). Subject gateways are also known simply as gateways or
digital collections. Each subject gateway may contain a variety of information such as: full text
documents, resource guides, bibliographies, directories, mailing lists and newsgroups, links to
other websites, news about events and so on (Dempsey 2000).

Many institutions provide access to institutional information via their institutional website. Given
the varied roles and activities that institutional stakeholders in the education sector carry out - from
managing budgets and delivering maintenance services to conducting research and educating
students - it is unsurprising that the information requirements of these groups are many and varied
and that when presented on mass such a range of resources leads to information overload 25.

Whilst the overabundance of institutional information is one reason for deploying an institutional
portal it is clear that institutional stakeholders also require access to coherent services and
resources. Eisler (2001) indicates the extent to which technical development has enabled portal
technology to address the issue of both information overload and coherence: “portal technology is

For an article o the use of phones in business for “receiving a range of information wherever you are”, see:
PORTAL project literature review

CREE Deliverable S1D1

driven by innovation in two primary functions: search/navigation and personalisation ...a portal
can help address to fundamental challenges facing users today, the overabundance and
discontinuity of information”.

The University of Nottingham began their Compass Project to provide an: “easy and coherent
route through to all the online information and services relevant to their [students] life” 26. The
emphasis on both ‘easy’ and ‘coherent’ reflects a need to counter the often confusing array of
information available. Olsen (2002) reveals that one – nameless – institution’s website review
exposed the fact that tuition fees appeared on six of the institutions 100,000+ web pages and that
each page cited a different figure.



The PORTAL project gives the JISC definition of an institutional portal, which describes the
various features as belonging to three categories – access to information resources (internal or
external), transaction-based services and collaborative tools:

“An institutional portal provides a personalised, single point of access to the online resources that
support members of an institution in all aspects of their learning, teaching, research and other
activities. The resources may be internal or external and include local and remote 'information
resources' (books, journals, Web-sites, learning objects, images, etc.), 'transaction-based services'
(room bookings, finance, registration, assignment submission, assessment, etc.) and 'collaborative
tools' (calendars, email, chat, etc.). Typically, access to many of these resources is restricted to
authenticated members of the institution”27

The restriction ‘to authenticated members of the institution’ referenced in the JISC definition
enables institutional portals to provide access to personalised information and services.
In an effort to better understand the preferences and requirements of those for whom institutional
portals are being constructed, the PORTAL project undertook a period of stakeholder consultation
between November 2002 and March 200328.

On the whole, respondents welcomed the role of an institutional portal as a means of gaining
access to both institutional and external resources. Personalisation was a major topic. At one level,
the ability to have the portal personalise displayed content by a users role would be welcomed and
useful. Beyond this, though, tensions appear between end users who might wish for a greater
degree of flexibility and the portal providers who wish to retain control over key aspects of
content, look and feel of the portal.

Since then a further period of consultation has been undertaken with users of the new portals.
Certain external information resources, such as the near-ubiquitous weather forecast, have been
rejected in favour of access to library resources, teaching materials, and personal information.
However, most users are also finding that the portal does not necessarily offer the functionality or
personalisation they would prefer.

The adoption of locally developed and commercial library portals in academic institutions is
having an impact on the use of quality information sources, as well as on internal library
workflows and efficiency. Library portals will interoperate with national services and institutional
portal and VLE developments. Building on previous studies and accounts, the JISC funded

EIS University of Nottingham 2002
JISC (2003) JISC Information Environment Architecture Portal FAQ. http://www.ukoln.ac.uk/distributed-systems/jisc-
Defining User Requirements for Institutional Portals: http://www.fair-portal.hull.ac.uk/WP3.html

CREE Deliverable S1D1

LibPortal 29 project reviewed current practice and future prospects. LibPortal defined a library
portal as: a network service that brings together content from diverse resources, including the
library catalogue, on-line subscription reference material, e-journals and learning and teaching
material. Dempsey (2003) argues that portals provide useful integration and presentation services,
but that they should be seen as one component of a broader set of services the library is building to
engage users and useful resources. Dempsey presents an example of how a portal is helping the
library at the University of Delaware to manage its digital resources. The article develops the
premise that portals are defined by their ability to cross-search many resources and that the end
results can be personalised by the library user. Dempsey also considers that the desire for a ‘one-
stop shop’ for information may be difficult to achieve given the proliferation of resource providers
over recent years. He suggests that library services may have to explore a range of services it is
appropriate to provide for their users, and consider delivery through a planned architecture, as the
JISC Information Environment is attempting to do. A further review of this area has been carried
out by the LibPortal project as a part of their final report30.

7. Interacting with information

This function addresses whether the user is able to interact with the information in an active or
passive way once it has been obtained. It is also interesting to consider whether the information
can be used proactively (e.g. using OpenURL), or edited once accessed.

Globally there are numerous operational projects, both private and public, that promote the
repurpose and reuse of learning objects within the community they serve. At present, Canada is
one of the leading countries in research and development for the design, production, sharing and
reuse of learning objects. There are a number of initiatives in place in Canada currently, and in
particular the eduSource project is focused on the creation of a network of linked and interoperable
learning object repositories across Canada. The project will look at defining the components of an
interoperable framework, the web services that will tie them all together and the protocols
necessary to allow other institutions to enter into that framework. The support for the eduSource
project is extremely strong, with a total of seven primary academic partners and over thirty-five
commercial and academic secondary partners.

One of the major initiatives from the US is MERLOT, a free and open resource aimed primarily
for teaching staff and students from higher education. The US is responsible for producing the
MERLOT repository 31, in an effort to promote the cataloguing and sharing of publicly available
learning materials. MERLOT is a collaborative effort of a consortium of higher education
institutions that is managed by the MERLOT administrative team. MERLOT stores virtual
locations of the learning objects and does not store the objects within the repository itself.
MERLOT's mission is to improve the effectiveness of teaching and learning by expanding the
quantity and quality of peer reviewed online learning materials that can be easily incorporated into
faculty-designed courses.

MERLOT is a dynamically designed, web-based software tool to support the development of

teaching and learning communities, collections, services, and research in higher education.
MERLOT’s success could be attributed to there being no requirement to register in order to use the
service. Its success is also often attributed to the peer review it provides for the learning objects it
points to. However, only 9% of materials are peer reviewed (Orhum 2004). Also since only the
LOs’ virtual locations are stored within the repository this reduces the amount of resources and
cost needed to maintain the service. Such as model of practice is effective although is limited
inasmuch as it does not support the long-term preservation of the learning materials.

MERLOT http://www.merlot.org

CREE Deliverable S1D1

Australia has been regarded internationally as being a leader in the delivery of distance learning,
and in recent years in elearning (Mason, 2003). The Australian government has funded a number
of initiatives across all education sectors, all of which are summarised in Mason’s paper. They
have also developed a portal and suite of web services, known as EdNA Online32, that brings
together many of these initiatives in a single interface. EdNA Online provides access to 16,000
Australian quality-assured education and training resources across all educational sectors. It allows
end users to search simultaneously from the EdNA Online resources, GEM from the US,
MERLOT and the Vocational Education and Training Database33 of technical and vocational
education and training research (VOCED) in Australia.



The aim of e-MapScholar was to promote and enhance the use of digital map data (e.g. Ordnance
Survey data available to the UK HE/FE through EDINA Digimap) in learning and teaching, by
developing resources applicable to any geo-spatial data. e-MapScholar offers a number of online
environments, for use in different learning and teaching approaches, including multi-mode or
blended learning. The learning resources incorporate interactivity.

The LRC supports the learning and teaching of concepts and skills about spatial data. A CMS
supports customisation of the resources (text, tools and geospatial content) to suit local contexts
and can be carried out by individuals or teaching teams. Customised resources can be added back
to the pool, enabling further sharing of ideas and good practice. The proof-of-concept Virtual
Work Placement (VP) demonstrator tool combines creative simulation with game-play
opportunities, and supports learning about real-world application of digital geographic information
in the work place.

The e-MapScholar teaching case studies are regarded by LTSN (now HE Academy) advisors as
“exemplars of how case studies on learning and teaching in general could be presented”34 and
allow sharing of pedagogical ideas and customisation of existing case study content for local
contexts of use. Case studies are available online at the e-MapScholar website35.

e-MapScholar tools also provide a way to stream real-time data to learning objects, and are a good
example of the interoperability between learning environments and digital libraries. e-MapScholar
tools would be suitable for use in various educational levels including undergraduate, FE/HE
progression and as an introductory resource for postgraduate students.

e-MapScholar interactive tools are not currently available to the HE/FE community.

In contrast to the potential interactivity allowed by e-MapScholar products, it is anticipated that

users of the JORUM+36 learning object repository will be able to:

• Search for and locate LOs and teaching materials

• Download LOs and teaching materials from the repository to their local PC or VLE
• Upload their own LOs and teaching materials from their PC into the repository
• Download and upload virtual objects/resource stubs to describe materials held elsewhere

EdNA Online http://www.edna.edu.au/
Vocational Education and Training Database http://www.voced.edu.au/
Professor B Chalkley and Dr Helen King, LTSN GEES: Submission to the JISC; available in an appendix to the final report
from e-MapScholar1
Available at: http://edina.ac.uk/projects/mapscholar/index.html

CREE Deliverable S1D1

Metadata within the JORUM+ repository will have two main functions: to allow the discovery of
objects, and to allow objects to be described in detail so that they can be re-used. A discovery
mechanism will be offered to remote applications (such as portals).

The interactive nature of much educational content also raises rights and privacy issues. Most
institutions engaged in the organisation, delivery, accreditation and validation of programmes of
HE and FE hold a licence to cover educational copying with Copyright Licensing Agency 37. In the
past, many academics have ignored issues of copyright and intellectual property when assembling
teaching resources and have freely photocopied and distributed journal articles, book extracts,
maps and diagrams to their students. In most cases there was an assumption (false) that this
multiple copying was allowed provided it was for educational purposes.

However, a hardening of attitudes by copyright owners and the ease with which resources can be
digitised, made available worldwide and located using the Internet mean that Universities can no
longer afford to turn a blind eye to such practices if they wish to avoid expensive and
embarrassing legal action. It is therefore important that tutors understand how copyright affects
their use of online learning resources and take steps to ensure that copyright materials are cleared
(paid for).

The underlying objective of Intellectual Property Rights (IPR) is to protect the creators right to be
appropriately acknowledged for his or her work, be it in the form of an invention, a manuscript, a
suite of software, or a business name. IPR puts in place a mechanism that affords the creator a
means of controlling how their protected work is exploited, thereby ensuring that they are properly
rewarded for their creative endeavours. Public dissemination of information across the Internet and
ever evolving information technologies and applications (especially those which allow interaction
with and alteration of the original material) has placed considerable pressures on the principles of
IPR, particularly with regard to the detection of infringement and the enforcement of IP owners
rights. Merely because material (for example Internet content, software, on-line databases or
journals) is openly available on the Internet does not necessarily imply that it may be freely used
or applied. The vast majority of such material or content is made available under certain terms and

8. Moving information

Once the information is available to the user, can it be moved between systems and/or software for
use elsewhere? This also leads to asking whether the information is interoperable and therefore
able to be moved, or what might need to be added to enable this?

The obvious utility of electronic information sharing has often been frustrated by the conflicting
format standards which exist in various software products. Use of universal Web data formats and
access technologies is replacing various proprietary interfaces or reliance on ordinary computer
file searching facilities. Work is underway to address a whole host of specific interoperability
problems, resulting in solutions that should be widely applicable and reusable. On the technical
front, for example, the investment of JISC and others in Z39.50 (Miller 1999) has helped the
protocol realise its potential as a valuable tool in linking distributed resources 39. With semantics
and terminology control, too, the MODELS outcomes 40 and work within the international NKOS
group41 offer important steps forward. The Library of Congress maintains a list42 of examples of
use and details of implementations of Z39.50.

For more information see: http://www.jisclegal.ac.uk/index.html
see JAFER project website: http://www.lib.ox.ac.uk/jafer/
The MODELS workshop page is at: http://www.ukoln.ac.uk/dlis/models/

CREE Deliverable S1D1

File formats for which the technical specification has been made available in the public domain, as
is the case with open standards such as JPEG, are useful in terms of interoperability and long-term
access. The developers of proprietary formats may also publish their specifications, either freely
(for example, PDF), or commercially (as is the case with the Adobe Photoshop format
specification). The advantages of some open formats come at the cost of some loss in structure,
context, and functionality (e.g. ASCII), or the preservation of format at the cost of usability (e.g.
PDF); proprietary formats frequently support features of their creating software which open
formats do not. The range and sophistication of open formats is increasing all the time.

The ability to exchange electronic records with other users and IT systems is frequently an
important consideration. Formats which are supported by a wide range of software or are platform-
independent therefore facilitate use and movement of information. The National Archives 43 also
indicate that this feature also tends to support the long-term sustainability of the data by
facilitating the migration of the data from one technical environment to another.

Standardised metadata is central to interoperability; at its best it is a powerful tool that enables the
user to discover and select relevant materials quickly and easily. At worst, poor quality metadata
can mean that a resource is essentially invisible within a repository or archive and remains unused.
Metadata quality has an important role to play in realising the goals of learning object repositories
and e-Print archives, and much effort has already gone into developing standardised approaches to
metadata structure, but as yet the issues surrounding the creation of good quality metadata within
that structure have received little attention (Barton et al. 2003).

There are various standards for the use of metadata for different materials. Recent years have seen
the publication of several standards designed to encode metadata for objects held within digital
library collections: what has been lacking is an overall framework within which these schemes can
be integrated (Gartner 2002). The Metadata Encoding and Transmission Standard (METS)44
attempts to fill this role: it is an emergent standard designed to encode metadata for electronic
texts, still images, digitised video, sound files and other digital materials within electronic library
collections. In doing so, it attempts to address the lack of standardisation in digital library
metadata practices which is currently inhibiting the growth of coherent digital collections.

Written in XML, METS offers a coherent overall structure for encoding all relevant types of
metadata (descriptive, administrative, and structural) used to describe digital library objects, and
necessary for both management of digital library objects within a repository and exchange of such
objects between repositories (or between repositories and their users). METS allows this metadata
to be either embedded directly within its own structure, or held in external files and referenced
from within it. An extensive system of internal identifiers is used to link all facets of the digital
object's components together. By virtue of being written in XML, METS is platform- and software
independent, robust and readily interchangeable with other schemes.

The Dublin Core is a metadata element set intended to facilitate discovery of electronic resources.
Originally conceived for author-generated description of Web resources, it has attracted the
attention of formal resource description communities such as museums, libraries, government
agencies and commercial organisations.

The key characteristics of the Dublin Core are:

• Simplicity
• Semantic interoperability
• International consensus
• Extensibility


CREE Deliverable S1D1

• Modularity

The Dublin Core is positioned as a simple information resource description format. However, it
also aims to provide a basis for semantic interoperability between other, often more complicated,


Exploit Interactive

The Exploit Interactive45 e-journal was funded by the EU's Telematics For Libraries programme to
disseminate information about projects funded by the programme. The e-journal was produced by
UKOLN, University of Bath. Exploit Interactive made use of Dublin Core metadata in order to
provide enhanced local search facilities.

Metadata needed to be provided in order to provide richer searching than would be possible using
standard free-text indexing. In particular it was desirable to allow users to search on a number of
fields including Author, Title and Description. The case study concluded that it would have been
desirable to store the metadata in a database. This would also have the advantage of allowing the
metadata to be managed and allow errors (e.g. variations of author's names, etc.) to be cleaned.

Use of a database as part of the workflow process would also have enabled greater control to be
applied for the metadata: for example, it would enable metadata such as keywords, article type,
etc. to be chosen from a fixed vocabulary, thus removing the danger of the editor misspelling such

This is a summary of Managing And Using Metadata In An E-Journal, Kelly, B., QA Focus case
study 01, UKOLN: http://www.ukoln.ac.uk/qa-focus/documents/case-studies/case-study-01/

In August 2004 the Museums, Libraries and Archives Council (MLA)46 and JISC signed a joint
agreement with the Dublin Core Metadata Initiative (DCMI) enabling the UK to become formally
involved in the standard’s continued development. Chris Batt, Chief Executive of MLA said47,
“Signing this joint agreement … enables those organisations at the cutting edge of Internet
developments to influence the development of the important Dublin Core standard and to ensure
the needs of the UK’s rich creative knowledge economy is well served globally … collections of
museums, libraries and archives have always been crucial to higher and further education, but once
digitised, knowledge can flow … so long as the underlying standards are common.”

Finally, IMS, which is concerned with standards for learning servers, learning content and the
enterprise integration of these capabilities, has developed a new metadata specification of key
relevance in this area. The Resource List Interoperability (RLI) specification details how
structured meta-data can be exchanged between systems that store and expose resources for the
purpose of creating resource lists and those that gather and organise those Resource Lists for
educational or training purposes. A typical example of such a resource list is a reading list, but
resource lists as a general concept can cover most anything between a list of URLs and a pile of
exhaustive metadata records. RLI offers a means of listing materials with enough metadata for
citation and retrieval purposes. The IMS Content Packaging specification48 wraps the resource list
to enable transfer between systems. How the retrieval exactly takes place depends on the local
system and the nature of the resource, but there is a place to stick resolvable locators such as the
popular OpenURL. The difference between a bibliographic management package's (such as


CREE Deliverable S1D1

EndNote49) format and IMS RLI is threefold: RLI is designed for use in bigger network
applications, it has a behavioural model making it comparatively easy for a wide range of other
tools to read, write and collate the lists and it is an open XML specification.

The CETIS Metadata SIG50 are producing case studies which document experience with
implementing educational metadata standards. The aim of producing these case studies is to raise
awareness of how people are implementing educational metadata specs, for what purpose and also
provide some reflection on what problems were encountered and what worked well. One of the
services being assessed is SearchLT, an online catalogue of Engineering learning resources for UK
HE. The metadata schema used for the SearchLT database has many elements in common with
IMS metadata and hence the IEEE LOM. A major benefit of using elements from the IMS schema
was to be able take advantage of the work already done on learning resource description. Further
details about the case studies can be found in Barker and Barker (2003).

9. Using information

Within the model the next consideration is how users use information once they have retrieved it
and what they do with it. We are also interested in whether the way they have retrieved it
influences the way in which they use it.

Use can take many forms, but essentially describes the step(s) taken once a user has discovered
something new; this may involve moving away from the retrieval system to elsewhere or may
involve further activity with the retrieval system. As people can only use what is available, use is
very heavily dependent upon provision and access (Nicholas 2000). Different levels of use can
also be distinguished. Nicholas (2000) states that the first involves determining whether
something is worth using; the second level is the use of information determined as being relevant.

Models of information use present and simplify the relationships among the theoretical processes
connected with the identification and satisfaction of a user’s information needs. Information
obtained by a user is processed, becomes part of their knowledge, and is used, directly or
indirectly, to influence their work and create new information needs (Wilson 1999). Information
activities form a cyclic process, in which individual elements of the context determine a person's
behaviour at all stages, and where information obtained becomes a new element in the system.

Fig 4: A new model of information behaviour (Niedźwiedzka 2003), based on Wilson (1999)

see section 2.9

CREE Deliverable S1D1

The model (Fig 4) shows two basic strategies of information seeking:

• A user seeks information themselves, or

• A user chooses to seek the help or services of other people

Niedźwiedzka (2003) indicates that a user can choose one, the other, or both of the strategies. A
fully independent user applies their own knowledge and available sources, then interacts with
search systems and information services (i.e. uses databases, catalogues, archives, search-engines
etc.). Such a user also selects and processes the acquired information personally. More often
people use various intermediaries and their services (information specialists, co-workers), and
utilise the effects of their information seeking and processing. A user can also almost entirely
depend upon intermediaries, and he or she acts independently only at the stage of mental
processing of information. Niedźwiedzka (2003) used 'almost' because in her view the economics
of information behaviour probably make an individual use sources that are at hand and appropriate
without using a mediator. But essentially, she concluded, it is an intermediary who engages in
systematic information activities: asking, seeking and searching, for this kind of user.

Clifford Lynch, executive director of the Coalition for Networked Information, at an E-Libraries
conference51 in 2002 stated that library management systems have remained "surprisingly static"
over the past 10 to 15 years, and only now are beginning to enter a period of instability that
indicates progress to truly digital systems. Starting around 1985, the last time library systems
underwent a significant paradigm shift, functional requirements—linkages, digital resources, and
graphical interfaces—drove the change.

According to Lynch, functional requirements will shift again in the coming decade, causing a
significant change in library services and systems, including:

• Links between course management systems and library finding tools in academic settings
• Portals giving access to content and increased user personalisation
• Tools to index licensed information (using standards such as the Open Archives
metadata project)

All of these functions will "have a huge impact on library collections," Lynch said, bringing
unprecedented issues as to how and what items are aggregated and who within a given
organisation is responsible for creating and maintaining them. In summary, Lynch said that though
we may not be working in digital libraries yet, "big changes are coming."

Littlejohn (2003) writes that, “New technologies are beginning to transform how higher education
is organised and delivered both on campus and at a distance. E-learning affords new opportunities
to increase flexibility in time and location of study, in forms of communication (for example
asynchronous discussions) and types of interaction (for example between teacher and student), in
how programmes are constructed (for example modules drawn from different universities) and in
access to, and availability of, information and resources through the World Wide Web.”

Developments in e-learning include the emergence of reusable Learning Objects, which allow
lecturers to share and re-purpose digital resources to be used in learning and teaching. While high
quality e-learning has high development costs, the costs of developing LOs can be offset against
the opportunities that appear in terms of sharing, customising and re-purposing materials so that
they can be used efficiently by a number of different people. Learning object repositories can
provide access to a wide range of learning materials for students and lecturers and also minimise
the need to reinvent the wheel across different institutions or subject areas.


CREE Deliverable S1D1



“JORUM” means a collecting bowl and can also be seen as meaning “the JISC Online Repository
for (learning) Materials”.

The project JORUM+ is seen as paving the way for, or crafting, the JORUM repository service on
behalf of the JISC and the FE/HE community in the UK, both in terms of providing this Report,
and also in supporting real-life re-use and re-purposing of content in the provisional collecting

The JORUM will be a key part of the JISC’s Information Environment (IE), which aims to provide
secure and convenient access to a comprehensive collection of materials, including learning and
teaching materials. The JORUM will be part of the infrastructure supporting submission, sharing
and re-use of content, in a context in which many more students are supported by fewer teaching
staff, and the co-authoring and sharing of materials, both intra- and inter-institutionally, and
between disciplines, are becoming more prevalent.

The JISC is responding to the need to facilitate access to and the sharing of UK FE/HE content.
One of the ways in which this is being done is by funding the JORUM+ (see Case Study 10)
project whose remit it is to report on the user requirements of a LOs repository and to identify
software systems suitable for the JISC’s purpose.

10. Managing information

Behind the scenes, how is the information managed, is it available at a distance or available
locally, how is it processed before being passed to the user and what requirements are there for
local presentation systems when presenting information to users?

10.1. General management of information

IT managers face problems resolving the conflict between two approaches to information systems

• Centralised: where decisions are taken at the most senior or central level.
• Decentralised: where decisions are taken at some level lower than the most senior;
typically by individual work units within the organisation or even by individual staff.

Heeks (1999) summarised that centralised approaches, can bring benefits of:

• Resource sharing: especially the cross-organisation sharing of data, technology and staff.
• Duplication avoidance: since, for example, any item of data is stored once and only once.
• Learning and control: because these have a central organisational focus.
• Scale economies: for both external purchases (hardware, training) and internal processes
(system development, management).

Whereas, the barriers to centralisation include:

• Technical barriers: that make interconnection between existing systems complex.

• Resource barriers: since centralised approaches require an overt, 'up-front' commitment of
money, time, people, and skills; all of which are in short supply.
• Political barriers: that make public servants unwilling to let "our data" or "our systems"
become "the organisation's data" or "the organisation's systems".

CREE Deliverable S1D1

• Structural barriers: that prevent the different parts of the organisation working together.
In addition to the structural 'stovepipes' that beset government, stereotypical structural
chasms still persist between: top managers who fail to understand information and IT; IT
staff who understand technology but not the organisation's business; computer illiterate
staff who still feel threatened by new technology; and computer literate staff who want to
pursue their own agenda without central interference.

Decentralised management promises to avoid such disadvantages (Heeks 1999). Driven on by

ever-increasing performance/usability/price ratios being delivered to the desktop, this is
synchronised with the push for greater autonomy and greater individual responsibility in the public

Decentralisation promises to deliver:

• Greater fit between systems and local needs: by reducing the 'distance' between users and
IT managers.
• Higher usage of computerised systems: especially where end users make a significant
contribution to system development.
• Faster system development: for the same reasons.

A portal unifies access to a number of internet-based resources in the form of a single Web page
(Clark, 2001; Katz et al. 2002; Miller, 2002). The portal is an extra layer between the client or
browser and server (Katz et al. 2002).

The first portals were rich home pages for commercial websites such as Yahoo or Netscape. The
home pages integrated a range of services such as search engines, information (e.g. links to the
stock market), news headlines, and weather or television listings (Clark, 2001). The user could
customise the page by selecting their preferred resources. The selections would be stored on the
computer in use at that time as a “cookie” which would be checked on future visits to those sites to
utilise the stored preferences whether consciously or unconsciously recorded. The major limitation
of this system is that, unless the same computer is always used, previous session details will be
unavailable. In addition there are security issues involved with storing session details on a client
computer especially those with multiple users.

Later more elaborate personalised home pages such as MSN (Microsoft Network) or My Yahoo
(Yahoo) were produced (Miller, 2001). Such sites allow an authenticated user to access their
personal resources, such as email, in addition to various other services promoted by the company,
which might include search engines, online shops and auctions. However, the user was now able to
discover and add resources to their personalised ‘portal’ home page.

Content Management Systems (CMS) separate the process of content maintenance from the
underlying programming and publishing technologies (CMSWorks Inc., 2003). Portals are purely
an access management layer and do not carry any data, so underlying content management is
important. A CMS performs two functions: to store the raw data (e.g. Web page content); and to
act as an application server (handling database functionality, html templates, server scripts, search
engines and other code).

CREE Deliverable S1D1



THE HERO website aims to be:

• The primary internet portal for academic research and higher education in the UK
• The natural entry point for enquiries about higher education in the UK for the widest
possible range of customers
• A showcase for the diversity and quality of research and higher education in the UK

The website aims to be quick and easy to navigate, providing the shortest route to the information
you require on higher education in the UK.

HERO uses a content management system (CMS) to produce a number of different websites.
The CMS, which runs in any web browser, allows HERO staff and contributors to edit the content
on any HERO website, from anywhere in the world with an Internet connection. Users can also
register to use the CMS in a number of different roles, to produce content or upload information to
HERO websites.

The Web Service concept builds upon the ubiquity and robustness of the Web to share computer
system functions across the Web. Services can range from simple requests to complex procedures.
For example one computer could provide course module data, another could store subject data and
another could amalgamate the information and present it as Web pages. The advantage of this
concept is that it builds upon existing protocols and languages (such as HTML, HTTP and XML).
Data and functions can be made available as Web Services and these can then be accessed and
consumed elsewhere subject to Web access and security restrictions. This process is possible
through the use of two protocols: Simple Object Access Protocol (SOAP) (Box, 2000) and Web
Services Description Language (WSDL) (Christensen, 2001) which formalise sharing of data
objects. The portal framework can benefit from Web Services as a mechanism for amalgamating
data objects from disparate sources.

Universities provide electronic access to information through online public access catalogue
(OPAC) services that provide users with direct access to the library’s holdings, and other
reference services which range from local CD-ROM sources to remote services managed by other
institutions and organisations. Subject portals evolved as a solution to the problem of integrating
access to the plethora of specialist online resources.

Semantic interoperability refers to the processes whereby subject portals integrate disparate
information sources. Work reported by Hunter (2001), from the University of Queensland in
Australia, built upon the strengths and overcame weaknesses in XSLT to develop a “metadata
term thesaurus” to “enable flexible and dynamic mapping among metadata standards”. Miller
(2000) surmised that “semantic interoperability presents a host of issues, all of which become
more pronounced as individual resources — each internally constructed in their own (hopefully)
semantically consistent fashion — are made available through 'gateways' and 'portals'…”

10.2. Local management of information

Library Subject Portals integrate various authentication processes (Clark, 2001) to make resources
available under campus agreements. A single generic username/password combination will be
made available to either the whole institution or large sub-groups. Library systems use different
metadata schemas to index data. The portal applies a search request in an appropriate syntax for
the various underlying systems, as well as “translate” various access protocols. Portal technology
not only facilitates a search across a range of sources but also intelligently interprets the results
and aggregates them in meaningful ways. The ability of such systems to integrate with
institutional authentication systems points the way to more sophisticated customisable services but

CREE Deliverable S1D1

this is not currently the norm. Clark (2001) also noted that another advantage of integrating user
level rather than generic authentication is that more complex licensing arrangements can be
catered for.

A number of UK Universities have begun implementing institutional portals52. The issues

surrounding the implementation of an institutional portal (see Case Study 8) are substantial since
portals cut across and need to interface with a host of other institution-wide systems. Also the user
interface is significant aspect of the system. There are a number of different approaches 53 to portal
implementation, and it is likely that system integration “will not be produced by a one off policy
decision and will require an evolving process of converging decisions over a period [of] time”
(McCredie 2000).

Reference management software is a tool designed to help researchers input, store, organise,
retrieve and format lists of references. Functions of the software include:

• Creating database of searchable, annotated references (including abstracts, keywords and

• Importing of references directly (from library catalogues, commercial databases, etc)
• Ability to search remote databases
• Production of bibliographic references (for article submission or independent
bibliographies based on publisher requirements) automatically while you write

Reference/bibliographic management systems of this sort (e.g. Endnote or Reference Manager)54

are characterised by the ability to handle references of books, articles, journals etc (reference
types) in the same database without having many fields defined that are only applicable to one
type of reference, for example journal details are not relevant to books. They typically employ
variable-length record structure, divided into a number of loosely organised components or fields.
Fields have repeating values, e.g. multiple authors. The structure of the database is often
predefined unlike more general text retrieval packages and database packages. This should make
the package easy to use since it has been tailored specially for a particular task but obviously loses
the flexibility of a more general package.

"Users need only to key an article into the database once, and then they never need to
type that reference again, no matter how many papers they write that cite that article.
This approach minimises typographical errors, and helps standardise indexing keywords
and reference formats."55

There is now an academic CHEST56 deal valid until December 2007, which enables HE and FE
institutions, and individual faculties within them, to purchase the software on behalf of their staff
for University owned computers. Alternatively, staff and students can purchase individual copies,
at heavily subsidised price, for home use.

11. Conclusions

A large amount of work has been done to ensure information provision to the HE community,
with much progress made, on bringing a wide range of information sources, both metadata and
full text, to the electronic reach of information users, whether researchers or teachers or students.
The technical infrastructure in HE (and increasingly within FE) is reliable and under constant
dynamic development. In recent years much attention has been paid to the need for agreed
standards to ensure optimum interoperability.

see: http://www.fair-portal.hull.ac.uk/ for more details
Deborah Fitzgerald and Greg Erianne, The Scientist, Jan. 7, 2002

CREE Deliverable S1D1

Yet the "information landscape", to borrow a phrase from UKOLN, is a confusing place for the
user. Users find that they may need a plethora of passwords to access various databases. They
need to familiarise themselves with a new interface every time they access a new resource, and
have to waste time re-keying the same search terms each time they look for an item in more than
one database. They find information about existing materials and resources, and maybe even
location information too, but they are still often far from actually handling the source material
they want. For that, there is a need to arrange either physical access to the resource at the
identified location or an inter-library loan request that will (eventually) bring the resource to them.

The overwhelming need now is for integration of the services in ways which help the user to make
more efficient and cost-effective use of the rich resources available and which extend the potential
user community for any given service through the simplification and integration process, thereby
creating better value for money all round.

12. References

Andrews, W. (1996). Search engines gain tools for sifting content on the fly. Web Week, 2(11),

Armstrong C. J., R. R. Fenton, R. E. Lonsdale, S. Spink, R.E. Thomas and C. J. Urquhart (2002).
JUSTEIS: JISC Usage Surveys: Trends in Electronic Information Services. Series of 5 reports
covering Strand A: A general survey of end users of all electronic information services; Strand C:
A general survey of electronic information services provision in FE and HE. Final Report - 2001-
2002 Cycle (Cycle Three). Aberystwyth: University of Wales Aberystwyth Department of
Information and Library Studies [and] Centre for Information Quality Management, 2002


METADATA STANDARDS: SearchLT—an online catalogue of Engineering learning resources
for UK HE Version 1.1 http://metadata.cetis.ac.uk/usage_survey/cs_searchlt.pdf

Barton, J., Currier, S. and Hey, J (2003). Building Quality Assurance into Metadata Creation: an
Analysis based on the Learning Objects and e-Prints Communities ofPractice, DC-2003,
http://www.siderean.com/dc2003/201_paper60.pdf [accessed 06/10/04]

Box, D., Ehnebuske, D., Kakivaya, G., Layman, A., Mendelsohn, N., Nielsen, H.-F., Thatte, S.
and Winer, D. (2000). Simple Object Access Protocol (SOAP) 1.1 W3C Note.
http://www.w3.org/TR/2000/NOTE-SOAP-20000508/ [accessed 13/10/04]

Brophy, P. (1998). It may be electronic but is it any good? Measuring the performance of
electronic services. In: Robots to Knowbots: the wider automation agenda. Proceedings of the
Victorian Association for Library Automation 9th Biennial Conference, January 28-30 1998,
Melbourne: VALA, pp 217-230.

Brophy, P. and Coulling, K. (1996). Quality management for information and library managers.
Aldershot: Gower.

Bruce, C. (1997). The Seven Faces of Information Literacy, Adelaide: Auslib Press. In summary
at: http://sky.fit.qut.edu.au/~bruce/inflit/faces/faces1.htm [accessed 04/10/04].

Chowdhury G.G. (2004) Introduction to Modern Information Retrieval, 2nd Edition

ISBN 1-85604-480-7. Library Association Publishing 480 pp.

CREE Deliverable S1D1

Chowdhury, G.G. and Chowdhury, S. (2001). Information sources and searching on the World
Wide Web. London, Library Association Publishing

Christensen, E., Curbera, F., Meredith, G. and Weerawarana, S. (2001). Web Services Description
Language (WSDL) 1.1W3C Note. http://w3.org/TR/wsdl [accessed 13/10/04]

Clark, J. (2001). Subject Portals. Ariadne 29 http://www.ariadne.ac.uk/issue29/clark/ [accessed


Cmor, D. and Lippold, K. (2001). Surfing vs. searching: the Web as a research tool. Presented at
the 21st Annual Conference of the Society for Teaching and Learning in Higher Education.
Memorial University of Newfoundland, 13-16 June, 2001.

CMSWorks Inc., (2003). CMS WATCH: an independent source of information, analysis, and
reports about web content management solutions. http://www.cmswatch.com/ [accessed 13/10/04]

Craven, J. and Griffiths, J.R. (2002). 30,000 different users…30,000 different needs? Design and
delivery of distributed resources to your user community. In: Brophy, P., Fisher, S. and Clarke, Z.
(eds.) Proceedings of Libraries Without Walls 4: The delivery of library services to distant user.
London: Facet.

Croft, W.B. (1995). What do people want from Information Retrieval. D-Lib Magazine,
November. http://www.dlib.org/dlib/november95/11croft.html [accessed 04/10/04].

Dempsey, L. (2000). The subject gateway: experiences and issues based on the emergence of the
Resource Discovery Network. Online Information Review, 24 (1): 8-23

Dempsey, L. (2003). The recombinant library: portals and people, forthcoming in Journal of
Library Administration.
http://www.oclc.org/research/staff/dempsey/recombinant_library/default.htm [accessed 15/10/04]

Dervin, B. and Nilan, M. (1986). Information needs and uses. Annual review of Information
Science and Technology, 21, 3-33

De Stricker, U. (2002) Keep me posted but not too much: challenges and opportunities for STM
current-awareness providers. Searcher 10(1), 52-56.

Eisler, D. (2001). Campus Portals: Are We Ready? Connections 2001 - Centre for Curriculum,
Transfer & Technology May 7, 2001 - Whistler, British Columbia, Canada

Ellis, D., Ford, N., and Furner J. (1998). In search of the unknown user: indexing and hypertext
and the world wide web. Journal of Documentation, 54(1), 28-47.

Fernandez, L. (2002). User Perceptions of Current Awareness Services: A Faculty Survey.

Issues in Science and Technology Librarianship, Vol.33. http://www.istl.org/istl/02-
winter/article3.html [accessed 12/10/04]

Gartner, R (2002). METS: Metadata Encoding and Transmission Standard. Techwatch report
TSW 02-05 http://www.jisc.ac.uk/index.cfm?name=techwatch_report_0205 [accessed 12/10/04]

Garvin, D.A. (1987). Competing on the eight dimensions of quality. Harvard Business Review: pp

CREE Deliverable S1D1

Griffiths, J.R. (1996). Development of a specification for a full text CD-ROM user interface.
MPhil thesis: MMU.

Griffiths, J.R. and Brophy, P. (2002). Student searching behaviour in the JISC Information
Environment. Ariadne Issue 33 http://www.ariadne.ac.uk/issue33/edner/intro.html [accessed

Heeks. R. (1999). Centralised vs. Decentralised Management of Public Information Systems:

A Core-Periphery Solution. Information Systems for Public Sector Management. Working Paper
Series, Paper No. 7. http://www.sed.manchester.ac.uk/idpm/publications/wp/igov/igov_wp07.htm
[accessed 13/10/04]

Hoelscher, C. (1998). How Internet experts search for information on the Web. In: World
Conference of the World Wide Web, Internet, and Intranet, Orlando, FL., 1998. Association for
the Advancement of Computing in Education.

Hunter, J. (2001). MetaNet - A Metadata Term Thesaurus to Enable Semantic Interoperability

Between Metadata Domains. Journal of Digital information, Vol. 1,8
http://archive.dstc.edu.au/RDU/staff/jane-hunter/harmony/jodi_article.html [accessed 13/10/04]

Jansen, B.J., Spink, A., and Saracevic, T. (2000). Real life, real users and real needs: a study and
analysis of user queries on the web. Information Processing and Management 36 (2), pp 207-227.
http://jimjansen.tripod.com/academic/pubs/ipm98/ipm98.html [accessed 12/10/04]

Johnson, F., Griffiths, J.R. and Hartley,R.J. (2001). DEvISE: a framework for the evaluation of
Internet Search Engines. Resource Report 100. http://www.cerlim.ac.uk/projects/devise/index.php
[accessed 04/10/04].

Johnson, F.C., Griffiths, J.R. and Hartley, R.J. (2003). Task dimensions of user evaluations of
information retrieval systems. Information Research, 8(4), paper no. 157.
http://informationr.net/ir/8-4/paper157.html [accessed 04/10/04].

Johnstone, D., Bonner, M., Tate, M. (2004) Bringing human information behaviour into
information systems research: an application of systems modelling. Information Research, 9(4)
paper 191. http://InformationR.net/ir/9-4/paper191.html [accessed 04/10/04].

Katz, R. N. & Associates (2002). Web Portals and Higher Education: Technologies to Make IT
Personal. Wiley. 208pp.

Kirkwood, A. (2003) Understanding Independent Learners’ Use of Media Technologies, Open

Learning, 18, 2, pp. 155-75

Koll, M. (1993). Automatic relevance ranking: A searchers complement to indexing. Proceedings

of the 25th annual meeting of the American Society of Indexers, p.55-60. Port Aransas, TX:
American Society of Indexers.

Kukulska-Hulme, A. (2002). Cognitive, Ergonomic and Affective Aspects of PDA Use for
Learning European Workshop on Mobile and Contextual Learning, University of Birmingham, 20-
21 June 2002. http://iet.open.ac.uk/pp/a.m.kukulska-hulme/PDAsBirmingham.htm [accessed

Larsen, R.L. (1997). Relaxing assumptions: stretching the vision. D-Lib Magazine, April 1997
http://www.dlib.org/dlib/april97/04larsen.html [accessed 04/10/04].

CREE Deliverable S1D1

Lee, S. and Morris, W. (2000). Making friends with e-journals. Library Association Record, 102
(7), 392-393

Littlejohn, A. (ed) (2003). Introduction to Reusing online resources, a sustainable approach to e-

learning, Kogan Page US, 2003, p1. http://www.reusing.info

Marchionini, G. (1992). Interfaces for end user information seeking. Journal of the American
Society for Information Science 43 (2), 156-163.

Mason, J. (2003). An Overview of Government-Supported E-Learning Activities in Australia Seoul

E-Learning Conference, 2003. http://www.educationau.edu.au/papers/elearning_seoul_jm.pdf
[accessed 06/10/04]

McCredie, J.W. (2000). Planning for IT in Higher Education: It’s not an oxymoron. Educause
Quarterly 23,4

McKay, S.C. (1999) Accessing Electronic Journals. Database 22 (April/May): 16-23

Miller, P. (1999). Z39.50 for all. Ariadne Issue 21. http://www.ariadne.ac.uk/issue21/z3950/

[accessed 06/10/04]

Miller, P. (2000). Interoperability. What is it and why should I want it? Ariadne Issue 24
http://www.ariadne.ac.uk/issue24/interoperability/intro.html [accessed 13/10/04]

Miller, P. (2002). The concept of the portal. Ariadne 30 http://www.ariadne.ac.uk/issue30/portal/

[accessed 13/10/04]

Navarro-Prieto, R., Scaife, M., Rogers, Y. (1999). Cognitive strategies in web searching.
Proceedings of the 5th conference of Human Factors and the Web.
http://zing.ncsl.nist.gov/hfweb/proceedings/navarro-prieto/index.html [accessed 04/10/04].

Nicholas D. (2000). Assessing Information Needs: Tools, techniques and concepts for the Internet
Age. 2nd Edition. ISBN 085 142 4333. London: Aslib

Nicholas D., Dobrowolski T., Withey R., Russell C., Huntington P., Williams, P. (2004). Digital
information consumers, players and purchasers: information seeking in the new digital interactive
environment. Aslib Proceedings 55 (1/2) pp23-31

Nicholas D., Huntington P., Watkinson, A. (2004). Digital journals, big deals and online searching
behaviour: a pilot study. Aslib Proceedings 55 (1/2) pp84-109

Niedźwiedzka, B. (2003). A proposed general model of information behaviour. Information

Research, 9(1) paper 164 http://InformationR.net/ir/9-1/paper164.html [accessed 13/10/04]

Olsen, F. (2002). The Power of Portals. The Chronicle of Higher Education. 09/08/02. [online]
http://chronicle.com/free/v48/i48/48a03201.htm [accessed 04/10/04].

Orhun, E (2004) Web Based Learning Materials for Higher Education: The MERLOT Repository.
The Turkish Online Journal of Educational Technology – TOJET 3, 3: Article 10

Powell, A. (2002). RSS FAQ, JISC Information Environment Architecture.

http://www.ukoln.ac.uk/distributed-systems/jisc-ie/arch/faq/rss/ [accessed 04/10/04].

CREE Deliverable S1D1

Powell, A. (2003). Mapping the JISC IE service landscape. Ariadne Issue 36

http://www.ariadne.ac.uk/issue36/powell/intro-alternative.html [accessed 04/10/04].

Powell, A. and Lyon, L. (2002). The JISC Information Environment and Web Services. Ariadne,
Issue31. http://www.ariadne.ac.uk/issue31/information-environments/ [accessed 04/10/04].

Proper, H.A. and Bruza, P.D (1999)., What is Information Discovery About? Journal of the
American Society for Information Science, 50(9):737-750

Quint, Barbara. 2001. New ISI Web of Knowledge integrates Web of Science with Current
Contents. Information Today 18(7), 42.

Rowley, J. (2001). JISC user behaviour monitoring and evaluation framework (incorporating
JUBILEE and JUSTEIS): 2nd Annual report. Summarised in Ariadne Issue 30, 2002
http://www.ariadne.ac.uk/issue30/jisc/ [accessed 04/10/04].

SCWEIMS group (2001). Building a university portal – the SCWEIMS experience.

http://www.scweims.ac.uk/public_documentation/building_a_portal.pdf [accessed 13/10/04]
Jansen, B.J. (2000). The effect of query complexity on Web searching results. Information
Research 6 (1). http://www.shef.ac.uk/~is/publications/infres/paper87.html [accessed 04/10/04].

Silverstein, C., Henzinger, M., and Marais, H. (1999). Analysis of a very large Web search engine
query log. SIGIR Forum 33 (1): pp 6-12.

Simmons, S. (ed.) (1999). Information insights : case studies in information management. London:
Aslib 176pp ISBN 0851424279 (pbk)

Smith, J. and Oliver, M. (2204) Presentation: Why Students can't use online resources. The Shock
of the Old 4: "e-Learning, The Student Experience". One-Day Conference on Educational
Technologies, University of Oxford, Said Business School, 15th April 2004.
http://www.oucs.ox.ac.uk/ltg/events/shock/talks2004/JS_Shock4.ppt [accessed 04/10/04].

Spink, A., Wilson, T., Ellis, D., and Ford, N. (1998). Modeling users successive searches in digital
environments. D-Lib Magazine, April 98. http://www.dlib.org/dlib/april98/04spink.html [accessed

Twidale, M.B., Nichols, D.M., Smith, G. and Trevor, J. (1995). Supporting collaborative learning
during information searching. Proceedings of Computer Support for Collaborative Learning
(CSCL '95), October 17-20, Indiana: pp 367-374.
[accessed 12/10/04]

Van de Sompel, H., Beit-Arie, O. (2001). Open Linking in the Scholarly Information Environment
Using the OpenURL Framework. D-Lib Magazine, 7 (3), 2001.
http://www.dlib.org/dlib/march01/vandesompel/03vandesompel.html [accessed 04/10/04].

Wilson, T.D. (1994). Information needs and uses: Fifty years of progress? In: B.C. Vickory (ed)
Fifty years of Information Progress: a Journal of Documentation Review (pp 15-52) London:

Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3),

Young, R (2004) Findings of the Altis Survey 2003-2004.

CREE Deliverable S1D1

http://www.portal.ac.uk/spp/documents/testing/phase2/altis/AltisSurvey20032004.doc [accessed

Zhang, H., Croft, W. B., Levine, B. and Lesser, V. (2004). A Multi-agent Approach for Peer-to-
Peer-based Information Retrieval Systems. AAMAS'04, July 19-23, New York.
http://www.aamas2004.org/proceedings/057_zhangh-p2pir.pdf [accessed 13/10/04]

Zipf, G. (1949). Human Behaviour and the Principle of Least Effort. Reading MA: Addison-

CREE Deliverable S1D1

Functional Literature Review - Update

Dr Caroline Ingram

The CREE literature review is a key element within the CREE project. The CREE functional
Literature review was initially carried out between August and October 2004. The following
sections were considered:

 Discovering information
 Locating information
 Requesting information
 Delivering/accessing information
 Presenting information
 Interacting with information
 Moving information
 Using information
 Managing information

Overall, the review considered how have, might or could these processes be brought together
within a portal environment and/or using portal technology.

This update will address, discursively, a review of recently (from October 2004 – May 2005)
published research in the area, as well as new resources that have come to light throughout the
CREE project. Research in this area is very much ongoing, and more in depth studies are likely to
be published over the next few years to supplement those summarised here.

1. Searching behaviour (discover)

“Twenty percent of all searching was sex-related back in 1997, now it’s about 5%,” said Amanda
Spink57, University of Pittsburgh professor, “it’s a little bit more in Europe, 8-10%, but in
comparison to everything else, it’s a very small percentage. People are using (the Web) more as
an everyday tool rather than as just an entertainment medium.”

What has not changed much over time is how hard people are willing to work at searching: i.e. not
very. In a seven-year survey, Spink and Jansen (2004a) found that people averaged about two
words per query and two queries per session. “…searches are taking less than five minutes and
they’re only looking at the first page of the results, that’s why people are wanting to get [the best]
results on the first page... I think people aren’t trained very well to use the search engines”

In a project aimed at finding better ways for us to organise and retrieve information for shared use,
researchers, led by Professor Tom Ormerod of Lancaster University, investigated how couples
catalogue and retrieve their digital photos now that the age of the shoebox full of prints and
negatives is gone58.

Professor Omerod summarised: "There is a widely held belief that people benefit from working
together to remember details of things – for instance, film storylines. However, research has
shown that when they try to recall information which they learned individually, the overall amount
remembered is less than if the same people were trying on their own.

“People mentally organise information in different ways, and cues that help one person recall may
inhibit another. So retrieving information from computer systems, such as a keyword search in a

Experts – web searches for sex declining http://www.msnbc.msn.com/id/6345251/print/1/displaymode/1098/
This article comes from PublicTechnology.net:

CREE Deliverable S1D1

library catalogue, may be impaired by a mismatch between the user’s mental organisation and the
cues provided by the system.”

As Novotny (2004) reports in a discussion of a research project that studied users interactions with
the Pennsylvania State University Libraries' online catalogue, most searchers, regardless of their
previous level of experience with research, do not use Boolen operators and do not understand
Library subject headings. Furthermore, he notes, many student searchers come to the interaction
with incorrect assumptions about the functionality of library catalogues and inflated estimations of
their own existing research abilities. Novotny quotes one student who in response to a question
about why he "selected the first remotely appropriate link that caught the eye," commented "I
don't think I click" (p.530).

In an article, Cole (2004) summarized from The Digital Future Report59 that “For users, the
Internet is now the most important source of information”. In the study, almost all users report that
the Internet is the first place they go for information, whether to settle a bet, contemplate a
purchase, or answer a complex question. The "always on" function of broadband has greatly
accelerated that trend by making the Internet easier to use than a telephone or reference book.
However, the Internet still trails television as a source of entertainment.

A recent study (Barnum et al., 2004) found that while readers preferred and expected to be able
to text search for topics in test PDF documents, they were more successful when using the (human
mediated) index. Barnum et al. (2004) also review the literature surrounding Human Indexing and
machine information retrieval, as well as usability testing of indexes and search engines.
Research conducted by User Interface Engineering 60 showed that user searches usually fail
because they either don’t understand the scope of the search, or they have trouble interpreting the
results. UIE concluded, “good indexing is a skill; humans do it better than machines. Full text
searches are different from looking something up in an index, but users didn’t seem to grasp this”.

Libraries are under increasing pressure to provide users with a single search box that provides
access to the diverse set of content and services available through the library website. Neither
library catalogues nor generic Web site search tools meet this need directly. At NC State, Morris
and Sierra61 reported at a recent conference that an analysis of library Web site search logs
indicated that a large percentage of user-submitted search terms target similar classes of content
(e.g., database names, journal titles, library information) to which the library could readily provide
a direct link. Concurrent with implementing a next generation metasearch tool, NCSU Libraries is
developing a new Web site search tool designed to provide users with quick and comfortable
access to distributed silos of library content. A "sponsored-links" component of this tool connects
users to relevant high-use library resources and information. A subject- identification component
provides contextual links to subject resource guides.

Shuttle (2004) surmises, “The really difficult thing I believe we must do as information
professionals is to let go the idea that librarians have to be the intermediary between users and
information. Those seeking information do not want to have to talk to a librarian … rather, they
just want information to be readily available and easily accessed. That's why they use Google and
why they will continue to do so until information professionals develop information organization,
access, and retrieval methods that are as easy to use as Google…”

Cole, J.I. (Founder and Organiser) 2004. The Digital Future Report: Surveying the Digital Future, Year Four. Copyright:
September 2004 University of Southern California [www.digitalcenter.org]

CREE Deliverable S1D1

2. Searching Library resources (locate)

Google Scholar is designed to make search and access to scholarly digital resources simpler62.
This need not only be applied to libraries, but could contribute to making all computing and
information transfer and use simpler. However, Google arose from a unified theory and is well
funded. Libraries tend to be institutionalised, can be feudal, have boundaries between them, don’t
market themselves well (if at all) and are generally not presenting information as users want it.

OCLC's Open WorldCat Pilot is “an initiative that integrates library records into popular Internet
search sites and tests the effectiveness of the Web in guiding users to library-owned materials. The
goal of the pilot: to make libraries more visible to Web users and more accessible from the sites
where many people begin to look for information.”63 The project aims to "open" WorldCat records
to present and potential library users through the familiar Web search engines Google and Yahoo!
Search. Enabling Web users to locate materials they need quickly and easily in libraries near them
will promote library use and reinforce the value of libraries. Ultimately even people who don't
often use libraries may come to consider libraries as a first source of information.

In the long term, OCLC wants to increase the amount of content available; increase the number of
partners by including other search engines, booksellers, and sites dedicated to books; enable
interlibrary loan requests through remote user authentication; and develop new user statistics and
configuration tools for libraries.

The Open WebCat project might advance faster if the search engines would put more effort into
dealing with non-Web content. Both Google64 and Yahoo!65 seem to be interested in opening up
new content avenues, and both have the research and development staff to deal with mechanisms
for better searching of original non-Web content.

3. Library vs. online searching (locate and request)

Many users see the Web as a substitute for libraries (Paschoud 2004), but many important online
resources cannot be freely located by search-engines, nor used without payment. Libraries can
license access to these, but need to develop methods of Access Management (AM) that can make
resource licensing more economic, and are technically sounder and easier to implement. The
current solution used by the UK academic community is Athens; the next generation of distributed
AM technologies is likely to be based on Shibboleth.

Paschoud stated that, “…respected commentators (Brophy 2002) have bemoaned (but
acknowledged) the trend for Google to become the first research port-of-call for students at all
levels. Readers seeking references, background material and evidence supporting these assertions
are advised to search the Web via Google.” In a recent study, Becker (2003) describes
undergraduate student search behaviour drawn from multiple disciplines. During interviews
conducted as part of the study, many students were able to articulate the importance of source
evaluation and describe electronically appropriate methods for assessing the authority and
reliability of Web-based information resources. In practice, however, these students frequently
abandoned source evaluation altogether and, following the path of least resistance, relied
exclusively on basic Google searching. This approach both compromised the quality of their
search results and contributed to frustration with the research process. This may not be unusual
behaviour, but is likely to be a cause for concern among information literacy instructors.

The Economist, 30 Oct 2004, “The conquest of complexity”, Vol.373, p.4
Open WorldCat Pilot: Using WorldCat to increase the visibility of libraries on the Web
Zeitchik, S. (2003) Google looks to add book content. Publishers Weekly, November 2003, p. 3.
Webb, C. (2004) Yahoo! Search will roll out Content Acquisition Program. The America's Intelligence Wire, 3 March
2004. http://www.infotrac.galegroup.com

CREE Deliverable S1D1

Paschoud (2002) suggested that many of the search skills needed are in many libraries, and that
this is demonstrated by the fact that academic libraries are leading access management projects,
because their purpose gives them a better overview of the problems and viable solutions. “New
skills and concepts will need to be mastered as we learn to coexist with Google; but there is still
plenty for us to do!”

Issues for resource discovery

Potential problems with seamless cross searching of different databases, indexes and other
information resources were raised in the Inspiral 66 project. Currier (2002) summarised that lack of
interoperability of search vocabularies, and a lack of awareness of and strategies to deal with this
in course design, could lead to confusing, ineffective resource discovery experiences for learners.
Some stakeholders in the Inspiral project user requirements analysis (including learners) were of
the opinion that this would not apply to experienced Internet users, who would not mind using
different databases if they are already familiar with them.

Currier (2002) also mentions concerns about all the library functions being offered online. These
included the potential diminishment of two important educational functions of traditional libraries:
serendipitous browsing and the social function as a place to meet fellow students and discuss
sources of information. The latter may be catered for by the VLE's online communication and
resource sharing tools, but users counselled that for this to work teachers would need to encourage
and enhance the use of these.


Tombros et al. (2004) investigated the criteria used by online searchers when assessing the
relevance of Web pages for information-seeking tasks. The intention of the study was to gain a
better understanding of what features make a web document useful for information seeking.
Tombros et al. concentrated specifically on information seeking tasks - finding web pages that
contain relevant or useful information – as this is one of the most common uses of web pages.

Twenty-four participants were given three tasks each, and they indicated the features of Web
pages that they used when deciding about the usefulness of the pages in relation to the tasks. The
tasks were presented within the context of a simulated work-task situation. Tombros et al. (2004)
investigated the relative utility of features identified by participants (Web page content, structure,
and quality) and how the type of information-seeking task performed can affect the importance of
these features and the stage of the search.

Nielsen (200567) reports on research that users expect on-line search to have three components:
• A box where they can type words
• A button labelled "search" that they click to run the search
• A list of top results that's linear, prioritised, and appears on a new page -- the search
engine results page (SERP)

In user testing results indicate that users want search on websites and intranets to work like X,
where X is their favourite major search engine. All three of the major engines (Google, Yahoo,
and MSN) work the same: exactly as stated in the list above.

Nielsen also reports that deviating from the expected design almost always causes usability
problems. Sites that separate out some search results and place them in boxes risk having these

see also JISC/RSLP funded project HILT (High Level Thesaurus) for a full analysis of this issue, at:
Jakob Nielsen's Alertbox, May 9, 2005: Mental Models For Search Are Getting Firmer

CREE Deliverable S1D1

links overlooked because users often assume they're adverts. Instead, such "best bets" should be
the first items in the linear list. Also, scoped search that covers only a current sub-site often
misleads users who rarely have to consider what's being searched.

4. The Digital Library Portal (delivering and accessing information)

Each of the elements of a digital library – its collections, the portals to those collections, and its
supporting services – plays an important role in determining how the library is used and what
impact it will have on users.

Manduca et al. (2005) explore how the portal of a digital library can be designed to influence the
behaviour of its users. From the study and interviews three ideas emerged that may be important
in guiding the design of on-line resources and digital libraries for academic staff: pedagogy and
content, the role of other colleagues as a trusted source of information, and the use of online
information in preparation for teaching. Similar to results reported in Chang et al. (2004), a
significant fraction of academics interviewed for the study (66%), and of those who responded to
the survey (50%), were interested in looking on the web for ideas for their teaching.

Founded in 2000 the BiosciEdNet (BEN) Collaborative68 sponsors a portal offering a growing
library of peer-reviewed teaching and learning resources for biology education in the US. As of
October 2004, the library contained more than 3100 resources, and was accessed by more than
4000 users. In a recent survey, Chang et al. (2004) asked respondents about their reasons for
visiting the portal. The majority of respondents (67% of all respondents) were seeking resources
for lectures. Resources for non-lecture presentations are also in demand (35% of all respondents).
Browsing for additional information for research or study was also a common activity at the BEN
portal and partner sites (45% of all respondents).

5. LibQUAL+ (presentation of information)

LibQUAL+ is a suite of services that libraries use to solicit, track, understand, and act upon
users’ opinions of service quality. The Association of Research Libraries (ARL) offers the
LibQUAL+ services to the library community. The program’s centrepiece is a rigorously tested
Web-based survey bundled with training that helps libraries assess and improve library services,
change organisational culture, and market the library. The goals of LibQUAL+ are to:

• Foster a culture of excellence in providing library service

• Help libraries better understand user perceptions of library service quality
• Collect and interpret library user feedback systematically over time
• Provide libraries with comparable assessment information from peer institutions
• Identify best practices in library service
• Enhance library staff members' analytical skills for interpreting and acting on data

More than 500 libraries have participated in LibQUAL+, including colleges and universities,
community colleges, health sciences libraries, law libraries, and public libraries -- some through
various consortia, others as independent participants. LibQUAL+ has expanded internationally,
with participating institutions in Canada, the U.K., and Europe. The growing LibQUAL+
community of participants and its extensive dataset are rich resources for improving library


CREE Deliverable S1D1

In October 2002 all SCONUL members were invited to participate in a UK pilot for
LibQUAL+69. The aim of the pilot was to test the methodology in the UK context and lay the
foundations for a standardised survey instrument for the UK higher education library sector. 20
member libraries agreed to participate in the pilot project as part of a SCONUL consortium group.
These represented a full variety of UK institutions, and the potential sample consisted of one-sixth
of the UK’s higher education students.

The UK 2003 participants were:

• Cranfield University
• De Montfort University
• Glasgow University
• Lancaster University
• Leeds Metropolitan University
• Liverpool John Moores University
• Robert Gordon University
• Royal Holloway University of London
• South Bank University
• University College Northampton
• University of Bath
• University of Edinburgh
• University of Gloucestershire
• University of Liverpool
• University of London Library
• University of Oxford
• University of the West of England, Bristol
• University of Wales College, Newport
• University of Wales Swansea
• University of Wolverhampton

The LibQUAL+ survey instrument makes it possible for libraries to canvas their users’ opinion
with minimal local effort. It employs a Web interface to ask users about their library service
expectations and experience. The UK 2003 survey used 25 core questions to measure library
users’ minimum, perceived, and desired levels of service quality on a nine-point scale in four key
• Access to information
• Affect of service
• Library as place
• Personal control

Other outcome questions, and five questions developed specifically for the SCONUL participants,
were also included. The final section of the survey enabled users to provide free-text comments
about the library. These were fed directly back to the library in real time enabling the library to
provide a prompt response.

Many US and some UK universities have given access to their results online70. At Glasgow
University Library 71 LibQUAL+™ gives users the opportunity to say where services need
improvement. The survey results allow the library staff to assess the quality of existing library
services and help them develop services that better meet users’ needs and expectations. According
to the LibQUAL+™ 2004 results, the five most important services by user group were:

Lock,S. and Town, J.S. (2003) LibQUAL+ in the UK: a brief report on the SCONUL Pilot. SCONUL Newsletter 29
Summer/Autumn 2003, 8-10
which can be found here: http://www.libqual.org/Information/Related_Sites/index.cfm

CREE Deliverable S1D1

• Undergraduate students:
o Quiet space to work
o Modern equipment to access information
o Haven for study, learning and research
o Library space inspiring study and learning
o Access to electronic resources from home

• Post-graduate students and academic/research staff:

o Print and/or electronic journals
o Access to electronic resources from home
o Electronic resources
o Library web site permitting self-help
o Library web page that is easy to navigate

Finally, related to this area, Spink and Jansen (2004b) have summarised their research between
1997 and 2003 that explored how people search the Web. The article reports selected findings
from many research studies conducted by the co-authors of the paper from 1997 to 2003 using
large-scale Web query transaction logs provided by commercial Web companies, including
Excite, Alta Vista, Ask Jeeves, and AlltheWeb.com. The researchers could not obtain Web query
data from Google, Yahoo or MSN.

6. Use of online information (use and management of information)

In the 10 years since online technology became generally available to the public as a
communication tool, the Internet has become one of the most important sources of information for
the vast majority of users. However, the importance of the Internet as a source of information
appears to be declining – along with the importance of all other forms of media as well. Among all
users in Year Four of the Digital Future Project 72, 55.2 percent consider the Internet to be a very
important or extremely important source of information for them. This number has been generally
declining since the first year of the study. Of respondents, only 59.1% considered books to be an
important information source.

The number of users who believe that information on the Internet is reliable and accurate has
declined slightly during Year Four of the Digital Future Project. In the current study, 50.1 percent
of users believed that most or all of the information online is reliable and accurate – a decline
from the three previous studies. The number of users who believe that only about half of the
information on the Internet is reliable and accurate is growing, and has now passed 40 percent for
the first time in the four years of the Digital Future Project. Non-users continue to report much
lower levels of belief in the reliability and accuracy of the information on the Internet; in the
current study, only 29.5 percent of non-users believe that most or all of the information on the
Internet is reliable and accurate.

Web sites mounted by established media (such as the New York Times: nytimes.com) ranked
highest in perceived accuracy and reliability; 74.4 of users said that most or all information on
established media Web sites is reliable and accurate. Government Web sites also fared well with
users in the current study; 73.5 percent say that most or all of the information on government Web
sites is reliable and accurate. Information pages posted by individuals have the lowest credibility;
only 9.5 percent of users say the information on Web sites posted by individuals is reliable and
accurate. Even though large percentages of users say that most or all of the information on Web
sites posted by established media and the government is reliable and accurate, it is worth noting
that significant numbers of users believe that only half or less of information on these sites is
reliable and accurate; 25.7 percent of users say that about half or less of news sites posted by

Cole, J.I. (Founder and Organiser) 2004. The Digital Future Report: Surveying the Digital Future,
Year Four. Copyright: September 2004 University of Southern California

CREE Deliverable S1D1

established media are reliable and accurate, while 26.5 percent of users judge that about half or
less of government Web sites are reliable and accurate.

Particularly with respect to medical information the Digital Future Report asked users how they
find information online. Unsurprisingly, most users find health or medical information online
through a search engine. However, a large percentage of users found health information through a
link from another Web site, a referral from a print article, or a site recommended by a friend –
experiences that were common among both new users and very experienced users.

Low numbers of users express concerns about their online searches for health or medical
information. Fewer than 20 percent of Internet users who go online for health or medical
information agreed or strongly agreed that they wanted more information but did not know where
to find it, or did not have the time to get the information they needed. Fewer than 15 percent of
these users said they felt frustrated during their search for information, or that it “took a lot of
effort,” or that they “did not have the energy” to get the information they needed. Only 6.3 percent
agreed or strongly agreed that there was a cost for the information they could not afford. Of
special note is the relatively low number of these users (21.8 percent) who agreed or strongly
agreed that they were concerned about the quality of the information they found.

Once Internet users find health information online, they have a variety of uses for it. Most
frequently, this information leads them to seek further advice or more information – actions that
are consistently high among all users, new users, and very experienced users. Large numbers use
this information to increase their comfort level with advice they have received from doctors or
other health professionals. More than half of users who have accessed health or medical
information within the last 12 months say that the information they found online led them to
contact a health care professional.

7. The future potential of searching and browsing?

There are new developments that aim to link together search and browse capabilities. A team at
Southampton University have developed the open source mSpace73 framework, one such model
designed to help explore relationships in information. mSpace helps people build on their
knowledge from exploring those relationships by offering several tools for organising an
information space to suit a person's interest. The basic premise of the design is linking the
searching for information capability of a search engine like Google to a browser with information
(like Amazon or i-Tunes):


CREE Deliverable S1D1

The idea is that the user will get a lot of associated information in one window. This would result
in not having to click through opening and closing a large number of links and windows, and
trying to remember what goes with what. Each element in the mSpace browser is connected to
information about that element, and can then be collected in a portfolio.

8. References

Barnum, C., Henderson, E., Hood, A., and Jordan, R., (2004), Index versus full-text search: a
usability study of user preference and performance. Technical Communication, 51(2): 185-206
http://www.spsu.edu/htc/home/p185.pdf [accessed 29/06/05]

Becker NJ (2003), Google in perspective: understanding and enhancing student search skills.
New Review of Academic Librarianship, 9(1): 84-100

Brophy P., (2002), Strategic Issues for Academic Libraries. Relay 2002 (52): 4-5

Chang, A., Matyas, M., Gough, N., George, Y. (2004), I Came, I Found It, I Used It, and It Made
a Difference. American Society for Microbiology, October 2004, 8pp
http://biosciednet.org/project_site/BEN_Survey_Article_October_2004.pdf [accessed 29/06/05]

Cole, J.I. (2004), Now Is the Time to Start Studying the Internet Age. Chronicle of Higher
Education. April 2, 2004

Currier, S. (2002), Libraries and e-learning: be inspired by INSPIRAL. Library and Information
Research News 26(82), Spring 2002, 4-15

Manduca, C.A., Iverson, E.R., Fox, S. and McMartin, F. (2005), Influencing User Behavior
through Digital Library Design: An Example from the Geosciences. D-Lib Magazine 11(5) (May
2005) http://www.dlib.org/dlib/may05/fox/05fox.html [accessed 05/08/05]

Novotny, Eric (2004), "I Don't Think I Click: A Protocol Analysis Study of Use of a Library
Online Catalog in the Internet Age." College and Research Libraries 65(6) (November 2004):

Paschoud J. (2002), Why Librarians should care about VLEs Relay 2002 (53): 4-5

Paschoud, J. (2004), It’s NOT on the Web! Why Users in the E-Age Still Need Libraries, and
Why Libraries Need Access Management.
final.doc [accessed 20/06/05]

Shuttle, J. (2004), Why Students Prefer Google and Use Library Resources as a Last Resort. The
Tennessee Librarian, 53(4)

Spink, A. and Jansen, B. J. (2004a), Web Search: Public Searching of the Web. Information
Science & Knowledge Management. Kluwer Academic Publishing 6(XIII), 199p.

Spink, A. and Jansen, B. J. (2004b), A study of Web search trends. Webology, 1(2) (December
2004) http://www.webology.ir/2004/v1n2/a4.html [accessed 29/0605]

Tombros, A., Ruthven, I., Jose, J.M. (2004), How users assess Web pages for information seeking.
Journal of the American Society for Information Science and Technology 56(4): 327-344.
http://www.dcs.qmul.ac.uk/~tassos/publications/jasist05.pdf [accessed 29/06/05]

CREE Deliverable S1D1

CREE Technical Literature Review

Chris Awre

1. Methodology

Underneath the range of services employed by users to address the areas described in the
functional review a wide range of technologies can be employed. The first emphasis of the CREE
project is on search, investigating how users wish to interact with search services. The second
emphasis of the project has been on the context of search, investigating where users would like the
interaction with search services to take place, both in portal and non-portal environments. This
technical review thus examines technologies and standards from two perspectives:

• The technologies and standards used within the JISC Information Environment74. Focus
has been placed upon those technologies most closely related to search. It is recognised
that those technologies not covered here are, of course, also important, but the review
focuses on those most closely related to the aims of CREE and the functionality of the
search tools and contexts being investigated. The sections of the Information
Environment Standards Framework 75 covered are as follows:
o distributed searching
o metadata harvesting
o news and alerting,
o context-sensitive linking
• The contexts in which search tools can be delivered are being examined within CREE. Of
key interest is the possibility of presenting search tools within a portal framework and the
standards that are becoming available to facilitate this, JSR 168 and WSRP

In relation to search, and CREE’s involvement of Google alongside more specific search tools,
and the use of metasearch functionality amongst the other search tools used within the project
merited sections on both of these areas to complement the more direct coverage of individual

2. Z39.50

The Z39.50 protocol has its origins in identifying a way to standardise communication between
different computer systems across a network. The first draft came out in 1988 as a result of work
within NISO, the National Information Standards Organization based in the US, and was
influenced by the Linked Systems Protocol developed for searching bibliographic databases and
transferring records by OCLC, the Library of Congress, the Research Libraries Group, and the
Washington (Western) Library Network 76. The latest version was released by NISO in 2003,
which defines Z39.50 as:

“… a client/server based service and protocol for Information Retrieval. It specifies procedures
and formats for a client to search a database provided by a server, retrieve database records, and
perform related information retrieval functions. The protocol addresses communication between
information retrieval applications at the client and server; it does not address interaction between
the client and the end-user.”77

See http://www.cni.org/pub/NISO/docs/Z39.50-brochure/ for a full online article about Z39.50 by William Moen,
including a section on its history
See http://www.niso.org/standards/standard_detail.cfm?std_id=465 for this definition and availability of the full-text of
the standard for downloading

CREE Deliverable S1D1

The Z39.50 standard has also been accepted internationally through the creation of ISO 23950 in
199878. The standard is currently maintained at the Library of Congress 79.

Z39.50 is commonly used in the library and information community to search distributed library
catalogues from a single point, and this enables far easier access to multiple diverse catalogues
without the need to learn how to access each one in turn80. It is not just catalogues that can be
searched using Z39.50, though: other bibliographic databases, such as abstracting and indexing
databases, scientific and technical data, geospatial data, museum information and thesauri can all
be accessed using the standard 81. Richardson and Powell (2003) experimented with searching
IMS metadata using Z39.50. The protocol also allows additional functionality such as inter-
library loans (ILL) and updating of databases to take place remotely82, although these functions
appear to have been obscured, and in the case of ILL superceded by ISO ILL (Prowse, 1998), by
the standard’s use for information retrieval. All of this functionality is enabled by a collection of
facilities, services, attribute sets, record syntaxes and profiles, all of which determine how the
connection between different systems should take place (Miller, 1999). As the definition of the
standard by NISO says, though, it does not address interaction between the client and the end-
user, simply the interaction between the systems.

Z39.50 has been extensively tested and used in a variety of scenarios. Particularly following the
release of version 3 of the standard in 1995, a range of projects explored its uses (Dempsey et al.
1998) and an increasing number of products and companies are now providing the means through
which to make use of the standard, although the level of conformance with the standard has long
been an issue that requires careful investigation when considering options (Schneider, 1996). As a
result, Z39.50 functionality can and has been presented to users in a variety of guises. Miller
(1999) highlights the use by both Melvyl and the AHDS Gateway (now superceded) of Z39.50 to
present a single, virtual resource, even though the data being searched is distributed. The user has
no idea that Z39.50 is being used, though, as it is kept hidden behind the user interface. Similar
services have been developed through the Large Scale Resource Discovery (Clumps) projects
funded by JISC (Stubley, 2000) (including CAIRNS, M25 Link, and RIDING), all providing
access to many different and diverse resources through a single interface.

Although a range of resources is offered by these services, they are pre-set by those running the
service, and users cannot add additional targets to the search unless they are available as options.
Z39.50 is more transparent through dedicated Z39.50 clients, where resources can be set up as
required (Soffer, 1997). For example, BookWhere83 is a dedicated Z39.50 client that allows the
user to configure his or her own Z39.50 targets. Bibliographic management software such as
EndNote84 offers a configurable Z39.50 client to supplement their main functionality, and use this
to allow references to be pulled into a local database directly (East, 2003). Notwithstanding this,
such clients now generally offer a wide range of pre-configured targets for the user to choose from
in addition to allowing configuration.

The work of the Clumps projects reflected a vision for using Z39.50 to create virtual union
catalogues (Stubley, 2000), permitting access to many different catalogues wherever they
happened to be located within a city, region, country or the world. Experience within these
projects, though, highlighted concerns about the variability in the implementation of Z39.50 and
its performance when compared to physical union catalogues (Stubley et al., 2001). Lynch (1997)
and Cousins (1999) had both addressed the pros and cons of physical vs. virtual catalogues and the

See http://www.iso.org/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=27446&ICS1=35&ICS2=240&ICS3=30
for details of availability of this standard, at cost through the ISO
http://www.loc.gov/z3950/agency/, a website containing many resources related to the protocol
http://www.biblio-tech.com/html/z39_50.html, as part of a full two-part introduction to Z39.50 by Peter Evans (last
update, 2001)

CREE Deliverable S1D1

role of Z39.50 within these, and identified a desire to see the best features of both used. Although
there have been some concerns about the scalability of Z39.50 in large union catalogue scenarios,
Hammer and Andresen (2002), in a study for the Danish Electronic Research Library project,
found that large-scale parallel searching of distributed catalogues was feasible, but only if all the
parts of the chain were tuned accordingly.

“… it is absolutely crucial that all components of such a service be given due attention. It
is not, ultimately, possible to think of a collection of Z39.50 servers as so many "black
boxes" on which you can build your service. Each individual server's reliability and
performance has a direct influence on the reliability and performance on the central
service, and each must be scaled and tested to handle the kind of load that will be imposed
on it – by local users and by the central service.”

Further work on Z39.50 response times has since been carried out by the COPAC/Clumps
Continuing Technical Cooperation Project (CC-interop 85), which also investigated the feasibility
of linking physical and virtual union catalogues, in their case utilising the JAFER toolkit86 to
provide Z39.50 to Z39.50 middleware (see Fig 4). This found that fast parallel searching was
feasible, though ‘quick and dirty’ Z39.50 implementations were often the cause of slowness
(Macgregor, 2005). The project further concluded that whilst technical interoperability between
the catalogues was feasible, semantic differences between them in the way the targets had been
configured and varied cataloguing practices were a large barrier to effective searching (Gilby and
Dunsire, 2004).

Fig 5.: CC-interop Z39.50 middleware using JAFER Z39.50 toolkit

The use of a standard profile, such as the Bath profile87 offers a means to configure Z39.50 in a
standard way (Lunau, 2000), but implementation has been more problematical and limited than
hoped (Nicolaides, 2003). Gatenby (2003) came to the same conclusion, noting that take-up of
the standard, whilst steady, has not been as widespread as hoped and recognising that inherent
differences in the way Z39.50 and the Web worked which are difficult to fully resolve may have
contributed to this. Needleman (2000) had also identified this gap between the Z39.50 and Web
communities and felt that if Z39.50 were to flourish in the future it would have to adapt. SRW is
an attempt to do just this and is considered in more detail below.


CREE Deliverable S1D1

Notwithstanding the doubts expressed about the value of Z39.50 in large-scale distributed
searching, its value as a tool in searching a number of distributed resources is still recognised.
Models of metasearching, one of the terms applied to searching across a range of resources and
explored further in a later section, commonly include Z39.50 as one of the protocols to be used
(for examples see Moen 88, and Bevan 89). Larson 90 describes the innovative use within the
Cheshire system91 of two lesser-known functions within the Z39.50 protocol: explain and scan.
These do not search the resources directly, but find out what is available from the resources
(explain) and then retrieve an index of the resource for later searching as required (scan). Some
commercial metasearch systems, such as MuseGlobal 92 also use Z39.50.

Z39.50 appears to still be a widely used protocol within systems. But what of the users’
perspective on the functionality it can offer? In an early examination of how users felt about
searching multiple databases using Z39.50, Payette and Rieger (1997) found high enthusiasm for
multiple bibliographic database access through a common interface amongst staff and students at
Cornell University. This was tempered with concerns about the response time, information
overload and irrelevancy, plus demands for functionality such as relevance ranking and de-
duplication that were more associated with single database searching. Students were stronger in
their preference for a single point of access, whilst staff indicated a slight desire to be able to fall
back on or resort to individual databases where possible. The ability to check holdings in the local
catalogue was also deemed valuable. Palmer and Robinson (2001) found that user case studies
within the Agora project93 concurred with this, whilst noting that users were frustrated by the
technology not being able to live up to their vision of what might, or should, be possible.

3. SRW/U

The Search and Retrieve Web Service (SRW) and its companion, the Search and Retrieve URL
Service (SRU), have been developed under the aegis of the Z39.50 International: Next Generation
(ZING) initiative94. SRW is defined on its homepage95 as,

“…a web service for searching databases containing metadata and objects, both text and non-

As its origins make clear, SRW is an initiative that is building on Z39.50’s background and aims
for use in the web environment, addressing the differences in approach identified above. Storey
(2004a) provides background to this development and describes OCLC’s role in its development,
particular Ralph LeVan. LeVan himself explains the relationship between the two protocols in a
presentation to the DSpace Conference in 200496, highlighting the weaknesses of Z39.50 and how
SRW can address some of these. SRW currently offers a subset of Z39.50 functionality, but based
on XML, XML Schema and SOAP; SRU also receives results using XML, but utilises URLs as its
means of sending requests. The SRW homepage states,

Presentation entitled ‘Users and Metasearch Applications: New Challenges for Usability Assessment’ given at Access
2003: Extending our Abilities, October 2003, Vancouver, Canada, available at
http://www.unt.edu/wmoen/presentations/Access2003_Usability_Oct2003.ppt [accessed 05/08/05]
Presentation entitled ‘Getting more value from your subscriptions: cross-searching and OpenURL links’ given at EDINA
Exchange event, May 2004, Edinburgh, UK, available at http://www.edina.ac.uk/events/edinaexchangefiles/cross-
searching.ppt [accessed 05/08/05]
Presentation entitled ‘Distributed IR for Digital Libraries’ given at ECDL 2003, August 2003, Trondheim, Norway,
available at http://cheshire.lib.berkeley.edu/ECDL_03.ppt [accessed 05/08/05]

CREE Deliverable S1D1

“Building on Z39.50 semantics enables the creation of gateways to existing Z39.50 systems; web
technologies reduce the barriers to new information providers allowing them to make their
resources available via a standard search and retrieve service.”

Version 1.0 of the SRW specification was released in November 2002, followed by the more
stable version 1.1 in March 200497. It was released in tandem with equivalent versions of the
Common Query Language (CQL) that underpins SRW and provides a standard means for
formulating the search queries 98.

Interest in SRW was apparent well before the protocol was released. Powell and Lyon (2002)
identified that SRW complemented the existing use of OAI-PMH and Z39.50 within the JISC
Information Environment nicely, and it has since been added to the standards framework
supporting the JISC IE for use within distributed searching99. In the same article, Powell and
Lyon also highlight how it is hard to make the case for Z39.50 to be considered a web service
according to W3C definitions. SRW has no such concerns. The EU-funded ARTISTE project100,
developing integrated content and metadata-based image retrieval, was an early implementer of
SRW in 2002 and examined the use of the protocol for image retrieval, finding it limited in its use
of only the initial subset of functionality available, but also flexible and able to allow the
presentation of a standard interface across image collections (Stevenson, 2002).

Because SRW is based on Z39.50 semantics, existing Z39.50 profiles can be re-expressed for use
with SRW, for example the Zthes profile for representing and searching thesauri 101. Existing
Z39.50 search systems are also planning to implement SRW as an alternative interface (e.g.,
GetRef from EDINA102). Commercial systems are also expressing interest, with Dynix intending
to implement SRW within its Horizon Information Portal product 103.

SRW is also being viewed as a valuable interface to library systems and institutional repositories.
The VTLS Virtua library system includes conformance to SRW for access to the collections it
holds (Anon, 2005b). The VTLS VITAL repository product, based on the open source Fedora
repository framework (Anon, 2004a) is also being used to underpin the architecture for the
ARROW (Australian Research Repositories Online to the World) project, which is including an
SRW and SRU interface alongside OAI-PMH and Google interfaces 104. SRW/U, selected over
Z39.50, is regarded as the means to enable real-time searching of the repository, to complement
the ‘proxy’ search services provided by OAI-PMH and Google, which rely on regular but not real-
time harvesting of records. There has been increasing recognition of the parallels between the
mechanisms used by SRW/U and OAI-PMH. Morgan (2004) compares OAI-PMH specifically to
SRU and highlights its functionality as a Web service, whilst LeVan (2005) proposes a number of
ways in which the two might be used together. OCLC has developed an SRW client that allows
SRW access to DSpace repositories, by converting the CQL queries into queries for Lucene, the
underlying DSpace search engine (See footnote 95).

The interest in SRW has been mirrored by interest in SRU, either in terms of providing interfaces
using both together, like Index Data’s YAZ Proxy toolkit105, which offers a gateway between
SRW/U and Z39.50, or as separate search technologies. Morgan (2004) describes SRW and SRU
as “brother and sister” Web service protocols. SRW is a SOAP-ful service, where the request and

Online document describing the use of Zthes with SRW, by Mike Taylor, November 2003, available at
Online paper entitled ‘Arrow targets: institutional repositories, open source and Web services’ by Andrew Treloar (2005)

CREE Deliverable S1D1

response use SOAP XML vocabulary; SRU is REST-ful, an alternative Web service flavour that
encodes requests as part of a URL whilst also receiving results as an XML stream.
Communication takes place over HTTP. As mentioned above, Morgan also highlights the
similarity between OAI-PMH and SRU, both of which use REST (Representational State
Transfer) and SRW. All offer ‘about’ and ‘browse’ type functions and mainly differ in the
granularity of retrieval, with SRW/U being more specific because of the use of CQL as the query

Morgan gives an example of how SRU is used within the OCKHAM project 106, to implement an
alerting service. SRU is also being used as the main search protocol in the European Library
(TEL) project (Woldering, 2004), a scheme to facilitate access to national libraries across Europe.
SRU was chosen over SRW in this project as it was deemed simpler to implement. A gateway has
also been developed to enable SRU queries to be translated to Z3.50 and back, allowing access to
existing Z-targets 107. SRU is also being used within the d+ project 108 at the University of
Edinburgh in combination with OpenURL, another means of structuring searches within a URL.

The recent advent of SRW/U has not yet led to user evaluations of its impact, and it will be
interesting to see how these differ, if they do, from evaluations of Z39.50 services.


During the 1990s a number or pre-print or e-print archives emerged in a variety of subject areas to
facilitate scholarly communication outside of the regular means through published journal papers.
As the number grew it became apparent that it would be valuable for them to cooperate to enable
easier access across them by researchers and others wishing to access their contents. In October
1999, a meeting was held in Santa Fe, USA to discuss this cooperation (Van de Sompel and
Lagoze, 2000). The meeting led to the establishment of the Open Archives Initiative109 and,
specifically, a series of organisational principles and technical specifications to facilitate a level of
interoperability between e-print archives. These became known as the Santa Fe Convention of the
Open Archives Initiative. The underlying mechanism to enable interoperability was metadata
harvesting, where metadata from across different e-print archives can be harvested into a central
service or services that can then be searched locally. This architecture was described by Bowman
et al. (1995) and also used in a prototype service presented to the Santa Fe meeting (Van de
Sompel et al., 2000).

The Santa Fe Convention has since developed and been renamed as the Open Archives Initiative
Protocol for Metadata Harvesting (OAI-PMH), which is now established at version 2.0 110. A
history of this development can be seen in Fig. 5 111.

The Santa Fe Convention established at an early stage two separate roles or participants: data
providers, which make available the data from an archive or collection for harvesting; and service
providers, which carry out the harvesting and provide end-user services based on these harvested
collections. An early, but now well-established service provider was Arc, developed at Old
Dominion University (Liu et al, 2001), which was based on the prototype demonstrated at the
Santa Fe meeting and is now used widely by other service providers, notably the RDN’s ePrintUK
initiative (Martin, 2003). The Arc service provider112 can in turn be harvested by other service
providers, and can thus act as an aggregator service as well as provide end-user search access.

Taken from Carpenter, L. et al. (2003), OAI for beginners – the Open Archives Forum online tutorial,

CREE Deliverable S1D1

The use of the metadata harvesting model by the OAI is in contrast to the distributed searching
model used widely as a way of achieving interoperability across digital library collections, often
using Z39.50 as the means to enable this. Whereas Z39.50 is highly functional but also highly
complex, OAI is designed to be simple and this gives it its strength (Nelson, 2001). Gatenby
(2002) offers a comparison between the distributed search versus harvesting model in the context
of building or enabling a union catalogue, contrasting these with the building of a physical union
catalogue and concluding that a combination of all three will be best.

Fig 6.: OAI development history

The simplicity of OAI, though, does still require that the necessary implementation of the OAI-
PMH is in place, something that organisations can enable, but not individuals. Two possible
solutions to this have been proposed. Firstly, the team behind the Arc harvester developed the
Kepler framework (Maly et al. 2001) makes use of ‘archivelets’ to enable individual researchers to
publish results and make them available for harvesting rapidly from their own PC rather than an
institutional server. The OAI itself have developed a specification for OAI Static Repositories 113,
which allows a locally stored XML file to be made available for harvesting by a remote service.

Systems supporting OAI-PMH have, though, emerged to enable institutions to easily expose
metadata of content they store within repositories. A full description of many of these is available
through the Soros Foundation114. It is notable that many of these are open source software.
However, the commercial sector is now also getting involved, with VTLS, for example, launching
its VITAL product, based on the open source Fedora repository framework (Anon, 2004).

The mission statement of the OAI was established at the Santa Fe meeting as:

“The Open Archives initiative has been set up to create a forum to discuss and solve matters of
interoperability between author self-archiving solutions (also commonly referred to as e-print
systems), as a way to promote their global acceptance.” (Van de Sompel and Lagoze, 2000)


CREE Deliverable S1D1

As the diagram above shows, though, development of the OAI-PMH has also coincided with an
acknowledgement that the protocol can also be used with a wide variety of other materials.
McKeown (2003) describes how OAI can be used for museum objects, acknowledging the
difficulties faced when dealing with such materials, whilst Copeland and Penman (2003) outline
how the same principles for e-prints can also be applied to e-theses, a topic also addressed by Fox
and Suleman (2003). Bird and Simons (2003) describe how OAI-PMH has enabled the sharing of
language resources within the Open Language Archives Community (OLAC). They also
highlight the possible limitations of one aspect of the protocol. As the table above makes clear the
metadata format used to determine what data is harvested is unqualified Dublin Core. The
protocol makes this mandatory, though only as a lowest common denominator, and leaves open
the possible use of more complex metadata schemas by others. OLAC have done just this in order
to facilitate full exchange of information about the resources held. Richardson and Powell (2003),
in addition to accessing learning objects using Z39.50, also experimented with exposing detailed
metadata about them using OAI-PMH, with some success. Van de Sompel et al. (2004) describe a
potential path to allow complex objects, including both content and metadata, to be harvested
using OAI-PMH.

The metadata to be harvested can also be used to transfer information within it on aspects of how
the content might be accessed or used. Gadd et al. (2004) examined the possibilities of
transferring rights metadata within OAI-PMH, and proposed a model for e-prints using Creative
Commons115. The OAI-Rights working group has extended this and a full specification is now

The impact of OAI on scholarly communication has been high, with wide uptake of the protocol
(Rusch-Feja, 2002). The JISC’s Focus on Access to Institutional Resources (FAIR) programme117
was launched in 2002 to further investigate how institutions might make use of OAI-PMH to
enable wider access to locally stored resources, including, but not limited to, e-prints (Awre,
2004). Similar initiatives have been launched in Holland118 and Australia119. Brody et al. (2003)
promote the idea of disseminating scholarly communications using OAI-PMH and the use of
institutional repositories as the data providers making their data available for others to harvest and
use accordingly. The use of e-print archives as alternative means of disseminating scholarly
materials has resulted in a widespread debate on open access publishing. However, the two do not
necessarily have to go together. It is possible to use OAI-PMH in a closed environment, within an
institution, for example to facilitate sharing within this, and it is possible to have open access
without using OAI-PMH (Pinfield 2003). The debate is continuing.

Using aggregations of metadata brought together by OAI-PMH is another area that requires
further work. The subject metadata used by one data provider may not match the terms used by
end-users when trying to search it. Shreeves and Kirkham (2004) found that use of heterogeneous
collections brought together using OAI-PMH was not straightforward and a number of challenges

5. Web search engines

The use of web search engines, and particularly Google, features large in surveys of users as the
choice of technology to use when searching for information and over two-thirds appear to feel
satisfied with the results gained in this way (Griffiths and Brophy 2002). In considering the types



CREE Deliverable S1D1

of services that users will want to use in the future and how these are presented, web search
engines are thus likely to be a major player.

Web search engines have been around as long as the web itself, enabling information presented
through this medium to be retrieved. They work by using spider programs that go out onto the
web and gather web information from wherever they can get it, creating huge, centrally stored
indexes that are then made available for searching. The spider programs used varied and were the
cause of some competition in the commercial web search engine market from an early stage.
Stanley (1995) took an early look at two big name search engines of the time, Lycos and InfoSeek,
to see how they compared. Quality of results was a major issue then as it might be argued it is
now. Poulter (1997) offered an early critical review of web search engines, describing their link
to pre-web technologies such as Gopher and Veronica.

However useful these studies were at the time, focussing on specific comparisons, they did not
represent a level playing field. A review of the literature of such evaluations by Oppenheim et al.
(2000) showed that there had been no consistency and that a standard set of tools should be
developed to allow a more useful comparison. In a comparison of six search engines Nowicki
(2003) found that the ability, or lack of it, of students in constructing search statements had a large
effect on the potential success of the search. In this vein, Ozmutlu et al (2003) examined the way
people are encouraged to ask questions in some search engines (e.g., Ask Jeeves) as opposed to
using keywords. They compared this with previous search data in two other search engines
(AllTheWeb and Excite) from 2001 and found that, in line with other studies at the time, search
queries were getting shorter, and that asking structured questions was not highly used. This
finding contrast, though, with a study of use of AltaVista between 1998 and 2002, which showed
query length lengthening, as well as a broadening of query topics (Jansen et al., 2005).

Professional evaluations of web search engines have been matched by studies on how users
decided which search engine to use. A study of library and information science students at North
Carolina University at Chapel Hill revealed that the students preferred search engines to search
catalogues – they did not see any advantage in pre-organised materials from the web – and that
they preferred search engines that gave them good results over those that were easy to use
(Vaughan, 1999). This would appear to favour specific online databases over search engines due
to the richness of the results they can offer. In contrast, a later study by Xie (2004) showed that
online databases were indeed welcomed for this level of detail, but that the ease of use of web
search engines was a far greater factor in affecting use than might have been the case before. The
popularity of their use is reflected in a study of subject searching within a library OPAC, where a
large number of enquiries used a style that was more suited to web search engines and which
would get better success with these over library catalogues (Young and Yu, 2004).

Chen et al. (2001) investigated the use of Web metasearch engines, tools that can search across
many individual search engines, and offer the case that such searching enables better precision by
having more sources to take results from. This is particularly effective when combined with a
categorisation facility that presents the results in an organised view. A specific example of this is
a ‘cancer meta spider’, developed to enable focussed access to cancer resources on the web (Chau
et al. 2003).

Mischo (2001) highlights the need to take into account web search engines when planning a
library portal, to ensure that these can be used alongside more specific resources and provide all
results in one place. Medeiros (2002) reports on the launch of a commercial offering attempting
to do exactly the same thing, Scirus, labelled the world’s first scientific search engine by its owner
Elsevier. This combines access to a web search engine and Elsevier’s own specific resources,
allowing access across these and combined results. Zhou (2003) wonders why many libraries
haven’t gone down the path of establishing web portals, following a trend started by many web
search engines in adding additional services to the basics of search (e.g., MyYahoo!), and offers
some guidelines to support such development. Dawson (2004) reflects on the use of metadata

CREE Deliverable S1D1

within libraries and how web search engines such as Google make little use of this in providing
their service. He makes recommendations on how library metadata can be used to support those
users making use of Google whilst still maintaining its value for more library-oriented activities
such as cataloguing and preservation.

Looking to the future, Lu et al (2002) consider the potential impact of the Semantic Web on the
retrieval of information by search engines and raise the possibility of semantic-based web search
engines. This potentially offers tremendous benefits in retrieval, though it remains to be seen
whether the web will move in this direction as a whole.

6. RSS

RSS, which has up to three meanings120, is not a technology that is normally used to enable
searching. It is a technology that primarily enables summaries of web content and links to this
content to be syndicated to whoever would like to access it. This can, for example, enable
announcements or news items to be distributed easily, but also allow users to be informed of
regular updates to resources that they might be interested in. This delivery of results from pre-
selected sources is akin to the end aim of using search engines and this warrants a brief
consideration of how RSS can be used in this review. It is also possible for RSS to be used more
directly as a search medium and this is described below.

RSS as a standard has been through a number of iterations since it was first developed. A good
summary of this history can be found in Wikipedia121. The various versions and incompatibilities
between them have meant that RSS is still a moving target and not yet stable (Wusteman, 2004).
Alternative syndication protocols have continued to emerge. Of these, Atom122 appears to be
gaining wide usage.

The idea of syndicating information has been picked up in a number of arenas. RSS has been seen
as a valuable asset for delivering information through portals. An early aim of developing portals
for the RDN included both RSS and OAI in its plans (Clark, 2001). RSS has also been considered
and promoted as a way of enhancing web pages. Storey (2004b) describes the use of RSS within a
personalised MyLibrary 123 webpage, where users can be allowed to specify the services they need.
Personalisation is also the long-term aim for the National Library of Health (Toth, 2004).
Librarians can make recommendations on appropriate RSS feeds, having vetted them first. This
would help avoid the experience of one website, where a third party scrapped the webpage content
and produced RSS feeds from it without first asking permission (Spence, 2004).

RSS has also been regarded as a way in which content updates can be filtered and customized to
provide more focussed information delivery (Arnold, 2004). As a result, a user can gain access
locally to the relevant contents of a website without necessarily having to visit it (Moffat, 2003).
With the huge amount of information available via websites, RSS allows a user to view just a
synopsis or snapshot of this information, making it easier to deal with (Hammond et al., 2004).
The websites still contain the detail of the information available and can be visited as required to
gather this. But RSS challenges the primacy of the website as the first port of call for information.

Hammond also points to a connection being made between RSS and the increase in the use of
Weblogs and wikis, and related developments of social environments on the Web. RSS can be
used to deliver the contents, or a synopsis of them, to readers via RSS readers or feeds embedded
in websites, increasing their dissemination and readership; this connection to the ‘blogosphere’ is
also reported by Conhaim (2004), who notes that many organisations are stating to join the
syndication ranks. He also notes, though, that even though RSS is aimed at spreading information


CREE Deliverable S1D1

widely, it can potentially widens the chasm between the information haves and have-nots, and that
work will be required to bridge this.

As well as news, announcements or Weblogs, RSS can be used to provide feeds of other types of
information, notably search results. The IMesh project produced a module that enables RSS to be
used for the sharing of information harvested using OAI-PMH, allowing such records to be fed
directly to the user and updated as fresh harvests take place (Duke, 2003).

7. OpenURL

When searching, a user can deal with results in a variety of ways. Linking from search results to
other associated resources (for example, an item’s location, an inter-library loan form, or similar
items in a database) enables the user to maintain their search without feeling they have reached a
dead-end. OpenURL124 offers a mechanism to achieve this and its origins and uses are considered
in this section.

The origins of OpenURLs lie in work carried out at the University of Ghent in the late 1990s.
Hitchcock et al. (1997) had attributed the explosive success of the World Wide Web to its linking
possibilities. A number of scholarly information content providers had taken advantage of this,
linking primary and secondary data together wherever this was possible. However, these linking
services were limited to data that was available to link to and this usually meant data within the
authority of the provider. They were closed linking frameworks when open linking frameworks
were needed, allowing many different types of content to be linked to regardless of ownership or
authority (Van de Sompel and Hochstenbach, 1999a). In a companion paper, Van de Sompel and
Hochstenbach (1999b) demonstrated their prototype SFX system, which aimed to provide a series
of links to users in the context of the whole range of services available to them, i.e., the user’s
context. This context also allowed SFX to address the appropriate copy problem of whether
linking actually took the user to the additional resource the user’s institutions wished them to
access (Caplan and Arms, 1999). SFX offered a ‘just-in-time’ approach to linking, showing
people where they could link to immediately, as opposed to a ‘just-in-case’ approach, where the
user may or may not be offered links depending on whether they are available. All the linking
was also dynamic, being created on request from the metadata available in the source (usually
bibliographic), rather than pre-configured or static as with many publisher-based linking schemes.
The metadata in the OpenURL is ‘resolved’ into a URL for the resource in question. A third
paper by Van de Sompel and Hochstenbach (1999c) highlighted the ability for the SFX solution to
used generically between libraries and remote resources.

Fig 7.: OpenURL context 125


CREE Deliverable S1D1

The SFX open linking framework was subsequently adopted by Ex Libris126 and is now available
commercially. The framework was further developed in association with Ex Libris and became
known as the OpenURL framework (Van de Sompel and Beit-Arie, 2001a). The technology was
also put forward for standardisation by NISO and a committee was set up in 2001 (Anon, 2001),
which has recently released version 1.0 of the standard 127. This standardisation process sought to
generalise the framework as much as possible so that it could be used in the widest possible
contexts, and was informed by adoption of the Bison-Futé abstract framework model (Van de
Sompel and Beit-Arie, 2001b).

OpenURL is not alone in the development of linking technologies that seek to enhance the ability
of the user to make connections between resources. The CrossRef128 system, which went live in
2000 (Anon, 2000), builds on collaboration between publishers and is a way of extending the
reach of resources that a content supplier has ‘authority’ over, or in this case a means of access to.
All publishers apply a Digital Object Identifier (DOI)129 to a work to ensure permanent
identification. This DOI and associated metadata are then placed in the central CrossRef database.
Bibliographic databases use their metadata to obtain a matching DOI, which can then be resolved
into a URL to the full-text from the publisher (Pentz, 2001). This system is still a closed linking
framework, though it can be combined with OpenURL to create an open-linking framework that is
capable of making use of DOIs (Van de Sompel and Beit-Arie, 2001c). The use of OpenURL and
CrossRef/DOI can thus be considered complementary and a valuable combination in meeting
library needs (Walker, 2002).

The availability of OpenURLs is dependent on the availability of an appropriate OpenURL

resolver. A number of these are now available commercially (Ferguson and Grogg, 2004), with
each offering a range of functionality built around the principle of resolving OpenURLs.
Established content suppliers are providing resolvers of their own as a way of enhancing the
services already offered (e.g., Cook and Dowling, 2003). Some libraries are building their own
(Dahl, 2004). Bibliographic resources also need to make their resources OpenURL compliant, so
that they can send OpenURLs to appropriate resolvers. More and more are adapting themselves
accordingly (e.g., Anon, 2004b). It is important that in implementing OpenURL, publishers are as
accurate as they can be, otherwise frustration and dead-ends can result (Chen, 2004).

OpenURLs do not need to be used solely for remote resources, but can be used to enable access to
internal resources just as much. The EnCoRe project 130 at the University of Derby used
OpenURLs to link into a large collection of copyright cleared locally digitised documents from
reading lists and the local catalogue (Owen-McGee and Swanson, 2003). This is a way in which
OpenURLs can help to support e-learning, dynamically extending the library into the delivery of
e-learning materials and courses. Randall (2004) discusses the possible uses of OpenURLs in this
arena, including the use of OpenURLs within virtual learning environments as a means of
embedding dynamic links to a range of materials from within these.

OpenURLs can also make use of different types of bibliographic records to enable linking.
Powell (2001) showed how Dublin Core metadata could be used to construct a partial OpenURL
and allow linking from Dublin Core records. For more complex records, Bekaert et al. (2004)
describes a mechanism for using MPEG-21 131 DIP and using OpenURL to make these available.

The flexibility and application of OpenURL continues to develop. Resolver suppliers are being
more proactive in their linking to inter-library loan forms to enable a request to be made even if
the full-text is not immediately available (e.g., Anon, 2005a). Google, too, is integrating


CREE Deliverable S1D1

OpenURLs into its Google Scholar132 service, to allow users to link to the full-text of an article
found when searching this resource (Chillingworth, 2005), albeit that this was in part a response to
concerns expressed by the academic community that general searching of Google Scholar would
lead to frustration when access to many of the resources listed would not be possible. CrossRef is
also working closely with Google to use the search engine for access to the central CrossRef

8. Metasearching

The term metasearching has been used in a number of different contexts. It indicates the ability to
carry out searching across a number of, usually distributed, resources at the same time. The
technique has also been called cross-searching or federated searching. With the availability of
multiple sources of information there has been long interest in being able to search across multiple
resources as a means of saving time and effort, whilst also potentially enabling better and more
targeted access to information.

The term metasearching was first applied to search tools that enabled a user to search across
multiple web search engines. This allowed the user to take advantage of the different indexes
compiled by the different web search engines and search across all of these. A study by Garman
(1999) concluded that whilst metasearch engines were no better than using a combination of web
search engines in terms of their precision (the metasearch engine is simply another way of
searching the same indexes), metasearch engines are valuable as they can act as a useful starting
point without having to visit many individual search engines. Repman and Carlson (1999) carried
out an evaluation of five metasearch engines, reviewing their different functionality. This
suggested that librarians may prefer metasearch engines due to their customisation and advanced
search features, whilst their relative speed of response would make them valuable to users in
general. Jacso (2002) describes a method for using the customisation features of one metasearch
engine to enable him to search a range of electronic journals that were freely available on the

In the scholarly information field, metasearching has also been a goal, and has been made
available in a number of guises. Tenopir (2001) describes the service available through ISI’s Web
of Knowledge product134, which allows access to the three main ISI Citation databases as well as
across a number of others. This is made possible through ISI licensing these additional products
and making them searchable through one interface, their own. The resources are, thus, all
centrally located and not distributed.

The work of the eLib Clumps projects 135 highlighted the possibilities of using Z39.50 as a tool
through which metasearching could take place of distributed library catalogues. This detailed
method of enabling the metasearching, though, was dependent on the level of cataloguing being
commonly adopted and good enough to guarantee appropriate accuracy of results (Dunsire, 2003).
Further use of Z39.50 as a mechanism for enabling metasearch is described in the earlier section
on this standard. The use of Z39.50 together with other standards described in this review,
particularly OAI-PMH, has enabled the National Library of Australia to host federated search
services, facilitating access to its own many collections rather than expecting users to access each
one individually (Campbell, 2004). Webster (2004) also considers that the use of standards can
help to break down information silos and allow access across these. However, he proposes that
OpenURL be used as the means for smooth transition and access across resources, rather than
searching directly across them.


CREE Deliverable S1D1

This combination of technologies has been at the heart of a number of commercial metasearch
products from library management system vendors. There has been much trumpeting of this
approach, allowing access to whatever resource is required. Luther (2003) considers the
possibility of such metasearch techniques competing with Google as a primary source of
information. They can particularly provide access to the deep web, that part that is not accessible
by general web search engines. Moen (2004) describes the use of metasearch engines to gain
access to the deep web at the University of Texas, combining techniques and technologies as
required. Configuring metasearch engines to achieve the desired goal, however, requires careful
attention (Sadeh, 2004).

The use of metasearching has also been at the heart of the development behind library portals, by
which a number of commercial library metasearch engines are known. The Scholar’s Portal
initiative136 in the US sought to use Fretwell Downing’s ZPORTAL product 137 to enable
metasearching services across a range of research libraries, but required careful planning to ensure
the most useful services were presented (Feeney and Newby, 2005).

Dempsey (2004) also describes this project alongside a report on the NISO Metasearch
Initiative138 to examine and possibly standardise metasearch and move forward from the various
combinations of searching techniques that currently come under the metasearch umbrella. A draft
standard is being issued (Anon, 2005d) and it remains to be seen how this work will progress.
Alongside the NISO initiative, a group of library management system vendors have formed the
VIEWS group139, who are examining the use of web services within library systems, have
themselves issued a paper on metasearch (Anon, 2005c) with particular emphasis on how web
services might enable and support this. The Research Libraries Group 140 surveyed its members on
expectations of metasearch in 2005 (Arcollo, 2005). They found that metasearch was of value,
but that implementation required support from target systems in order to make it work effectively.
The benefits were also more related to bibliographic resources than, for instance, image resources.
De-duplication and ranking were not among the most important considerations in setting up a
metasearch service.

There is no doubt that metasearch across many different information resources will continue to be
developed. The concepts behind metasearch are also being further examined to assess how they
can be best developed to serve a library’s users. Christenson and Tennant (2005) investigated the
principles, technologies and approaches to integrating information resources, of which metasearch
is one approach, and have produced sets of content discovery and integration principles to help
guide future development and implementation of integrated search services.

9. Usability

A literature review on the topic of usability could extend far and wide. This section considers
some of the work that has taken place in considering the usability and presentation of search
interfaces. There is not necessarily much direct connection with the individual technologies
covered so far, though the search services they provide will have interfaces to which usability and
presentation is highly relevant if they are to be successfully used.

The value of presenting electronic information in a usable format goes back a long way. Jackson
(1988) examined the way in which data downloaded from commercial online search services
could be manipulated, presented and managed to enhance its information content and value to the
user. The advent of the World Wide Web brought this need to consider how electronic
information is presented to users more into focus. A special issue of the International Journal of


CREE Deliverable S1D1

Human-Computer Studies in July 1997 141 focussed on usability within websites in general, with
the article by Shneiderman (1997) offering a good overview of design at the time and includes
recommendations for search and navigation within websites. Considering search as one of the
tasks to be undertaken within a website can help with analysing what users need to do in order to
carry out that search and how they should deal with the results. At a similar time, Jakob Nielsen
was producing a regular Alertbox 142 on web usability issue that continues to this day, offering
much advice on this topic. His book ‘Designing Web usability: the practice of simplicity’
(2000)143 has sold over 250,000 copies in 22 languages, showing how much interest and
importance there is in this area. Another popular book, ‘Don’t make me think!: a common sense
approach to Web usability’ by Steve Krug (2000)144, also recommends the simple approach,
stressing that the less a user has to think about what they are doing on a website, the more likely
they are to do it. Having said that, knowing your audience and what they expect is also key to
knowing might be considered simple and what not.

In the digital library field, considerations of usability and presentation have also been the topic of
discussion. Pope (1998) identified and explored four broad areas for future development in the
digital library environment, of which interfaces, presentation and display are one. He describes
the benefits of Z39.50 in providing a common interface to a range of different resources, with all
being searched from the same place. He also considers the possibilities of visualisation techniques
and raises awareness of the needs of disabled users, whilst also warning that digital library
systems have a tendency to add functionality rather than develop usability, leading to a lack of
intuitiveness and a greater need to train users. As mentioned by Pope, the testing and improving of
usability often needs to be considered alongside the testing of accessibility, to ensure that disabled
users can also gain access to the services on offer. Kirkpatrick (2003) argues that focussing on
accessibility can bring about improvements in usability by default, as accessibility guidelines aim
at making a website and service as simple to use as possible, following the exact advice from the
usability books described above.

The design of library and digital library websites can take on board the recommendations made in
the texts highlighted above. But it is equally valuable to get feedback from users through usability
testing. Buffalo University Libraries found such testing to be an excellent way of evaluating and
improving the effectiveness of their website (Battleson et al., 2001). Many other case studies are
available that express the same sentiment (e.g., Callicott, 2002/Cockrell and Jayne, 2002). It is
not just library web sites that can benefit from this. An examination of the usability of subject-
based portals in Denmark revealed that design problems had made the portals difficult to use, and
that users need to be involved at an early stage in the development process (Nyborg, 2002). Such
early usability testing as part of development was used at the University of Malaya Library to
design their new website (Ramly, 2002). It is necessary, though, to make sure that the tests being
used within usability testing are realistic to the users, and do not just reflect librarians’
interpretation of “good” library website usage, especially when so many users are more used to
using web search engines such as Google to find their information (Callicott and Vaughn, 2003).
This so-called “broccoli librarianship” (as in, eat your broccoli, it’s good for you!) can do more
harm than good. Having said that, users don’t always know what is best for them. A test of
searching a collection of Adobe PDF files by Barnum et al. (2004) revealed that whilst users
preferred to use the full-text search within Adobe Acrobat, they actually obtained more accurate
results to pre-set queries by using the electronic back-of-the-book index.



CREE Deliverable S1D1

Work has also taken place on the usability and presentation of web search engines. The DEVISE
framework 145, developed at Manchester Metropolitan University, focussed on a definition of the
construct of user satisfaction of using such search engines, allowing a full evaluation of these to be
carried out (Johnson et al., 2001). Another study considered the use of navigational aids as part of
a web search engine interface and found that including these led to more accurate answers in less
time compared with users making use of web search engines without such aids (Mat-Hassan and
Levene, 2001). Chen et al. (2005) found that cognitive styles can play a part in how we use search
engines, with different people with different cognitive styles focussing in on different aspects of
the web search engine, suggesting different needs will need to be met for different users.

10. Portal frameworks

The literature available on portals is huge and increasing. The term ‘portal’ has been applied in a
wide range of arenas, often simply as the current term to apply to a website or service that existed
beforehand under a different guise. Defining a portal as a place which brings together a range of
services and information, most are able to adhere to this in some way or other. How it brings
these services together can vary, though, and the flexibility and freedom available to combine
what you would like even more variable. One increasingly popular route to enabling a high level
of flexibility appears to be the use of portal frameworks (see Figure 8). These pieces of
infrastructure act as scaffolding for a range of core services and additional individual information
channels or portlets (covered in the next section) to be combined in different ways, and allow very
specific and targeted portals to be presented to the user. This section reviews the literature
available on the development of these frameworks.

Fig 8.: Portal framework architecture overview (Pinto and Fraser, 2003)

In the late 1990s, commercial enterprise system suppliers along with other sectors started to re-
cast their products as portals. Karpinski (1999) reports on the delivery of a customer-relationship
management system as a portal. However, this wasn’t just a relabelling of an existing system.
The services were to be delivered through a server-delivered portal framework. At the same time,
Gilbert (1999) reports on the announcement that Oracle would be developing a portal framework,
which would be usable with a library of portlet services to be combined as the user saw fit.
Further announcements (e.g., Whiting, 2000) revealed the ongoing trend. Sanborn (2000) argues
that such a proliferation is beneficial to business. Companies were competing to add additional
functionality, and also starting a move towards the use of standards (XML) instead of their own
proprietary technologies. The idea of syndication, to allow content to be shared between portals,


CREE Deliverable S1D1

was also addressed. Moore (2002a) discusses the move to integrate identity management systems
with portal frameworks, to allow for better authentication of services presented through the portal.
For many enterprise system vendors, the portal framework is now the cornerstone of their
offering, upon which all other services are built (Moore 2002b), but whilst there are moves to
develop the portal frameworks further, allowing for more advanced services to be delivered
through them, there is also concern that the current systems might not be up to the job (Grant et
al., 2003).

The use of portals and portal frameworks within educational institutions has also emerged over
recent years. Looney and Lyman (2000) reported that in late 1999 educational institutions
themselves were being contacted by their enterprise system suppliers and asked if they would be
interested in a portal. Some saw this as an opportunity to expand the range of services that the
institution could offer; others were not impressed by what they saw as an invasion from the e-
commerce world and did not see the relevance to education. The authors conclude that,

“Why have a portal? We believe that a portal is a means of renewing and extending a sense of
academic community – learning communities, alumni or development communities, faculty
communities – using the Internet.”

This is at first sight very different from the e-commerce aims of the commercial sector and their
use of portal frameworks, and yet enables the university to take advantage of e-commerce
opportunities where available. The consideration of portals has slowly increased. Olsen (2002)
reports a survey where institutional portals (a term that has been applied to enterprise-wide portals
in universities) ranked fourth among the top 10 technology issues that college administrators
anticipated increasing in importance during 2002/3.

Jacobsen (2000) reviewed some of the existing portals put in place by universities in the US. He
also raised awareness of non-commercial possibilities for portal frameworks, including the
uPortal146 initiative from the (then titled) Java in Administration Special Interest Group (JA-SIG)
of Educause147. Over 100 institutions around the world have since picked up this framework.
Other portal frameworks are also in use now amongst universities148. The use of such portal
frameworks has been largely based on the institution’s perceived needs. Pearce (2003) reports on
a UK survey of users and potential users of institutional portals, carried out as part of the JISC-
funded PORTAL project 149, that revealed a wide range of services that users would like to see
within their institutional portal, including access to library services and resources.

Portal frameworks appear to be becoming established. Research is ongoing to find what else they
might be able to support. One key area is that of the Grid, supporting e-science activities. Feng et
al (2004) and Sipos et al. (2005) describe exemplars of portal frameworks for the Grid. Novotny
et al. (2004) describe their work on the GridSphere project150 (the name deriving from IBM’s
WebSphere portal framework product). Much as foreseen by Looney and Lyman for universities,
the aim of this project is to help foster collaborations and communities within e-science activity,
by providing a framework that can be adapted for a variety of activities and not just for a specific
project, which is where much previous Grid-enabled portal work had been concentrated.

11. JSR 168 and WSRP

As mentioned in the previous section, portal frameworks act as scaffolding upon which portals can
be built from component channels or portlets. As portal frameworks have been released, they

For information on a range of implementations and work in this area see presentations at the Pan European Portals
Conference (PEPC) 2004, held at the University of Nottingham, http://www.nottingham.ac.uk/pepc2004/

CREE Deliverable S1D1

have come with their own portlet APIs, their own different ways of developing and structuring the
portlets required to build these portals and means for portal frameworks and portlets to talk to
each other. Hence, each portal framework vendor had their own way of building portals. Gilbert
(1999) makes reference to Oracle planning a library of portlet services that could be used within
the Oracle portal framework. Indeed, initiatives such as Campus EAI151 have developed to
facilitate the exchange of portlets, primarily in this case for the Oracle and uPortal portal
frameworks. Following Oracle’s model, other suppliers are also offering their own libraries of
portlets as part of a contract, for example Sungard SCT offering a range of portlets as part of its
Luminus product built around uPortal152. The availability of such libraries is very helpful.
However, having a standard way of developing portlets, a standard portlet API, would broaden
this out to allow portlets to be used across many different portal frameworks. This would enable
greater interoperability and exchange of functionality and reduce development costs.

Schaeck and Hepper (2002) reported on initial plans in this direction. Both a Java Portlet API and
Web Services for Remote Portals were being investigated, seeking to increase interoperability
between portal frameworks and portlets. The latter was also more outward facing, seeking to
overcome problems of communication between the portal framework and the Web. Moore (2003)
reported on the ongoing development of these standards. The Java Portlet API, now known as
JSR 168 153, focused on a standard portlet programming API, establishing a standard way for the
portlet and portal framework to communicate. The Web Services for Remote Portals standard, the
development of which was being overseen by the industry standards body, OASIS (Organization
for the Advancement of Structured Information Standards)154, was focussing more on leveraging
Web services to integrate remote content and applications into portals through portlets. The
emphasis on portlets led to a change of name for the standards to Web Services for Remote
Portlets (WSRP)155 prior to release. Commercial portal framework suppliers, the source of the
standards initiatives in these cases, were quick to adopt the standards, with both Sun and IBM
promising JSR 168 implementation and others (Vignette and Plumtree) offering WSRP
(Callaghan 2003). Vendors themselves set up a Portlet Open Source Trading (POST)156 website to
facilitate the sharing of portlets built to the two standards (Savvas, 2003), both of which were
ratified and released toward the end of 2003.

Some commentators initially saw these two developments, which had run in parallel, as
competitive, as both are concerned with standardising the way portals can be built from portlets.
Zeichcik (2004) comments that whilst WSRP was not only designed to allow remote portlet-to-
portal communications, but also accommodate cross-platform portlets, JSR 168 was focused on
standardising the way in which Java-based portal frameworks communicated within themselves.
However, it was noted that due to this narrower focus, JSR 168 was easier to implement. Other
commentators saw this complementarity between the two standards, allowing both to be used
according to the purpose of the portal. Petersen (2004) concludes that using the standards should
ensure that portal projects are a success and enjoy long-term viability.

The use of Web services by the WSRP standard raises awareness of one specific use of Web
Services technology and standards in particular. OASIS has many such standardisation processes
in play, and many other Web Services standards exist, which could easily make up a whole
literature review of their own. But it is their potential for the future as a way of delivering
services across sectors wherever they are needed that is key. The ability of applications to be
broken down into individual services that can be provided over the Web offers much promise.
Joshi et al. (2004) discuss the benefits of Web services as a non-proprietary and cross-platform
route toward achieving the benefits of distributed computer architecture. Papazoglou and Yang


CREE Deliverable S1D1

(2004) also see benefits in being able to combine individual Web services in order to provide
greater value-added services. However, they warn that the technologies for combining Web
services are currently immature and propose their own framework for managing this. Ongoing
standardisation efforts on the orchestration of Web services are also taking place157. In individual
sectors, Web services are also seen as offering great promise. Bonaventure (2004) foresees a large
impact of Web services on the development of the Grid and e-science activity. The Grid is all
about providing access to applications over the network. Web services thus offer a route to enable
this. In the library field Felstead (2004) concludes that Web services may enable a new approach
to procuring library management systems in the future, as services become available for delivery
individually rather than as a whole package.

12. Conclusions

This technical literature review has sought to provide an overview of the development of a variety
of technologies applicable within the library and portal fields, how they are being used and
possible future directions and trends. CREE is seeking as a project to show how these might be
brought together to enable access to library services and resources through the medium of the Web
and portals. This review has helped greatly in the thinking within the project and hope it will be
of use to others wishing to gather background information in these areas.

13. References

Anonymous (2000), CrossRef goes live. Advanced Technology Libraries 29(7): 1-2

Anonymous (2001), NISO creates OpenURL standard committee. Advanced Technology

Libraries 30(5): 8

Anonymous (2004a), VTLS launches VITAL, VORTEX. Advanced Technology Libraries 33(2):

Anonymous (2004b), CABI adds openURL to databases. Information World Review Issue 204: 2

Anonymous (2005a), Endeavour integrates Voyager ILL with LinkFinderPlus. Advanced

Technology Libraries 34(1): 5

Anonymous (2005b), VTLS announces Virtua Release 45, support for SRU/SRW. Advanced
Technology Libraries 34(3): 8

Anonymous (2005c), VIEWS issues with paper on metasearch. Advanced Technology Libraries
34(3): 9

Anonymous (2005d), NISO Metasearch Initiative to issue paper, draft standard. Advanced
Technology Libraries 34(5): 9-10

Arcollo, A. (2005), RLG Metasearch Survey report.

http://www.rlg.org/en/page.php?Page_ID=20750 [accessed 12/10/05]

Arnold, S. (2004), Data as pure spring water. Information World Review Issue 199: 17-8

Awre, C. (2004), The JISC FAIR Programme: opening up access to institutional assets.
ASSIGNation 21(3): 7-10

For example, http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsbpel

CREE Deliverable S1D1

Barnum, C. et al. (2004), Index versus full-text search: a usability study of user preference and
performance. Technical Communication 51(2): 185-206

Battleson, B, Booth, A. and Weintrop, J. (2001), Usability testing of an academic library Web site:
a case study. Journal of Academic Librarianship 27(3): 188-98

Bekaert, J. et al. (2004), Using MPEG-21 DIP and NISO OpenURL for the dynamic
dissemination of complex digital objects in the Los Alamos National Laboratory Digital Library.
D-Lib Magazine 10(2) (February 2004)
http://www.dlib.org/dlib/february04/bekaert/02bekaert.html [accessed 11/10/05]

Bird, S. and Simons, G. (2003), Building an Open Language Archives Community on the OAI
foundation. Library Hi Tech 21(2): 210-8

Bonaventure, O, Hutchison, D. and Mathy, L. (2004), Network support for Grid computing.
Computer Communications 27(14): 1355-6

Bowman, C.M. et al. (1995), The Harvest information discovery and access system. Computer
Networks and ISDN Systems 28(1/2): 119-25

Brody, T. et al. (2003), The impact of OAI-based search on access to research journal papers.
Serials 16(3): 255-60

Callaghan, D. (2003), Vignette, Plumtree back WSRP. eWeek 20(37): 48

Callicott, B. (2002), Library web site user testing. College and Undergraduate Libraries 9(1): 1-

Callicott, B., Vaughn, D. (2003), Broccoli librarianship and Google-bred patrons, or what’s wrong
with usability testing? College and Undergraduate Libraries 10(2): 1-18

Campbell, D. (2004), How the use of standards is transforming Australian digital libraries.
Ariadne Issue 41 (October 2004) http://www.ariadne.ac.uk/issue41/campbell/ [accessed 12/10/05]

Caplan, P. and Arms, W.Y. (1999), Reference linking for journal articles. D-Lib Magazine 5(7/8)
(July/August 1999) http://www.dlib.org/dlib/july99/caplan/07caplan.html [accessed 11/10/05]

Chau, M. et al. (2003), Testing a cancer meta spider. International Journal of Human-Computer
Studies 59(5): 755-76

Chen, H. et al. (2001), MetaSpider: meta-searching and categorization on the Web. Journal of the
American Society for Information Science and Technology 52(13): 1134-47

Chen, S.Y., Dimakopoulos, D. and Magoulas, G.D. (2005), A flexible interface design for Web
directories to accommodate different cognitive styles. Journal of the American Society for
Information Science and Technology 56(1): 70-83

Chen, X. (2004), Assessment of full-text sources used by serials management systems, OpenURL
link resolvers, and imported e-journal MARC records. Online Information Review 28(6): 428-34

Chillingworth, M. (2005), Google gets OpenURL links. Information World Review Issue 214: 8

Christenson, H. and Tennant, R. (2005), Integrating information resources: principles,

technologies, and approaches. Report in partial fulfilment of National Science Foundation Award

CREE Deliverable S1D1

#0333710. http://www.cdlib.org/inside/projects/metasearch/nsdl/nsdl_report2.pdf [accessed


Clark, J. (2001), Subject portals. Ariadne Issue 29 (October 2001)

http://www.ariadne.ac.uk/issue29/clark/ [accessed 11/10/05]

Cockrell, B.J. and Jayne, E.A. (2002), How do I find an article? Insights from a web usability
study. Journal of Academic Librarianship 28(3): 122-32

Conhaim, W.W. (2004), Wake up to the Blogosphere. Information Today 21(8): 39

Cook, A. and Dowling, T. (2003), Linking from index to primary source: the OhioLINK model.
Journal of Academic Librarianship 29(5): 320-6

Copeland, S. and Penman, A. (2003), E-theses: recent developments and the JISC ‘FAIR’
programme. SCONUL Newsletter Issue 28 (Spring 2003): 39-42
http://www.sconul.ac.uk/pubs_stats/newsletter/28/ [accessed 05/10/05]

Cousins, S. (1999), Virtual OPACs versus union database: two models of union catalogue
provision. Electronic Library 17(2): 97-103

Dahl, M. (2004), Building an OpenURL resolver in your own workshop. Computers in Libraries
24(2): 6-8, 53-4, 56

Dawson, A. (2004), Creating metadata that work for digital libraries and Google. Library Review
53(7): 347-50

Dempsey, L., Russell, R. and Murray, R. (1998), The emergence of distributed library services: a
European perspective. Journal of the American Society for Information Science 49(10): 942-51

Dempsey, L. (2004), Pick up a portal. Library + Information Update 3(10): 36-7

[accessed 12/10/05]

Duke, M. (2003), Delivering OAI records as RSS: an IMesh toolkit module for facilitating
resource sharing. Ariadne Issue 37 (October 2003) http://www.ariadne.ac.uk/issue37/duke/
[accessed 11/10/05]

Dunsire, G. (2003), Joined up indexes: interoperability issues in Z39.50 networks. International

Cataloguing and Bibliographic Control 32(3): 47-9

East, J.W. (2003), Z39.50 and personal bibliographic software. Library Hi Tech 21(1): 34-43

Feeney, M. and Newby, J. (2005), Model for presenting resources in Scholar’s Portal. Portal:
Libraries and the Academy 5(2): 195-211

Felstead, A. (2004), The library systems market: a digest of current literature. Program 38(2): 88-

Feng, Z. et al. (2004), Research on integrating service in Grid Portal. Lecture Notes in Computer
Science Edition 3032: 821-4

Ferguson, C.L. and Grogg J.E. (2004), OpenURL link resolvers. Computers in Libraries 24(9):

CREE Deliverable S1D1

Fox, E.A., and Suleman, H. (2003), Leveraging OAI harvesting to disseminate theses. Library Hi
Tech 21(2): 219-27

Gadd, E., Oppenheim, C. and Proberts, S. (2004), RoMEO studies 6: rights metadata for open
archiving. Program 38(1): 5-14

Garman, N. (1999), Meta search engines. Online 23(3):74-8

Gatenby, J. (2002), Aiming at quality and coverage combined: blending physical and virtual union
catalogues. Online Information Review 26(5): 326-35

Gilbert, A. (1999), Oracle to offer customer portals. InformationWeek Issue 754: 28

Gilby, J. and Dunsire, G. (2004) CC-interop Final Report

http://ccinterop.cdlr.strath.ac.uk/documents/CCiFinalReportVersion1.pdf [accessed 05/08/05]

Grant, B., Lemon, S.C. and Modersitzki, M. (2003), Next generation portals. Computer
Technology Review 23(4): 30

Griffiths, J.R. and Brophy, P. (2002), Student searching behaviour in the JISC Information
Environment. Ariadne Issue 33 (October 2002) http://www.ariadne.ac.uk/issue33/edner/intro.html
[accessed 11/10/05]

Hammer, S. and Andresen, L. (2002), Issues in Z39.50 parallel searching

arching.htm [accessed 05/08/05]

Hammond, T., Hannay, T, and Lund, B. (2004), The role of RSS in science publishing:
syndication and annotation on the Web. D-Lib Magazine 10(12) (December 2004)
http://www.dlib.org/dlib/december04/hammond/12hammond.html [accessed 11/10/05]

Hitchcock, S. et al. (1997), Citation linking: improving access to online journals. Proceedings of
the 2nd ACM International Conference on Digital Libraries, edited by Robert B. Allen and Edie
Rasmussen (New York, USA: Association for Computing Machinery), pp. 115-22, web version
available at http://journals.ecs.soton.ac.uk/acmdl97.htm [accessed 11/10/05]

Jackson, A.E. (1988), After downloading: enhancing the value of your data. Aslib Proceedings

Jacobsen, C. (2000), New horizons: institutional information portals. EDUCAUSE Review 35(4):
58-9 http://www.educause.edu/apps/er/erm00/articles004/horizons.pdf [accessed 12/10/05]

Jacso, P. (2002), Cross-searching electronic journal archives. Information Today 19(6): 34-5

Jansen, B.J., Pedersen, J. and Spink, A. (2005), A temporal comparison of AltaVista web
searching. Journal of the American Society for Information Science and Technology 56(6): 559-

Johnson, F.C., Griffiths, J.R. and Hartley, R.J. (2001), DEvISE. A framework for the evaluation
of Internet search engines. Library and Information Commission Research Report 100
http://www.cerlim.ac.uk/projects/devise/devise-report.pdf [accessed 12/10/05]

Joshi, P., Phippen, A.D. and Singh, H. (2004), Web services: measuring practitioner attitude.
Internet Research: Electronic Networking Applications and Policy 14(5): 366-71

CREE Deliverable S1D1

Karpinski, R. (1999), Portal opens for customer management. InternetWeek Issue 771: 15

Kirkpatrick, C.H. (2003), Getting two for the price of one: accessibility and usability. Computers
in Libraries 23(1): 26-9

LeVan, R., Sanderson, R. and Young, J. (2005), SRW/U with OAI: expected and unexpected
synergies. D-Lib Magazine 11(2) (February 2005)
http://www.dlib.org/dlib/february05/sanderson/02sanderson.html [accessed 05/08/05]

Liu, X. et al. (2001), Arc: an OAI service provider for Digital Library Federation. D-Lib Magazine
7(4) (April 2001) http://www.dlib.org/dlib/april01/liu/04liu.html [accessed 01/10/05]

Looney, M. and Lyman, P. (2000), Portals in Higher Education. EDUCAUSE Review 35(4): 28-
36 http://www.educause.edu/apps/er/erm00/articles004/looney.pdf [accessed 12/10/05]

Lu, S., Dong, M. and Fotouhi, F. (2002), The Semantic Web: opportunities and challenges for
next-generation Web applications. Information Research 7(4) http://informationr.net/ir/7-
4/paper134.html [accessed 11/10/05]

Lunau, C. (2000), The Bath Profile: an international Z39.50 specification for library applications
and resource discovery. D-Lib Magazine 6(10) (October 2000)
http://www.dlib.org/dlib/october00/10inbrief.html#LUNAU [accessed 05/08/05]

Luther, J. (2003), Trumping Google? Metasearching’s promise. Library Journal 128(16): 36-9

Lynch, C. (1997), The Z39.50 information retrieval standard. Part I: a strategic view of its past,
present and future. D-Lib Magazine 3(4) (April 1997)
http://www.dlib.org/dlib/april97/04lynch.html [accessed 05/08/05]

Macgregor, G. (2005), Z39.50 broadcast searching and Z-server response times: perspectives from
CC-interop. Online Information Review 29(1): 90-106, e-print available at
http://eprints.rclis.org/archive/00004105/ [accessed 05/08/05]

McKeown, R. (2003), Accessing the virtual museum: bringing museum information into
cyberspace. New Review of Information Networking 9:40-53
=parent&backto=issue,4,12;journal,3,3;linkingpublicationresults,1:110339,1 [accessed 11/10/05]

Maly, K., Zubair, M. and Liu, X. (2001), Kepler: an OAI data/service provider for the individual.
D-Lib Magazine 7(4) (April 2001) http://www.dlib.org/dlib/april01/maly/04maly.html [accessed

Martin, R. (2003), ePrints UK: developing a national e-prints archive. Ariadne Issue 35 (April
2003) http://www.ariadne.ac.uk/issue35/martin/ [accessed 05/10/05]

Mat-Hassan, M. and Levene, M. (2001), Can navigational assistance improve search experience?
A user study. First Monday 6(9) (September 2001)
http://www.firstmonday.org/issues/issue6_9/mat/index.html [accessed 12/10/05]

Medeiros, N. (2002), Introducing Scirus: Elsevier’s shot at the title. OCLC Systems and Services
18(3): 121-4

Miller, P. (1999), Z39.50 for all. Ariadne Issue 21 (September 1999)

http://www.ariadne.ac.uk/issue21/z3950/intro.html [accessed 05/08/05]

CREE Deliverable S1D1

Mischo, W.H. (2001), Library portals, simultaneous search, and full-text linking technologies.
Science and Technology Libraries 20(2/3): 133-47

Moen, W.E. and Murray, K.R. (2004), The deep Web: resource discovery in the Library of Texas.
Texas Library Journal 80(1):16-9, 22-4

Moffat, M. (2003), RSS: a primer for publishers and content providers. New Review of
Information Networking 9: 123-44

Morgan, E.L. (2004), An introduction to the Search/Retrieve URL Service (SRU). Ariadne Issue
40 (July 2004) http://www.ariadne.ac.uk/issue40/morgan/ [accessed 05/08/05]

Moore, C. (2002a), Geared for portal patrol. InfoWorld 24(22): 15

Moore, C. (2002b), Portal players forge processes. InfoWorld 24(47): 1

Moore, C. (2003), Portal standards set for takeoff. InfoWorld 25(30): 20

Needleman, M. (2000), Z39.50: a review, analysis and some thoughts on the future. Library Hi
Tech 18(2): 158-65

Nelson, M.L. (2001), Better interoperability through the Open Archives Initiative. New Review of
Information Networking 7:133-45

Nicolaides, F. (2003), The Bath Profile four years on: what’s being done in the UK? Ariadne Issue
36 (July 2003) http://www.ariadne.ac.uk/issue36/bath-profile-rpt/ [accessed 05/08/05]

Novotny, J., Russell, M and Wehrtens, O. (2004), GridSphere: a portal framework for building
collaborations. Concurrency and Computation: Practice and Experience 16(5): 503-13
http://www.gridsphere.org/gridsphere/wp-4/Documents/RioBabyRio/gridsphere.pdf [accessed

Nowicki, S. (2003), Student vs. Search engine: undergraduates rank results for relevance. Portal:
Libraries and the Academy 3(3): 503-15

Nyborg, F. (2002), Usability test at DEF-fagportaler. Usability test on DEF subject portals. DF
Revy 25(1): 7-8

Olsen, F. (2002), The power of portals. The Chronicle of Higher Education 9th August
http://chronicle.com/free/v48/i48/48a03201.htm [accessed 12/10/05]

Oppenheim, C, et al. (2000), The evaluation of WWW search engines. Journal of Documentation
56(2): 190-211

Owen-McGee, D. and Swanson, L. (2003), Project EnCoRe and the development of electronic
course pack creation at the University of Derby. ASSIGNation 20(3): 38-42

Ozmutlu, H.C., Ozmutlu, S. and Spink, A. (2003), Are people asking questions of general Web
search engines? Online Information Review 27(6): 396-406

Palmer, D. and Robinson, B. (2001), Agora: the hybrid library from a user’s perspective. Ariadne
Issue 26 (January 2001) http://www.ariadne.ac.uk/issue26/case-studies [accessed 05/08/05]

Papazoglou, M.P. and Yang, J. (2004), Service components for managing the life-cycle of service
compositions. Information Systems 29(2): 97-125

CREE Deliverable S1D1

Payette, S. and Rieger, O.Y. (1997), Z39.50: the user’s perspective. D-Lib Magazine 3(4) (April
1997) http://www.dlib.org/dlib/april97/cornell/04payette.html [accessed 05/08/05]

Pearce, L. (2003), “Apart from the weather, I think it’s a good idea”: stakeholder requirements for
institutional portals. Ariadne Issue 35 (April 2003) http://www.ariadne.ac.uk/issue35/pearce/
[accessed 12/10/05]

Pentz, E. (2001), Reference linking with CrossRef. Interlending and Document Supply 29(1): 20-

Petersen, D.M. (2004), Portlet standards accelerate web portal development. Business
Communications Review 34(7): 57

Pinfield, S. (2003), Open archives and UK institutions: an overview. D-Lib Magazine 9(3) (March
2003) http://www.dlib.org/dlib/march03/pinfield/03pinfield.html [accessed 05/10/05]

Pinto, F. and Fraser, M. (2003), Access management, the key to a portal. Ariadne Issue 35 (April
2003) http://www.ariadne.ac.uk/issue35/SPP/ [accessed 12/10/05]

Pope, N.F. (1998), Digital libraries: future potentials and challenges. Library Hi Tech 16(3-4):

Poulter, A. (1997), The design of World Wide Web search engines: a critical review. Program
31(2): 131-45

Powell, A. and Apps, A. (2001), Encoding OpenURLs in Dublin Core metadata. Ariadne Issue 27
(March 2001) http://www.ariadne.ac.uk/issue27/metadata/ [accessed 11/10/05]

Powell, A. and Lyon, L. (2002), The JISC Information Environment and Web services. Ariadne
Issue 31 (April 2002) http://www.ariadne.ac.uk/issue31/information-environments/ [accessed

Prowse, S.W. (1999), A review of interlending in the UK in 1998. Interlending and Document
Supply 27(2): 80-8

Ramly, R. (2002), Navigating the WWW: a Web usability study. Kekal Abadi 21: 1-5

Randall, S. (2004), Learning systems and us. Library Journal 129(16):34-5

Repman, J. and Carlson, R.D. (1999), Surviving the storm: using metasearch engines effectively.
Computer in Libraries 19(5): 50-5

Richardson, S. and Powell, P. (2003), Exposing information resources for e-learning – harvesting
and searching IMS metadata using the OAI Protocol for Metadata Harvesting and Z39.50. Ariadne
Issue 34 (January 2003) http://www.ariadne.ac.uk/issue34/powell/ [accessed 05/08/05]

Rusch-Feja, D. (2002), The Open Archives Initiative and the OAI Protocol for Metadata
Harvesting: rapidly forming a new tier in the scholarly communication infrastructure. Learned
Publishing 15(3):179-86

Sadeh, T. (2004), The challenge of metasearching. New Library World 105(3/4): 104-12

Sanborn, S. (2000), Portal proliferation benefits e-business. InfoWorld 22(17): 30

CREE Deliverable S1D1

Savvas, A. (2003), Suppliers unite to create portal tools website. Computer Weekly 18th
November: 4

Schaeck, T. and Hepper, S. (2002), Portal standards for Web services. Network World 19(35): 33

Schneider, K.G. (1996), Z39.50: beyond your wildest dreams. American Libraries 27(6): 86-9

Shneiderman, B. (1997), Designing information-abundant web sites: issues and recommendations.

International Journal of Human-Computer Studies 47(1): 5-29

Shreeves, S.L. and Kirkham, C.M. (2004), Experiences of educators using a portal of aggregated
metadata. Journal of Digital Information 5(3) http://jodi.tamu.edu/Articles/v05/i03/Shreeves/
[accessed 05/10/05]

Sipos, G. et al. (2005), Workflow-oriented collaborative Grid portals. Lecture Notes in Computer
Science Edition 3470: 434-43

Soffer, S.B. (1997), Z39.50 clients, a comparative study. Library Technology Reports 33(5): 515-

Spence, B. (2004), Feed(ing) frenzy. Information Today 21(8): 16

%20SPENCES%20#top [accessed 11/10/05]

Stanley, T. (1995), Searching the World Wide Web with Lycos and InfoSeek. New Review of
Information Networking 1: 191-202

Stevenson, A. (2002), Report on implementation of SRW: Technical Note ARTISTE 58

http://www.it-innovation.soton.ac.uk/ftpdata/artiste/srw/T58_Report_on%20SRW.pdf [accessed

Storey, T. (2004a), Moving Z39.50 to the Web. OCLC Newsletter Issue 263 (Jan/Feb/Mar 2004):
8-9 http://www.oclc.org/news/publications/newsletters/oclc/2004/263/srw.html [accessed

Storey, T. (2004b), Personalizing the Web experience. OCLC Newsletter Issue 266 (Oct/Nov/Dec
2004): 7-10 http://www.oclc.org/news/publications/newsletters/oclc/2004/266/personalize.html
[accessed 11/10/05]

Stubley, P. (2000), What have the clumps ever done for us? Ariadne Issue 23 (March 2000)
http://www.ariadne.ac.uk/issue23/stubley/ [accessed 05/08/05]

Stubley, P., Bull, R. and Kidd, T. (2001), Feasibility study for a National Union Catalogue
http://web.archive.org/web/20040924113213/http://www.uknuc.shef.ac.uk/ [accessed 05/10/05]

Tenopir, C. (2001), The power of citation searching. Library Journal 126(18):39-40

Toth, B. (2004), An integrated library for the NHS. Library + Information Update 3(10): 28-9

Van de Sompel, H. and Hochstenbach, P. (1999a), Reference linking in a hybrid library

environment. Part 1: Frameworks for linking. D-Lib Magazine 5(4) (April 1999)
http://www.dlib.org/dlib/april99/van_de_sompel/04van_de_sompel-pt1.html [accessed 11/10/05]

CREE Deliverable S1D1

Van de Sompel, H. and Hochstenbach, P. (1999b), Reference linking in a hybrid library

environment. Part 2: SFX, a generic linking solution. D-Lib Magazine 5(4) (April 1999)
http://www.dlib.org/dlib/april99/van_de_sompel/04van_de_sompel-pt2.html [accessed 11/10/05]

Van de Sompel, H. and Hochstenbach, P. (1999c), Reference linking in a hybrid library

environment. Part 3: Generalizing the SFX solution in the ‘SFX@Ghent and SFX@LANL’
experiment. D-Lib Magazine 5(10) (October 1999)
http://www.dlib.org/dlib/october99/van_de_sompel/10van_de_sompel.html [accessed 11/10/05]

Van de Sompel, H. and Lagoze, C. (2000), The Santa Fe Convention of the Open Archives
Initiative. D-Lib Magazine 6(2) (February 2000)
http://www.dlib.org/dlib/february00/vandesompel-oai/02vandesompel-oai.html [accessed

Van de Sompel, H. et al. (2000), The UPS prototype: an experimental end-user service across e-
print archives, D-Lib Magazine 6(2) (February 2000)
http://www.dlib.org/dlib/february00/vandesompel-ups/02vandesompel-ups.html [accessed

Van de Sompel, H. and Beit-Arie, O. (2001a), Open linking in the scholarly information
environment using the OpenURL Framework. D-Lib Magazine 7(3) (March 2001)
http://www.dlib.org/dlib/march01/vandesompel/03vandesompel.html [accessed 11/10/05]

Van de Sompel, H. and Beit-Arie, O. (2001b), Generalizing the OpenURL Framework beyond
references to scholarly works. The Bison-Futé model. D-Lib Magazine 7(7/8) (July/August 2001)
http://www.dlib.org/dlib/july01/vandesompel/07vandesompel.html [accessed 11/10/05]

Van de Sompel, H. and Beit-Arie, O. (2001c), Open linking in the scholarly information
environment using the OpenURL Framework. New Review of Information Networking 7: 59-76

Van de Sompel, H. et al. (2004), Resource harvesting within the OAI-PMH framework. D-Lib
Magazine 10(12) (December 2004)
http://www.dlib.org/dlib/december04/vandesompel/12vandesompel.html [accessed 05/10/05]

Vaughan, J. (1999), Considerations in the choice of an Internet search tool. Library Hi Tech
17(1): 89-106

Walker, J. (2002), CrossRef and SFX: complementary linking services for libraries. New Library
World 103(1174): 83-9

Webster, P. (2004), Breaking down information silos: integrating online information. Online
28(6): 30-4

Whiting, R. (2000), Vendors add power to portals. InformationWeek Issue 778: 83

Woldering, A. (2004), The European Library: integrated access to the national libraries of Europe.
Ariadne Issue 38 (January 2004) http://www.ariadne.ac.uk/issue38/woldering/ [accessed

Wusteman, J. (2004), RSS: the latest feed. Library Hi Tech 22(4): 404-13

Xie, H. (2004), Online IR system evaluation: online databases versus Web search engines. Online
Information Review 28(3): 211-9

CREE Deliverable S1D1

Young, M. and Yu, H. (2004), The impact of Web search engines on subject searching in OPAC.
Information Technology and Libraries 23(4): 168-80

Zhou, J. (2003), A history of Web portals and their development in libraries. Information
Technology and Libraries 22(3): 119-28

Zeichick, A. (2004), Portlet standard predicament. InfoWorld 26(18): 47