EINS-Web: User Interface Evaluation in Digital Libraries
by Silvana Mangiaracina and Pier Giorgio Marchetti
The construction of new user interface paradigms, or metaphors,
in the interaction of humans with huge collections of information
is currently a hot topic in both HCI and Digital Libraries (DL).
We describe the EINS-Web user interface, designed and evaluated
in the frame of the BRIDGE and CIME projects, co-financed by the
European Union and coordinated by the European Space Agency and
the EINS consortium (European Information Network Services). The
Library of the Italian National Research Council (CNR) in Bologna
was selected as the test site for the evaluation of EINS-Web.
The EINS-Web interface enables users to access distributed collections
of bibliographic and textual databases, and provides a seamless
interaction with the World Wide Web. The interface design has
been guided by a heuristic evaluation, using a spiral design approach.
This methodology was adopted as it is largely software-independent
and proactive. It thus makes it relatively easy to incorporate
suggested adaptations during the design and testing process.
The Design-Evaluation Spiral Process
The Library of the Italian National Research Council (CNR) in
Bologna was selected as the test site for the evaluation of EINS-Web.
The evaluation team consisted of a number of researchers from
the CNR campus with their information problems in a varied set
of disciplines, such as chemistry, material science, electronics,
physics, geology, environment, etc., plus a mixed group of evaluators
consisting of information specialists and user interface experts.
In the construction of the EINS-Web interface, we reused the design
efforts employed in the development of the previous version of
the interface (BRAQUE PC where BRAQUE = BRowse And QUEry), developed
for the Windows environment.
A heuristic evaluation method then assessed user satisfaction,
ease of learning, ease of use, error prevention and efficiency
of the interface and was used as a feedback tool to drive a spiral
design process. The evaluation began in April 1996, when BRAQUE
1.2 was released. Problems identified by our set of users were
analysed against the above heuristic criteria. A number of issues
were signalled, mainly of an aesthetic nature. Some were serious,
heavily influencing the user interaction. Where possible, suitable
solutions were proposed. As the results of the BRAQUE PC evaluation
were greater than expected, it was decided to assess and improve
the methodology in the design of the next generation Web interface.
This new version was called EINS-Web, as the international EINS
consortium had decided to adopt and test this interface.
All the information collected during the BRAQUE evaluation sessions,
such as evaluators’ opinions and implementers’ replies, was used
as input to drive the design process for the EINS-Web interface.
Two different evaluation sessions were conducted: one involving
a mixed group of experts (evaluators) and users; one with experts
only. A new heuristic evaluation form was used by the evaluators:
for each ‘usability’ problem, a rating value was assigned - we
agreed to assign values from 1 (the interface does not take this
problem into consideration at all) to 5 (this problem has been
completely solved).
The evaluators were requested to identify potential usability
problems and to link each problem to the specific heuristic it
violated. Multiple heuristics could be linked to any given violation.
In the experts-only evaluation session, the same evaluation -
covering the same information problems - was run by four different
groups of evaluators.
Assessment of the evaluation methodology and lessons learned
The evaluation was useful in detecting certain design issues that
had been neglected to some extent. In particular two problem areas
were identified. The first related to feedback and visibility
of system status. The second regarded user background knowledge
and the user conceptual model. When the results of the four different
groups of evaluators were compared, it could be seen that they
had used all the heuristics present in the evaluation form, either
in positive matches (ie a score >= 3), or in negative matches
(ie one or more problems recognized were linked to the heuristic).
This result shows that all the pre-selected heuristics were relevant.
We noticed slight differences in the evaluation results, depending
on the presence of real users or not. It appeared in fact that
in a ‘pure’ heuristic evaluation session (only interface experts,
no real users) it was possible to detect problems relating more
to the interactive behaviour of the interface, such as users’
behaviour problems, conceptual user model, aesthetic design. The
evaluation with real users made it possible to examine concrete,
real-world information seeking interaction problems. Matching
between interface design and user expectation is difficult when
information space is dispersed over very large collections: expert
users want to increase interface functionality to achieve their
goals, whilst non-expert users want to reduce interface functionality
in favour of intuitive and simple features.
As a preliminary conclusion we feel that the interaction among
evaluators, implementers and designers has contributed significantly
to the success of the spiral design methodology, and is very necessary
to cope with the requirements of designing interfaces targeted
at the rapidly evolving Internet world.
Please contact:
Silvana Mangiaracina - CNR, Bologna
Tel: +39 051 639 8026
E-mail: mangiaracina@area.bo.cnr.it
