Sunday, All Day
Metrics for the Evaluation of Visual Analytics
||Jean Scholtz, Pacific Northwest National Laboratory|
|Georges Grinstein, University of Massachusetts Lowell|
|Catherine Plaisant, University of Maryland|
The field of visual analytics is now recognized as a research area in
many universities and organizations. As new fields develop ways of
assessing progress in those fields also emerge. In the field of visual
analytics, we are fortunate in that we already have lessons learned
about evaluating visualizations. Unfortunately, these lessons still
point out that this is a difficult problem. Visual analytics compounds
this problem by adding more dimensions; not only are we concerned with
some measure of the visualizations, but we are concerned with evaluating
the impact these visualizations have in helping analysts in their work.
User-centered evaluations are vital in visual analytics as they
contribute greatly to adoption of research software. The issues we face
in developing user-centered evaluations for visual analytics are selecting:
- The task: the tradeoff is between simple tasks that are easily
evaluated and developing a more realistic task that consumes more time
and is much less straightforward to evaluate.
- The corresponding dataset: the same issues as above plus the issues
of developing a publicly releasable dataset that resembles a realistic
- The system and environment: how much does the system or environment
play a role in the utility or success of the task.
- The participants: using senior analysts or junior analysts and
ensuring that analysts are open to new technology
- Training: how much training to provide to analysts or whether
analysts should be paired with technologists to operate the software
- The metrics: what combination of quantitative and qualitative
measures will be accepted? How can we ensure that qualitative measures
meet are collected with some rigor? How can we measure insights that
were derived from the visualization and interactions with the
visualization? This is especially problematic as not all analysts
approach problems in the same fashion. Most importantly, what measures
are most helpful to the analytic community and to the research community?
Monday, All Day
|Main Organization Team:
||Gerik Scheuermann, Leipzig, Germany (Workshop chair)|
|Kwan-Liu Ma, University of California, Davis (Program co-chair)|
|Robert van Liere, CWI/Eindhoven University of Technology(Program co-chair)|
|Ming Chen, Swansea University (Steering committee co-chair)|
|Hans Hagen, Technische Universität
Kaiserslautern (Steering committee co-chair)|
|Contact Person:|| Min Chen, email: email@example.com
Over the years, a number of researchers
have introduced various
techniques for visualizing complex structures in scientific data by
relying on information abstracted from the
data. Such information typically includes:
- Topological relationships between features and events in the data
(e.g., a contour tree of a volume
dataset, vector field topology, a tracking graph for a time-varying
Statistical indicators of the data, for example, histogram,
correlation, importance and certainty (e.g.,
entropy based viewpoint selection and optimized lighting placement).
Abstract geometric and temporal representations, such as skeletons,
features, and events.
Information about a visualization process, such as user interaction data.
Information about the visualization results (e.g., color histogram,
level of cluttering).
Information about users perception.
Such work provides an intrinsic interface between the scientific
visualization and information visualization
communities. With the increasing size and complexity of data, the use of
information to aid scientific
visualization will inevitably become a necessity rather than an option.
Furthermore, abstraction information
related to different datasets and users is much more suitable for being
collected together (i.e., knowledge
base), combined together to form global knowledge (i.e., knowledge
fusion), and used as a basis for
intelligent reasoning (i.e., new knowledge creation). It is hence highly
desirable for both the scientific and
information visualization communities to work together, and to explore
this new knowledge-based frontier by
building on and expanding the existing information-guided visualization
techniques. It has also been
suggested that one should consider the connection with knowledge-based
medical diagnosis as it also
involves information and knowledge acquisition and processing.