Sunday, All Day
Metrics for the Evaluation of Visual Analytics

Organizers:	Jean Scholtz, Pacific Northwest National Laboratory
	Georges Grinstein, University of Massachusetts Lowell
	Catherine Plaisant, University of Maryland

The field of visual analytics is now recognized as a research area in many universities and organizations. As new fields develop ways of assessing progress in those fields also emerge. In the field of visual analytics, we are fortunate in that we already have lessons learned about evaluating visualizations. Unfortunately, these lessons still point out that this is a difficult problem. Visual analytics compounds this problem by adding more dimensions; not only are we concerned with some measure of the visualizations, but we are concerned with evaluating the impact these visualizations have in helping analysts in their work.

User-centered evaluations are vital in visual analytics as they contribute greatly to adoption of research software. The issues we face in developing user-centered evaluations for visual analytics are selecting:

• The task: the tradeoff is between simple tasks that are easily evaluated and developing a more realistic task that consumes more time and is much less straightforward to evaluate.
• The corresponding dataset: the same issues as above plus the issues of developing a publicly releasable dataset that resembles a realistic dataset
• The system and environment: how much does the system or environment play a role in the utility or success of the task.
• The participants: using senior analysts or junior analysts and ensuring that analysts are open to new technology
• Training: how much training to provide to analysts or whether analysts should be paired with technologists to operate the software
• The metrics: what combination of quantitative and qualitative measures will be accepted? How can we ensure that qualitative measures meet are collected with some rigor? How can we measure insights that were derived from the visualization and interactions with the visualization? This is especially problematic as not all analysts approach problems in the same fashion. Most importantly, what measures are most helpful to the analytic community and to the research community?

Monday, All Day
Knowledge-Assisted Visualization

Main Organization Team:	Gerik Scheuermann, Leipzig, Germany (Workshop chair)
	Kwan-Liu Ma, University of California, Davis (Program co-chair)
	Robert van Liere, CWI/Eindhoven University of Technology(Program co-chair)
	Ming Chen, Swansea University (Steering committee co-chair)
	Hans Hagen, Technische Universität Kaiserslautern (Steering committee co-chair)
Contact Person:	Min Chen, email: m.chen@swansea.ac.uk

Over the years, a number of researchers have introduced various techniques for visualizing complex structures in scientific data by relying on information abstracted from the data. Such information typically includes:

• Topological relationships between features and events in the data (e.g., a contour tree of a volume dataset, vector field topology, a tracking graph for a time-varying volume dataset).
• Statistical indicators of the data, for example, histogram, correlation, importance and certainty (e.g., entropy based viewpoint selection and optimized lighting placement).
• Abstract geometric and temporal representations, such as skeletons, features, and events.
• Information about a visualization process, such as user interaction data.
• Information about the visualization results (e.g., color histogram, level of cluttering).
• Information about users perception.

Such work provides an intrinsic interface between the scientific visualization and information visualization communities. With the increasing size and complexity of data, the use of information to aid scientific visualization will inevitably become a necessity rather than an option. Furthermore, abstraction information related to different datasets and users is much more suitable for being collected together (i.e., knowledge base), combined together to form global knowledge (i.e., knowledge fusion), and used as a basis for intelligent reasoning (i.e., new knowledge creation). It is hence highly desirable for both the scientific and information visualization communities to work together, and to explore this new knowledge-based frontier by building on and expanding the existing information-guided visualization techniques. It has also been suggested that one should consider the connection with knowledge-based medical diagnosis as it also involves information and knowledge acquisition and processing.