Insight Based Method for Evaluating VR Viz

Spring 2023, Dave Song 

<Key Ideas>

• current visualization tool for bioinformatics research → mix feedback

• The purpose of the visualization of data is to generate insights and discovery.

  • thus, effective data visualization → ability to generate unpredicted new insights.

  • visualization evaluation → done through accuracy or performance measuring on given tasks

  • should focus more on “recognition and quantification of insights gained from actual exploratory use of the visualizations”

<related topics>

• Metrics, Heuristics, and models

  • evaluation

    • inspection of user interfaces by experts with heuristics

<Study>

• insight definition should be quantifiable and reproducible for the sake of the study

  • insight characteristics

    • Definition: “An individual observation about the data by the participant, a unit of discovery”. Standard think-aloud protocol and it is feasible to notice when the user makes an insight

      • characteristics of insights

• Observation: data observation

• Time: duration taken to reach the insight

• Domain value: significance of the insight. On a scale of 1 to 5. 1-2 points mean trivial observations, intermediate values (recognizing particular process) earn 3. An insight that can spark, deny, or support hypotheses earn 4 or 5.

• Directed vs. unexpected

• Correctness: correctness reviewed by experts

• Breadth vs. Depth:

• category: four categories

1. overview

2. patterns

3. groups

4. details

→ in the context of gene and bioinformatics data visualization

Key Takeaways from the research regarding collaborative evaluation

• When targeting scientific data visualization, insight can be a measure of vr system’s effectiveness in presenting the data in a collaborative environment.

• Different types of insights and quantifiable definitions of insight can be useful when evaluating collaborative vr user experience.

Source: https://ieeexplore.ieee.org/document/1432690