Embedded Files
written by Justin Park
Connected is a Spring 2026 project from Brown University's VR Visualization course. It builds a passthrough AR brain tractography viewer for Meta Quest 3 and evaluates it against a web-based 2D viewer in a controlled user study. The central question is whether AR spatial exploration improves connectivity identification accuracy and comprehension compared to a standard flat-screen tool — a question with direct relevance as AR headsets begin entering clinical neuroscience and surgical planning workflows.
Research Question
Does viewing white matter tractography data as an AR spatial overlay in physical space improve anatomical connectivity identification accuracy compared to a standard 2D web-based viewer?
Hypotheses going into the study:
H1: AR participants will identify target tract connectivity with higher accuracy than 2D participants.
H2: AR participants will report greater confidence in spatial orientation of tracts relative to brain anatomy.
H3: AR participants will prefer the modality for exploratory tasks even if 2D is faster for targeted identification.
System Description
The AR condition was built in Unity 6.3 LTS using the Meta Mixed Reality template for Meta Quest 3. White matter tract data was sourced from DSI Studio's built-in HCP tractography atlas, exported as .tt.gz files, decoded using a custom Python parser (nibabel cannot read DSI Studio's proprietary TinyTrack format), normalized to a 0–1 coordinate space, and exported as a JSON file containing 2,928 streamlines across 8 named tract systems. Directional color coding was applied per point — red for left-right fibers, green for front-back, blue for up-down — matching DSI Studio's standard convention.
In the AR headset, the brain floats in physical space anchored to a fixed world-space position. Users scale the brain using the right controller thumbstick (from palm size up to room scale), walk around the volume freely using Quest locomotion, and isolate individual tract systems using controller buttons (A for next tract, X for previous, B for show all). A screen-space UI label shows the currently active tract name.
The 2D condition was built as a standalone HTML file using Three.js, loaded from the same tractography JSON. Participants rotated the brain with mouse drag, zoomed with scroll wheel, and isolated tracts via on-screen buttons. The same directional color scheme was applied. The web viewer required no installation and ran on any laptop via a shared URL.
Study Design
8 participants completed a within-subjects counterbalanced study. Half began with the AR condition and then completed the 2D condition; the other half did the reverse. Each participant completed both conditions in a single session during the class activity on April 28, 2026.
Participants were given a printed reference sheet containing a labeled brain anatomy diagram showing major regions (frontal lobe, parietal lobe, temporal lobe, occipital lobe, cerebellum, brainstem) and a color key explaining the directional color convention. They were instructed to refer to this sheet freely during both conditions.
Each condition consisted of three connectivity identification tasks, one per tract system. Tasks were presented via Google Form and required participants to select one of four multiple choice answers based on what they observed in the visualization.
Task 1 — Corpus Callosum: "Which two parts of the brain does this tract connect?"
Task 2 — Corticospinal Tract: "Where does this tract begin and end?"
Task 3 — Arcuate Fasciculus: "Where is this tract located in the brain?"
After completing both conditions, participants rated ease of navigation, perceived understanding of tracts, and confidence in answers on a 1–5 scale, and indicated overall modality preference.
Quantitative Results
Overall accuracy was 88% for the 2D condition and 84% for the AR condition across all 8 participants and 4 questions.
Per-question accuracy:
Q1 Corpus Callosum connects — AR 75%, 2D 88%.
Q2 Corpus Callosum location — AR 88%, 2D 100%.
Q3 Corticospinal Tract — AR 75%, 2D 62%.
Q4 Arcuate Fasciculus — AR 100%, 2D 100%.
AR outperformed 2D specifically on Q3 — the Corticospinal Tract task — which asked participants to identify a vertically running tract's endpoints. This is the task most dependent on depth judgment and up-down spatial reasoning, consistent with AR's theoretical advantage for perceiving depth natively rather than inferring it from a flat projection.
Subjective ratings on a 1–5 scale:
Ease of navigation — AR 3.50, 2D 3.75.
Perceived understanding of tracts — AR 3.50, 2D 3.88.
Confidence in answers — AR 3.75, 2D 3.50.
2D rated higher on ease and perceived understanding. AR rated higher on confidence, meaning participants felt more certain about their answers in the AR condition despite slightly lower accuracy scores.
Modality preference among the 8 participants who completed both conditions: 4 preferred AR overall, 3 preferred 2D, 1 had no preference. When asked which condition was better specifically for understanding connectivity, 4 selected 2D, 2 selected AR, and 2 said about the same.
Order Effect Analysis
The most analytically interesting finding was the order effect. Participants who completed the AR condition first scored 88% on AR, then improved to 100% on 2D second — a gain of 12 percentage points. Participants who completed 2D first scored 75% on 2D, then improved to 81% on AR second — a gain of only 6 percentage points.
Both groups improved on their second condition, which is expected due to familiarity with the tasks and data. However, the AR-first group showed a substantially larger improvement when switching to 2D than the 2D-first group showed when switching to AR. This asymmetry suggests that spatial exploration in AR may build stronger mental models of tract geometry that transfer to subsequent flat-screen tasks — even when AR does not directly produce higher accuracy scores on its own.
One participant explicitly noted after completing both conditions that their AR answers felt wrong in retrospect, suggesting the AR exploration had given them spatial intuitions they were then able to apply more precisely in the 2D condition.
Qualitative Results
AR condition: participants described the spatial exploration as genuinely useful for understanding where tracts sit in the brain relative to each other. Being able to scale the brain to room size and walk through the fiber tracts was described as providing depth cues that the flat screen could not replicate. The main complaints were navigation friction — controller input had a learning curve that affected early task performance — and the absence of an in-headset reference sheet, as switching between the headset and a printed diagram was disruptive to task flow.
2D condition: mouse-based rotation was described as smoother and more immediately intuitive than the Quest controller scheme. However, several participants noted that the flat view collapsed spatial information in ways that made depth and hemisphere position judgments harder. The most commonly cited limitation was that 2D felt adequate for recognizing a tract by shape but insufficient for developing a spatial sense of where it sat inside the brain relative to other structures.
Key tension: preference and utility diverged. The majority preferred AR experientially and rated their confidence higher in the AR condition, yet most credited 2D with stronger functional comprehension on the specific connectivity tasks. This divergence — enjoying a tool more versus feeling it helps you learn more — is a finding worth examining in larger studies.
Evaluation Scenarios Addressed
This project addresses five of the seven evaluation scenarios from Shneiderman et al.:
UP (User Performance): accuracy, confidence, and ease ratings measured quantitatively across both conditions.
VDAR (Visual Data Analysis and Reasoning): the order effect analysis examines whether AR supports stronger mental model formation beyond task-level performance scores.
UE (User Experience): subjective ratings and open-ended qualitative feedback captured after each condition.
CTV (Communication through Visualization): the study tasks were designed to be answerable by non-expert participants using only the visualization and a labeled reference sheet, testing whether the tools effectively communicate spatial connectivity information.
UWP (Understanding Work Practices): DSI Studio evaluation documented standard tractography workflows and where 2D tool friction occurs, informing the design of the AR condition.
Limitations
Sample size was 8 participants, which is sufficient for a course-level study but too small for statistical significance testing. Order effects were partially controlled through counterbalancing but were not analyzed with inferential statistics. Navigation friction in the AR condition — particularly the controller learning curve — likely suppressed accuracy scores in ways that do not reflect the visualization's underlying spatial advantage. The absence of an in-headset reference UI required participants to remove the headset to consult the printed diagram, adding overhead that disadvantaged the AR condition. Tasks were limited to three tract systems, which may not generalize to more complex connectivity identification scenarios.
Future Work
A larger counterbalanced study with 20+ participants would allow proper statistical testing of the accuracy and order effect results. In-headset anatomical labels and a floating reference diagram would reduce the headset-removal overhead that hurt the AR condition in this study. A third condition using a fully immersive VR viewer (no passthrough) would disambiguate whether the spatial benefit comes from AR's physical grounding or from 3D spatial navigation generally. Longitudinal testing — measuring whether AR users retain better spatial mental models one week later — would directly test the mental model formation hypothesis suggested by the order effect.
References
Yeh, F.C. et al. (2022). Population-based tractography atlas. Nature Communications 13, 4933.
Human Connectome Project. WU-Minn HCP Consortium. db.humanconnectome.org.
Rheault, F. et al. (2020). Common misconceptions, hidden biases and modern challenges of dMRI tractography. Journal of Neural Engineering.
Basser, P.J. et al. (2000). In vivo fiber tractography using DT-MRI data. Magnetic Resonance in Medicine 44(4), 625–632.
Shneiderman, B. et al. (2006). Strategies for evaluating information visualization tools. BELIV Workshop.
Page updated
Google Sites
Report abuse