Melvin's Journal
Hello and welcome to Melvin's Journal! I'm an Applied Math and Computer Science student at Brown. My fascination for Virtual Reality (VR) was sparked after visiting the Immersive Van Gogh exhibit in Boston, where I saw his paintings come to life in a 360° immersive setting. My interest with data visualization also emerged after examining breast cancer data biases in National Cancer Institute histology datasets in my Computer Vision class.
Together, this is my first time exploring Virtual Reality software and technologies for immersive data visualization. Join me as I delve into the exciting world of immersive and collaborative VR data visualization!
before after
---- ----
1 | 4 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities;
1 | 3 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research;
1 | 5 | Goal 3: execute tool evaluation strategies;
0 | 3 | Goal 4: build visualization software packages;
1 | 4 | Goal 5: comparatively analyze software tools based on evaluation;
1 | 5 | Goal 6: be familiar with a number of VR software tools and hardware;
1 | 5 | Goal 7: think critically about software;
1 | 4 | Goal 8: communicate ideas more clearly;
0 | 4 | Goal 9: demonstrate VR tools to non-users;
PROJECT CONTRIBUTIONS & DELIVERABLES
Project 1 Proposal: Proposal Plan Doc
Presentation for Project 1 Proposal: Project 1 Proposal - Immersive VR Visualization of Air Quality Data
In-class Activity: Project 1 In-Class Activity
In-class Activity Results: Project 1 In-Class Activity Results
End Presentation for Project 1: Project 1 End Presentation
Poster/Public Demo: Project 1 Poster
Project 2 Proposal: Project Plan Doc
Presentation for Project 2 Proposal: Project 2 Proposal - Evaluating Collaborative VR for Finance and Economics
Project 2 In-class Activity: Project 2 In-Class Activity Project 2
Project 2 In-Class Activity Board: Project 2 Activity Board
In-class Activity Results: Project 2 In-Class Activity Results
End Presentation for Project 2: Project 2 End Presentation
Flash Talk: VR Flash Talk
Poster: Project 2 Poster
Public Demo 1: VR Demo 1
Public Demo 2: VR Demo 2
ALL OTHER WIKI CONTRIBUTIONS
CONTRIBUTION 1: Home Page: Fixed Outdated Links, Button HTML, and Cheat sheets
CONTRIBUTION 2: A Page Explaining VR Applications in Environmental Science
CONTRIBUTION 3: A Page Evaluating UX Methods for Data Visualization
CONTRIBUTION 4: A Page Explaining Collaborative VR Storytelling
CONTRIBUTION 5: A Page Evaluating Flow Immersive Software for Collaborative Data Visualization
CONTRIBUTION 6: Additions to Evaluating Glitch WebVR Software
CONTRIBUTION 7: A Page Comparing WebVR Mapping Projects with Evaluations AFrame City and Aframe Globe Component
CONTRIBUTION 8: Additions to Scientific Data Page
CONTRIBUTION 9: A Page Comparing Environmental API Data Collection Methods
CONTRIBUTION 10: A Page Explaining How to Make Calls using Postman & Parse Data from Ambee API
CONTRIBUTION 11: A Page Evaluating Sketchfab for VR Modeling
CONTRIBUTION 12: A Page Streamlining the Process of VR Model Rendering: Sketchfab -> Blender -> Unreal Engine
CONTRIBUTION 13: A Page on VR Applications in Economics
CONTRIBUTION 14: A Page Evaluating VR Visualization Software for Finance
CONTRIBUTION 15: A Page Evaluating Immersed
CONTRIBUTION 16: A Page Evaluating Meta Horizons Workrooms
CONTRIBUTION 17: A Page Tutorial on Setting up Immersed
CONTRIBUTION 18: A Page Tutorial on Meta Quest 2 VR Hand Tracking
HOURS SUMMARY
Total: 224 hours
HOURS journal
1/26, 2 hours
Journal set up page for HW1
Joined slack channel
Introduced myself on slack channel
Explored website, syllabus, and previous research.
1/27 - 3 hours
Read Kenny Gruchalla's bio, preparing for discussion sessions.
Questions for Kenny Gruchalla, Tuesday's NREL visitor, who will give an introduction to VR assisted research software at NREL
What were some challenges you faced while becoming an expert in using VR assisted research software?
Do you have any advice for someone interested in researching VR?
What are some skills that you think are important for scientific research that you've experienced throughout your career?
Reviewed research papers and website overview, made 3 changes to website
Nine Separate Changes (Applied 3):
Takes 10 minutes:
Google Sites cheat sheet needs to be updated on main page (outdated)
Jump back to top button points to wrong page (html code goes to 2018 page)
"Into to WebVR" link on homepage does not respond when clicking
Takes 1 hour:
Create chart of most popular VR hardware and software (include prices, popularity, etc)
Add comparison table for VR development software
Update the Joining the Team page with proper details
Takes 10 hours:
Make an introduction to Brown VR Resources page that includes helpful tutorials to campus resources such as the YURT, CAVE, and other clubs, organizations, labs, etc.
Update the overall user interface of the website to be of a more techy background with interactive VR graphics.
Change the left sidebar to have less tabs cramped together, maybe figure out a way to open up to new page or have a better design.
1/31 - 2 hours
Set up notes docs for classroom notes and ideas:
Exploring VR Software to research and evaluate for final project:
Unity, Unreal Engine, Google VR, Paraview, Adobe Photoshop 3D
2/1 - 3 hours
Set up notes docs for classroom notes and project ideas:
Exploring Oculus with Jakobi during office hours
Set up Paperspace, explored VR apps & Google Earth
2/4 - 5 hours
Explored Google Earth VR and Google Home Web (+ Google Form)
Downloaded and surveyed DinoVR application
2/5 - 1 hours
Installed and surveyed Unity and Unreal Engine
2/6 - 5 hours
Surveying technologies and reviewing wiki/web to solidify your 3-5 project ideas
Project 1: Virtual Virtual Reality Museum using Unity/Unreal Engine
3 things I'll do:
Surveying 3-5 virtual reality software including Unity, Unreal Engine, and VR tools/frameworks for museum architecture building.
Research data on virtual reality usage and explore CSCI 1951T site for useful data and comparisons to include in 3D museum.
Use existing data on software technologies on site to form 1st draft and collect user feedback on rudimentary museum.
1 class activity: invite classmates to try to use the different VR tools/frameworks and collect their feedback.
potential deliverables: a very detailed wiki section that presents current VR applications for museum building and list/comparison of software; a better user interface for the Brown wiki page
a more detailed wiki section presenting current VR applications on architecture and a list of related software; an architecture data VR representation work.
Project 2: Explore negative health effects of extended VR usage
3 things I'll do:
Research ergonomics, mental health effects, and motion sickness data on extended VR usage.
Collect college student opinions on VR technology pros and cons (visualization, motion sickness, eye strain, etc).
Compare and make wiki page on data and research on
1 class activity: invite classmates to try out different frenzy VR applications to see if they get dizzy
potential deliverables: a wiki page that has more data on ergonomics and motion sickness of using VR, comparing various applications of VR to see which cause more strain.
Project 3: VR Software for visualizing Partial Differential Equations
3 things I'll do:
Send out email to Professor Tabrizian at Brown about potential PDE visualization ideas as well as research available data on web.
Survey tools such as DinoYURT and explore the YURT at Brown to see if 3D visualization would help user understanding of concepts.
Collect data and user feedback on whether 3D visualization of PDE's helps them understand more abstract concepts easier.
1 class activity: invite VR and PDE classmates to explore YURT visualization and give feedback.
potential deliverables: a wiki page on using VR visualization for mathematics data & education, a 3D visualization model that helps pde and other math students better understand abstract concepts
2/7 - 5 hours
Surveying technologies and reviewing wiki/web to solidify your 3-5 project ideas
Project 1: Virtual Virtual Reality Museum using Unity/Unreal Engine
3 things I'll do:
Surveying 3-5 virtual reality software including Unity, Unreal Engine, and VR tools/frameworks for museum architecture building.
Research data on virtual reality usage and explore CSCI 1951T site for useful data and comparisons to include in 3D museum.
Use existing data on software technologies on site to form 1st draft and collect user feedback on rudimentary museum.
1 class activity: invite classmates to try to use the different VR tools/frameworks and collect their feedback.
potential deliverables: a very detailed wiki section that presents current VR applications for museum building and list/comparison of software; a better user interface for the Brown wiki page
a more detailed wiki section presenting current VR applications on architecture and a list of related software; an architecture data VR representation work.
Project 2: Explore negative health effects of extended VR usage
3 things I'll do:
Research ergonomics, mental health effects, and motion sickness data on extended VR usage.
Collect college student opinions on VR technology pros and cons (visualization, motion sickness, eye strain, etc).
Compare and make wiki page on data and research on
1 class activity: invite classmates to try out different frenzy VR applications to see if they get dizzy
potential deliverables: a wiki page that has more data on ergonomics and motion sickness of using VR, comparing various applications of VR to see which cause more strain.
Project 3: VR Software for visualizing Partial Differential Equations
3 things I'll do:
Send out email to Professor Tabrizian at Brown about potential PDE visualization ideas as well as research available data on web.
Survey tools such as DinoYURT and explore the YURT at Brown to see if 3D visualization would help user understanding of concepts.
Collect data and user feedback on whether 3D visualization of PDE's helps them understand more abstract concepts easier.
1 class activity: invite VR and PDE classmates to explore YURT visualization and give feedback.
potential deliverables: a wiki page on using VR visualization for mathematics data & education, a 3D visualization model that helps pde and other math students better understand abstract concepts
2/6 - 1 hours
Update Google Earth Videos and comment Project topics
Read over articles on: Data Visualization,
2/8 - 8 hours
Update Google Earth Videos and comment Project topics
Articles:
Tools/Frameworks:
Netflix VR
Hulu VR
Youtube VR
Select one project and create a plan.
Project: Exploring Video Development and Cinematic User Experience in VR
The goal of this project is to explore the potential of Virtual Reality (VR) technology in enhancing the future of video graphics development and introducing interactive cinematic experiences. The project will focus on surveying current tools and frameworks for VR video development, exploring the feasibility of VR interactive film cinemas through user experience research, and building a brief movie industry data display (make web VR video interactive, if time).
In-class activity: Explore user experience by comparing interactive cinematic shorts and collecting classmates' feedback
Milestones:
2/14 - General Video VR main wiki page, prepare for powerpoint, (wiki page evaluating streaming VR software, if time)
2/16 - Guide on frameworks for visualizing data or film in VR, explore film related datasets (host webVR site, if time)
2/23 - Tutorial on setting up VR streaming (integrate film data set on webVR site, if time)
2/28 - Further research on VR film data, design in-class interactive activity
3/02 - Able to launch in-class interactive activity, collect feedback. (host interactive activity on webVR site, if time)
3/07 - Finish software evaluating and comparing based on my own experience.
3/09 - Integrate classmate and users' feedback to contribute to wiki. Touch up wiki, put together poster/presentation.
2/11 - 4 hours
Drafting project 1 feedback email
Researching new topics in Collaborative Data Visualization
MRI/CTE Scans
Fluid Dynamics through recording from NREL Paraview data visualizations
Geographic data: air quality data, traffic data, climate change data
2/12 - 4 hours
Project Proposal
Finalized Project Topic: Immersive VR data visualization of air quality pollution data
The goal of this project is to explore immersive VR visualization for air pollution data. The project will extensively research VR applications in Environmental Science, VR software for pollution particle visualization, and UX for various modes of data displays. All research will be displayed through the web wiki pages. Finally, the project will focus on hosting collaborative VR visualization experiences through WebVR and software frameworks to integrate collaboration and interaction into air pollution visualization.
Existing Data sets on Air Quality (CSVs and APIs):
Relevant Research:
Real-time Providence Air Quality: https://www.breatheprovidence.com/
Associations between Air Quality and Covid-19: https://www.sciencedirect.com/science/article/pii/S0013935121012251
Associations between Air Quality and historically red-lined areas: https://pubs.acs.org/doi/full/10.1021/acs.estlett.1c01012
Brown research faculty to contact: https://vivo.brown.edu/display/mhasting#Research
Explored Technologies & Frameworks to use for data visualization:
Postman API: API client that makes it easy for developers to create, share, test and document APIs.
A-Terrain: Cartography component for A-frame GL for creating VR environment
Vr-Wiz: Collaborative multiplayer VR client and server framework for data visualization powered by A-Frame and Socket.IO
Vr-Map: Framework platform for visualizing
Project Proposal Plan: https://docs.google.com/document/d/1uRiudOqFz6ziz_NLQ8m530UQfT-tuqOYhiwbwAMhfek/edit
2/12 - 10 hours
Wiki Page: VR in Environmental Science
Survey software/data/research: Postman API, A-Terrain, VR-Wiz, VR-Map, Kaggle/Ambee/EPA/Airnow.gov data sets, Breathe-Providence Research
Presentation for Project 1 Proposal Presentation: Project 1 Proposal - Immersive VR Visualization of Air Quality Data
2/13 - 4 hours
Further research for Project Proposal Plan: stakeholders, technologies, etc.
Finalize Project Proposal Plan Doc: https://docs.google.com/document/d/1uRiudOqFz6ziz_NLQ8m530UQfT-tuqOYhiwbwAMhfek/edit
2/15 - 7 hours
Finished VR tutorial exploring time-series data using Flow Immersive.
Created VR immersive data stories (posted on YouTube =>)
Added Flow Immersive wiki page
Activities logging rubric:
(5) Journal activities are explicitly and clearly related to course deliverables
(5) deliverables are described and attributed in wiki
(5) report states total amount of time
(5) total time is appropriate
2/15 - 7 hours
Attended Office Hours with Jakobi: learned about python batch data scripting
Data Cleaning Ambee Pollution Data using Postman API
10,000 lattitude and longitude &Ambee limits 500 requests per business email account (rather time consuming).
Rendering data visualization points in Flow immersive
Created Flow Immersive wiki page
2/22 - 2.5 hours
Set up and explored Volume Rendering & Paraview for in class activity.
Reviewed wiki pages, documentation, and articles for volume rendering pollution particles:
2/25 - 2.5 hours
Began designing up Pollution VR Data Visualization and Storytelling UX template in class activity for Project 1
2/26 - 7 hours
Cleaned 1500+ Cities from Ambee Air pollution Data
Set up Flow Data Visualization for class activity.
2/27 - 5 hours
Built Slide deck for Project 1 Update for in-class presentation.
Finished Pollution VR Data Visualization and Storytelling UX in class activity
3/1 - 4 hours
Prep for Lexi's in-class activity
Prep for Mohammed's in-class activity
Review results from Melvin's in-class activity in forms and on Slack
Added Wiki Page: Evaluating UX Methods for Data Visualization
3/6 - 4 hours
Prep for Vincent's in-class activity
Prep for Yifei's in-class activity
Prep for Dave's in-class activity
Prep for Yuanbo's in-class activity
More data parsing and research for Project 1
Added Wiki Page: A Page Evaluating Flow Immersive Software for Collaborative Data Visualization
3/8 - 4 hours
Prep for Brian's in class activity
Prep for Ashley's in class activity
Prep for Austin's in class activity
Worked on Aframe WebVR Design
Added Wiki Page: Additions to Evaluating Glitch WebVR Software
3/10 - 4 hours
3/11 - 3 hours
Office hours with Jakobi
Exploring Glitch, VRmap, Aframe Registry
3/12 - 5 hours
Added to Wiki Page: Additions to Scientific Data Page
Added Wiki Page: A Page Comparing Environmental API Data Collection Methods
3/13 - 4 hours
Prep for Liza's in class activity
Prep for Camilo's in class activity
Prep for Vincent's in class activity
Added Wiki Page: A Page Comparing WebVR Mapping Projects with Evaluations AFrame City and Aframe Globe Component
3/14 - 5 hours
Tidy up Wiki Pages
Draft Email on Journal Reviewing Evaluation
Prepare Project 1 Presentation
3/20 - 2 hour
Read through Empirical Studies in Information Visualization: Seven Scenarios
The scenarios for understanding data analysis are:
Understanding environments and work practices (UWP)
Evaluating visual data analysis and reasoning (VDAR)
Evaluating communication through visualization (CTV)
Evaluating collaborative data analysis (CDA)
The scenarios for understanding visualizations are:
Evaluating user performance (UP)
Evaluating user experience (UE)
Evaluating visualization algorithms (VA)
Project 2 Initial Ideas Plan
Not entirely set on specific idea of what data I will use but could explore forest fire data visualizations
Explore Unity/Unreal and Blender side of things, then compare with WebVR
Get ideas from classmates who've done Unity/Unreal & Blender
Project 2 Proposal: Collaborative Financial and Quantitative Research Data Visualizations in VR
3/23 - 4 hour
Revised Project 2 Plan
Evaluate your project plan using the rubric from project 1:
Included evaluation questions for seven scenarios to consider in project
Includes In-Class Activity Ideas, Wiki Deliverables, and Collaboration Ideas
Included milestones for 4/04, 4/06, 4/11, 4/13, 4/18, 4/20, 4/25, 4/27, 5/02, 5/04.
Includes Proposal Self-Evaluation
04/03/23 - 4 Hours
Extensive Review of the Seven Senarios paper
Understanding data analysis:
1) Understanding environments and work practices (UWP)
How does VR in economic visualizations differ for Personal, Educational, Business, vs Academic environments?
What are the work practices of these four environments and how might that affect VR data analysis?
What types of analyses should the visualization tool support?
What are the characteristics of the identified user group and work environments?
What data are currently used and what tasks are performed on it?
What kinds of visualizations are currently in use?
How do they help to solve current tasks?
What challenges and usage barriers can we see for a visualization tool?
2) Evaluating visual data analysis and reasoning (VDAR)
Can the VR software effectively support visual analysis and reasoning about the data?
Data exploration? Knowledge discovery? Decision making?
3) Evaluating communication through visualization (CTV)
Do people learn better and/or faster using the visualization tool?
Is the tool helpful in explaining and communicating concepts to third parties?
How do people interact with visualizations installed in public areas? Are they used and/or useful?
Can useful information be extracted from a casual information visualization?
4) Evaluating collaborative data analysis (CDA)
Does the tool support effective and efficient collaborative data analysis?
Does the tool satisfactorily support or stimulate group analysis or sensemaking?
Does the tool support group insight?
Is social exchange around and communication about the data facilitated?
What is the process of collaborative analysis? What are users’ requirements?
Understanding visualizations:
5) Evaluating user performance (UP)
What are the limits of human visual perception and cognition for specific kinds of visual encoding or interaction techniques?
How does one visualization or interaction technique compare to another as measured by human performance?
6) Evaluating user experience (UE)
What features are seen as useful?
What features are missing?
How can features be reworked to improve the supported work processes?
Are there limitations of the current system which would hinder its adoption?
Is the tool understandable and can it be learned?
7) Evaluating visualization algorithms (VA)
What features are seen as useful?
What features are missing?
How can features be reworked to improve the supported work processes?
Are there limitations of the current system which would hinder its adoption?
Is the tool understandable and can it be learned?
4/6 - 3 hour
Refresh Bloom's Taxonomy & Seven Scenarios
Researched and Evaluated several VR Finance Visualizations:
D3 VR (best existing option that actually works, closed sourced, requested a demo)
ShoppyVerse (not very usable)
CashPrinter (fun, not very useful
Flow immersive (solid, but closed sourced, costs money beyond trial)
Stock City (only really proof of concept/not too developed for individual use; https://www.youtube.com/watch?v=YQ2-8_2Vwpw)
4/10 - 4 hour
Prep for Lexi's Activity
Updated to Unreal Engine 5, read Austin's tutorials and video links
Surveyed Sketchfab and Blender for financial visualizations (very sparse)
Surveyed internet for VR Finance visualizations (most proof of concept, working ones closed sourced)
More research:
Finalizing towards:
Sketchfab + Blender + Unreal Engine 5
Flow Immersive Data Story
D3 VR (still waiting to hear back)
Adding to Wiki pages: VR in Economics Page, Importing to Blender from Sketchfab
Started piecing together poster (considering displaying environmental visualization for for ANOCH)
4/11 - 6 hour
Exploring and updating new version for Unreal Engine for modeling Financial Visualization
Somewhat of a pain. Mac is too slow and keeps on heating up. Had to set a bunch of configurations to even get Unreal Engine updated with VR...
Using Blender for Data Visualization
Rendering image for museum-like VR visualization
Unfortunately not much info added by incorporating it into Unreal Engine
4/12 - 6 hour
Adding & updating wiki pages: Evaluating Immersive VR Visualizations for Finance
Exploring existing demos for VR financial visualization:
Everything is old, closed sourced (and for businesses), or only proof of concept! :/// SADGE
Set up new flow Finance VR Visualizations (worst case scenario, have to demo flow story again, but ideally not)
Did add additional info according to 7 scenarios evaluation, but I also want to not demo same technologies as last time.
Requested demos, still waiting to here back from D6 VR, Bloomberg Terminal, Virtualtrics 🤞
4/14 - 6 hour
Preparing, Formatting, and Printing Environmental Poster Presentation for ADOCH:
EcoVRViz: Evaluating Air Pollution and Environmental Science Data Visualization and User Experience in Virtual Reality
4/14 - 2 hour
Presenting and Sharing Brown's VR Research @ ADOCH for incoming undergraduates
4/17 - 4 hour
Created new wiki tutorial: A Page Tutorial on Setting up Immersed
Evaluating Immersed VR for Drawing Economic Graph Charts & Sharing Financial Data Screens
Prep for Camilo's In-Class Activity
4/19 - 6 hour
Prep for Dave's In-Class Activity
Prep for Brian's In-Class Activity
Researched Hand Tracking Features on Meta Quest 2
Tried hand-tracking features set up in coordination with Immersed VR
Compiled together a Tutorial on Meta Quest 2 VR Hand Tracking
4/21 - 4 hour
Presenting and Sharing Brown's VR Research @ ADOCH for incoming undergraduates
Exploring Immersed VR for Business Economical VR Setting
Contributed to Immersed VR wiki page
4/22 - 3 hour
Setting up Activity Board for In-Class Activity Project 2:
4/24 - 5 hour
Finished compiling Melvin's In-Class Activity Project 2
Prep for YiFei's In-Class Activity
4/26 - 2 hour
Prep for Mohammed's In-Class Activity
Prep for Yuanbo's In-Class Activity
Prep for Liza's In-Class Activity
A few contributions to: A Page Evaluating Meta Horizons Workrooms
4/31 - 5 hour
Prep for Ashley's In-Class Activity
Prep for Austin's In-Class Activity
Prep for Vincent's In-Class Activity
Analyzing and Collecting feedback from Activity Board for VR activity
Additions to Flow immersive and Blender VR rendering models
5/2 - 3 hour
Presenting Environmental VR Poster Presentation for CS Research Symposium:
EcoVRViz: Evaluating Air Pollution and Environmental Science Data Visualization and User Experience in Virtual Reality
5/3 - 3 hour
Polishing research and feedback results
Compiling 3 minute Project 2 end presentation
5/8 - 5 hour
Polishing research and feedback results
Reorganizing wiki page
Made <1 minute Project 2 Flash Talk
5/10 - 2 hour
Polishing research and feedback results
Moved hackmd to google sites
Made <1 minute Project 2 Flash Talk
Beginning Project 2 poster and printout
5/10 - 2 hour
Polishing research and feedback results
Moved hackmd to google sites
Made <1 minute Project 2 Flash Talk
Beginning Project 2 poster and printout
5/14 - 5 hour
Collecting research conclusion results on immersive and collaborative insights financial VR visualization.
Re-read several cutting-edge research papers and cited on project presentation
Rehearsed 2x Project 2 Flash Talk
Polishing research and feedback results
Moved hackmd to google sites
Made <1 minute Project 2 Flash Talk
Beginning Project 2 poster and printout
5/15 - 5 hour
Developing first Financial Data Storytelling visual on Flow Immersive (1st QR code).
Data story consisted of Project 2 presentation and corresponding immersive data visualization.
Rehearsed 2x Project 2 Flash Talk
Continued Editing Project 2 Poster
5/16 - 7 hour
Developed second Financial Data Storytelling visual on Flow Immersive (2nd QR code).
Data story consisted of Project 2 presentation and corresponding immersive data visualization.
Rehearsed 1x Project 2 Flash Talk
Rehearsed 1x Project 2 Presentation slides
Continued Updating Project 2 Poster
5/17 - 5 hour
Added Augmented Reality XR Features to Project 2 Demo
Added QR codes, conclusions, and citations to Project 2 Poster
Printed out Poster 2!
Voice recorded Audio Features to add to Project 2 Demo