Jakobi Haskell's Journal

Personal Website

Exploring collaborative Web VR one dataset at a time...

Hello reader! I'm Jakobi, a Compute Science student at Brown. I got interested in VR after I visited the traveling Immersive Van Gogh exhibit and saw Van Gogh's paintings come to life in VR (I highly recommend this if they are still offering it!). This class is my first exploration of VR in-depth and also my first time doing data visualization.


Coming into this class I new I wanted to explore web VR because I didn't know it was possible to have VR experiences on a website, and I want to get more experience with web development. Below you will find my web VR related projects, including:

  • Github repos for a basic multiuser webVR implementation with A-frame

  • The proposals, in-class activities, and presentations for my two semester projects

  • A list of my wiki contributions, including a general page on webVR as well as more specific ones on A-Frame components

  • Log of my hours worked this semester


PROJECTS Plans

Project 1 Proposal - Link to Project 1 Proposal Doc

Project 1 ORIGINAL Plan - Link to Project 1 Original Plan Doc

Project 1 ONGOING Plan - Link to Project 1 Plan Doc

Presentation for Project 1 Proposal - Link to PowerPoint

Project 1 Update Slides - Link to PowerPoint

Project 1 Final Presentation Link to PowerPoint

Project 1 In-Class Activity - WebVR 1st Experience Tutorial


Project 2 Proposal - Link to Project 2 Proposal

Project 2 Original Plan - Link to Project 2 Original Plan

Project 2 Proposal Presentation - Link to Presentation for Project 2

Project 2 In-Class Activity Page - Project 2 In-class activity

Project 2 Update Presentation - Project 2 Update Presentation

Project 2 Final Presentation - Link to Project 2 Final Presentation

Flash Talk Presentation - Link to Flash Talk

Poster - Link to Poster

Public Demo - Link to Public Demo Page

WIKI CONTRIBUTIONS

Project 1:

CONTRIBUTION 1 [Wrote Tutorial on Setting up A-Frame Hosted Static Webpage] Link to Tutorial

CONTRIBUTION 2 [General Page on WebVR] Link to Wiki WebVR page

CONTRIBUTION 3 [VR Live Test Website] Link to Client Repo, Link to Server Repo

CONTRIBUTION 4 [Tutorial on Setting up WebVR Collaboration] Link to Collaboration Tutorial

Project 2:

CONTRIBUTION 1 [Comparison of A-Frame Compatible Visualization Libraries] Link to Comparison

CONTRIBUTION 2 [Software Evaluation of VR-Viz] Link to Evaluation

CONTRIBUTION 3 [Datasets that work with VR-Viz] Link to Datasets

CONTRIBUTION 3 [Page on WebVR for Visualization] Link to WebVR Visualization Software

CONTRIBUTION 4 [Added Miscellaneous, Logistics section to Project Ideas] Link to Project Ideas

CONTRIBUTION 5 [Overview of GeoJSON Data in Scientific Data Page] Link to Scientific Data

CONTRIBUTION 6 [Comparison of D3.js, VR-Viz, BabiaXR Page] Link to Comparison

CONTRIBUTION 7 [Repo for COVID Bar Charts Exploration Website] Link to Github Repo

CONTRIBUTION 8 [Small Repo for A-Frame Webpage with Portals to Classmate's In-class activities] Link to Portal Github Repo

CONTRIBUTION 9 [Repo for Climate Change Exploration Website] Link to Climate Change Github Repo

HOURS Journal

Total: 159.35 Hours

1/27/21 - 2 Hours

  • Read the course homepage for this year and last year

  • Joined the slack

  • Came up with 9 changes I would make to the wiki

  • As part of the ten-minute change, I Implemented a course google calendar with descriptions of class plans, homework and when it is due (taken from course syllabus page).

1/29/21 - 1 Hours

  • Read wikipedia entry for VR and took notes

1/30/21 - 1.5 Hours

  • Read the section of the course wiki about VR and took notes

  • Watched the demo video for unreal engine 5

1/31/22 - 0.7 Hours

  • Revised my journal notes

  • Started page on VR for web

2/1/22 - 2 Hours

  • Setup Oculus Quest 2

  • Explored Oculus TV, websites that support webXR, and other features of Quest 2

2/2/22 - 2 Hours

  • Setup Virtual Desktop, Paperspace, and SteamVR

  • Researched VR Software, Brainstormed Project Ideas

2/3/22 - 1 Hour

  • Added in depth to-do list at top of page for starting the project.

Total: 10.2

2/5/22 - 3.5 Hours

2/6/22 - 1.5 Hours

  • Watched and read NGINX tutorials

  • brainstormed ways to setup website over https

2/7/22 - 0.7 Hour

  • Set-up DinoVR

2/8/22 - 2.5 Hours

  • Took designated screenshots for DinoVR

  • Set-up RPI

  • Installed Raspbian on RPI

2/9/22 - 2 Hours

  • Troubleshooting Raspberry Pi OS

  • Borrowed Pi Equipment from IT Service Center

  • This Raspberry Pi setup is frustrating, but I know I will get through it.

Total: 20.4

2/10/22 - 5 Hours

  • Switched to using AWS from RPI

  • Set up server on AWS Lightsail with HTTPS using Let's Encrypt

2/11/22 - 1.5 Hours

  • Tried to setup webpage with NGINX (talk about what it is?), accidentally deleted Let's Encrypt Certificate

  • Debugged Certificate issue

  • Uploaded sample Github repo w/ existing A-Frame Projects onto https://www.vrwikitest.com

Tota: 26.9

2/13/22 - 3 Hours

  • Set-up own Github repo using A-frame, uploading it to website

  • Added controller support

Total: 29.9

2/14/22 - 3 Hours

  • Created schedule for Project 1, with deliverables and in-class activities, and contributions to Wiki

  • Prepared presentation

Total: 32.9

2/15/22 - 1.7 Hours

  • Started draft of webpage setup tutorial, researched about workings of web infrastructure

2/16/22 - 1 Hour

  • Added to draft of webpage tutorial, explained how web infrastructure used in tutorial works

Total: 35.6

2/17/22 - 1 Hour

  • Tidied up Project Plan, Linked it on course page

2/18/22 - 2 Hour

  • Read photon web documentation

  • Read about Vizible - VR Application for Collaboration -- it would be convenient to have this on the web.

  • Read about ways to prevent cybersickness from VR wiki, after experiencing it myself

2/19/22 - 1 Hour

  • Watched tutorial on using websockets for multiplayer game, took notes

    • I think I will start with websockets, as photon was really made for Unity

2/20/22 - 2 Hours

  • Watched end of tutorial, took more notes

  • Looked at getter methods for position data in A-Frame

2/21/22 - 2 Hours

  • Read about A-Frame event handling

  • initialized repo for ws testing

  • watched freeCodeCamp tutorial

2/22/22 - 4 Hours

  • Tested getter for pos data in repo

  • Used WS to send pos data

Total: 47.6 Hours

2/23/22 - 2 Hours

  • Tracked rotation as well as pos

2/24/22 - 1 Hour

  • Realized that website does not work for multiple people

  • brainstormed ways to architect communication to solve this

2/25/22 - 2 Hours

  • Updated journal with missing deliverables such as Google Earth, Peer Review

2/26/22 - 1 Hour

  • Planned Server architecture

2/26/22 - 3 Hours

  • Wrote code for revised server architecture

2/27/22 - 2 Hours

  • Debugged Code

Total: 58.6 Hours

2/28/22 - 5 Hours

  • Debugged Code more (took forever)

  • Wrote draft on in-class activity

  • Prepared for Beatrix's in-class activity

  • Tested website with others for first time

Total: 63.6 Hours

3/2/22 - 1.5 Hours

  • Created presentation for progress update

  • Fixed small bug for ID's of controllers

3/4/22 - 1 Hour

  • Refined code

  • Started transition to TS

3/6/22 - 1 Hour

  • Commented code

  • Debugged code

Total: 66.1 Hours

3/8/22 - 3 Hours

  • Downloaded software for Lucia's Architecture demo

  • Tidied up code for collaborative demo

  • Changed data structure of client storage of other clients

  • Tried out Amanda's virtualitiks demo after class

3/9/22 - 2 Hours

  • Cleaned up code for repo (still need to remove todos)

  • added link to repositories for collaborative demo to wiki

3/13/22 - 2 Hours

  • Removed todos from code

  • Wrote full draft of collaborative webVR tutorial

Total: 73.1 Hours

3/14/22 - 0.5 Hour

  • Prepared for Shashidir and Maia's Tutorials

3/15/22 - 0.5 Hour

  • Explored Hand Tracking Experience (Idea: add to webVR page about that, create a section on existing features)

  • Prepared for Paul's Tutorial

  • Refactored organisation of A-Frame page for ease of access

3/18/22 - 1.5 Hours

  • Added constants as environment variables for repo

Total: 75.6 hours

3/19/22 - 1 Hour

  • Finished env variable configuration to repo

  • simplified steps for collaborative webVR tutorial accordingly

3/21/22 - 1.5 Hours

  • Setting up twitter API

3/22/22 - 1 Hour

  • Drafted project plan

  • Rewrote it because Google sites didn't save

3/23/22 - 1 Hour

  • Project 2 Plan Revision

  • Created Slides for Project 2 Idea

  • Preparation for Aakansa's Activity

Total: 80.1 hours

4/04/22 - 2.5 Hours

  • Researched Data visualization frameworks with A-Frame integration. Includes D3, VR-Viz, and BabiaXR

    • wrote intro to comparison page between them

    • wrote spreadsheet comparator

4/05 - 1 Hour

  • Prepared for Aakansha's, Nick's, Jennifer's in-class activities

  • Reinstalled Oculus and SteamVR Apps but VR on remote desktop is still not working

4/06 - 2 hour

  • Posted screenshots for in-class activites (didnt complete them in-class bc my paperspace machine was not working)

  • Installed software for Sayan's activity

  • Downloaded VR-Viz repo and imported sample point cloud dataset

Total: 85.6 hours

4/07 - 2 Hour

  • Imported stock data into Rectangle chart (got it to work)

  • Tried importing point cloud data, did not work

  • Read more of VR-Viz Documentation

  • Tried importing scatter plot of healthiest cities data, did not work yet. Tried debugging it.

4/08 - 1 Hour

  • Tried uploading gltf file to VR-Viz

    • had to validate the gltf file first

      • but it uses bin, which is not supported

4/11 - 2.5 Hours

  • Revised D3, VR-Viz, BabiaXR comparison page

  • Created VR-Viz evaluation page

  • Added author name to pages I contributed to.

  • Refactored organisation of Journal, added missing links and moved other sections to google docs (as they are easier to edit without risk of not saving), updated hours

Total: 91.1 Hours

4/13 - 0.5 Hours

  • moved Google Earth Screenshots, Evaluations of Journal below and into collapsible components, as they are not as important

  • Answered Doodle for VR Project presentations time

4/14 - 4 hours

  • converted existing project 1 repo to TS with Nick

  • debugged cone movement not changing in a-frame

4/16 - 0.5 Hours

  • Took in feedback of WebVR page from last class activity and made changes (including adding logos, and personal recommendations) accordingly.

4/17 - 1 Hours

  • Continued changes to WebVR website, including a section on React 360, and added notes to other React 360 pages saying that the project had been archived.

  • Worked on debugging cone movement

4/18 - 6 Hours

  • Wrote draft of slides

  • Researched types of data(data cubes, time series, flow data) for potential investigation for project 2

  • Finished Progress Report on Project 2 Slides

  • Updated my Project Plan

  • Looked at interesting datasets on kaggle to explore (travel data for flow)

  • Researched WebXR framework, WebGL, Three.js

  • Updated Web VR Page with Web GL section and more accurate explanations of each technology

  • Finally fixed position bug of VRWiz repo with Nick (it was due to everything being wrapped in a div, which prevented full VR mode)

  • Migrated VRWiz repo from vanilla JS to React, for integration with VR-Viz

Total: 103.1 Hours

4/19 - 2 Hours

  • Continued working on migrating repo to React

Total: 105.1

4/20 - 1 Hour

  • Prepared for Amanda and Mandy's in class activity

    • Could not set up android installer for Amanda's activity, so will have to try with somebody else's Oculus when I'm back in person

4/23 - 3 Hours

  • Continued trying to setup multiplayer repo with React and VR-Viz

4/24 - 2 Hours

  • Continued setting up repo with react

    • Not sure if this is possible, as using socket.io with react makes state sharing difficult

  • Tried using VR-Viz with A-Frame networked instead, but no luck

4/25 - 5 Hours

Total: 116.1

4/26 - 6 Hours

  • Worked on getting bar charts and scatter plots to show up in VR-Viz

  • Added 3D text as a guide through exploration

  • Deployed website to a Glitch and AWS server unsuccessfully, then to Vercel server succcessfully after numerous times

  • Created Google forms for feedback on VR-Viz (with emphasis on user experience)

  • Unfortunately joystick movement was not working so tried to debug that

Total: 122.1

4/27 - 0.25 Hour

  • Installed Engage for Shashidir's in class activity

  • Tried booting paperspace again for Sayan's activity (since it didn't work first time)

Total: 122.35

5/2 - 3 Hours

  • tweaking dataset of COVID flight map for VR Viz

  • debugging failed air traffic location dataset import

    • Data Processing Steps:

      • Opened in Mathematica

      • Selected all with a specific date (4-03)

      • Got their latitude, longitude, and PercentOffBaseLine

      • Convert it to a new dataset, which will then be exported as a csv (can change headers later)

      • Import it to Map Chart with Bar graph in VR-Viz

      • Not working when imported to VR-Viz (and do not know why)

  • added adjustments to babiaXR, and VR-Viz comparison

5/3 - 2 Hours

Total: 127.35

5/4 - 1.5 Hours

Total: 128.85

5/5 - 2 Hours

  • Worked on slides for final in class presentation

  • Brainstormed focus of final project, and wrote about takeways from in class activity

5/6 - 2 Hours

  • Finished Slides for final project 2 presentation (remember to link to them AND IN CLASS ACTIVITY FOR WIKI CONTRIBUTIONS) AND ADD PHOTO OF CAVEMAN AND VR)

  • Started Poster in Figma with dimensions in terms of pp

    • Not sure what my actual dimensions will be but for now I am playing around

    • I like to start by writing all the titles and from there I will start filling sections in with text and visuals)

5/7 - 3 Hours

  • Changed slides for in-class activity (because I misinterpreted them)

  • Worked on Poster in Figma

    • Added some screenshots

    • Revised layout

  • Changed message in course timeline to be more clear

Total: 135.35

5/8 - 2 Hours

  • Revised Poster, added more content

    • Updated section on collaborative features and user experience, added some annotations to images

  • Added Nick's Collaboration features to my in-class activity repo

  • Updated instructions for public demos and final presentations using feedback from Ross and David in the Slack

5/9 - 3 Hours

  • Added "How can I implement Collaboration?" to WebVR General Page

  • Added some project ideas that I didn't get to to Project Ideas Page

    • The hope is that future students in the course can build upon the projects that we would have continued if we had time. Notably, I would've liked to explore Paraview Web

  • Added links to Contributions Page in my Journal

5/11 - 2 Hours

  • Revised Public Demo Poster

    • changed layout

Total: 142.35

5/12 - 2 Hours

  • Tried importing different point cloud, map datasets to no avail

  • Cleaned up Tutorials

5/17 - 5 Hours

  • Came up with idea to use climate change data for public demo

  • Found average annual temperature data on Kaggle

  • Processed data using Jupyter Notebook into a VR-Viz compatible form

  • Visualized it as a waterfall plot for public demo

5/18 - 8 Hours

  • Revised typos in poster

  • Printed poster in CIT

  • Got TopoJSON map to work in VR-Viz

  • Updated Hours Journal with correct number of hours

  • Added Logistics section and refactored Miscellaneous section to Project Ideas Page

    • Motivated by the fact that google sites failed to save changes three times, and I lost my work.

  • Set up Public Demo

5/19 - 2 Hours

  • Finalized Wiki Contributions, including more pages to github repos and data files that can be used with VR-Viz

Total: 159.35

Project Ideas - 2/08

Jakobi

Project Idea: Using 3D graph nodes to predict social media post popularity in a collaborative environment .

Software: Three.js for 3D graphics for the web, which supports VR (with WebXR) and real-time collaboration. LGL (Large Graph Generation) for graph generation which can then be imported into three.js.

Three things I will do: In-class search for specific tweet with partner, contribute to wiki page on webVR, explore backend multiplayer frameworks for web.

Class Activity: Exploration of twitter data as a network of graphs in 3D. Creating a room on the webstite, sending a link to partner, locating a specific tweet with partner and person who created room sends a screenshot of their partner pointing to the tweet, post-experience survey of collaborative web experience, ease of use.

Potential Deliverables: Evaluation of 3D graph nodes visualization effectiveness in VR, in-depth wiki page on resources for webVR, tutorial on creating collaborative environment for webVR.

Software Evaluation Metrics: Ease of set-up rating, Effectiveness of visualization, level of collaboration (interactivity or just visuals?)



Project Idea: fMRI and EEG data for medical sleep studies

Research on how mental illness affects sleep patterns, could use VR for real-time analysis with annotations on brain patterns

Three things I will do: demo VR visualization with sleep researcher and with their existing dataset, install and evaluate ParaView and other software for fMRI visualization, record user feedback in survey from sleep research project.

Potential Deliverables:Interview with medical researchers about their experience using VR, documentation of a survey or experment, contribution to analysis of existing medical VR software.

Software Evaluation Metrics: Ease of use for medical researchers who have never used VR before, Time in setting up software, Comparison of time used to write down observations necessary for sleep analysis with VR vs. time it takes with software that researchers currently use.



Project Idea: Mars/Space Mission/Exploration / Remote sensing Mars Data / less explored planets like Venus or Exoplanets

Software: Open Space, Photon and Unity for Collaboration

Three things I will do: Downloadable extension applicaiton on unity to explore Venus collaboratively, post survey with results of in-class activiy and its implications, download data from an exoplanet into Unity to see what its like to create new visualization.

Potential Deliverables: analysis of collaborative features of OpenSpace, additions to wiki page on VR for Space.

Visualizing where missions went wrong, planning future missions, how effective is it vs plain 3D graphic simulation without VR


Google Earth Screenshots

From top to bottom, left to right:

My dorm (Andrews Hall)

A place I used to look out from when I visited my uncle in the Bay Area:

My house in Washington D.C.

My favorite trail from Running Club

Self-Evaluation of Journal

Activities logging rubric -- fill in in your journal

key for each criterion:

5 == exceeds expectations

4 == meets expectations

3 == mostly solid with some gaps

2 == shows some progress

1 == some attempt

0 == not found

Criteria:

Journal activities are explicitly and clearly related to course deliverables

3 - Tutorials that I have done are generally about webVR in general, could use some more data visualization specific stuff. could also spend more time reading through wiki

deliverables are described and attributed in wiki

1 - I hint at them but don’t link to them, or describe specifically which one I am working on

report states total amount of time

4 - usually up to date

total time is appropriate

4 - usually up to date



Self-Evaluation of Project Proposal

TODO: Self-Evaluation and compile written plan from slides. WHERE IS MY PROJECT PLAN?

  • Project Evaluation

Below are a set of questions that should help in evaluating project ideas. Answer each with one of:

          1. strongly disagree

          2. disagree

          3. neither agree nor disagree

          4. agree

          5. strongly agree

The questions are:

o The proposed project clearly identifies deliverable additions to our VR Software Wiki

4 - Agree although I could be more specific in what I am adding, I guess for me I kind of have to see how the project progresses first, see what gaps need to be filled, and go from there.

o The proposed project involves collaboration in VR

5 - This is what my project will focus on. The methods for dealing with collaboration right now are very tedious and I plan to research easier ways.

o The proposed project involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use

5 - Twitter Network data is in the wiki page, the text and metadata surrounding it is a specific type.

I plan to use A-Frame for visualization for now (and I will follow the tutorial), but might use Paraview later.

o The proposed project has a realistic schedule with explicit and measurable milestones at least each week and mostly every class

5 - I think its pretty realistic and I have backup plans if I don't get to what I want in time.

o The proposed project includes an in-class activity

4.5 - Could have a more detailed in-class acitivty, but do have one planned.

o The proposed project has resources available with sufficient documentation

3 - Unfortunately there isnt a ton of documentation about using webVR collaboratively but that is partly why I am doing this project, to contribute to the gap that needs to be filled about this.

9 Changes

9 Changes I would make to the wiki

10 minutes:

  • Remove the ability for anyone on the website to publish or edit the wiki

  • Set it so that only those on the course list can edit, but not publish

  • Set up (but don’t fill out) a course calendar, that shows dates of homework when they are due


1 hour:

  • Give an overview of three.js

  • Add a ‘hello world’ guide to three.js

  • Create an 'I’m feeling curious' button, that takes a user to a random page on the wiki


10 hours:

  • Create a section for general trends in VR, and projects to look out for

  • Create a plain collaborative real-time environment on webXL

  • Create a section for VR for the visual arts under the what is VR for? section that discusses its applications for visual arts



Course Calendar

This is a calendar for the course that includes the time of classes, a description of what the plan is for that class (taken from the course timeline page), and a list of homework for the next class (also taken from course timeline). This makes it convenient to integrate the workload for this class for users of google and apple calendar.