PROPOSALS

Project 1 Proposal

Presentation for Project 1 Proposal

In-class Activity

Project 1 Update Presentation

End Presentation for Project 1

Project 2 Proposal

Presentation for Project 2 Proposal

Project 2 Update Presentation: rough draft of poster:

Flash Talk

Poster

In-class Activity

Public Demo

ALL OTHER WIKI CONTRIBUTIONS

CONTRIBUTION 1 : VR Point Cloud Editor Application page added under VR Modeling Softwares - VR Point Cloud Editor 

CONTRIBUTION 2: Tutorial for UNH's VR Point Cloud Editor - UNH Tutorial 

CONTRIBUTION 3: Masterpiece Studio Pro page added under VR Modeling Softwares - Masterpiece Studio Pro 

CONTRIBUTION 4: Masterpiece Studio Pro 3d Model Import Tutorial - Masterpiece Studio Tutorial: Importing and Editing 3D Models 

CONTRIBUTION 5: Verto Studio VR page added under VR Modeling Softwares - Verto Studio VR

CONTRIBUTION 6: Updated Tilt Brush page to include information about OpenBrush - TiltBrush/OpenBrush 

CONTRIBUTION 7: Comparison of 3D modeling softwares for model masking (removing parts of models) - Comparison of softwares for Model Masking 

CONTRIBUTION 8: Added section on MIDI data to Data, Examples, and Collaborators

CONTRIBUTION 9: Quickstart Tutorial for Maestro Midi Player Tool Kit - MIDI data visualization using Maestro Midi Player Tool Kit 

HOURS journal

1/28/22 - 10:30-11:30pm - 1 HR

• Set up journal

• Joined Slack

• Read over home page

1/29/22 - 6:30- 10:00pm - 3.5 HR

• Explored old wiki and thought about changes

• Attempted to implement "jump to" section on the course timeline page:

  • Google sites doesn't want people to rely on html so their embed function is stumping me a little on how to implement a custom "jump to" section.

  • Update 10pm: Able to link to headings on the page but can only open them on new window, self target doesn't work for some reason. This was the embed I was testing out:

• Instead of implementing that then, which took a lot more than the 10 minutes I thought it would and would be inconvenient if it opened a new tab every time, I will just change something else

• "jump to" idea moved from 10-minute change to 1-hr change

1/31/22 - 8:30- 11:30pm - 3 HR

• finished changes list

• Read part of Shared Surfaces and Spaces: Collaborative Data Visualisation in a Co-located Immersive Environment Publisher: IEEE Cite This paper

Assignment 1 // 9 changes:

Three 10-minute changes:

1) Complete list of links to hardware pages on hardware overview

2) Make course archives folder public so it is easier to find

3) Hardware page: make spreadsheet more readable (text color changes)

Three 1-hour changes:

1) Add Contents to every article with subheaders so that they will remain navigable as they grow

2) Change course archive pages to be slightly different from current pages (for example change header image background) so they do not get confused for each other

3) Course Timeline and Homework page: add a "jump to" section to jump to certain weeks on the page

Three 10-hour changes:

1) Add a "bookmark" interface where viewers can star specific pages for quick future access.

2) Make all overview tabs consistent- right now, for example, the "VR Visualization Software" page contains an overview and links to all the individual pages, while the "VR Development Software" page contains information summaries for only some of the children pages. I would combine the summaries into the children pages and let the parent page just be a description of development software and links to each child page.

3) Add more specific wiki editing guidelines and update all pages to consistently follow them- ex. citation requirements (link each piece of information to a resource via footnotes)

2/3/22 - 2:40- 3:40am & 7:30-10am - 3.5 HR

• set up quest part 1

• Software exploration: MinVR, DataVizVr, ParaView

  1. MinVR: barebones development platform/cross-platform VR toolkit that can facilitate a variety of data visualization and virtual reality research projects, including game design and scientific visualization. Provides API for user input and location tracking, but all other aspects of development left to the developer (read: code-heavy)

  2. DataVizVr: allows data visualization for up to 6 variables at a time in VR. Selecting variables/axes is intuitive but currently only has basic features. Could have more features such as multiple variables and a larger variety of plot types.

  3. ParaView: scientific visualization tool. Existing GUI allows for modeling without code

• potential project ideas:

  1. CV Camera calibration point cloud viewer of depth estimates from multi-view images (photogrammetry)

     • inspo: camera calibration photogrammetry assignment in CV class that viewed 3D point clouds in 2D space - would have been helpful to view in 3D space to analyze outliers better and to compare versions easier by sight

     • compare the features different data modeling VR softwares offer/ease of use/output options/amount of time it takes to render point cloud

  2. Comparing different VR capture methods and/or which softwares support

     • 360 video/photos

     • light field capture (veloview?)

     • volumetric 3d capture

     • photogrammetry

  3. Comparison of inside-out and outside-in tracking methods, for example comparing latency, compatibility and easy of use (how much set up time, what software you need, etc)

  4. Tool to aid in 3D scanning of poverty points - tools to help align multiple scans of the same object, automated cleaning up of objects

  5. visualizing ai fashion models/fabrics on various mannequins

2/7/22 - 11:30pm-2:30a - 3 HR

• Fleshed out project ideas

2/9/22 - 11:30pm-1:30a - 2 HR

• Proposal complete on above idea 1: Exploring VR Tools to aid Historical Archiving

• Researched more VR modeling softwares

  • UNH's VR Point Cloud Editor Application - Pros: Free

  • Kodon - Extensive modeling software but paid ($50)

  • Arena4D - extensive but very expensive package

• Researched unity tools for feature matching https://link.springer.com/chapter/10.1007/978-3-030-90176-9_55

2/13/22 - 11:00a-3:00p - 4 HR

• Proposal Presentation and outline created

• Researched ParaView's PCL Library to load point clouds

• Got 3D scans of multiple objects (lego bird, spray bottle)  - 2 profiles for each object

• Looked at and used UNH's VR Point Cloud Editor Application: Built from their Unity Point Cloud Asset (and free!)

  • notes: crashes when editing a dense point cloud (raw point cloud setting on polycam - tried optimized file type and still crashed but was able to work on it for longer)

2/16/22 - 8:00p-12:00a - 3 HR

• Added deliverable with tutorial on UNH's VR Point Cloud Editor Application

• Added hidden deliverable page on obtaining 3D scans using Polycam app

• Researched other software with point cloud editor capability (to add to wiki)

  1. Adobe Medium - only supports Rift, so need to figure out how to host it on Quest

  2. VrMesh (free 30 day trial doesn't allow you to save or export results)

2/22/22 - 1:45p-3:45 - 2 HR

• Added UNH Point cloud app link to software recommendation by types

• looked at http://www.rslab.se/pointcloud-xr/ - a vr point cloud viewer (but doesn't allow point editing)

• read paper on a point cloud viewer - https://dl.acm.org/doi/fullHtml/10.1145/3480433.3480437#sec5 - but can't find the application

2/25/22 - 12:15p - 1:15p - 1 HR

• drafted class activity prep

2/26/22 - 12:45a - 3:15a - 2.5 HR

• finished class activity tutorial steps

• created survey

2/27/22 - 1:30p - 6:00p - 4.5 HR

• tried to set up Oculus link to download Adobe Medium - unable to set up Oculus app because of incompatible Windows system (paperspace's is 2016 version, need Windows 10)

• downloaded and tested Verto VR demo - unable to edit vertices of models

• downloaded and tested gravity sketch - able to import OBJ files but unable to edit model with too many vertices

• downloaded VRMesh

2/27/22 - 9:30p - 3:00a - 5.5 HR

• tried to test VRMesh - only works on windows 10 machines :-(

• downloaded and tested Masterpiece Studio Pro - great tool! can erase parts of model and has many other functions

• added deliverable about masterpiece studio pro + tutorial

• update project presentation

• looked at openbrush - import model function is in beta, not released yet

• tested Kanova

• looked at https://arxiv.org/pdf/2103.02690.pdf, a comprehensive survey on point cloud registration

3/7/22 - 7:30a - 8:30a - 1 HR

• prepared for Lucia's class activity

• tried gravity sketch's toolbelt

3/10/22 - 12a - 1:30a - 1.5 HR

• prepared for Tongyu's class activity

• Peer journal review for Shashi

3/12/22 - 5:45p - 7:45p - 2 HR

• Began deliverable for Verto Studio VR

3/13/22 - 12:30p - 5:00p - 4.5 HR

• Finished deliverable for Verto Studio VR

• Updated TiltBrush page to include information about OpenBrush

• Updated Masterpiece Studio Pro page and tutorial page

3/20/22 - 1:00p - 3:30, 5:00p - 6:00p - 3.5 HR

• Wrote up comparison between modeling softwares

3/21/22 - 8:30p - 12:00a - 3.5 HR

• Project 1 final presentation completed

• read 7 scenarios

• Project 2 idea - 3D synthesia - third axis visualizes volume

  1. I'm a piano teacher, and some of my students learn best at home visually with youtube videos that use synthesia/synthesia-like reactive lighting scenarios. These visuals often lack information about dynamics/volume in music, which is really important in shaping the mood of a piece.

  2. I want to create a 3D synthesia program in Unity that visualizes volume on the third axis.

3/23/22 - 9:45p - 12:15a - 2.5 HR

• Reviewed 7 scenarios

• Sample MIDI data that includes velocity (volume/dynamics): https://www.kunstderfuge.com/piano-rolls.htm

• Looked at unity plugins/C# libraries that may be helpful for manipulating MIDI data:

  1. https://paxstellar.fr

  2. https://www.youtube.com/watch?v=Br3bjj0Kh2w (look at documentation for each app for helpful tips)

  3. https://www.codeproject.com/Articles/8506/Simple-VB-NET-MIDI-Wave-Play-Class

  4. https://www.codeproject.com/Articles/6228/C-MIDI-Toolkit

  5. https://stackoverflow.com/questions/8763/best-way-to-play-midi-sounds-using-c-sharp

  6. https://assetstore.unity.com/packages/tools/audio/maestro-midi-player-tool-kit-free-107994

• https://www.youtube.com/watch?v=qZJNccWKuNo&t=108s AR piano playing

  1. The scenarios for understanding data analysis are

     • Understanding environments and work practices

     • evaluating visual data analysis and reasoning

     • evaluating communication through visualization

     • evaluating collaborative data analysis (CDA).

  2. The scenarios for understanding visualizations are

     • Evaluating user performance (UP),

     • evaluating user experience (UE), and

     • evaluating visualization algorithms (VA).

• Project 2 proposal presentation:

3/27/22 - 12:30p - 3:30p - 3 HR

• completed model comparison page

• looked into AR piano visualizers with hand detection

4/5/22 - 12:30a - 2:00a - 1.5 HR

• Setup class activities

• Project 2 proposal done

4/7/22 - 7:30a - 8:30a - 1 HR

• Setup class activities

4/11-12 - 10:45p - 12:45a, 7:00a - 8:30a - 3.5 HR

• Set up unity steamvr environment

• sketched piano model and environment ideas

4/16/22 - 9:30p - 3:00a - 5.5 HR

• Set up normcore in Unity

• Finished piano model build with Fusion

  1. white key width: 6.5/7 in

  2. white key length: 6 in

  3. white key height: 7/8 in

  4. gap between white keys: .05 in

  5. white key hole gap width: 15/32 in

  6. black key width: 7/16 in

  7. black key height: 7/8 + 1/4 in

4/18/22 - 9:00p - 2:00a - 5 HR

• Imported note OBJs into Unity

• explored MIDI package vis options - chose to go with Maestro Midi Player Toolkit

• created full piano in Unity

• looked into unity's xr input system - vs steamvr?

• halfway functional piano vis, notes currently don't go back down after played

• checkpoint presentation made:

4/22/22 - 9:00p - 11:30p - 2.5 HR

• debugging attempts

4/26/22 - 5:45p - 11:45p - 6 HR

• tried to clean and load 1999 midi file from garageband but didn't exactly work

• https://www.youtube.com/watch?v=44SEReNPSx4&list=PLa739g934GxzKAgp_VPAf4niwI68_5Jq2&index=4

• removed color changing from piano

• added hand controller models - a little wonky with normcore but theyre there

• started implementing controller inputs

4/28/22 - 5:30a - 8:00a - 2.5 HR

• set up for aakansha and shashi's class activities

• loaded more midi files onto unity

• read Tongyu's audio wiki pages

5/2/22 - 6:30p - 3:30a - 9 HR

• added pause/resume functionality - took a while to debug :-(

• added load song functionality

• in class activity page completed

5/8/22 - 6:30p-8:30p - 3 HR

• researched more midi unity packages/libraries/assets, like Helm, Minis, NAudio, and MIDI Unified

• Added MIDI data to Data, Examples, and Collaborators 

5/9/22 - 8:30p- 2a- 5.5 HR

• finished adding MIDI data assets/packages to Data, Examples, and Collaborators

• project 2 poster

• started maestro midi tutorial page

5/9/22 - 8:30p- 2a- 5.5 HR

• finished adding MIDI data assets/packages to Data, Examples, and Collaborators

• project 2 poster

• started maestro midi tutorial page

5/14/22 - 7:30p- 12a- 4.5 HR

• continued maestro midi page - official Maestro mptk website is slightly unorganized and hard to navigate, took a while to find all the necessary components to form a solid tutorial

5/15/22 - 3:00p- 11:30p- 8.5 HR

• created new unity audio vis demo for the maestro midi page - a bit of weird debugging

• finished maestro midi page

5/18/22 - 1:00p- 3:00p, 8:00p-5:30a - 11.5 HR

• finished poster

• finished model comparison page

• Piano vis app:

  1. added slow down and reset speed

  2. added haptic feedback

  3. added more songs

• slow down song has bug right now where duration isn't also changed - would have had to rework the midi player structure in order to do this, ran out of time :-(

Peer Review of Journal (by Shashidhar Pai)

Journal activities are explicitly and clearly related to course deliverables

5 - Yes, activities are clearly expressed and on point with the expectations

deliverables are described and attributed in wiki

3 -  Deliverables are mentioned in the presentation of project and missing in the wiki or project proposal page

report states total amount of time

4 - Yes (total time to be updated in banner/heading)

total time is appropriate

5 - Yes, hours spent and work done is logged accurately

Key for each criterion:

5 == exceeds expectations

4 == meets expectations

3 == mostly solid with some gaps

2 == shows some progress

1 == some attempt

0 == not found