Zak Wegweiser journal

Week 1 - 10.5hrs

1/24, Watched two ARKit videos from Apple, which discussed the release of ARKit and the updated features. Also hinted at the direction of AR.

Video 1, WWDC 2017: https://developer.apple.com/videos/play/wwdc2017/602/

Video2, WWDC 2018: https://developer.apple.com/videos/play/wwdc2018/602/

(2.5 hours)

1/25, began toying around with my own ARKit app in Xcode. I was very slow, as it is hard for me to test, but I was able to get it to recognize a surface to place objects. (2.5 hours)

1/26, Installed and played with Paraview (1 hour)

1/27, Read over Brandon's notebook and setup my own. Reading through his notebook I found this article: https://online-journals.org/index.php/i-jet/article/view/7773/4759 about the Microsoft Kinect, which discussed the merits of using the Xbox Kinect as opposed to traditional DAQ systems in educational laboratories. Reminded me of my experiences with the Kinect and what turned me away from using it. (2 hours)

1/28, Read another article from Brandon's notebook. I actually read over Brandon's notes about the article more in depth as I found them to be much clearer and more concise.

Article: What's Real About Virtual Reality? Brandon's notes: What's Real About Virtual Reality? Notes (2 hours)

Week 2 - 10.5hrs

1/31-2/1: Visited yurt for the first time in class. I would love to dig deeper into the architechture of how it was built, how the projectors avoid the person inside, and how it is running a computer program. (1 hour)

1/31-2/1: Read an interesting article recommended by my HTA of a different class called "Gamified ICT Tools for Higher Education", which discusses the values and ethics of using games in general (as well specifically in VR) to influence education. I took notes, and then we had an informal debate about why games are useful, but can be ethically dangerous if they are not designed correctly. My notes can be found here: https://docs.google.com/document/d/1p233LPlf15u1F6yay7ueLP6TuGbzS-JPH29ccPBr-FY/edit?usp=sharing (1 hour reading, 1 hour writing, 1 hour debating = 3 hours)

2/1: Spent time looking for potential projects. Read over a few ideas on Professor Laidlaw's website and the visualization research lab site. Nothing jumped out at me too much, I really enjoyed the bloodflow project found here: http://vis.cs.brown.edu/areas/projects/artery.html. I feel like I am still potentially thinking about Augmented Reality, so I might explore other realms than the listed projects. (1.5 hours)

2/2: Downloaded Unity3D. Wasn't too sure what I was doing, but just played with the drag and drop interface. Will have to research more tutorials if I go down this route. (1 hour)

2/4: First Project Ideas:

  • 1. AR Glasses. Simple goal of building my own glasses that can run some sort of augmented/virtual reality software and display objects in front of your eyes. This seems like a stretch, but I have been researching glasses and it sounds very enticing. I was hoping someone else might have ideas https://www.lifewire.com/best-smart-glasses-4172796

  • 2. Unity3D game. I would love to designate a good portion of this class to learning Unity3D, even if my final project was completing a VR game I think that would be a solid accomplishment.

  • 3. ARKit in swift. I think it would be really valuable to learn ARKit, and make a real-use application utilizing augmented reality. I think this might be helpful for hearing-impaired individuals or aiding other disorders.

  • 4. Unreal Engine game. Like Unity3D, it would be nice to learn another software that is wildly used in the industry. I started some research at their website and it looks hard but feasible: https://www.unrealengine.com/en-US/what-is-unreal-engine-4

(2 hours)

2/4: Searched for various tutorials tailored towards VR development in UnrealEngine4. Seems like it may be too complex for me to learn on my own, and I may put it aside for now. Mostly read Epic Games tuts. (2 hours)

Week 3 - 11 hours

2/5 - Researched usability principles and explored how those principles can be applied to VR/AR with David and Gabi https://www.usability.gov/what-and-why/usability-evaluation.html (2 hours)

2/7 - Project Draft:

  • Title: ARDraw (Working title)

  • Description: Track user hands to draw in Augmented Reality. Almost like an augmented white board.

  • Pre-Activities: Get situated with ARKit. Make a simple app and read through the documentation to get a understanding of how the technology works.

  • Goal/Objective: Obtain a working understanding of ARKit in Swift. The actual product is a little less important to me than the ability to be comfortable with ARKit after this project.

  • Contributions: Make a product that can track real objects to create objects in AR. Possibly make a tutorial on how to make it.

  • Deliverable: Functioning ARKit App. I think it would be really cool to teach a virtual class room this way.

(1.5 hours)

2/9 - Project Research (1.5 hours)

2/10 - Research outside project. Professor Laidlaw recommended comparing ARKit to ARCore, so I thought it would be smart to look into ARCore (especially or iOS) before writing up my final project description: https://developers.google.com/ar/develop/unity/quickstart-ios (1.5 hours)

2/11 - FIRST PROJECT DESCRIPTION:

Title - ARDraw

Description: The goal of my project is to build an Augmented Reality app for the phone that lets people draw in AR. I believe we are heading towards a new revolution in technology where VR is ready to grow, and with this new significant pivot, it could eventually find its home market, allowing augmented reality app development to become extremely important. I feel like this will allow me to get a solid foundation for developing this rising technology. However, my end goal is to create a tutorial that I can add to the wiki so that anybody interested in learning can have the opportunity to have another student that has been through the struggles, lighten the process.

Contributions: my main contribution is to create a tutorial for the wiki for other students to learn how to develop augmented reality. My main focus is for students to learn ARKit, but if time permits, I will also make a comparison to ARCore and possibly other augmented reality libraries to help students gauge the differences. This can add a lot to the project, giving other students the opportunity to quickly compare two rising technologies in the wiki.

Prior Knowledge: I have a lot of experience with iOS development. However, I have never worked with ARKit. Thus, I think this is the perfect time to expand my current knowledge and learn a new important library: ARKit.

Milestones:

  • Week 1

    • Understand how to use ARKit:

    • Construct ARKit app that detects surfaces

    • Add objects to that surface

  • Week 2

    • Create app that can track your finger

      • This will include significant research into how this works. I know it can be done, but no idea how. ARKit might not be helpful in this aspect but other libraries might.

  • Week 3

    • Combine apps. Make ARKit detect finger and place basic object (such as black dot) where fingers are drawing.

    • Again, this may take significant more research for finger detection.

  • Week 4

    • Unsure how the current timeline is. If time permits, maybe attempt 3D drawing!?

    • Otherwise, play catch up on unexpected hurdles

  • Week 5

    • Start creating tutorial on how to make the app

    • Document process results. What was easy? What was challenging? What do you recommend for others? Put it all in the wiki!

  • Week 6

    • Experiment with ARCore.

    • Make comparisons from the app developed in ARKit to the possibility of developing in ARCore

Deliverable: My project will have three concrete deliverables. First, and foremost, a working ARKit app that uses finger detection to draw in space (even if its just on 2D surfaces). Second, a tutorial, for others, to add to the wiki. Hopefully, other students will learn from my experience and find this path helpful! And finally, a comparison to ARCore, especially in iOS. Professor Laidlaw made a great point about the importance of comparing technologies.

Risk: Every project contains a bit of risk. There is a chance I can't make this work. However, I remain optimistic that I will be able to create an augmented reality app that is useful for both myself and the rest of the class!

(2.5 hours)

2/11 - Read a couple VR articles from Science Daily. Quick reads. Fun!

https://www.sciencedaily.com/terms/virtual_reality.htm (1 hour)

Week 4 - 11 hours

2/12 - Attended AR/VR Facebook Talk at Macmillan (2 hours)

2/12 - PreProject Milestones:

  • Make list of biggest issues and questions

  • List of softwares to compare

  • Reach out to Lawrence Huang

(0.5 hour)

2/13 - Worked on GigapixelViewer images with David (1 hour)

2/14 - Viewed the GigaPixelViewer in the YURT! (1 hour)

2/14 - Talked to David about my project (1 hour)

2/14 - Pre-Project self-evaluation (0.5 hour):

  • The proposed project clearly identifies deliverable additions to our VR Software Wiki

    • 5 : Tutorial

  • The proposed project will inform future research, ie, advancing human knowledge

    • 4 : I'm hopeful that the tutorial and comparison will help people advance their knowledge about AR

  • The proposed project involves large data visualization along the lines of the "Data Types" wiki page and identifies the specific data and software that it will use

    • 1 : As of now, I am struggling to find a good way to vizualize datasets. Could use help on this front.

  • The proposed project has a realistic schedule with explicit and measurable milestones at least each week and mostly every class

    • 3.5 : I think there is a reasonable outline for milestones

  • The proposed project includes an in-class activity

    • 2 : Not really. We could all go through the tutorial or actual app itself?

  • The proposed project has resources available with sufficient documentation

    • 5 : LOTS of docs available

2/14 - Journal Self-Evaluation (0.5 hour):

Journal activities are explicitly and clearly related to course deliverables

5 - I think I have been doing well at finding engaging material related to the course.

deliverables are described and attributed in wiki

3 - I haven't been as attributing to the wiki, but I will make sure to change that

report states total amount of time

5 - Always keep track of time

total time is appropriate

4 - Sometimes I am a little slow, but I always keep track of my work

2/16 - Progress on first milestone: Construct ARKit app that detects surfaces (1.5 hours)

Week 5 - 11 hours

2/18 -Progress on the first milestone: Studied Apple's documentation and examples and was able to follow their examples to construct an app that detects surfaces and highlights it with what they call "a focus square". Learning a lot, lots more to go! Once I get models to be placed down I will add progress to the WIKI! (2.5 hours)

2/19 - After project feedback from Professor Laidlaw, I found that I might have to shift my project a little bit. My milestones still stand for the first week, getting objects to display in AR. However, I did some research on how I can visualize planet and data and came across this Udemy course: https://www.udemy.com/ar-solarsystem/

Starting playing around with the material in the course. It's a bit long but it looks really cool and has some nice guidelines! Added it to the WIKI under AR Development Software for others to try out as well! (2 hours)

UPDATE: Gave class presentation and decided to stick with original project idea!

2/20 - Completed first Milestone! Constructed ARKit app that detects surfaces, add objects to that surface, and be able to manipulate those objects with UIGestures.

Wiki Link: https://sites.google.com/s/1x3I4pruD53DlbdCWJoTIDtM_JW5SouHp/p/1JgSBph7ZOUo9WNMe6PRzlgB2ifCPw0qu/edit

GitHub Link: https://github.com/zweg25/ARExample

(2.5 hours)

2/21 - I realized after my presentation my next biggest hurdle is finger tracking. So I started researching different vision APIs. Nothing solidified yet, but just got to see what's out there (1 hour).

2/22 - Learned about Vision: a high-level framework that provides an easy to use API for handling many computer vision tasks. Watched this hour WWDC session in depth, coding along and made substantial progress. Posting the video in the WIKI for others to see, but you can also check it out here: (2 hours)

Week 6 - 11.5 hours

2/26 - Completed Second Mile Stone of finger tracking! Took way more research than I anticipated, but works well! Check out the video: (5 hours)


2/26 - In class was assigned to make tutorial. Starting research what to make and how (1 hour)

2/28 - Researched tutorials in the wiki. Found good starting ARKit tutorial HERE! Made good notes and mapped out what I wanted to do for my tutorial. (1.5 hours)

2/29 - Made fully fledged beginner ARKit tutorial on the WIKI: https://sites.google.com/view/brown-vr-sw-review-2018/related-technology/tutorials/arkit/first-arkit-app?authuser=0! (4 hours)

Week 7 - 10.5 hours

3/5 - Explored ParaView in class. Used the supernova dataset, which I describe further in the WIKI. Viewed in the Vive and the YURT! (2 hours)

3/7 - Loudon demoed Spacial Audio in VR. Very interesting. Smoothe for the most part, however audio beneath a user seemed flawed (1 hour)

3/8 - Back to working on my project. Hitting a small hiccup on placing objects in intervals to look like a smooth pen stroke. Been researching with some videos, finally found a path that looks promising (2 hours)

3/10 - FINALLY GOT IT! WHOO! Code is a little hacky and messed together, but was able to make smooth drawing in AR! Check out the demo video! (4.5 hours)

Week 8 - 10 Hours

3/11 - Ran through my own tutorial (https://sites.google.com/view/brown-vr-sw-review-2018/related-technology/tutorials/arkit/first-arkit-app?authuser=0) to make sure it was ready for class. Also developed a form for feedback for afterwards. I hope it runs smoothly! (1 hour)

3/12 - Held class tutorial! Excited about all the feedback! (1 hour)

3/12 - Read through all the feedback! So helpful for my upcoming comparison. Will do more research, but great starting place (1.5 hours)

3/13 - Began researching 3D drawing. There's a lot of interesting talks on this subject. Mostly in Virtual Reality as opposed to Augmented Reality, but stilll very interesting. Not too many leads on how to progress, but I have some ideas.. Seems like a stretch goal, but I think its good to push myself for it! (1.5 hours).

3/15 - Spent almost 3 hours trying to get 3D drawing to work. Get a lot of crashes. Had to take a break and cool off. Might have to go a different route. Will give it another shot in a few days, maybe something will come up (3 hours).

3/16 + 3/17 - Watched Jeff Huang's PHD student video. Amazing! Started experimenting with some of it. Obviously outside the scope of my programming knowledge, but quite amazing! Started making brief comparisons to my project. Might end up changing my comparison to this project. Will continue to explore both routes. (2 hours)

Week 9 - (SPRING BREAK!) - 11 hours

3/19 - VTK.js and React360 tutorials. Honestly, not for me, but cool to learn different perspectives (2 hours).

3/21 - WHAT HAVE I ADDED TO THE WIKI?

3/25 - Completed project and made final video! (4 hours)

3/27 - Wrote final review! (4 hours)

3/28 - Completed presentation for first project and started thinking about second project (1 hour)

Week 10 - New Project! - 8.5 hours (First week back from spring break)

4/1 - Met with Jing to discuss research opportunities for new project. Both of use are excited. I think I am going to pose the research question that compares ARDrawing with his project to 3D drawing in Unity. (We met for about 1.5 hours).

4/2 - PRESENTED MY FIRST FINAL PROJECT! 1 DOWN 1 TO GO! (1 hour)

4/3 - Looked into AR case studies. Found a cool article. https://dzone.com/articles/arkit-development-case-studies-augmented-reality-i (1 hour)

4/4 - Helped David prepare for his presentation ! (1 hour)

4/8 - Met with Jing about goals for project. (1.5 hours)

4/8 - Played around with his project to get myself situated. (1.5 hours).

Week 11 - 10 hours

4/12 - Scheduled out rest of semester (1 hour)

4/13 - Set up Oculus with Brandon. Ran through install, setup, tutorial in about an hour or two. Then played with Google Tilt Brush to begin comparison to AR drawing software. General consensus, VR is so much better of an experience. Thus, need to change comparison to make it more interesting. Overall spent 5 hours working on project (5 hours)

4/13 - Played with dprom's audio visualization tool in Google tiltbrush. Learned enough to help teach him and help with tutorials (2 hours)

4/14 - Began comparison. Talked with Jing about evening out comparison. I think its mostly about "overall experience" since VR introduces a whole new world. Could make people uncomfortable and even dizzy. (2 hours)

Week 12 - 10 hours

4/18 - Performed an hour with both Jing's project and Tiltbrush to formulate my own opinion on the tools and noted brief thoughts throughout (3 hours)

4/20 - Set up tutorial with Dprom for class. Each student will use tiltbrush and respond on a feedback form I made (2.5 hours).

4/21 - Set up presentation and tutorial for class (1.5 hours)

Week 13 - 10.5 hours

4/23 - Ran tutorial with Dprom in class (1.5 hours)

4/24 - Went through Tutorial's feedback and began outlining user study comparison (3 hours)

Some of the results displayed below:

4/25 - Began writing comparison (2 hours)

4/26 - Invited a couple friends to try Jing's project and Google tiltbrush! (3 hours)

Week 14 - Last Week! (12 hours)

4/28 - Finished personal comparison fully published on WIKI: AR vs. VR (2 hours)

4/29 - Finalizing write up from all the feedback found in class. Here are some of the "gem" quotes I've been basing my comparison on:

  • HOW DO YOU FEEL ABOUT THE VR EXPERIENCE VS AR EXPERIENCE?

    • "I prefer VR, since there is still a big disconnect between what looks real in AR vs what is actually real"

    • "I like VR with a big field of view better than AR on a tiny phone screen. I have done some AR with a bigger display (like hololens), but VR feels much more ready-to-use."

  • WHAT DO YOU LIKE AND DISLIKE ABOUT VR?

    • “I like how you’re immersed in a completely different world, but I dislike how you can’t move properly in VR"

    • "I like how immersive it feels, dislike how hard to access it is."

  • WHAT DO YOU LIKE AND DISLIKE ABOUT AR?

    • "like: still being in the real world dislike: tiny display space, not so great implementations (laggy, etc.)"

    • "More comfortable because I am still in the real world, but it is too low res right now."

Compiling all this information from class and friends' surveys to make my comparison (3.5 hours)

4/30 - Prepared final presentation for Thursday! (2.5 hours)

5/1 - Finished comparison write up! Need to go over it and put it on the WIKI! (3 hours)

5/2 - Did final presentation for class! (1 hour)

WHAT HAVE I ADDED TO THE WIKI?

All of this can be found in the navigation on the wiki!

Project 1

Project 2

Reviewed by Loudon Cohen, David Promisel