H+-+Sunset+2+-+AR+You+Experienced+(Apple)

Session convened and presented by Victor LaBozzetta < vlabozzetta@apple.com> Notes recorded by Akbar Ali Herndon  (please add, correct and enhance)

Victor Introduced himself Polled how many use VR goggles of any sort? How many are using AR? (Not many).

An emerging #|technology. Used to be cost prohibitive. Used the “Virtual Vatican” - an early approach to VR in a previous era.

What is AR or VR?

VR - Everything you perceive and interact with will be in those goggles. Device tracks where you are. Totally immersive. Drawback, only one student can use it at a time.

AR - Blended reality. Interactive with world around. Uses a device (cell) with camera. Device is window to augmented world. Glasses that will show a mixture of worlds. Can be distributed and uses a device they already have.

Advantages and disadvantages to both experiences.

We (Apple) just released new version of final Cut Pro which now. allows 360 camera input - feed goes directly into FCP as single track. Dashwood stitches the combination of camera views. Old Final Cut can be upgraded to new version.

[Chart shown]. By the time current students graduate, VR/AR is projected to be a $200 billion market.

Why do it? We want to enhance options. Does not replace good teaching. Example: I am afraid of falling. I am not willing to zip on a zip line. Last summer I zipped over a volcano virtually.

VR dissection of human heart is great and practical, allowing medical school practice without risk. Apps have been released showing how to work on a breaker box with AR. We want to give experiences that made safe but cannot be practiced in real life safely. No major difference between various AR training systems.

Measurement app (Plnar) Extremely accurate. Similar to IKEA app. Shows how to measure a complicated shaped space.

What are the functional pieces of AR?


 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">#|Tracking **
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Movement of the device in space
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Uses multiple sensors to figure out location and precise device orientation
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Visual inertial odometry
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">AVFoundation and Core Motion

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">[Video WWDC2017 world tracking shown]

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Choose an object, close one eye and then the other over and over. 300 times per second triangulated demonstrates what is happening in AR system

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Creates a three dimensional grid <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Light estimation
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Scene Understanding **
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Recognizing features in environment **
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Particular planes
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Adds anchors used as reference points


 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Rendering **

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">ARKit supports
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Scenekit
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Spritekit
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Metal2
 * <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Unreal and real unity

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">#|Samples of AR <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">IronmanHUD - shoots lasers

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">(Interesting question: What if we could use GPS and WIFI positioning?)

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Inkhunter <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Draws registration mark on his arm. (Smiley face). Matches tracking symbol. Overlays tatoo to same location. 2D objects.

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Transparent Earth <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Shows what’s on the opposite side of the earth

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Jigspace <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Adds an overlay of a selected object <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Lock and key used as example chosen from a library of objects or uploaded objects created in #|sketchup <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Showed inner working of lock <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Lunar module model <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Atomic bomb

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">How to create an AR app for yourself:

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Using Swif tplayground <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">There’s a AR template. Go through early swift coding so that you can navigate and use variables. <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Could create a sketchup object and #|upload to swift file or use sift object

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Did not work (due to recent update)

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Switched to XCode.

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Created new project <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Use AR template

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Can create an APP in android, python, chromebook

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">App store restricts use of Apple assets e.g., logo, G-rating, and questions is this code “borrowed” (can be checked because it’s tagged with Apple ID). Files copied will not be accepted by App store.

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Template fills in the 4 required features mention in AR scene kit. Cannot be tested on a simulator. Requires actual computer environment to run.

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Using iPhone. Brings jet to AR overlay of room.

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Required steps to create a AR overlay in XCode: <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Must be on a Mac <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Go to https://developer.apple.com/ <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Download xcode <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Your Apple ID will be assigned developer status <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Choose AR template <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Choose an object <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Launch it

<span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">List of available AR or VR apps <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Plnar <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">IKEA <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">IronmanHUD <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">VRtube <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Skyview <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Nightsky <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">anatomyARVR <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">elements4D <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Haircolor <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">Mazewalk <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">VRmovies <span style="font-family: &#39;Times New Roman&#39;; font-size: 11pt; vertical-align: baseline;">paintspaceAR

Victor from Apple Scott Adamson sadamson@friendsseminary.org Final Cut updated with AR Function (Hero 5 in cage). Stitching built into the Final Cut Final Cut update vs new purchase 6 Cameras can pull the images together (vs. 15+ of days of past) Plug in Headset into iMac (what kind of headset) HTC Vive? (Oculus does not seem to #|work yet)

[]

AR is blended reality (interaction with environment) Enhance the environment NOT replace good teaching environment

#|Tracking (iPhone/iPad can do x/y/z) Video from WWDC https://developer.apple.com/videos/play/wwdc2017/102/ Mathematic calculating the environment

Apple xCode Frameworks for AR and other Android/Windows etc AR Template-Crossplatform IOS Template (AR Frameword) Mac, xCode Download, AppleID builds developer account .dae file .png texture file

Apple PlayGround [|__https://www.apple.com/swift/playgrounds/__] Sketch Up Models can be brought into PlayGround

LEGO Studio (#|app) LEGO® AR Studio by LEGO System A/S [|__https://itunes.apple.com/us/app/lego-ar-studio/id1296734986?mt=8__] [|__https://www.lego.com/en-us/aboutus/news-room/2017/december/lego-ar-studio__] Planar (app) PLNAR by Smart Picture #|Tech [|__https://itunes.apple.com/us/app/plnar/id1282049921?mt=8__] Google Expedition [|__https://edu.google.com/expeditions/__] INKHUNTER Try Tattoo Designs by Kateryna Khotkevych [|__https://itunes.apple.com/us/app/inkhunter-try-tattoo-designs/id991558368?mt=8__] Transparent Earth -See the other side of the #|earth by Hogere [|__https://itunes.apple.com/us/app/transparent-earth-see-the-other-side-of-the-earth/id342988344?mt=8__] JigSpace by JigSpace Inc. [|__https://itunes.apple.com/us/app/jigspace/id1111193492?mt=8__] Sketch up to build a model and put in Jig Space Pokémon GO by Niantic, Inc. [|__https://itunes.apple.com/us/app/pok%C3%A9mon-go/id1094591345?mt=8__] Exploration (walking through real-world)