Augmented Reality

Joel Gethin Lewis

Session 1: Introduction

What I'm going to talk about today:

  1. Introduce myself.
  2. The aim of this course.
  3. What is Virtual Reality? What is Augmented Reality?
  4. How can one experience VR or AR?
  5. Why web based AR is the way to go.
  6. The current web based AR ecosystem and demonstrations.
  7. Examples of Augmented Reality done right.
  8. Paper prototyping.

1. Introduce myself.

2. The aim of this course.

  • I am going to teach you about some technologies, but only to allow you to think practically and creatively about AR.
  • Just creative thinking or just practical thinking is boring. The intersection is interesting.
  • No stupid questions, only stupid answers. So please ask in class or via email.

3. What is Virtual Reality? What is Augmented Reality?

  • Virtual Reality (VR) is a technology that uses headsets and other devices to replace one or more human senses with artificial ones.
  • Augmented Reality (AR) is a technology that uses headsets and other devices to augment one or more human senses with artificial ones.
  • Other terms include Mixed Reality (MR, or the continuum between VR and AR) and Extended or Cross Reality (XR, which encompasses all perceptions).

4. How can one experience VR or AR?

  1. Window on the world.
  2. VR headsets.
  3. AR headsets.

4. How can one experience VR or AR? Window on the world

  • The cheapest, simplest most direct method.
  • Works for both VR and AR.
  • Simply hold your smart phone or tablet and pan it around to explore the VR or AR world.
  • Both Apple and Google have released platforms for this functionality on iOS and Android (ARKit and ARCore).

4. How can one experience VR or AR? Window on the world

  • ARCore Demo:

4. How can one experience VR or AR? Window on the world

  • ARKit Demo:

4. How can one experience VR or AR? VR headsets

  • The most widespread method after window on the world.
  • Four main desktop platforms, but many others available: Vive (HTC and Valve), Rift (Oculus and Facebook), Playstation VR (Sony) and Windows VR (Microsoft).
  • Three main mobile platforms: Cardboard (Google backed, but works on iOS too), Gear (Samsung and Occulus) and Daydream (Google).

4. How can one experience VR or AR? VR headsets

  • Job Simulator Demo (available on Vive, Rift and Playstation VR).

4. How can one experience VR or AR? VR headsets

  • Lone Echo Demo (available on Rift).

4. How can one experience VR or AR? AR headsets

  • The most hyped method.
  • Two main platforms, but many others rumoured (Apple particularly): Hololens (Microsoft) and Leap One (Magic Leap).
  • Hololens has been available since 2016 to developers only, Leap One is shipping this year. Magic Leap is currently valued over $4.5 billion with over $1.8 billion invested to date (including Google, Alibaba and Warner Bros).

4. How can one experience VR or AR? AR headsets

  • Hololens Demo.

4. How can one experience VR or AR? AR headsets

  • Magic Leap Demo.

5. Why web based AR is the way to go.

  • Future proof - compatible with every major platform, open source and non proprietary, no lock in.
  • Widest possible audience - every recent smart phone and tablet, deployable to the entire world and accessible.
  • Cost effective - development uses open source tools, has the biggest community of developers and doesn't require purchase of new hardware.
  • Measurable - using existing web based analytical tools.

6. The current web based AR ecosystem.

  • A-Frame. Supported by Mozilla. Mainly VR.
  • AR.js. Developed by Jerome Etienne and others. Not really AR, more marker tracking.
  • three.ar.js. Developed by Google, built atop three.js (Ricardo Cabello aka Mr. Doob and others) ARKit (Apple) and ARCore (Google). Demos.
  • three.xr.js. Developed by Mozilla, built atop three.js, ARKit (Apple) and ARCore (Google). Demos.
  • aframe-xr. Developed by Mozilla, built atop three.xr.js and A-Frame. Demos.
  • aframe-ar. Developed by chenzlabs, built atop three.ar.js and A-Frame. Demos.
  • I ended up using A-Frame-XR, your mileage may vary. You'll need XRViewer (custom browser from Mozilla) to view demos on your device.

7. Web based demonstrations.

  1. A-Frame on Cardboard and iOS.
  2. A-Frame on Daydream.
  3. AR.js on iPhone.
  4. WebXR on Android.
  5. WebXR viewer on iOS.

8. Examples of Augmented Reality done right.

9. Paper prototyping.

Paper Prototyping lets you ignore budget, ability and experience to get to the core of an idea quickly.

N.B. Technology is NOT an idea.

I want to teach you my method:

  1. Find an interesting technology/method/approach/context.
  2. Apply it to an experience in the real world.
  3. Get the money/ability to make it happen.

2. is what you are going to learn today

I'll find you a technology and I'm afraid I don't have any budget. (-;

  1. Find an interesting technology/method/approach/context.
  2. Apply it to an experience in the real world.
  3. Get the money/ability to make it happen.

I want elevator pitches!

Technology usually flows "downhill":

Military -> Commercial -> Art -> The street.

This is changing - see the Singularity.

You are reality hackers!

Paper prototyping time!

Brief 1: Smile

and the whole world smiles with you...

Auto Smiley

Project: Auto Smiley by Theo Watson

The Brief:

You can detect smiles. What are you going to do with that?

Where could a smile be a useful switch?
What if many smiles were collected at once?

Top tips:

  1. Think of the setting
  2. Sum it up in a sentence
  3. Act the story
  4. Write down all your ideas first - but only pick one idea - be an editor
  5. Only one minute to present!

Demonstration

The Magic Hat.

Go!

15 minutes thinking time, 1 minute presentation.

Brief 2: Drones

I for one, welcome our robot overlords.

Nanocopters

Experiments performed with a team of nanocopters at the GRASP Lab, University of Pennsylvania.

Drones for Performance

Marshmallow Laser Feast installation for Advertising Director festival. Cannes, France, 2012.

Drones for delivery

Amazon Prime Air

Boston Dynamics Big Dog

The new sound of terror?

Boston Dynamics Handle

With guns?

Boston Dynamics Spot Mini

With a hand!

What kind of drone would you like?

Go!

15 minutes thinking time, one minute presentation.

Brief 3: Human Hacking

How would you redesign yourself?

What if you had a tail?

New balance. Photo by Tethus925.

What if could change your sex in real time?

Male cuttlefish (right) courting a female (left), he displays a male pattern facing the female, and a female pattern facing away. Photo by Culum Brown.

What if could change your sex in real time? Part 2

What if you didn't have to eat any more?

Photosynthesis! See Cassiopeia. Photo by Chris Hind.

What if you could see directions like birds can?

Augmented reality in nature! See Cryptochrome and Magnetic Sensing Research at University of Illinois at Urbana-Champaign.

You can splice qualities of other organisms onto yourself.

What quality?

Why?

Go!

15 minutes thinking time, one minute presentation.

Brief 4: Augmented Labels

  1. How could you do the job of labelling in the museum using AR?
  2. How would this change the museum?
  3. What content would the labels have? Who would make it?

Go!

15 minutes thinking time, one minute presentation.

Brief 5: Collection objects in the museum.

  1. How you could you display the collection in a museum using AR?
  2. How would this change the museum?
  3. How would visitors interact with the collection? Curators? Researchers?

Go!

15 minutes thinking time, one minute presentation.

Brief 6: Collection objects in the city.

  1. How you could you display the collection outside of the museum using AR?
  2. How would this change the museum?
  3. How would visitors interact with the collection? Curators? Researchers?

Go!

15 minutes thinking time, one minute presentation.

Discussion and wrap up.

  1. Is this interesting?
  2. Were the methods interesting?
  3. What do you all think?

Thanks!