WebAR: A Framework for Mobile Augmented Reality is my master's project from Georgia Tech. It's a javascript library for initiating Augmented Reality (AR) experience directly on the mobile web without downloading any plugin or native application.

The story begins with a technical report about Digital Life in 2025 from Pew Research Center. It gathers over 2,500 experts and technology builders and ask them to predict the digital life in 2025. Overall there are four points most of them agreed on, one of which is that there will be "Augmented Reality enhancements to the real-world input that people perceive through the use of portable/wearable/implantable technologies." If AR wants to be pervasive instead of gimmicks, I personally think web is the most ideal platform to let people experience immserive AR and it can create convincing use cases.

The graph shows a trend in web media. In media 1.0, users perceive whatever content provider gives them which forms only one-way communication. Later, technologies make web applications such as Facebook, Twitter, and Flickr became possible. In media 2.0, users began to "move forward" and contributed user-generated content such as comments, images, or videos. During this phase, two-way communication makes the involvement higher. In media 3.0, which I believe is happening right now, users can jump into the digital world. By jumping it means being temporarily immersed in the context. It reminds me of acquisition of Oculus Drift by Facebook for 2 billion. Why would a social networking website want to buy a company that is making virtual reality device? I think because they see the potential of immersive web context. Both virtual reality and augmented reality can bring engaging experience to web audience.

Currently, there are over a dozen of AR browsers and over a hundred of AR applications. However, they are used once or twice only becuase after the first "wow" experience, they don't really want to use that applcaition or browser again. For developers, it's relative inefficient to create cross-plaform experience because of various proprietary SDK based on each mobile platforms. WebAR, on the other hand, provides a web-based approach. Initiating AR experiences can be as simple as a click on the web. The goal of WebAR is to create a new medium on the web which augments web's capability of sensing physical environment. Normal website doesn't preceive physical movement, but with WebAR, a web can understand how users physically interact with their devices. It means WebAR can transform environmental information (such as device orientation or live camera view) as digital input into the web. The web can based on these inputs and generates digital experience correspondently.

It supports location-based tracking (GPS and DeviceOrientation API), marker-based tracking (JSARToolkit), and 3D content (Three.JS). Here are some of my implementations using WebAR.

It could be embedded as an element just like image or youtube video. It's suitable for preview online purchasing, for example, buying a LV handbag and gets the idea of its actual size.

It could an entertaining experience as well. Experince the same effects in a movie.

As a remote monitoring tool for construction that doesn't need to be on the spot.

WebAR's integration with Argon2 so that it can run on web browsers on Android as well.

Marker tracking performance test: static 3D model, animated 3D model, occlusion effect, and physics simulation.

Demo for a full-capability web app using public block-building installation as an example


This is the website with selected chapters from my design document for WebAR

DEVELOPMENT SCALE
Individual
SKILLS USED
WebGL using Three.JS
Node.JS
Socket.io
HTML5 native APIs
JSARToolkit