Start coding “Augmented Reality”

You don’t need one of those expensive Virtual Reality (VR) headsets to learn how to write “Augmented Reality” (AR) programs. All you need is a mobile device that has a built-in camera and a few built-in sensors. And most of the smart phones today (iphone, android phones, etc.) are equipped to handle the basic AR.

In this post, I will share a quick summary of how I started to learn AR and I am still very early in the journey. So a disclaimer first, some of the conclusions and comparison among different AR frameworks might not be correct! In that case, please leave me a comment and I will correct them.

Starting Point

What is AR? According to the Oxford dictionary, it is “A technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view”.

The first thing to learn anything is to do an internet search and read a few technology summary articles. One of the first I read was this. This list of AR frameworks mentioned in this article is no way close to being comprehensive. Especially since it was written at the end of Y2017, a lot of advances have been made since then in the area of AR. In addition, it didn’t cover some of the web based AR frameworks (such as AR.js with AFrame).

Hands-on Practice

The second thing to learn anything is to have an easy-to-follow hands-on exercise. The first AR example I follow is this article “Creating Web-Based Augmented Reality With Just 10 Lines of HTML Code (For Beginners — AR.js)”. You can experience a simple “marker” based AR with either no code at all (using their hosted link) or with literally 10 lines of code. Of course, I expect this simple example will lead you to read more about AR.js, AFrame, three.js, babylon.js, WebAR. But this example is quite self-explanatory even with no prior knowledge of these frameworks. If you are interested in WebAR / WebXR experimental framework, this post also provides a good tutorial.

Which AR Framework to Choose?

After the initial fun and thrill of hands-on AR, now we need to think seriously about which framework to use for development? There are so many choices! Web based or Native App based? Open Source or Commercial?

I am always a big fan of web. So at first, my natural inclination was to use AR.js with Aframe. Even though the basic AR functionalities worked well and pretty easy to learn, I had encountered a certain amount of trouble to integrate it as part of the Angular framework as AR.js is as of now not a very well encapsulated package/component. In addition, AR.js is mostly “marker” based and it doesn’t have built-in implementation of plane detection of the real world.

We might still get back to Web based AR. But for now, we started to look into native app based AR. For that, the obvious choice is using Unity as the development platform. There we again had a choice of the Vueforia Engine (developed by PTC) versus the ARFoundation (developed by Unity). Both can be used within the Unity Editor. Both can be deployed on many different devices and can dynamically rely on ARCore (by Google) on Android devices and ARKit (by Apple) on iOS devices.

Vueforia has a certain amount of built-in AR functionalities to have the choice of not relying on ARCore/ARKit. As a result, Vueforia based AR app can be used on some devices that ARCore/ARKit don’t support (e.g. Huawei Honor 8). However, Vueforia doesn’t have the full functionalities of vertical and horization plane detection (it only has a basic ground detection) comparing to ARFoundation. ARFoundation based app on the other hand relies on ARCore and ARKit and doesn’t even load at all if the device is not one of the ARCore/ARKit supported devices.

In addition to functionalities, of course, there are also other factors in choosing the right framework, e.g. ease of development, pricing.


It is very easy to get started with AR. You can start today! There are many available frameworks to use. Depends on the specific requirements of your use case, you can choose what’s the best for you. For my specific use case, it seems so far Unity/ARFramework (on top of ARCore/ARKit) is a winner. But the landscape of AR seems to be changing very quickly, I will definitely keep an eye on the pure web based AR and hope it will mature soon.

beginning a new journey by launching my own startup in the middle of the pandemic. I am passionate about sharing my learning.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store