In the past few years the terms Virtual Reality and Augmented Reality have become common talk among the general public yet there are still many issues making widespread use of the technologies in the AEC sector still challenging. I’d like here to try and put some structure and definitions to make it clear to all who want to take an active part in the dialogue and understand what is it that we are getting all excited about.
First some definitions
the computer-generated simulation of a three-dimensional image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.
a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view.
In a nutshell when we are talking about Virtual Reality we are typically talking about simulating an entire environment, whereas in augmented reality we are superimposing a limited subset of the virtual on top of or within the existing. So what challenges do we face when trying to implement these ‘reality’ solutions on our projects?
Let’s take a look first at some of the aspects of building a Virtual Reality solution. The big challenge here for a typical AEC project is not the 3D physical description of the proposed designs, which are rather well defined in today’s 3D based design software, but rather a good enough and practical simulation of the environment which provides the context for the project in question. Generally when we build a virtual reality environment we look at the relevant 3D context of the terrain, vegetation and built environment which provides the backdrop for the proposed construction project. So modeling our Virtual Reality solution must budget the effort in acquiring and building the context of the proposed design. Typically we use digital elevation and/or digital terrain models to describe the heights and shape of the landscape. As long as the elevation data is good enough (and often even if it isn’t) we can overlay or texture the terrain model with ortho-rectified aerial images which are easily obtained today for projects in high resolution on the order of centimeters per pixel. That alone will provide a fairly reasonable VR environment for a flight simulator type of experience as long as you don’t get too low.
To provide a better low-level experience we always try to include vegetation simply by placing 3D or 2D trees on top of the terrain models. Lastly we identify and model to whatever level of detail is appropriate for our goals elements from the built environment which add to the 3D experience, such as existing buildings, and other man-made objects which are not captured in the terrain model. Once we have the entire environment defined we then simply display it in real time on our preferred viewing hardware. A computer screen whether a desktop or laptop, VR goggles with motion sensing or even a smartphone or tablet all are reasonable candidates. The advantage of this approach is that the entire visual environment is available and defined in 3D and as long as the viewing engine can handle the complexity and refresh rates the user can be convincingly immersed in the synthetic 3D environment. The down side is typically found in the considerable effort needed in creating the 3D environment to an acceptable level of realism.
Augmented reality solutions on the other hand do not necessarily require modeling any of the existing environment as that is provided for free in real time through the camera lens of the device being used which in many cases is as simple as a standard smartphone. The challenge here is to provide the required 3D design information and be able to successfully orient the 3D model on the screen in sync with the natural movements of the camera and blend the two into a believable visual experience. This technology is still very much in its developing stages and there are many hurdles still to be crossed.
There are at least two big issues which need to be addressed in order for the solution to be satisfactory for use in design visualization for AEC projects. The first issue is placing the proposed 3D model in the proper orientation and scale with respect to the existing 3D environment. One approach is to have an accurate understanding of the geographical location on the face of the earth through the device’s GPS and then using the sensor information for the accelerometer and gyroscope to provide the orientation of the smartphone to generate an overlay image of the proposed 3D model on the pixels captured through the lens of the camera. This needs to be continuously updated and refreshed at around 30 times per second for the experience to seem life-like. A major drawback of the approach just mentioned is that while the location accuracy through GPS technology today is getting better and better with solutions already in the meter to sub-meter range, even these small errors can lead to lackluster end results.
Over the past few years a genre of solutions which have relied on some type of visual marker have been popular. In this case the device visually scans some known visual pattern (such as a dollar bill for instance ) which is placed into the environment and is replaced in the augmented picture with a 3D model which moves and rotates in the view based on the orientation of the dollar bill. Obviously a robust solution can’t be predicated on being able to place dollar bills everywhere you want to see your proposed design – it could end up very costly. A different approach to solving the position and orientation problem which has a rudimentary solution in the recently popular Pokemon GO game is for the smart device to understand its 3D environment by analyzing the video feed and ‘understanding’ the 3D topography. Certainly this game is turning heads and gives us a glimpse of what the future holds for A/R technology but still has many challenges to overcome.
The second issue which makes A/R so challenging after the question of pose and orientation are solved is how do you composite the two images (the real life camera, and the 3D design) in such a way that the final result is believable. Today most A/R solutions can easily overlay the 3D model over the pixels from the camera, but still is very difficult to determine on video alone is the pixel from the camera should be behind of or in front of the 3D model. To help solve that problem a new generation of smartphone technologies are in development where the smart phone can not only identify a pixel in the image but can as well calculate the distance from the camera to the pixel. I have already seen phones that can do this for close range (a few meters) with a high sense of accuracy.
Over time as these techniques are refined I would expect we will see a shift from Virtual reality solutions in the AEC market to Augmented Reality solutions which will provide value to 3D design models at every stage of the project life-cycle.