Below is my interview with David Menard. David Menard is a product manager at Unity, working on Reflect, an AEC tool for generating real-time 3D visualizations of BIM data. David was kind enough to discuss the potential for AR/VR in the construction world, and the future of AEC design tools. Note: David was speaking for himself, and not representing Unity.

Brian: To start, do you want to talk a little bit about your background, and how you ended up at Unity?

David: Yeah, absolutely. So my background is actually in software engineering, I went to school for software engineering and, like most people back then, I wanted to make video games. It's kind of classic in Montreal, where I come from. So I did exactly that, I went into video games for a few years, as a programmer. But the games industry can be rough and overworked.

So I decided to take a job at Autodesk, and I found that real-time 3D video game technology was starting to be used in every other industry. When I joined Autodesk, my project was to bring this new technology into AEC [1], mainly for virtual reality purposes. After a few years, the opportunity showed up at Unity to lead this new product called Reflect. It was barely a thing back then, and I just jumped on it.

Reflect and AR/VR in Construction

Brian: For those who don’t know, can you give a high-level overview of what Reflect is, and what it can do?

David: Yeah. We kind of bounced between a few ideas when we first started, and where we landed was that it's very hard to bring Revit [2] data or AEC data into a real-time environment, because the data's just not formatted in the right way. There's a lot of manual work to be done. And when you try to do that process, you lose a ton of that data. That's where the idea for Reflect came from - let's just find a better way to make that workflow work.

As it evolved, we realized that most people don't want to just bring in the data for one project, but they want to be able to repeat that workflow on every single project that they have. So Reflect eventually evolved into this development platform that allows you to create these apps for repeatable results, not only one-off projects so that you can scale it, not just to the biggest projects, but also to the smallest ones at the push of a button.

Brian: When you say real time data, can you talk a little bit about what you mean by that, and the kind of the things that allows you to do?

From what I can see, there's a lot of neat stuff where Reflect basically allows you to take a Revit model and turn it into something that you can navigate the way you would walk around in a video game. And you can then plug that into a piece of AR or VR hardware, so instead of navigating with a controller, you're projecting it in front of you, you’re physically walking around the building. Can you maybe talk a little bit more about that?

David: The big difference between something like Revit and real-time 3D is in real-time 3D, the rendering happens so fast that it looks like an interactive world. So things can move, you can interact with objects, open doors, etc. This happens 60 times a second, 90 times a second, so fast that it looks fluid. The first applications of real-time 3D in AEC were to enable virtual reality, because you need this dynamic nature in the world. You can't pre-render everything in virtual reality, you need this real-time aspect.

As the technology got better at handling larger models and doing more complex things, new technologies like augmented reality came along, and the use cases for real-time 3D kind of grew proportionally. People are now using real-time 3D to do design review on their desktop. It looks better, it feels better, it's more fluid and more interactive.

They’re also bringing that technology into coordination because you can interactively measure distances, see if there's clashes, see if there's enough clearance for your machines. And you're seeing that translate all the way to the construction site where augmented reality can be used to help guide your construction or verify that the as-built matches the as-designed.

Brian: Can you talk a little bit about some of the things you can kind of do with Reflect in the AR and VR world currently, and maybe where you see those capabilities going?

David: Like I mentioned, the first use case for virtual reality is typically in design review. You'll have architects who want to present to their clients to try and get a feel for the space, if it fits their typical workflows. One example that we always see is you have a doctor, do they have enough reach or enough space to move around?

Where we're seeing a lot more traction in virtual reality is mostly in training, before you go on the construction site, can you spot hazards, can you spot potential safety issues that will be on site that can actually affect your wellbeing? You might save someone from breaking a leg or falling off a building. We're seeing so much more of these training scenarios in virtual reality. That is super exciting.

Brian: I wouldn't have considered that, but it makes sense.

Thinking about the future, the process of design now, it's just a series of paper drawings. You have to take your design intent and put it onto 2D plans, and then somebody else has to take your 2D intent and build it back into 3D. It's this really noisy process of translation.

You can imagine how in AR it would be so much smoother, where every single worker on site has AR glasses, and they can just see the entire building projected just in front of them. They don't have to try to figure out where the beam needs to go, it'll just be shown in front of them and then they can put the beam into place, and it will flash green when it’s aligned. Right, something like that. Just eliminating all that friction that's in that process and is responsible for so many errors.

David: One of the barriers there is that most firms, the deliverables are still 2D plans. Some firms are trying or are starting to have actual BIM models as a deliverable, but they're still fairly rare. So at the end of the day, people need to produce those 2D plans. What we're trying to bring on top of that with Reflect and what Autodesk tried to do with Revit and what everyone's trying to do is add a layer of data richness to that. So not only do you deliver to the plans just to meet the standard, but deliver a Reflect project or a 3D mock-up or anything to the construction company so that they can actually deploy this onsite as well. And that's the real challenge, to break that barrier.

Brian: For VR and AR, what kinds of things do you see being unlocked as hardware gets better?

David: Right now, for AR you have two options. You have the HoloLens that sits on your head, it’s kind of heavy, clunky, underpowered. Or you have the new generation iPad - that is kind of the standard for AR at this point. If you can get what the iPad offers into a HoloLens device, I think that would break the adoption barrier to have more adoption on site. The new iPads, I was super impressed by Apple and what they did with their new silicon and the LIDAR devices.

It's all stuff that is incredibly useful, but you still have to hold it with two hands. So you can only do one thing at a time. To merge together the power of an iPad and the form factor of something a bit better than the HoloLens, it'll start getting some adoption.

Videogames vs Revit

Brian: For those who don't know, the original Unity program is a tool set and engine for building video games. What was it like to adapt that kind of tool set for use in kind of the AEC world? When a game designer is building a building or environment, how does that compare to how an architect will build something up?

David: When someone builds a video game, everything in the world is curated. Every single blade of grass is optimized to be able to render super fast on your device, 60 times a second. When you look at a video game, every single back wall isn’t in existence, because you never see those things. There's no piping under the walls because you don't need them. The player will never see the opposite side.

When you come into Revit, every single detail of the potential of the building is in there. That creates so much data that these game engines, they’re just not used to handling that. That was the biggest difference for me. I've seen cities, entire cities in video games, why can't entire cities have a Revit model? Well, there, there you go. That's why. And so the optimization of that data and just keeping it consistent was the biggest challenge.

Brian: What about the tool sets that somebody would use to build an environment in a video game, are there similarities to how Revit works or AutoCAD works? Or is it a completely different sort of workflow?

David: It's a bit different because when you're in the Revit world, everything is what we call parametric. So everything is math equations, right? You can move a door and essentially that door will move in the wall. No problem. But when you're in the video game world, everything is tessellated and geometry and triangles. So that door, if you want to move it, you can't just move the door in the wall, you have to remake the wall and repunch a door of it. So there's a clash there with how the different renderers work. It's very similar to vector graphics versus pixelated graphics, Photoshop versus illustrator.

Design Tool Capabilities and Limitations

Brian: An anecdote that I heard once, allegedly a former Autodesk executive said he could walk through a building and just by looking at the building, know what version of AutoCAD was used to design it. In that vein, do you have any thoughts about how designers are constrained or shaped by their tools, and how that limits the sorts of buildings they’ll tend to design? And how different tools might relax those constraints?

David: I love that anecdote by the way, because it sounds so true and believable.

As far as trends go, what you see in more modern architecture nowadays is a lot more organic forms, because the software can support it - I'm thinking of the parametric software like Rhino and Grasshopper. Not only can those software now produce those kinds of plans, but we're starting to get the know-how to construct them as well. Previously even if someone could draw them, there was no way of actually constructing the thing. So I love seeing those new organic buildings, you see a lot of them built almost entirely out of glass. I think they're beautiful.

Brian: As design tools have gotten more capable, able to produce more, it hasn't eased the burden on the designer. It's just made the design deliverable larger and larger. If you compare a set of construction documents today to one 50 years ago, it will be 10 or 20 times as many drawing sheets or something like that.

You can imagine that if the new expectation becomes that you need to have like a beautifully rendered 3D model that you can walk around, that’s very, very high fidelity, even if your design tools get more capable, you're just having to spend even more time than you did previously. Is there any thought given to how you can handle the additional work that having to create a really high fidelity model is going to create?

David: There's a lot of promising tech on the design front that's coming up, such as generative design, to ease the burden on the designer. Instead of having to create everything manually, you input your constraints and generate those solutions. That's an idea. It's not quite mature yet, especially not in AEC. On the visualization side, automation is king at this point. It was very manual in the past with 3DS Max, and even in the first days of Unity and Unreal, you had to put a lot of work in to get a visualization. Now with things like Inscape, they come out and just do gorgeous rendering. It just becomes easier and easier to create those visualizations.

The next step is that these will not only be visualizations, they're going to become productivity tools. Not necessarily on the design front, but down the line in the construction process. So in coordination, it will become more and more efficient to create that process just because upstream you have those 3D models ready. So if you have that very detailed architectural model and structural model, once you're onsite, you have that data to be that much more efficient on the construction site. And you're going to see probably the bigger firms benefit from that, because they're a bit more integrated.

Brian: Right now, with the design process, information flows from the designer out to the construction site, but getting information back to the designer from the construction site is much much harder. Is any thought given to software tools or that might be able to easily incorporate site information, and help information flow back the other way?

David: We hear about that a lot. Bi-directionality is frequently asked for. Unfortunately it's not an easy problem to solve. The way it's solved today, someone will create a change order or put a comment somewhere. It goes back to the designer and they have to manually tweak that.

Typically there has to be a review process around that, because some human has to check it. You might have a contractor who moves a wall because he can't install it at the right place, or just something as simple as a pipe, you need some kind of human confirmation there to make sure that data flows back in the proper way, and it was verified.

So AR is definitely an enabler there. But you also see a lot of reality capture tech to be able to daily do a full scan of your construction site. Right now, it's kind of expensive to do, you'll use a drone, or something. We've seen people try and put that process in place, but it's still a very flawed process. I do think in five, 10 years, you might have some more maturity there and be able to reconstruct a 3D model or BIM model from the data you capture on site, and then have tools to maybe compare those two a bit more efficiently. But it's still in the experimental phases.

Brian: To kind of wrap up, can you talk a little bit about what you see for the long term trends and opportunities in AEC design tools?

David: Design tools are interesting because you kind of see waves. First CAD, then BIM. I'm always wondering what that next wave will be - to be honest, I don't really know. I do believe it's going to be real-time based, because that interactivity brings so much value. It's not something that's super easy to achieve because there's so many constraints in Revit. But there is a lot of opportunity there as well with stuff that helps you iterate along the way and share those models and gain feedback, because it's one thing designing on your own, but being in an environment where everyone can collaborate at the same time, it is so beneficial to not only iteration time, but getting that real-time feedback while you're designing That's something that's missing in those design tools, just that iteration speed and that collaborative aspect.

[1] - Architecture, Engineering, and Construction

[2] - Revit is a 3D modeling software for designing buildings and producing 2D building blueprints. It’s by far the most popular 3D building design software.

Feel free to contact me!

email: briancpotter@gmail.com

LinkedIn: https://www.linkedin.com/in/brian-potter-6a082150/

Construction Physics

Interview with David Menard, Product Manager…