Google’s Project Starline Videoconference Tech Wants to Turn You Into a Hologram

Our reporter tests out a “video booth” to glimpse the future of telepresence.
Video: Google

Over the past several years, Google has been working hard to craft software experiences that make you feel like you’re present with another human being, even if they’re several time zones away. On one end of the spectrum is boring Google Meet, the company’s Zoom competitor. On the other, more daring end is the now discontinued virtual-reality platform Daydream, complete with goggles and hand controllers.

But nothing within that gamut was cutting it for Clay Bavor, the high-energy Googler who heads up the company’s augmented- and virtual-reality efforts. He wanted full-on photo-realistic, volumetric video meetings that make it look, sound, and feel like the other person is sitting across the table from you—no headset required.

So his team built a video booth. The new prototype machine for face-to-face meetings is named Project Starline. Call it hyper-telepresence. Call it whatever you want. Either way, it’s pretty wild.

The phrase “video booth” really is the simplest way to describe Starline in its current form: It's a large booth, like the kind you’d find in a diner, just way more technologically complex. I had the chance to test-drive it in early May. After an initial conversation with Bavor outside of Google’s campus in Mountain View, California, I was led inside the almost empty building and escorted to a private office. There was the Starline booth, part wood-paneled and partly encased in gray fabric, with a built-in bench on one side and a 65-inch display on the other. I was instructed to sit opposite the display. There were lights, cameras, and not a whole lot of action until a product manager sat down across from me. From a very specific angle, he looked as though he was sitting across from me. But he was on a different floor of the building, piping into our meeting through Starline.

This is Google’s idea for the future of videoconferencing, a giddy vision that only a small group of Googlers have had access to, and one that has apparently gotten a thumbs-up from chief executive Sundar Pichai. You couldn’t be blamed for thinking that Starline must have been developed during the pandemic, while desk workers were umm-ing and muting and unmuting their way through an endless stream of Meets and Zooms. But Bavor says there wasn’t really any aha moment that led to Project Starline. In fact, it’s been in the works for over five years.

“What has always excited me about virtual and augmented reality is this idea that these technologies can take you other places, and they can make you feel present somewhere else,” Bavor says. “But it didn’t seem like there was a way of bringing the most important things in the world to you, namely, the people you care about.”

The company gave the public a virtual glimpse of Project Starline today during its annual IO conference. But Starline is still just a concept, unlikely to liven up your Google Meet meetings for several years.

Starline's booth. Not authorized by FCC rules. No sales or leases permitted until authorized. 

Photograph: Google
I’m Right Here

Andrew Nartker, the lead product manager for Project Starline, has been bringing an apple into meetings. It’s a way of showing how objects present in Project Starline, and, perhaps more creepily, a way of tracking eyes. “I can show you this apple from Whole Foods, and I can see exactly what you’re looking at,” Nartker says to me after I take my seat in the booth. The image of Nartker is jarring: He is life-size and sitting directly in front of me, with volume and depth and shadows. And the apple. Nartker and the apple appear to be trapped in a clear box.

Nartker is conferencing in from another Project Starline booth identical to the one I’m using. The imagery is remarkable, and the visuals are complemented by spatial audio. What I’m actually looking at is a 65-inch light field display. The Project Starline booths are equipped with more than a dozen different depth sensors and cameras. (Google is cagey when I ask for specifics on the equipment.) These sensors capture photo-realistic, three-dimensional imagery; the system then compresses and transmits the data to each light field display, on both ends of the video conversation, with seemingly little latency. Google applies some of its own special effects, adjusting lighting and shadows. The result is hyper-real representations of your colleagues on video calls.

Volumetric video—sometimes called holographic video, or just 3D video—is often captured in large, multi-camera studios like Microsoft’s Mixed Reality Capture Studios or this now-shuttered volumetric capture stage from Intel. On the one hand, Project Starline likely isn’t going to have a place in your ad hoc home office anytime soon. Google also hasn’t said how much it costs to build a Project Starline booth; my best guess is “a lot.” On the other hand, Project Starline is a distillation of much larger volumetric capture studios. And what’s more notable is that the video isn’t being rendered after the fact. It’s all happening live.

All of the data is being transmitted over WebRTC, the same open-source infrastructure that powers Google Meet, the company’s main video conferencing app. What Google claims is unique is the compression techniques it has developed that allow it to synchronously stream this 3D video bidirectionally. And while it’s hard to imagine this kind of tech working seamlessly over your shoddy home Wi-Fi connection—Google did confirm that the booth I was sitting in was hardwired to the building’s network—one of the engineers on Project Starline insisted that the tech would work using Google’s standard-speed office network, no fiber required.

I met with three separate Googlers in Project Starline (all of them men), and some of the surreality faded each time I shifted in my seat. Move to the side just a few inches and the illusion of volume disappears. Suddenly you’re looking at a 2D version of your video chat partner again, similar to the way a sporting event looks great on your big-screen TV until you move too far out of the sweet spot on your couch.

There were also a few random artifacts fluttering onscreen, broken bits that served as occasional reminders that the person in front of me was not really there. And when Nartker started casting a web page onto the light field display, as an example of how two people might collaborate in Starline, we both just stared over each other’s right shoulders at a not-quite-interactive page. Starline is impressive, but there’s still work to be done. Will it take five more years? The Google men did not seem inclined to answer.

Video: Google
Welcome Present

Google says that around a hundred employees have used Starline, as the technology has been tucked away in secret offices in Mountain View, Seattle, and New York. Bavor himself has been using it for most of his recent meetings with colleagues in Seattle and New York, spending what he estimates to be around 50 hours in the booth. But he swears his interactions in Starline have painted a bolder brushstroke on his brain, that he has better recall of details and walks away from meetings with the sense he actually met with the person.

“I know that the person I’m sitting across from is not checking his phone during the meeting, and that’s nice,” Bavor says. “But the crazy thing is, I would wake up the next morning and have the memory of, ‘Oh, I saw Steve yesterday.’ Not like, ‘I had a video call with Steve yesterday.’ And there’s just something different about how our memories are laid down.”

Bavor’s latter observation is somewhat subjective, and of course it's a sample of one. The former is a phenomenon that’s actually being studied, now that office workers have spent the past 14 months taking meetings through screens. We are more distracted during video meetings, according to research from Microsoft, and it’s partly a coping mechanism to protect ourselves from the mental strain of too many video meetings. In this context Google’s Project Starline seems especially overengineered, an amalgamation of accessible tech (Google Meet), nerd tech (computer vision! compression algorithms!), and an intricately constructed, unmovable mini studio, all for the sake of … more video meetings.

But of course, it’s the realism of photo-realism that technologists are striving for, and they’re operating under the premise that sometimes connecting through screens really is the only option. But then, maybe, try a VR headset?

Google has tried making VR headsets. People didn’t really use them. It has also made AR glasses; who could forget Google Glass? And if you happen to write an article suggesting Google Glass is no longer a thing, the company’s public relations team will be quick to remind you that it still sells a product called Glass Enterprise Edition 2. One has to wonder if Bavor (who often sports regular, not-smart glasses) is a little less interested in head-up displays these days, at a time when seemingly every other consumer tech company is making face computers.

Bavor says he still thinks VR is “very powerful in its ability to take you somewhere else,” and that there’s a through-line from AR and VR to Project Starline. But also, he admits, “I think one of the other learnings with AR and VR is that it’s nice to be able to just sit down, as you, and not put anything on. I think it speaks to the importance of comfort and lightness and ease. So that’s kind of how I’m thinking about it.”

For now, Google will attempt to shrink Project Starline while it continues to perfect the tech. The booths will likely be sold to businesses, if and when they go on sale. Bavor says Google will run trials of the technology later this year with a handful of early customers: enterprise cloud businesses, telemedicine apps, or media companies, though he declined to name these early testers.

And some of the optics developed for Starline could be deployed sooner in everyday tech. The illusory parts—the lighting and shading or the spatial audio—could be applied in the videoconferencing apps we already use. Until then, Project Starline booths will be used primarily by Googlers, the ones who are going into offices, who will marvel at the realism, hold up their apples, and temporarily ignore the gap between realism and reality.


More Great WIRED Stories