The Brooklyn Hologram Studio Receiving Millions from the CIA

Last summer, Looking Glass Factory, a company based in the Greenpoint neighborhood of Brooklyn, New York, revealed its latest consumer device: a slim, holographic picture frame that turns photos taken on iPhones into 3D displays. Linus Sebastian, an affable YouTube personality behind the immensely popular technology channel Linus Tech Tips, gave his viewers a preview of the technology.

Sebastian praised the Looking Glass Portrait as “freaking awesome,” especially considering the progress the company had made since Sebastian had toured their office two years earlier, after $2.5 million in money from a Kickstarter campaign. “For the price, for the amount of development work, and how niche this thing is, it honestly looks like a pretty compelling value for the right customer,” marveled Sebastian. “Which raises the questions, who is that exactly?”

Sebastian suggested the product would be a perfect fit for those who wanted to “flex” with a novelty piece of artwork or a designer seeking to preview their own work.

But Looking Glass Factory’s other customers went unmentioned in any of the splashy coverage of the new device: the Central Intelligence Agency and the Department of Defense. The military was interested in holographic technology, but the price was a potential obstacle. “The high cost of assembling holographic display devices are restraining market growth,” noted International Defense Security & Technology, a trade publication, last year. One of the growing players in the market, IDST added, is Looking Glass Factory.

Looking Glass received $2.54 million of “technology development” funding from In-Q-Tel, the venture capital arm of the CIA, from April 2020 to March 2021 and a $50,000 Small Business Innovation Research award from the U.S. Air Force in November 2021 to “revolutionize 3D/virtual reality visualization.”

With a brick and black metal facade, Looking Glass looks, from the outside, like another art studio in New York City, but its connection to the intelligence community is not disclosed on its website or public facing materials. Looking Glass Factory did not respond — in person, by phone, or by email — to requests for comment. In-Q-Tel also did not respond to a request for comment.

Across the various branches of the military and intelligence community, contract records show a rush to jump on holographic display technology, augmented reality, and virtual reality display systems as the latest trend. According to its advocates, augmented reality goggles will allow soldiers to see through buildings and mountains to visualize enemies on the other side of the battlefield, or even serve as the primary interface for pilots of unmanned drones, tanks, or underwater vehicles.

Critics argue that the technology isn’t quite ready for prime time, and that the urgency to adopt it reflects the Pentagon’s penchant for high-priced, high-tech contracts based on the latest fad in warfighting.

“It’s kind of the culture,” said Dan Grazier, a former Marine and now a fellow at the nonprofit Project on Government Oversight, investigating military waste and abuse. “The Pentagon always wants to find a technological solution, particularly one that can generate contracts and subcontracts spread all over the country.”

“A lot of these fancy electronic systems end up being more of a distraction than they are actually useful in helping soldiers do their jobs,” added Grazier.

Eight years ago, the 2014 edition of IQT Quarterly, the publication of In-Q-Tel, notes that we are still “far away from a true Star Trek Holodeck experience,” yet:

A perfect simulated reality that is indistinguishable from real life will ultimately take one of two forms: it will either manipulate real light and real matter, like the Star Trek Holodeck, or it will remove the “middleman” of wearable VR inputs and instead directly manipulate our perceptions through a machine-brain interface, like that envisioned in The Matrix. Between those perfect simulations and the current state of the art, we envision the emergence of hybrids, such as the manipulation of real light (holograms) combined with haptic gloves, or the direct manipulation of the brain’s sense of touch combined with VR/AR contact lenses, or many other such combinations involving other senses. Given where VR is now compared to just 10 years ago, and the historical pace of technological change since the Industrial Revolution, it’s astounding to consider how VR might continue to evolve. We think these “perfect systems” may emerge within the next two centuries, and that the current state of the art provides a strong foundation to build upon.

The rapid investments from government sources in augmented and virtual reality reflect the vision laid out eight years ago. Mojo Vision, a startup based in Saratoga, California, is developing an augmented reality contact lens using “tiny microLED” displays the size of a grain of sand to project images directly onto the retina. The company is backed financially by the Defense Advanced Research Projects Agency, or DARPA, a research and development arm of the Pentagon.

In 2018, In-Q-Tel backed a startup called Immersive Wisdom that provides communication and data sharing interfaces. “Allowing multiple users to be anywhere in the world, while still being connected via the same virtual space containing shared maps, video feeds, and real-time data, offers a significant new edge,” In-Q-Tel said in a statement at the time.

In-Q-Tel also invested in DigiLens, the producers of a low-cost holographic lens used for augmented and virtual reality glasses. Last year, the Air Force awarded $1.2 million in contracts to the company.

According to the new disclosure, In-Q-Tel also invested over $1.9 million in Dreamscape Immersive, a Los Angeles-based virtual reality company. Co-founded by former Disney executive Bruce Vaughn, the company provides story-based virtual reality experiences, including the VR-based “Men in Black” feature released in 2019. As previously reported by Forbes, former Raytheon executive Dave Wajsgras — who became a member of Dreamscape Immersive’s board — has said, “The US department of defense is aggressively increasing spending on synthetic digital training which prepares personnel for real-life situations.”

Military interest in holographic imaging, in particular, has grown rapidly in recent years. The IDST article reported that military planners in China and the U.S. have touted holographic technology to project images “to incite fear in soldiers on a battlefield.” Other uses involve the creation of three-dimensional maps of villages of specific buildings and to analyze blast forensics.

Perhaps no investment is as illustrative of the industry’s commitment to production despite potential red flags as the Defense Department’s flagship augmented reality project: the Army’s Integrated Visual Augmentation System, or IVAS, goggles. IVAS provides headsets to guide soldiers through unfamiliar territory, along with machine learning to instantly distinguish between friend or foe, and targeting systems for tracking vehicles and objects on the battlefield. (Investor presentations show the IVAS contract is also supported by Ultralife Corp., a battery provider, and Intevac, which produces night vision sensors and cameras for the defense industry.)

The Army has touted the system as a way to “fight, rehearse, and train” using “advanced eyewear that places simulated images in a Soldier’s view of real-world environments,” allowing soldiers to see through smoke or complete darkness.

The initial contract for the IVAS system, which is based on Microsoft’s HoloLens technology, was awarded to Microsoft and a number of other firms.

But the IVAS contract, which could one day cost upward of $22 billion, has faced chronic delays and failures. Last October, the Army pushed the official launch date to September 2022 for the product for operational fielding and testing.

The Defense Department’s 2021 annual report from its Director of Operational Test & Evaluation, or DOT&E, painted a sobering picture. According to the report, “Soldiers continue to lack confidence in their ability to complete the most essential warfighting functions effectively and safely while wearing the IVAS in all mission scenarios.” At worst, the goggles led to soldiers being unable to “distinguish enemy from friendly forces.”

But all such damning details were — despite congressional appeal — redacted from the publicly available version of the DOT&E’s report through being labeled as “controlled unclassified information,” or CUI. And while the Department of Defense Office of Inspector General made headlines last month through a report based on DOT&E’s critique, the underlying details remained redacted.

The CUI version of the Defense Department’s testing report was only made public through a leak to the nonprofit Project on Government Oversight, where Grazier works. POGO’s primary reason for releasing the document was to help expose the failure of the F-35 program, but Grazier stated that he ultimately released the entire document in hopes of crowdsourcing the analysis of broader instances of fraud, waste, and abuse.

“If you can’t distinguish between a friendly troop and an enemy troop, then you’re going to have a very difficult time distinguishing a civilian,” noted Grazier. “If a soldier can’t identify friends or enemies with their vision system,” he added, “that tells me that system probably needs a lot more work or possibly needs to be scrapped altogether.”

Soon after Congress put $349 million of its IVAS funding on hold in March, Insider reported on a leaked Microsoft memo in which a manager wrote that the company “expect[s] soldier sentiment to continue to be negative as reliability improvements have been minimal from previous events.” HoloLens boss Alex Kipman reportedly described his team as “[s]o depressed, so demoralized, so broken.” The Wall Street Journal reported that roughly 100 Microsoft HoloLens employees left to work at Facebook parent company Meta Platforms Inc. in 2021.

In February, Secretary of the U.S. Air Force Christine Wormuth also poured cold water on the project, at an event for the Center for a New American Security, tempering expectations.

“Remember early satellite phones from the 1980s that wealthy people had in their cars?” said Wormuth during the event. “They were big and clunky and now we have iPhones. It took us some time to get there. The first iteration of IVAS may not be quite as streamlined as we want it to be ultimately, but it’s the alpha version, and we need to start there.”

The military, including soldiers at Fort Benning, have found some productive applications of the IVAS system. Rather than forming the basis of futuristic cyborg warriors, the HoloLens goggles have been an expensive thermal sensor for rapidly detecting soldier temperatures — as a way to screen Covid-19 cases.

In-Q-Tel has a history of working closely with companies that have commercial success providing consumer products while developing innovations with military applications. The investment fund, for instance, backed a skin care company with a line of popular beauty products that had created a method for removing biomarkers that could be used for intelligence purposes. AR and VR technology appears to follow the same track, where consumer products are helping fuel the advancement of innovations that can be one day used for the military.

Before founding Looking Glass Factory in 2014, their CEO, Shawn Frayne and then-CTO Alex Hornstein had each run separate pieces of the Ocean Invention Network, an interconnected group of inventor labs. Press coverage in early 2013 described the network as “an indie rock supergroup of the cleantech scene; Haddock Invention, which opened shop in 2006, is on lead guitar, while Mantis Shrimp Invention, opened in Manila (Philippines) by Alex Hornstein in 2012, hits the drums. The Solar Pocket Factory … is their hit single.”

Frayne and Hornstein had both completed bachelors of science at Massachusetts Institute of Technology in the early 2000s: Frayne in physics in 2003 and Hornstein in electrical engineering in 2007. Their third partner was Jordan McRae, who finished his B.S. in aerospace engineering at MIT in 2005 (and spent two years working for Lockheed Martin before joining the Ocean Invention Network as CTO of Humdinger Wind Energy under Frayne).

Frayne and Hornstein’s Haddock and Mantis Shrimp labs would collaborate on Solar Pocket Factory, a “coffee-table size machine that makes panels small enough to power pocket-size devices.” As reported by Fast Company, the effort raised $78,000 via Kickstarter in 2012 but by the end of 2013, Frayne and Hornstein had pivoted to purchasing solar panels from a factory in Dongguan, China, adding the ability to control them with a cellphone, then renting them out for $1.50 to $2 per week. They would partner with a utility company in the Philippines as part of a trial on top of 20 homes in the island of Alibijaban. (This last incarnation was called Tiny Pipes.)

While Frayne and Hornstein would move on from the Ocean Invention Network to focus on Looking Glass Factory’s holographic devices in 2014, McRae’s component of the network, Octo23, would continue on until 2017. Octo23 began with a mandate as broad as the Ocean Invention Network’s: “clean energy, clean water, ocean conservation, and robotics.” But it later focused on OCTOtalk, a “proprietary technology to reinvent the diving/snorkelling mask enabling recreational divers and snorkelers to talk underwater” — a product that McRae then repackaged for the military.

McRae’s Mobilus Labs, founded in 2017, produces bone conduction communication technology that would later be combined with Microsoft’s HoloLens 2 in Trimble’s XR10 hardhat and, reportedly, tested by the British Army.

Hornstein ultimately departed Looking Glass in February 2021 — his LinkedIn status remains “tbd” — and both Mobilus and Looking Glass were listed among Time magazine’s “Best Inventions of 2021.”

McRae did not immediately respond to a request for comment.

One of the loudest voices in the startup scene evangelizing the promise of augmented and virtual reality systems for modernizing the military is Palmer Luckey, who founded the technology startup Anduril Industries after Facebook bought his first company, Oculus VR.

During a Ronald Reagan Presidential Foundation and Institute interview in December titled “Into the Metaverse,” Luckey claimed Facebook’s rebranding to Meta resulted directly from the company’s 2014 acquisition of his virtual reality headset company. “Facebook paid a lot of money for Oculus — a few billion dollars — all the way back then because they saw that our technology was the key for unlocking the Metaverse,” said Luckey. “In a way, Oculus took over Facebook for the Meta rebranding,” he added.

Anduril has been promoting a relatively restrained approach to military adoption of augmented and virtual reality interfaces that emphasizes its usefulness for training. Its central product is Lattice, an artificial intelligence operating system. “I think soldiers are going to be superheroes who have the power of perfect omniscience over their area of operations, where they know where every enemy is, every friend is, every asset is,” Luckey exclaimed, during a 2018 speech.

Last year, in the Reagan Institute talk on defense issues, Luckey was brimming with excitement. “We’ve gotten some really big contracts with DoD including with the U.S. Air Force on the Advanced Battle Management System,” he said. “We’ve integrated our system as part of life fire exercises where we were tying together naval assets, air assets, ground assets, shooting cruise missiles out of the air with real gun systems on the ground — all command and controlled through a virtual reality interface.”

Just last week, Anduril executive David Goodrich posted on LinkedIn about the possible use of the company’s virtual reality headsets as part of its recent Underwater Autonomous Vehicle partnership with the Royal Australian Navy. The video that prominently plays on the company’s website suggests the use of virtual reality headsets for monitoring a fleet of its tube-launched Altius drones.

Anduril maintains a large team of lobbyists, spending roughly a million dollars a year influencing congressional budgets and Pentagon planners, and last year formed an advisory board filled with former top government officials. The board now boasts former CIA chief strategy officer Constantine Saab, retired Adm. Scott Swift, and Kevin McAleenan, President Donald Trump’s acting secretary of homeland security.

Luckey has received secretive Air Force contracts to develop next-generation artificial intelligence capabilities under the so-called Project Maven initiative, as The Intercept reported. Similar to Google — whose participation as a subcontractor on Maven was confirmed by the Pentagon — there is no known public procurement record of Anduril subcontracting on Maven. While it is unclear whether Anduril was serving as a prime or subcontractor, Google worked underneath Virginia-based staffing firm ECS Federal, which was named as the major Project Maven prime contractor in a heavily redacted report released by the Defense Department’s Office of Inspector General in January.

Anduril and Looking Glass share an early investor, Lux Capital co-founder Josh Wolfe. Wolfe has become a prominent advocate of Silicon Valley venture capital playing a more prominent role in the Department of Defense; he even recruited the former head of U.S. Special Operations Command, Tony “T2” Thomas, as a venture partner at the beginning of 2020.

Wolfe has periodically tweeted updates on the progress of Looking Glass’s holographic technology, using a “Star Wars” X-wing fighter as an example in February 2018. The Pentagon’s now-scrapped cloud computing initiative, JEDI, would be unveiled the following month and — according to reporting from ProPublica — the C3PO acronym had been blocked from use in the project.