Microsoft's whitepaper for the U.S. Army's 'fighting google' and integration with Maven through Project Convergence
As the US Army considers a recompete of Microsoft's $22 billion augmented reality goggles contract, IVAS Next, the public deserves to see Microsoft's full pitch for the Army's Project Convergence.
I’ll admit to sitting on this story for more than a year. Frankly, most of what I wanted to write I am still withholding due to needing at least a second source for confirmation, causing the uncomfortable delay.
But with the U.S. Army currently considering a recompete of its $22 billion ceiling augmented reality goggles Other Transaction Agreement (OTA) with Microsoft, known as the Integrated Visual Augmentation System (IVAS), I have decided to publish the underlying information directly, perhaps for usage by journalists with more sources inside Microsoft. According to reporting from Breaking Defense, the U.S. Army on January 22 “published its new Integrated Visual Augmentation System (IVAS) Next request for information … giving interested companies until Feb. 26 to respond with their solutions for the combat-suitable heads-up display.”
The primary document exclusively published in this article, entitled “IVAS: Convergence of Warfighter Modernization,” was produced on July 3, 2020, according to its metadata, and contains Microsoft’s full pitch document to the U.S. Army on how IVAS can produce a “fighting google” used by “the entire [U.S.] dismounted force, which is approximately 125,000 Soldiers and Marines,” including through the incorporation of drone video feeds from the Pentagon’s Project Maven AI surveillance program.
As first reported by the author based upon public procurement subaward analysis, Microsoft received at least $31.6 million as part of Project Maven contracts led by the staffing company ECS Federal between July 2019 and October 2020, including for the artificial intelligence analysis of electro-optical and infrared “Full Motion Video.” Microsoft’s “Convergence” pitch thus fits squarely into this date range. (Of the $31.6 million, $12.4 million were through the contract codenamed “Pavement” which was confirmed to the author by a Pentagon spokesperson to have been deleted from public record using a national security provision of the Federal Acquisition Regulation (FAR).)
The project’s title is also a clear allusion to the U.S. Army’s “Project Convergence” effort towards accelerating the pace of warfare through the incorporation of commercial artificial intelligence and machine learning. Begun in 2020 as an outgrowth of the previous year’s Project Quarterback, Microsoft’s IVAS headsets became a central component of the public marketing of Project Convergence. A video published by the U.S. Army in October 2022 includes a squadron of soldiers from the 82nd Airborne Division wearing IVAS headsets atop a hill, with a helicopter hovering in the background.
As noted in a history of Project Convergence (PC) authored by a member of the Next Generation Combat Vehicles team of the U.S. Army Futures Command, one of the earliest experiments involved a U.S. Army autonomous weapons system named “Origin” being tested at Yuma Proving Ground in Arizona on August 25, 2020. The first of four goals of PC’s predecessor, Project Quarterback, was also listed as “AI-enabled Aided Target Detection, Recognition (AiDTR) that geo-locates and identifies the enemy.” Project Convergence is widely understood to be the U.S. Army’s component of the inter-service Joint All-Domain Command and Control (JADC2) effort, whose international extension is known as Combined Joint All-Domain Command and Control (CJADC2), which is periodically stress-tested with international allies through the Global Information Dominance Experiments (GIDE).
Roughly three weeks before the test of the Army’s “Origin” autonomous weapons system at Yuma Proving Grounds, then U.S. Army Secretary Ryan D. McCarthy thanked Microsoft CEO Satya Nadella for sending him “such an informative paper on Warfighter Convergence.” McCarthy continued that “I have shared this paper with Army Futures Command and key members of my staff and it was well received … I found thought-provoking the paper’s argument that in this era of great power competition and near-peer conflict, information dominance can provide combat overmatch rather than military’s long-held reliance on standoff distance.”
When former U.S. Army Secretary McCarthy was reached by phone and email in November 2023, he confirmed key details of the email, including his attempted coordination with his peer at the United Kingdom Ministry of Defence for the purchase of IVAS units, but did not wish to provide comment. Microsoft “politely” refused to comment on either Mr. McCarthy’s email or the contents of their pitch for Project Convergence.
The originally optimistic relationship between Microsoft and the U.S. Army and Congress largely soured as a result of ongoing complications with the technology, including the severe headaches common amongst its users. As noted on pages 115 to 117 of a Controlled Unclassified Information (CUI) version of the U.S. Army Director of Test & Evaluation’s 2021 annual report, which was originally published by the nonprofit Project On Government Oversight (POGO), “hardware discomfort still leads to headaches, nausea, pressure on the face, and neck strain.” The report further noted that: “Soldiers continue to lack confidence in their ability to complete the most essential warfighting functions effectively and safely while wearing the IVAS in all mission scenarios.”
As a result of the Army’s critical testing conclusions, 2022 became a difficult year for IVAS, with Congress freezing $349 million in funding for the effort that March. A subsequent bill report published by the Senate Armed Services Committee slashed the IVAS budget in favor of the more traditional nightvision goggles program partially produced by L3Harris, known as the Enhanced Night Vision Goggle-Binocular (ENVG-B). The Senate report argued that “not all personnel in a close-combat formation should be equipped with IVAS and that equipping select soldiers with alternative night vision and situation awareness equipment could make the formations more combat effective.”
In August 2023, Breaking Defense reported that the IVAS tool for training U.S. soldiers for in-door battle, known as the Squad Immersive Virtual Trainer (SiVT), was unable to handle buildings with doors. “We are working on it. We’ve identified it as a real issue,” Lt. Col. Mindy Brown told Breaking Defense.
After the U.S. Congress advertised a $200.5 million cut to Microsoft’s IVAS budget in August 2024, the company announced a partnership with the up-and-coming defense technology contractor Anduril the following month. Though it appears to have gone unnoted except by the author, the final version of the 2025 National Defense Authorization Act (NDAA) quietly reallocated $190.5 million of the $200.5 million cut back to Microsoft’s IVAS effort in December.
Beyond Anduril, another potential competitor for IVAS Next is Palantir Technologies, which became the lead contractor for Project Maven through its Maven Smart System (MSS) product. Palantir hired away former IVAS program director David Marra to head Palantir’s mixed reality division circa January 2023 to March 2024 before transitioning him into an advisory role that appears to have ended earlier this month, according to his LinkedIn profile. (Mr. Marra is now newly listed as the CEO of the mixed reality software development company Rivet Technologies.)
One of the further noteworthy aspects of the 28 page Microsoft pitch document — whose introduction was signed by the then-head of Microsoft’s IVAS program, Alex Kipman, prior to him resigning in response to sexual harassment allegations — is that it attempts to shield itself from Freedom of Information Act (FOIA) requests through appeals to the Trade Secrets Act. In particular, the vast majority of pages contain the footer disclaimer of: “Microsoft Confidential and Proprietary Information – Protected from public disclosure by the Trade Secrets Act, 18 U.S.C. § 1905 and 5 U.S.C. § 552(b)(4).”
Thankfully, anonymous sources do not have to agree to let corporations subvert the public’s access to basic information relating to how U.S. technology giants are being granted their tax dollars.
A full copy of then-U.S. Army Secretary Ryan D. McCarthy’s August 3, 2020 email to Microsoft CEO Satya Nadella follows:
From: McCarthy, Ryan D HON USARMY HQDA SECARMY (USA) <ryan.d.mccarthy3.civ@mail.mil>
Sent: Monday, August 3, 2020 12:17 PM
To: Satya Nadella <satyan@microsoft.com>
CC: Martin, Joseph M GEN USARMY HQDA VCSA (USA) <joseph.m.martin22.mil@mail.mil>; McPherson, James E HON USARMY HQDA SECARMY (USA) <james.e.mcpherson.civ@mail.mil>; Murray, John M (Mike) GEN USARMY FUTURES COMMAND (USA) <john.m.murray26.mil@mail.mil>
Subject: [EXTERNAL] Warfighter ConvergenceSatya [Nadella],
Hope this note finds you and your family well.
Many thanks to you and your team for taking the time to provide such an informative paper on Warfighter Convergence. I have shared this paper with Army Futures Command and key members of my staff and it was well received. It is apparent to me, that an immense amount of knowledge and expertise has brought us to where we are with the IVAS program and a lot of thought was spent on building the white paper. I found thought-provoking the paper’s argument that in this era of great power competition and near-peer conflict, information dominance can provide combat overmatch rather than military’s long-held reliance on standoff distance. AFC is digging further into these implications, and we look forward to learning alongside you on these technology-driven concepts.
I think we are both in congruence that DoD’s movement to open source software and open standards will enable a connected and collaborative DoD development community. It is absolutely critical that we are creating an open architecture to extend the value of IVAS for future development. To ensure the full value of IVAS is realized on the battlefield, the Army must have a high-bandwidth low-latent [sic] tactical network. The Army and Microsoft must continue to acknowledge these challenges and partner on the necessary solutions to address them in order to deliver on the immense value the IVAS program will provide to our soldiers.
On another note, I wanted to give you a heads up that I recently traveled to the United Kingdom to meet with my counterpart, Minister for the Armed Forces, James Heappey. The U.S. and UK signed a bilateral Modernization memorandum of agreement that provides the foundation for future cost sharing collaboration opportunities in the years to come. The UK was especially interested in the capabilities that IVAS brings Soldiers. I personally invited James and several key UK Defense Leaders to attend Soldier Touchpoint 3 this fall. If the UK is interested in procuring IVAS for their Army, it would it [sic] be a great opportunity for Microsoft, the U.S. Army and UK Army. My staff will coordinate with your team as this potential visit matures.
I really enjoyed our conversation and I look forward to meeting with you again in the near future.
Army Strong!
RM [Ryan D. McCarthy]
Maybe the Chinese can work this out as a side project?