Satellites have just learned to find things on their own — what does this mean?

For the first time, an Earth observation satellite found what it was looking for on its own, without human analysts on the ground. This milestone, which occurred in April, marks the first reported use of vision language models in orbit and provides a glimpse into how AI can fundamentally change the capabilities of space-based sensors and their value.

Typically, satellites download large amounts of data to analysts on the ground, who use machine learning algorithms or their own eyes to figure out what’s going on. But on the Yam 9 spacecraft, built by space infrastructure company Loft Orbital, a software package built by NASA’s Jet Propulsion Laboratory responded to natural language queries to identify areas of interest.

Google DeepMind’s Gemma 3, the vision language model or VLM that powered the demonstration, is purpose-built for edge applications and is designed to run on limited hardware far from data centers. VLM combines the contextual understanding of a large-scale language model with the ability to analyze images. Researchers asked the model to classify sensor data where the natural environment and human development meet, for example, or to identify infrastructure around rail hubs. And it happened.

This demo is important for two reasons. In the short term, triaging initial data in orbit could greatly increase the usefulness of space sensors and reduce the amount of raw data that analysts currently have to examine. In the long term, this will prove the point in running large-scale AI infrastructure in space.

“This opens the door to an always-on patrol layer in space,” Paul Lasserre, Loft’s head of AI, told TechCrunch. “With VLM, you can build logic and interact with the satellites like, ‘Watch this perimeter and let me know if something looks suspicious.’”

Loft Spaceship is designed as a platform for third-party customers. This business model is closer to infrastructure-as-a-service than traditional satellite manufacturing. The recent contract will build, launch and operate six new satellites for EarthDaily, and will analyze and sell the data collected on the spacecraft. Yam-9 will launch in the fall of 2025 as Pathfinder for the company’s orbital AI project and is powered by the Nvidia Jetson Orrin AGX GPU, one of the leading chips used in space computing.

Juan Delfa Victoria, a technology leader in NASA JPL’s AI group, led the development of NAVI-Orbital, a software package that serves as an effective harness for the Gemma 3 VLM. Although Gemma 3 is commercially available, software engineers had to streamline the software package to reduce the amount of libraries and memory required.

Although this is the first reported use of VLM in orbit, we expect others to follow suit. Planet Labs flies satellites powered by Jetson Orin processors. For now, it’s using them for simpler object detection tasks, but a spokesperson said research is underway on other AI applications, including VLM.

Kepler Communications, which operates the largest GPUS group in space, declined to say whether it had deployed VLM into space under NDA agreements with partners, but said it had “several undisclosed use cases for our computing environment” since the spacecraft launched in January.

“Now that we have proof of concept, that’s really the direction we’re going,” Laserre said. The goal is to build a constellation of satellites that can cover every corner of the planet in real time, which would require 50 to 100 satellites like Yam-9, he said. (Loft currently operates 12 spacecraft in orbit.)

Lessons learned from deploying these small models into orbit will inform how companies seek to deploy large-scale computing infrastructure in space, especially how they approach the mundane but important areas of power and memory management.

It could also pave the way for new scientific tools. The idea for NAVI-Space began with JPL researcher Taran Cyriak John, who was thinking of a digital assistant for astronauts exploring the Moon and Mars.

“We think we have astronauts in pressurized suits, but we know they can’t hit a keyboard. Whatever they want to do is complicated,” Delfa Victoria said. “So why not give us an interactive AI-powered assistant, like in video games and movies?”

Please don’t call it HAL 9000.