Interaction Models, Gemini Omni and SpaceXAI announced

Google announced on May 12, 2026 a set of coordinated product moves that expand how models interact with people and devices. The company introduced Interaction Models, rolled out Gemini Omni surfaces for visual and spatial experiences, and the same update window highlighted a new effort called SpaceXAI from SpaceX focused on on-orbit and mission automation.

Interaction Models are presented as a new class of model endpoints that combine multimodal inputs with action-oriented outputs. Google described the capability as a way for agents to maintain stateful interactions across text, image, audio, and live video, and to map those interactions to discrete actions such as tool calls, device control, or UI updates. The announcement said Interaction Models will be available to developers in a staged release, with early access for enterprise customers and partners.

Gemini Omni surfaces are a companion set of developer primitives for rendering and interacting with visual content. Google positioned Omni surfaces as a set of standardized UI layers for mixed reality, live camera overlays, and embedded video experiences. The surfaces aim to give developers a consistent runtime for displaying model outputs, capturing contextual signals, and routing user intents back into agents. Google presented examples including live translation overlays, gesture-driven search, and contextual visual suggestions inside mobile camera apps.

What’s new for developers and products

The update bundles three elements: the Interaction Models themselves, connectors to existing model tooling, and Omni surfaces for client-side rendering and input capture. Interaction Models include APIs for session management, multimodal context windows, and an actions interface that lets models request external services. Google said the interfaces are designed to reduce glue code between models and product logic, and that the company will publish SDKs for major platforms.

Gemini Omni surfaces are shipped as a set of UI components and runtime guidelines. They support synchronized video streams, layered annotations, and input capture for touch, gaze, and simple gestures. Google showed demonstrations that pair Omni surfaces with Interaction Models so an agent can point at a live feed, annotate a frame, then trigger a backend workflow without a separate integration step.

SpaceX unveiled SpaceXAI as a distinct program focused on autonomy, on-orbit operations, and mission planning. The announcement emphasized models tuned for spacecraft telemetry, rendezvous planning, and anomaly detection. Initial tests will run in simulation and selected mission prototypes. SpaceX framed the program as internal infrastructure work that may inform future flight systems and mission control tooling.

All three moves emphasize tighter coupling between model outputs and real-world actions. Google highlighted enterprise pilots and developer previews as immediate availability milestones. SpaceX described staged testing and simulation runs before wider operational use.

Interoperability and controls

Both companies said they are building controls to govern when models can take actions. Google described permissioning and policy hooks for Interaction Models so product owners can audit and restrict agent behavior. SpaceX said SpaceXAI will operate under strict safety and verification layers in simulated environments before any live deployment. Neither company released exhaustive technical specifications for verification methods or governance at the time of the announcements.

Why it matters

These announcements push models from passive responders to connected actors inside apps and systems, which raises engineering and safety tradeoffs. Developers will gain tools to build richer multimodal experiences, while operators and regulators must confront new questions about action authorization, testing, and auditability. The shift affects product teams, field operators, and anyone building systems that let models take or suggest real-world actions.