One example provided by Apple involves asking Siri what a family member suggested doing during an upcoming visit. Siri can search messages for the answer, find a related recipe in email and then add the ingredients to Reminders.
Onscreen awareness is another major part of the update. Siri can now answer questions about information already visible on a device and take actions based on it. While viewing a sports website, for example, a user could ask Siri to find upcoming home games and add them to a calendar.
Apple is also extending Siri’s reach beyond its own apps. The company says users can already perform actions in services such as WhatsApp and Audible, with additional support planned for apps including Microsoft Outlook, Notability and Tripsy.
Users can continue conversations with follow-up questions rather than starting from scratch each time. Apple has also created a dedicated Siri app where conversation history can be revisited, with iCloud syncing allowing discussions to continue across supported Apple devices.
The assistant is being integrated into more parts of Apple’s operating systems as well. On iPhone, users can invoke Siri through voice, the side button or the Dynamic Island. On iPad and Mac, Siri is integrated with Spotlight, while Apple Vision Pro users can place a Siri interface in their spatial environment and interact with it visually and by voice.
Visual Intelligence expands those capabilities further. Siri’s multimodal features are now built into the Camera app on iPhone through a new Siri mode, allowing users to ask questions about what the camera sees and take actions from that information.
Apple gave the example of pointing an iPhone at a music festival poster, asking for set times and then having Siri create calendar events. Visual Intelligence is also integrated into screenshots on iPad, available through a keyboard shortcut on Mac and extended to Apple Vision Pro.
Writing tools are also becoming part of Siri. Users can ask the assistant to create text from scratch or revise existing writing based on natural-language instructions. In Mail and Messages, Apple says Siri can adapt to how a user typically communicates with individual recipients.
The wider Apple Intelligence update adds several features outside Siri. Photos gains Spatial Reframing, an Extend tool and an improved Clean Up feature. Image Playground can now create photorealistic images and make changes based on text instructions, while Safari adds tab organization, website-change notifications and the ability to generate custom extensions from a description.
Shortcuts is becoming more conversational as well. Users can describe the automation they want, rather than assembling each step manually. Calendar can create or modify events from simple descriptions, while Mail and Messages gain more personalized Smart Reply capabilities.
Apple is also preparing new intelligence features for Apple Watch and Health later this year. Audio Intelligence on Apple Watch Series 12 and Apple Watch Ultra 4 will include Sound Recognition, Live Rewind and Siri Recap. The Health app is also getting new AI-generated summaries and personalized guidance based on user data.
Apple says the new system is built around a combination of on-device processing and Private Cloud Compute. The company’s latest foundation models were created with Google and Gemini technology, while Siri also uses an on-device system orchestrator to access features such as the Spotlight index and App Toolbox.
The architecture gives Apple access to more capable models while keeping much of Siri’s interaction with personal information tied to the device or Private Cloud Compute.
For Apple, the result is the broadest Siri expansion in years: a shift from a voice assistant built around individual commands toward a system that can understand personal context, follow conversations, interpret what is on the screen and take actions across multiple apps.
This analysis is based on reporting from Apple.
Image courtesy of Apple.
This article was generated with AI assistance and reviewed for accuracy and quality.
