I tried Apple’s Siri Recaps. These are surprisingly useful, but also a bit unsettling

On Tuesday, Apple introduced a set of potentially controversial audio intelligence features to testing in the first developer betas of iOS 27.3 and watchOS 27.3. This beta offers four new AI features that enable Apple Watch Series 12 wearers to detect important sounds, identify songs, follow conversations, and take notes.

After just a day of using Apple’s Siri Recaps feature on my Apple Watch, I can already see that its value may ultimately outweigh any spooky elements that AI-powered voice recording and summarization brings. However, there is room for improvement, as AI-assisted note-taking can go too far and end up recording content that is not worth remembering.

The first two, sound and song detection, do not indicate any potential privacy concerns. It’s designed to detect important sounds like alarms and doorbells, as well as identify songs using Shazam technology. The former helps hearing-impaired users navigate the world. The latter is something we’ve been able to do with the iPhone for nearly 20 years now.

Two other features are causing concern: Live Rewind and Siri Recaps. This feature records conversations that occur with and around the Apple Watch owner and can be transcribed or summarized. Live Rewind allows you to go back and see what you said in the last 15 seconds. Siri Recaps, on the other hand, summarizes audio throughout the day and saves notes of important things said.

Apple has been careful to ensure the privacy of these AI features by processing audio and running AI models on-device where possible, not identifying the speaker, storing raw audio files, and redacting personal information such as government ID numbers or personal account information.

Still, some balk at the now-normalized idea that technology is always listening.

I feel sympathy for both camps. I often lament how technology has become so ingrained in our daily lives, but I still think there’s value in what Apple has built.

I haven’t had a good time to test Live Rewind yet, and I don’t think it will be a frequently used feature. You’re more likely to say, “Sorry, what?” or “Can you repeat that?” before asking your Apple Watch for help if you missed what was said.

But with Siri Recaps, I quickly found that it became a feature I relied on regularly to save important notes for later.

I was able to test this feature in a real-world scenario yesterday when I took my teenager’s car in for repair and picked up my rental. At both locations, employees shared a lot of information about terms, prices, and policies that were communicated verbally. The information came out so fast that I couldn’t stop and write anything down. But when I returned to Siri Recaps later in the day, it had captured important details that I might need to refer to later.

For example, it listed deductibles, repair periods, how much gas to use at the start of the rental, and other things to keep in mind.

One complaint was that Siri Recaps mixed this information with other topics that were later discussed in the household. For example, what errands we had to run or what time the babysitter would have this week were things we talked about to create events on our family calendar with appropriate alert time slots.

I think these items should have been saved under a different heading in a separate note titled “Family Plans and Errands” rather than in the same note as the car repairs.

I set Siri Recaps to run constantly, so it captured even unimportant things I wouldn’t normally write down in my notes, like my “plan” to watch midterm election coverage on TV. It was also a stark reminder that this feature is eavesdropping on our everyday conversations. This can feel invasive in various scenarios if the subject matter being discussed is more personal in nature.

So when I’m home in the evening, I’ve considered turning off the summary feature so that family dinner table chatter doesn’t feed into my notes.

I appreciated that Recap picked up reminders about errands I had to run from family conversations, but the other tidbits it saved (like what I was watching on TV) are only valuable if you export your Siri Recap notes to Apple’s Journal app. That’s because it was something I wanted to record for personal reasons.

I’d like to see these extraneous details saved as separate notes, or at least as a dedicated section that can be independently exported or deleted at the end of the day.

I’d also like to be able to swipe left on individual bullet points to completely remove them from Siri Recap, rather than going into edit mode, hovering over them, and starting deleting unwanted text.

Another option I’d like, rather than an extra step, is a setting that allows you to automatically export saved Siri Recaps (those saved by clicking the bookmark icon) to Apple’s Notes app.

These summaries can be placed in their own folder in the Notes app for easy reference. Given the variety of ways people use generic apps like Notes, it’s understandable why this option isn’t the default. However, it’s useful for people who want to keep Notes as a central starting point for all their reminders and to-dos.

While I wasn’t immediately sold on the idea that the Watch was constantly listening, so to speak, I could see the value of this kind of automatic note-taking in everyday life.