Everyone is talking about Instinct, the AI personal assistant. This is an AI personal assistant that still has private access. Besides its amazing features, there are also potential privacy concerns. A true taskmaster, the agent has a “magical” feel and has been hailed as one of the “most exciting releases” since OpenClaw. However, some testers have expressed concerns about the AI agent’s security model and its disturbing terms of service.
To be fair, Instinct is still being tested privately at this point, so these concerns haven’t reached the broader public yet.
Founded by a small team led by former Sierra research scientist Noah Shinn, San Francisco-based Instinct is operated by Spear Street Technologies in accordance with its terms and conditions and California business filing. According to PitchBook, it is currently operating in stealth mode.
The AI agent itself works by connecting to applications and devices, including email, messaging apps, calendars, device audio, location, screen, and more. You can text or call agents via text message or WhatsApp to schedule an appointment or reservation, schedule a ride to the airport, organize your inbox, organize important information, take care of your purchases, find cheap flights, and more.
Although Instinct has been rated by testers as performing above expectations, some have expressed concerns about the company’s approach to customer privacy and security. This raises a timely question. Is the trade-off of giving AI this level of access and autonomy really worth it?
For example, several people have distributed screenshots of the company’s terms of service, which give Instinct a broad “perpetual and irrevocable” license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” your material, including to train its AI models. This term also details how Instinct receives information from your device, such as screen captures, cursor movements, and keyboard input.
The Terms also permit Instinct to enter into binding “agreements, undertakings, or transactions” on your behalf.
Peter Yang, one of the early adopters, pointed out that Instinct did not delete Gmail records when asked. (The team later fixed the issue by adding a tool to settings to remove external data, he said.)
Another person, Claire Vo, noticed that Instinct continued to summarize her inbox even after she cut off access. When she asked Instinct what happened, the bot confirmed that the email was saved in clear text for later retrieval.
Many others also had doubts about the security model. One tester was a little concerned when they noticed that Instinct was able to retrieve sign-up codes from email inboxes to complete certain tasks (in their case, reserving a table at a restaurant through Resy). And Hello Patient co-founder Alex Cohen wrote that he deleted his account after learning how easy Instinct was to phish.
In addition, Moxxie Ventures founder Katie Jacobs Stanton said Instinct violated her trust by sending emails on her behalf without checking with her.
“We are trading privacy and control for hyper-personalized AI tools (AI note-takers, personalized AI agents, etc.), often without fully understanding the transaction,” she said of X, summarizing the dilemma that personal AI agents pose. “The more powerful these agents become, the more important trust becomes. Each successful action increases trust a little bit. One rogue action can reset that trust to zero,” she said.
Michael Mignano, founder of Anchor, which was acquired by Spotify and now a GP at Union Square Ventures, said products like Instinct will “change modern security standards for consumers,” adding: “People are increasingly handing over their passwords to 3P (third-party) apps without knowing what they’re storing and how.”
Interest in Instinct and personal AI has grown since the introduction of OpenClaw’s personal AI assistant, which gained popularity for its powerful capabilities, and its founder joined OpenAI to help develop the next generation of personal agents. Poke, another messaging-based assistant, also just migrated to Cognition.
Amid the criticism, the Instinct team has yet to respond to anyone’s concerns or complaints about X, preferring to keep a low profile. (The bot itself identifies Luca Borletti, also formerly of Sierra, as involved in the company, but that has not been confirmed.) TechCrunch has heard from multiple investors that Kleiner Perkins and Conviction invested in the startup, and those rounds have now closed.
Requests for comment sent directly to the startup’s primary email address and Shinn have not yet been returned.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
