A Developer Builds a Free iPhone Shortcut That Lets a Grok Bot See Your Screen and Answer Questions About It
grok bot can now see what's on your phone and answer anything about it just hold the action button and ask what you need: > ask anything about the video you're watching > tell it to build what you saw in a reel for your business > ask it to break down the article you're reading it can watch videos, read posts, check the web, basically everything. i built it as a free iphone shortcut and made it open source, you can grab it below
Context
It shows how people are wiring AI assistants into their phones themselves, giving an agent a view of whatever is on screen, with clear privacy tradeoffs.
Grok Bot lets people build personal AI agents, and users can share their setups as templates. The iPhone Action Button is a programmable side button that can run a Shortcut. This project comes from an individual developer, not from xAI.
Deep dive
A developer who posts as Chris, handle everestchris6, said on Friday that he built a free iPhone shortcut that lets a Grok Bot see what is on your phone and answer questions about it. He said users hold the Action Button and ask what they need, for example about a video they are watching or an article they are reading, and that the bot can watch videos, read posts and check the web. He said the shortcut is open source.
The template page for the project, called Action Button Bot, describes it this way: hold the Action Button on a reel, video or page, ask a question out loud, and get a silent answer as a notification. The answer is built from a screenshot, the post, its comments and a web search.
The same page lists several cautions. Each screenshot goes to a public host for 10 minutes, including whatever else is on screen, such as messages or banking apps. The clipboard is also sent on every press, and a webhook key sits inside the shortcut. Text from the screen goes straight into the bot's prompt, so it could carry injected instructions.
This is a community project built on top of Grok Bot, not a feature xAI has announced.
What to expect next
Watch whether xAI or Apple offer built in ways for assistants to see the screen, which would make shortcuts like this less necessary. The privacy cautions on the template page are the main thing for users to weigh before installing it. Expect more people to share similar phone automations as personal agents become easier to build. Security researchers may also test how such setups handle injected text from web pages.