Skip to content
All tool news

Tool desk · Updated daily

Tool news·PostHog·

PostHog Asks 9,000 Users Which AI Prompt Led Them to Sign Up.

PostHog added one question to its signup flow and collected 9,214 AI prompts in three months, turning user responses into strategic data.

CW

Create With tool desk

1 source checked · 3 min read · 3 sections

ShareLinkedIn
PostHog Asks 9,000 Users Which AI Prompt Led Them to Sign Up

PostHog added a single question to its signup flow three months ago: which AI prompt led you here? So far, 9,214 people have answered it, and the analytics platform is using those responses to inform strategy in ways traditional answer engine optimization tools can't match.

The question appears during onboarding, and according to Ian Vanagas, a growth engineer at PostHog who announced the approach on LinkedIn, users answer in surprising detail. Some responses are long paragraphs describing exactly what they typed into ChatGPT, Claude, Perplexity, or other AI tools before landing on PostHog's site.

Real prompts, not panel data

Every morning, a PostHog subscription delivers a report of 50 to 100+ prompts from the previous day's signups. The company uses PostHog AI to cluster the patterns, then feeds actionable insights directly back into strategy. Vanagas describes it as one of the most valuable strategic inputs in their entire reporting stack, and it cost roughly zero dollars to set up.

The approach sidesteps the core problem with answer engine optimization: there's no reliable prompt-volume data. Most companies operate on vibes, guessing what users type into AI interfaces. Third-party prompt trackers only show you the prompts you thought to track, and they rely on panel data from users who installed browser extensions or other tracking tools. PostHog's method is first-party, collected directly from people who converted, with no vendor black box or extrapolation.

One user found PostHog through a Darth Vader NPC in Fortnite voice chat while asking it to analyze the Fortnite economy. No AEO tool would have surfaced that.

How it works

The implementation is straightforward. PostHog added a single open-text field to its signup flow, and users volunteer the information as part of onboarding. The responses are tied to user profiles, so the company can correlate prompts with product usage, feature adoption, and retention.

Because the data comes from actual signups, not a sample or panel, it reflects the prompts that drive conversions. That's different from tracking all prompts that mention your product, which may include research queries, competitive comparisons, or curiosity that never converts. PostHog is capturing the language that leads to action.

The company routes the data through its own analytics platform, which means it can segment by geography, device, referrer, and any other dimension PostHog tracks. The AI clustering step helps surface patterns at scale, turning hundreds of freeform responses into themes the team can act on: common pain points, unexpected use cases, or language that resonates with buyers.

PostHog analytics dashboard
PostHog analytics dashboard

Why this matters for AEO

Answer engine optimization is still emerging, and most companies lack visibility into how users interact with AI interfaces. Search engines provide query data through tools like Google Search Console, but AI chatbots don't expose prompt logs to the public. That forces marketers to rely on third-party trackers, which use panel data from a small subset of users, or to guess based on keyword research and content trends.

PostHog's approach flips the model. Instead of inferring what users might ask, it captures what they actually asked, in their own words, at the moment of conversion. The method works because users are already filling out a signup form, the question feels relevant in that context, and the responses are voluntary.

The data can inform content strategy, landing page copy, and product positioning. If a cluster of users describes the same workflow or pain point in their prompts, that's a signal about what to emphasize in documentation, tutorials, or marketing pages. If certain language appears frequently, it suggests how to frame product benefits in ways that match user intent.

PostHog has been testing AI-driven product improvements for months, including agents that generate pull requests from analytics data and a setup wizard that uses an AI agent to automate developer onboarding. Collecting prompt data fits into that broader strategy of using AI both as a product surface and as an input for product decisions.

Other companies can replicate the approach by adding a similar question to their own signup flows, checkout forms, or onboarding surveys. The key is to ask at a moment when users are already sharing information and the question feels natural, not intrusive. Open text fields work better than multiple choice because they capture the full range of language users employ, including phrasing and context that wouldn't fit into predefined categories.

For now, PostHog is one of the few companies publicly sharing how it uses first-party prompt data at scale. The approach may become more common as answer engine optimization matures and marketers look for alternatives to third-party prompt trackers.

Sources

1 checked

How we cover tool news: Create With's tool desk drafts these reports with AI from the sources listed above and checks them against those sources before publishing.

Worth passing on?

ShareLinkedIn

Latest tool news

What else changed this week.

All tool news
VercelDigest

What Vercel Shipped in Late October

Vercel shipped quicker static deployments, four AI Gateway models, more free domain choices and new CLI controls in late October. The changes cover developers testing sites…

Vercel

Vercel Adds Submit_feedback to Its MCP

submitfeedback is a new Vercel MCP tool that lets agents and harnesses send bug reports, friction reports and feature requests directly to the Vercel team. Feedback can include…

The Create With Briefing

Don't watch forty changelogs. Read one email.

Every Tuesday: the tool changes worth knowing, real business use cases, and what's on near you. Free, unsubscribe any time.