Claude, GPT, Gemini install Sentry and Playwright every time

Sentry for error monitoring and Playwright for end-to-end tests were the first install in all 30 answers each on InstallRate’s leaderboard. Its November 2 run gave Claude, GPT and Gemini build tasks that never name a product and logged what they installed across 600 answers. Where the models split, the model picks the vendor: a task that leaves feature flags open got Vercel Flags as Claude’s top choice, LaunchDarkly as GPT’s and PostHog as Gemini’s, on samples of about 10 answers per model per category.

In 13 of the 20 categories all three models put the same tool first. Neon leads Postgres hosting at 67% overall and 80% for Claude. Stripe takes 80% of payments answers from every model, and PostHog takes 93% of product analytics, with GPT and Gemini at 100%. Drizzle, Resend, AWS S3, shadcn/ui and Meilisearch also lead for all three. On the findings pages, Resend took 16 email answers to Postmark’s 4 and Mailgun’s none, and Vercel took 12 deploy answers while Netlify, Railway, Render and Fly.io got none.

Share counts the product an answer installed first. So a shared leader at 40%, as Vercel is for hosting, BullMQ for queues and a generic cron library for scheduled jobs under every model, was the first install in fewer than half of the answers.

Gemini’s top pick differs from the overall leader in five categories: Clerk over Auth.js for auth, OpenTelemetry for logging, Pinecone at 20% where pgvector leads, LangSmith at 50% where Langfuse leads LLM observability, and a Git-based CMS over Sanity. Claude splits in three: Vercel Flags, OpenTelemetry at 50% for logging where GPT puts Grafana, Loki and Prometheus first at 60%, and Ably at 20% for realtime. GPT parts from the field in only one, with LaunchDarkly. InstallRate calls the samples small and points readers to the uncertainty bands on each category page.

The leaderboard doubles as a storefront. For $19, delivered within 24 hours, a vendor gets its product set against three competitors it chooses on all three models, with the exact prompts where it loses and what to fix. The site also asks for missed tools, wrong-looking picks and new prompts, and says it fixes the data in the next run.

Its findings pages track a second gap, between what an answer names and what it installs. Supabase is mentioned and never installed, while Neon takes the Postgres picks. Radix UI is named 11 times and never installed, with shadcn/ui taking the pick, and answers that name Cloudflare R2 go on to install the AWS SDK. The vendor in a reply’s text and the one in the dependencies it leaves behind can differ.

Datadog was named nine times and installed zero times.