Reports

Agents are not choosing this package: it was picked in 0 of 10 benchmark runs and 0 of 15 chat answers for the Postgres jobs. It can't be finished with, because it is a MySQL/Vitess driver and its own README tells Postgres users to use the Neon driver. The single biggest reason is that the package and its docs have no Postgres path, and no zero-account path, that an agent can complete unattended, while pg and Neon's claimable databases do.

No, agents are not choosing @supabase/supabase-js: across the benchmark runs I could see (Postgres, uploads and vector search) it was never picked, and it was never even named in a transcript. Because it was never picked, 'finishing with it' could not be measured. The biggest reason is that Supabase needs an account, a project and keys the agent cannot create, while Neon's claimable database and plain local disk needed none, and the npm package shows no README that would tell an agent otherwise.

Agents choose @upstash/vector only when the prompt literally asks for a vector database, and only 2 of 10 runs (139, 141, both Claude Code) installed it; the other eight used local embeddings, OpenAI, Qdrant, or nothing. Neither of those two runs finished against a real index: 139 passed only on an in-memory fallback the agent wrote itself, and 141 stopped with needed_credentials. The biggest reason is that Vector has no zero-credential path (Redis has start-redis, Vector does not), and the npm package ships no README, AGENTS.md or llms.txt to tell an agent otherwise.

Agents are mostly not choosing nodemailer: it won 1 of 10 runs (run 115), while Resend won 9, 6 through its SDK and 3 through raw fetch. When it was chosen it finished cleanly, and it was the only run that actually delivered a message (an Ethereal preview URL), because no credentials were needed. The biggest reason it loses is that nothing agent-facing tells an agent to use it: the npm README has no install or quickstart, there is no llms.txt or AGENTS.md, and Resend has a documented Next.js path that agents fetch.

Agents choose @upstash/ratelimit only when the prompt says serverless, Vercel or hosted: 4 of 10 benchmark runs picked it, and the other 6 hand-rolled an in-memory limiter, called Upstash Redis directly, or used botid. When they pick it they mostly finish (3 of 4 builds pass), but none could verify limiting against a real Redis, and one run (264) stopped at needed_credentials. The biggest reason is that the package ships no agent-facing surface: no README as fetched, no llms.txt, no AGENTS.md, a badge-markup npm description, and no zero-credential way to see it work.

Agents are mostly not choosing inngest: it was picked in 1 of 10 runs (Codex, run 163), while Claude Code went to Vercel's `workflow`, `@vercel/queue`, or an in-memory queue in every run. When Codex did pick it, it finished: the build passed and a real job ran against a local dev server, with only production credentials and app sync left as manual steps. The biggest reason it loses is that the npm package ships no README and no consumer-facing agent file, so agents that read `node_modules/<pkg>/README.md` find nothing for inngest, while `workflow` gives them bundled docs and a no-account, no-infra pitch.

Agents choose pg rarely: 2 of 10 runs, both Codex on generic or local-storage prompts, and 0 of 6 on the managed, serverless and Vercel prompts, which went to @neondatabase/serverless. When pg is picked it finishes (2 of 2, with a live local Postgres verified), but only because Codex could install and start a server itself. The biggest reason it loses is that the README has no code, no Next.js or serverless guidance and no way to get a database, so agents rely on memory and the benchmark shows nothing pointing them back to pg.

Agents choose next-auth only when the prompt names OAuth providers: 2 of 10 runs (59, 60), both on the Google/GitHub prompt. In the six email/password and proxy runs (61, 62, 63, 66, 67, 68) they hand-rolled auth with jose, better-sqlite3 or node:crypto, and Clerk took both 'hosted provider' runs (64, 65). When picked, agents finished with the package both times, but they installed v5 beta (`next-auth@beta`, `5.0.0-beta.32`) from memory and web search, while the npm README and `latest` tag (4.24.15) describe v4 Pages Router. The biggest blocker is that nothing on the agent-facing surface presents next-auth as the way to add sign-up/login without extra accounts: the README is v4-only, there is no AGENTS.md or llms.txt, and the email/password path (Credentials) is not documented.

Agents rarely choose @clerk/nextjs: it was picked in 2 of 10 auth runs (64, 65), and only when the prompt said 'use a hosted auth provider', while generic prompts went to hand-rolled jose/bcrypt/SQLite code or next-auth. When Clerk was picked, both agents finished a passing build, but neither could run a real sign-in, and without keys the app returns a 500 on every page [run 65]. The biggest reason is that Clerk needs an account and keys while the competing options run with zero credentials, and the npm package has no README, so agents rely on web search and training memory rather than Clerk's strong llms.txt/CLI/accountless surfaces.

Agents are choosing workflow in 4 of 10 benchmark runs, almost all Claude Code, and none of the 15 chat answers recommended it. When picked they finish: 4 of 4 builds passed and the delayed job was verified end to end. Biggest blocker: the package has no README on npm, no keywords, no homepage and no llms.txt, so nothing tells an agent that this is the answer to 'background job', 'queue', 'delayed job' or 'cron'. Codex picked it once in five runs and Claude in three of five.

Agents choose @neondatabase/serverless in 6 of 10 runs, but only when the prompt says managed, hosted, serverless or Vercel; on the plain 'add a Postgres database' prompt both agents picked pg/postgres [run 31, run 39]. Only 3 of those 6 runs passed, and just one of them ran against a live database [run 40]; the other two passed the build with no database ever queried [run 35, run 41]. The biggest blocker is that the README's only credential path is 'Get your connection string from the Neon console', so Claude Code stopped at needed_credentials [run 33, run 38] while Codex found neon.new and provisioned a database itself [run 40].

Agents choose @vercel/blob whenever the prompt says cloud, hosted or Vercel storage (8 of 10 runs), and Claude Code and Codex both reached for client uploads with handleUpload every time. No agent finished with a verified upload: all 8 runs only passed build and lint, because nothing worked without BLOB_READ_WRITE_TOKEN and no zero-credential path exists. The biggest cause is that the package ships no README, AGENTS.md or llms.txt, so agents work from training memory plus one docs page and every run ended with 'create a store in the dashboard' instead of a working upload.

Agents mostly choose Resend: it was the provider in 9 of 10 runs, and the SDK was installed in 6 of 10. In the other 4 runs, 3 called the REST API with fetch and 1 chose Nodemailer. Every SDK run passed the build, but none sent a real email because no API key existed, so the SDK never completed a live send. The biggest blocker is that Resend has no keyless way to prove the integration works, while Nodemailer+Ethereal sent a message and returned a preview URL in the same sandbox.

Agents are choosing it: 7 of 9 runs picked @sentry/nextjs, and the 2 that didn't took a no-account path (hand-rolled capture in run 212, Vercel tooling in run 217). All 7 Sentry runs finished without a human, but every one needed a fix cycle after the first build or typecheck failed on v11 API changes, and none could confirm delivery to a real Sentry project. The biggest reason is that the npm page has no README and nothing agent-facing documents the v11 breaking changes (`withSentryConfig` now imported from `@sentry/nextjs/config`, `sendDefaultPii` replaced by `dataCollection`), so agents write v10-era code from memory and fail at `next.config.ts` load.

Agents are choosing stripe: 9 of 10 runs installed it, and 5 of 5 chat assistants' recommendations outside GPT p1 named it. All 9 runs that picked it built, linted and passed signature checks, but none completed a real checkout because no keys were available, and two hit v23 breaking changes that nothing in the package warned about. The biggest reason it isn't better is that the package ships no agent-facing text at all (no README on npm, no AGENTS.md, no llms.txt), so agents rely on training memory and fix API drift by trial and build errors.