Evil Martians · Ruby

Ruby & Rails LLM discoverability scorecard

Ruby and Rails are a great default, for humans and AI agents alike. Yet models rarely reach for them: across the open whichlang benchmark, 13 models picked Ruby 0 times in 1,267 solutions. That is a discoverability problem, and it starts with the docs. We score 92 ecosystem resources on whether agents can find them and whether they would reach a model's training data. How docs get into training →

47/92ship an llms.txt 14/92do content negotiation 22/92serve .md docs 52/92have a sitemap 7block AI crawlers

01The scorecard

Measured over HTTP, June 2026, against each project's documentation page ( good, missing). Columns split into two questions: Retrieval, can an agent find the docs at request time, and Training, would they reach a model's corpus, either through the code corpus (docs in a public, permissively-licensed repo, which skips the web filters) or the web corpus (Common Crawl coverage, then best-of-5 FineWeb-Edu quality). Where the code corpus already qualifies, the web cells are dimmed as secondary. Click any heading to sort; what each column means →

Showing 92 of 92
Resource (docs) Retrieval can an agent find it at request time Training would it reach the corpus
robots
allows AI
crawlable
(no WAF)
sitemap llms.txt content
negotiation
.md
routes
code corpus
docs repo license
Common
Crawl
quality
best of 5, ≥3
Core (Ruby Central / Rails Foundation / community-run)
Ruby (language)custom4,097/—3.7
Rails GuidesMIT442/—3.8
Rails APIMIT4,956/—3.6
RubyGems Guidescustom35/—2.8
Bundlerno license119/1,440
RubyDoc.infoblocks ccbot, gptbot, claudebot, google-extended, applebot-extended2/—3.8
Frontend & View
Hotwireno license8/—1.9
Turbono license3/—2.0
Stimulusno license0/—2.8
ViewComponentMIT7/—2.8
PhlexMIT4/403.1
Ruby UIno license105/—
Inertia RailsMIT15/—1.9
LookbookMIT18/—1.6
Vite RubyMIT0/—2.1
Web Frameworks
RodaMIT8/—2.0
Sinatrano license33/—2.2
Hanamino license14/—1.9
BridgetownMIT16/1392.1
JekyllMIT52/2101.9
Rageno license5/341.7
Data & ORM
SequelMIT359/—2.0
ROMno license16/—1.7
dry-rbno license72/—1.8
AI
RubyLLMMIT15/232.4
Background, Realtime & Deploy
Sidekiqno license2/—1.9
AnyCableMIT16/772.0
Kamalno license19/—1.5
Falcon404MIT0/—2.0
Karafkacustom1/2092.6
Heroku0/—1.7
Fly.ioApache-2.053/—1.7
Render0/—2.1
RailwayMIT0/—2.2
DigitalOcean0/—1.8
AWS Elastic BeanstalkCC-BY-SA-4.038/—2.4
PlanetScaleApache-2.00/—2.3
SupabaseApache-2.00/—2.1
Neonno license0/—2.2
Tooling & Types
SorbetApache-2.021/862.8
RuboCopMIT486/1,5881.9
RSpecblocks gptbotno license150/—1.6
TestProfMIT11/—2.1
Pryno license2/—1.7
Brakeman000no license0/—
StandardMIT0/—1.8
SentryFSL-1.1-Apache-2.063/—2.8
AppSignal27/—2.0
New RelicCC-BY-NC-SA-4.023/—1.8
DatadogBSD-3-Clause0/—1.2
Honeybadger10/—2.5
Rollbarno license0/—1.6
Bugsnag7/—1.6
Scout APMno license2/—1.4
SkylightCC-BY-NC-SA-4.023/—1.5
Better Stack0/—1.9
Papertrail1/—
GitHub ActionsCC-BY-4.00/—2.7
Libraries
GraphQL RubyMIT1,801/—2.5
RodauthMIT35/—2.0
Action PolicyMIT2/—2.6
ShrineMIT4/1252.4
Avono license46/—1.6
ActiveAdminMIT6/—1.8
Ransack404MIT0/—1.4
Pagy404MIT1/—1.5
NokogiriMIT12/—1.8
Faraday404MIT0/—
CapistranoMIT2/—2.2
Trailblazerno license38/—
imgproxyno license191/5291.4
FlipperMIT32/701.7
DeviseMIT4/—1.8
PunditMIT0/—1.4
CanCanCanMIT0/—1.4
Community & Resources
GoRails2,027/—2.7
Drifting Rubyblocks gptbotWAF block0/—1.6
RubyEventsblocks ccbot, gptbot, claudebot, google-extended, applebot-extendedWAF blockno license1/16,3281.1
Rails at Scale (Shopify)21/1143.3
Ruby Weekly301/6211.2
Short Ruby228/2931.2
This Week in Rails30/—1.4
Evil Martians132/—1.7
Hotwire Weekly0/—1.2
Write Software Wellblocks ccbot, gptbot, claudebot, google-extended, applebot-extended2/—2.8
ThoughtbotWAF block1,161/—1.7
AppSignal Blog283/1,0682.2
Riding Railsno license51/—1.6
Joe Masilottiblocks ccbot, gptbot, claudebot, google-extended, applebot-extendedno license1/1341.6
Code with Jason765/—2.6
Maintainable60/4801.6
Ruby Newsno license153/—3.0

The second gate: would the docs survive the quality filter?

Being crawled is the first gate. The second is a quality classifier. We scored up to five pages per resource with the open FineWeb-Edu filter (kept at score ≥ 3). Even counting each resource's best page, only 19 of 92 clear the bar (top: RubyDoc.info 3.8, Rails Guides 3.8, Ruby (language) 3.7); for 48 the best of five scores below 2. The filter rewards educational prose and penalizes reference and code, exactly the docs developers need.

There is a way around it. Docs in a public, permissively-licensed repo reach the code corpus (The Stack) and skip the quality filter entirely, the Training column shows which resources qualify. And once a snippet is in public code and copied widely, models reproduce it verbatim: Supabase Auth and Resend's quickstart both fail this filter yet Claude recites them from memory. Read the full guide →

02What will move the needle

Four levers, ordered by depth, each acting on the same number. Rails is plural by design (omakase defaults and swappable adapters), so the job is to strengthen the default and agree on shared conventions. Each layer shows its goal as a live gauge; together they feed the final boss below.

Layer 0: get into the corpus at all ship now

Crawlable, unblocked85/92
Sitemaps52/92
  • Unblock AI crawlers (CCBot, GPTBot, ClaudeBot, Google-Extended) in robots.txt and at the WAF. The RubyEvents one-line fix alone unlocks ~15,775 pages of talks.
  • Add sitemaps, server-render, link internally, earn high-authority backlinks.
  • CC-license and transcribe conference video; Google trains Gemini on YouTube transcripts and CC-licensed talks flow into open corpora.

Layer 1: win retrieval and publish comparisons (content) ship now

Win retrieval

Why: an agent fetching a Rails or gem doc at request time should get current Markdown it can read, instead of HTML it has to scrape.

Content negotiation14/92
.md routes22/92
llms.txt47/92
  • Serve Markdown via content negotiation and .md routes (Mime::Type.register "text/markdown", :md). A real HTTP standard agents already use, the durable bet. Ship llms.txt too, cheaply.
  • Make rdoc emit Markdown + content negotiation by default; the keystone that lifts every gem at once.

Publish the missing comparisons

Why: a model reaches for what the corpus argues for, and today almost nothing argues, with numbers, that Rails is the better build for these product shapes. 0 of 12 comparisons are solid (current, task-specific, with real numbers); the rest are generic framework takes or absent.

Build … in Railsvs JS fullstack Next.js / Nodevs Python FastAPI / Django
B2B SaaS multi-tenant, team invites, billingmissingmissing
Online store catalog, cart, checkout, admingeneric1generic1
Team inbox shared inbox, collaborative repliesmissingmissing
Issue tracker projects, issues, statuses, commentsgeneric2missing
Approval workflow upload, route for sign-off, trackmissingmissing
Internal admin staff-login CRUD dashboardgeneric2generic3

Sources: 1 monterail.com · 2 kunalganglani.com · 3 goudeketting.nl

Snapshot 2026-06, by web search per task × stack, then judged. solid current + task-specific + numbers · generic framework pros/cons or boilerplates · missing nothing credible. It's a web search, refreshed each pass.

Layer 2: make agents fluent in the gems (tools) ship now

Shared gem/agent conventionnone yetAgent Skills conventionfragmented
  • Agree a convention so any gem maintainer ships agent-discoverable tooling, an MCP endpoint or a skill, the way they already ship a README.
  • Converge a Rails MCP server: let agents introspect the app (gems, versions, schema, routes) and pull current per-gem docs on demand.
  • Agree a shared Agent Skills convention so skill packs interoperate.
  • Copy Laravel Boost (official MCP, version-pinned guidelines, on-demand skills, tools). Rails has the parts (fast-mcp, Tidewave, rails-mcp-server) on the official Ruby MCP SDK, and a Boost-shaped bundle of MCP + skills + guidelines is emerging in rails-hyperdrive new.

Why: standard Rails is in the training set; the gems, and anything past the cutoff, are where agents guess. A maintainer convention is what scales the fix across that long tail.

Layer 3: change the training default long game

Ruby in Multi-SWE-benchabsentOpen idiomatic-Rails datasetnone yet
  • Contribute real Rails repos to Multi-SWE-bench (the repo-level agentic benchmark, which takes open contributions) and publish an open idiomatic-Rails eval. Ruby is in MultiPL-E's HumanEval/MBPP puzzles but absent from the agentic benchmarks, where modern coding ability is measured; adding a language to an eval measurably improves models on it (MultiPL-T, Bridge-Coder).
  • Grow Ruby's share of the training corpus. Code models also learn from curated GitHub archives (Software Heritage → The Stack), where Ruby is ~6.8 GB against Python's ~60 and JavaScript's ~65, and capability tracks that share. Publish an open, idiomatic-Rails instruction dataset and contribute permissively-licensed Ruby to open corpora like Common Corpus; rebalancing a corpus toward under-represented languages measurably lifts them.
  • Keep the public whichlang benchmark as the scoreboard for the final boss below, and re-run it on each new model.

★ The final boss

Frontier models reach for Ruby more often. The single metric every layer above serves, measured by the public whichlang benchmark: given a free choice of language across 13 models, Ruby was picked 0 times in 1,267 generated solutions (the defaults are Python, JavaScript, and Go). Win condition: that zero starts climbing, model after model, as agents pick Ruby whenever it is the better fit.

Ruby picks0/1,267

03Methodology

All indicators probed over HTTP, June 2026, against each project's documentation URL: robots.txt parsed for AI user-agents (CCBot, GPTBot, ClaudeBot, Google-Extended) with Disallow: /; crawlability tested by fetching as CCBot (to catch Cloudflare/WAF blocks); content negotiation via Accept: text/markdown; .md routes and llms.txt checked for a 200. The language-choice figure is from the open whichlang benchmark (13 models, 1,267 classified solutions, 0 Ruby; github.com/chad/whichlang). That the same models write Rails competently when instructed is our own informal observation.

Why Common Crawl? It's the open web crawl that seeds most LLM pretraining corpora (C4, RefinedWeb, FineWeb, behind GPT, Llama, and others). Common Crawl is a sampled, English-biased slice of the web: it picks domains and pages by harmonic centrality under a fixed budget, so a project's CC coverage is a proxy for whether a model has seen its docs at all. Inclusion is necessary but not sufficient: the corpora are built from heavily filtered, deduplicated derivatives, so a crawled page must also clear a quality filter to reach training. It's the one column you cannot fix this quarter (it reflects crawls already taken), which is why getting in, via sitemaps, internal links, backlinks, and unblocking bots, is Layer 0.