Evil Martians · Ruby

Ruby & Rails LLM discoverability scorecard

Ruby and Rails are a great default, for humans and AI agents alike. Yet given a free choice, 13 models picked Ruby 0 times in 1,267 solutions (the open whichlang benchmark). Models reach for what they can see. So we score 93 ecosystem resources on two questions: can an agent find the docs, and would they reach a model's training data? How docs get into training →

51/93ship an llms.txt 16/93do content negotiation 23/93serve .md docs 56/93have a sitemap 6block AI crawlers

01The scorecard

Measured over HTTP, June 2026, against each project's documentation page ( good, missing). Retrieval: can an agent read the docs at request time? Training: would they reach a model's corpus, through code or through the web? The code column holds the corpus's own answer: every docs repo is checked against the official Am I in The Stack? index. A repo already in The Stack skips the web filters, so its web cells dim. Click any heading to sort; what each column means →

Showing 93 of 93
Resource (docs) Retrieval can an agent find it at request time Training would it reach the corpus
robots
allows AI
crawlable
(no WAF)
sitemap llms.txt content
negotiation
.md
routes
in The Stack v3
observed; docs repo license
Common
Crawl
quality
best of 5, ≥3
Core (Ruby Central / Rails Foundation / community-run)
Ruby (language)NOASSERTION4,756/—3.7
Rails GuidesMIT276/—3.8
Rails APIMIT1,845/—3.6
RubyGems GuidesNOASSERTION = CC-BY-SA (Creative Commons Attribution-ShareAlike)12/982.8
Bundler(no license file)212/—
RubyDoc.infoblocks ccbot, gptbot, claudebot, google-extended, applebot-extended4/—3.8
Frontend & View
Hotwire(no license file)6/—1.9
Turbo(no license file)5/—2.0
Stimulus(no license file)3/—2.8
ViewComponentMIT12/—2.8
PhlexMIT44/—3.1
Ruby UI(no license file)96/—
Inertia RailsMIT21/601.9
LookbookMIT16/—1.6
Vite RubyMIT6/—2.1
Web Frameworks
RodaMIT6/—2.0
Sinatra(no license file)37/—2.2
Hanami(no license file)14/—1.9
Bridgetownopted out68/1392.1
JekyllMIT40/2101.9
Rageafter cutoff6/351.7
Data & ORM
SequelMIT125/—2.0
ROM(no license file)15/—1.7
dry-rb(no license file)36/—1.8
AI
RubyLLMMIT65/—2.4
Background, Realtime & Deploy
SidekiqLGPL-3.03/—1.9
AnyCableMIT36/852.0
Kamal(no license file)32/—1.5
Falcon404MIT3/—2.0
KarafkaNOASSERTION = custom, 'All Rights Reserved' for original wiki content136/2092.6
Heroku0/—1.7
Fly.ioApache-2.014/—1.7
Render0/—2.1
RailwayMIT0/—2.2
DigitalOcean0/—1.8
AWS Elastic BeanstalkCC-BY-SA-4.046/—2.4
PlanetScaleApache-2.00/—2.3
SupabaseApache-2.00/—2.1
Neon(no license file)0/—2.2
Tooling & Types
SorbetApache-2.021/862.8
RuboCopMIT415/1,6811.9
RSpecblocks gptbotopted out159/—1.6
TestProfMIT11/—2.1
Pry(no license file)0/—1.7
Brakeman000(no license file)0/—
StandardMIT0/—1.8
SentryFSL-1.1-Apache-2.045/—2.8
AppSignal126/—2.0
New RelicCC-BY-NC-SA-4.028/—1.8
DatadogBSD-3-Clause0/—1.2
Honeybadger27/—2.5
Rollbar(no license file)0/—1.6
Bugsnag9/—1.6
Scout APM(no license file)5/—1.4
SkylightCC-BY-NC-SA-4.023/—1.5
Better Stack0/—1.9
Papertrail4/—
GitHub ActionsCC-BY-4.00/—2.7
Libraries
GraphQL RubyMIT1,804/—2.5
RodauthMIT46/—2.0
Action PolicyMIT17/—2.6
ShrineMIT6/1272.4
Avo(no license file)168/—1.6
ActiveAdminMIT8/—1.8
Ransack404MIT2/—1.4
Pagy404MIT5/—1.5
NokogiriMIT11/221.8
Faraday404MIT0/—
CapistranoMIT6/—2.2
Trailblazer(no license file)19/—
imgproxy(no license file)141/5291.4
FlipperMIT43/701.7
DeviseMIT0/—1.8
PunditMIT0/—1.4
CanCanCanMIT0/—1.4
Community & Resources
GoRails1,167/—2.7
Drifting Rubyblocks gptbotWAF block7/—1.6
RubyEventsblocks ccbot, gptbot, claudebot, google-extended, applebot-extendedWAF block(no license file)2/16,6181.1
Rails at Scale (Shopify)43/1163.3
SF Ruby14/—1.2
Ruby Weekly219/6251.2
Short Ruby279/2931.2
This Week in Rails32/—1.4
Evil Martians114/—1.7
Hotwire Weekly0/—1.2
Write Software Wellblocks ccbot, gptbot, claudebot, google-extended, applebot-extended2/—2.8
Thoughtbot43/—1.7
AppSignal Blog333/1,0752.2
Riding Rails(no license file)57/—1.6
Joe Masilottiblocks ccbot, gptbot, claudebot, google-extended, applebot-extended(no license file)2/1341.6
Code with Jason165/5322.6
Maintainable330/4821.6
Ruby News(no license file)203/—3.0

The second gate: a quality filter

Being crawled is the first gate. A classifier is the second. We scored up to five pages per resource with the open FineWeb-Edu filter, which keeps score ≥ 3. Counting each resource's best page, 19 of 93 clear the bar (top: RubyDoc.info 3.8, Rails Guides 3.8, Ruby (language) 3.7); for 49 even the best of five scores below 2. The filter rewards tutorial prose and penalizes reference docs and code, exactly what developers need most. Docs in a public repo skip this filter entirely; that is what the "in The Stack v3" column measures. Why the code channel decides this →

Licenses, as GitHub reports them

The cell prints the license string GitHub's API returns, verbatim, because that string is what tooling reads. It is lossy: every license GitHub cannot match becomes NOASSERTION ("Other"), so a bespoke permissive license and a source-available one look identical there. Across the 70 resources with a public docs repo:

The reading predicts; the train set decides. The Stack v3 filters file by file, and only the result is observable: 66 of 70 docs repos are in, including karafka/wiki, whose license file says "All Rights Reserved", while karafka/karafka (LGPL) is out and sidekiq/sidekiq (also LGPL) is in. A missing license is no barrier: The Stack keeps unlicensed files. Opting out works: the rspec and bridgetownrb orgs asked to be removed and are absent org-wide. Where we read a license file ourselves, that row's tooltip carries the terms. The full license guide →

02Rails products, tested against the harvest filters

The table above tracks documentation. Models also learn Rails from complete applications, and agentic RL pipelines mine them mechanically. The SWE-bench family selects top-starred repos, derives the test command from CI config, rebuilds the environment in a container, and turns each merged PR that closes an issue and touches tests into one verifiable training task; SWE-bench Multilingual discards about 30% of candidate repos whose tests will not build or run too slowly. Below: the top-starred open-source Rails products, measured against exactly those filters over the GitHub API.

Product stars Code corpus would the code reach a pretraining set RL harvest could a SWE-bench-style pipeline mine it
license
as GitHub reports it
SWH
archive
in The Stack v3
observed
test CI median
run
container
recipe
issue-linked
PRs w/ tests
of last 50 merged
Maybepersonal finance app; archived 2025-07-2454,340AGPL-3.0no test workflow0/50
Mastodonfederated social network50,182AGPL-3.0Ruby Testing14 min0/50
Huginnself-hosted automation agents49,766MITCI8 min0/50
Discoursecommunity forum47,624GPL-2.0Tests14 min0/50
Metasploitpenetration-testing framework38,773NOASSERTIONExtended Tests<1 min5/50
Chatwootcustomer support platform35,659NOASSERTIONFrontend Tests7 min4/50
GitLabDevOps platform (GitHub mirror; development on gitlab.com)24,529NOASSERTIONno test workflow0/50
Foremcommunity platform behind dev.to22,762AGPL-3.0CI9 min3/50
DocuSealdocument signing18,227AGPL-3.0CI5 min0/3
Postalmail delivery platform16,725MITCI3 min2/50
OpenProjectproject management15,811GPL-3.0Test suite24 min0/50
Diasporafederated social network13,657AGPL-3.0CI19 min3/50
Fizzykanban board by 37signals8,044NOASSERTION = O'Saasy License Agreementafter cutoffCI (Main)3 min0/50
Canvas LMSlearning management system6,764AGPL-3.0no test workflow0/50
Redmineproject management (SVN mirror; patches via redmine.org)6,014NOASSERTIONTests26 min
Zammadhelpdesk and ticketing5,837AGPL-3.0CI93 min2/50
Lagousage-based billing (the Rails API behind getlago/lago)431AGPL-3.0Run Spec9 min0/50

Copyleft keeps the Rails flagships out of the permissive code corpus. 6 of 17 products are observed in The Stack v3 train set, and every AGPL- and GPL-licensed product is absent, because v3 keeps permissively licensed and unlicensed files. All six that are in report MIT or NOASSERTION terms. Most products also fail the PR-to-issue linkage filter. 6 of 17 merged at least one issue-closing, test-touching PR in the sampled window, and 5 of 17 have GitHub issues disabled (Discourse and OpenProject triage on their own forums; GitLab and Redmine are mirrors of development that happens elsewhere), so a pipeline mining GitHub finds no harvestable instances there. The runtime filter bites too: Zammad's suite runs a 93-minute median, the kind SWE-bench Multilingual discards. The direct workaround is Multi-SWE-RL: its task pipeline is open, it accepts community contributions, and Ruby is absent from it today.

03What will move the needle

Four levers, ordered by depth, all pushing the same number. Rails is plural by design, so the work is shared defaults and shared conventions. Each layer shows its goal as a live gauge; all of them feed the final boss below.

Layer 0: get into the corpus at all ship now

Crawlable, unblocked87/93
Sitemaps56/93
  • Unblock AI crawlers (CCBot, GPTBot, ClaudeBot, Google-Extended) in robots.txt and at the WAF. One line at RubyEvents frees ~15,775 pages of talks.
  • Add sitemaps, server-render, link internally, earn backlinks.
  • CC-license conference video and publish transcripts: Gemini trains on YouTube, and CC-licensed talks flow into open corpora.

Layer 1: win retrieval and publish comparisons (content) ship now

Win retrieval

Why: an agent fetching a doc should get Markdown it can read. Today it gets HTML to scrape.

Content negotiation16/93
.md routes23/93
llms.txt51/93
  • Serve Markdown by content negotiation and .md routes (Mime::Type.register "text/markdown", :md). Plain HTTP, already used by agents, the durable bet. Ship llms.txt too; it is cheap.
  • Teach rdoc to emit Markdown and content negotiation by default: one change that lifts every gem at once.

Publish the missing comparisons

A model argues for what its corpus argues for, and almost nothing argues, with numbers, that Rails is the better build for these products. 0 of 12 comparisons are solid; the rest are generic takes or missing.

Build … in Railsvs JS fullstack Next.js / Nodevs Python FastAPI / Django
B2B SaaS multi-tenant, team invites, billingmissingmissing
Online store catalog, cart, checkout, admingeneric1generic1
Team inbox shared inbox, collaborative repliesmissingmissing
Issue tracker projects, issues, statuses, commentsgeneric2missing
Approval workflow upload, route for sign-off, trackmissingmissing
Internal admin staff-login CRUD dashboardgeneric2generic3

Sources: 1 monterail.com · 2 kunalganglani.com · 3 goudeketting.nl

Snapshot 2026-06: one web search per task and stack, then judged. solid current, task-specific, with numbers · generic framework pros and cons · missing nothing credible.

Layer 2: make agents fluent in the gems (tools) ship now

Shared gem/agent conventionnone yetAgent Skills conventionfragmented
  • Agree on a convention so a gem ships agent tooling (an MCP endpoint, a skill) the way it ships a README.
  • Converge on one Rails MCP server: agents introspect the app (gems, versions, schema, routes) and pull current docs on demand.
  • Agree on a shared Agent Skills convention so skill packs interoperate.
  • Copy Laravel Boost: official MCP, version-pinned guidelines, on-demand skills. Rails has the parts (fast-mcp, Tidewave, rails-mcp-server, the official Ruby MCP SDK), and a Boost-shaped bundle is emerging in rails-hyperdrive new.

Why: models know standard Rails. The gems, and anything past the training cutoff, are where agents guess. A convention scales the fix across that long tail.

Layer 3: change the training default long game

Ruby in Multi-SWE-benchabsentOpen idiomatic-Rails datasetnone yet
  • Contribute real Rails repos to Multi-SWE-bench and publish an open idiomatic-Rails eval. Agentic benchmarks are where coding ability is now measured, Ruby is absent from them, and adding a language to an eval measurably lifts models on it (MultiPL-T, Bridge-Coder).
  • Grow Ruby's share of the corpus. In The Stack v2, Ruby is ~6.8 GB to Python's ~60, and capability tracks share. Publish an open idiomatic-Rails instruction dataset; contribute permissive Ruby to open corpora like Common Corpus.
  • Re-run the public whichlang benchmark on each new model; it is the scoreboard below.

★ The final boss

Frontier models reach for Ruby more often. One metric, fed by every layer above: given a free choice, 13 models picked Ruby 0 times in 1,267 solutions (they default to Python, JavaScript, and Go). Win condition: the zero starts climbing, model after model.

Ruby picks0/1,267

04Methodology

Probed over HTTP, June 2026, against each project's documentation URL: robots.txt parsed for AI user-agents; crawlability fetched as CCBot, which catches WAF blocks; content negotiation asked with Accept: text/markdown; .md routes and llms.txt checked for a 200. The Stack v3 membership is read from the official Am I in The Stack? index, one lookup per repo owner. An absence is attributed only when the reason is verified (an opt-out issue, a repo public after the 2025-08-07 crawl cutoff) and says "not established" otherwise. The language-choice figure is whichlang's: 13 models, 1,267 classified solutions, 0 Ruby. The Rails-products table is measured in August 2026 over the GitHub API (stars, license, workflows and their run durations, the last 50 merged PRs via GraphQL), the Software Heritage origin API, and the same Am-I-in-The-Stack index.

Why Common Crawl? It seeds most open web corpora (C4, FineWeb, behind GPT and Llama), samples by domain centrality, and never copies a site in full, so coverage is a proxy for whether a model saw the docs at all. It also reflects crawls already taken: the one column you cannot fix this quarter, which is why Layer 0 exists. How it samples →