CI configuration: what AI coding agents learn from your pipeline
This check looks for a continuous integration pipeline defined in the repo: GitHub Actions workflows, or a config file for another CI system.
How many repos pass
Of the 371 repos on the leaderboard, 354 pass in full (95%), 8 pass in part (2%) and 9 fail (2%).
Why agents need it
A CI config is a written record of how the project is built and tested. An agent can read it to learn which commands a change has to pass and run them before opening a pull request.
How we score it
- Full pass: at least one
.ymlor.yamlfile in.github/workflows/. - 90%: a config for GitLab CI, CircleCI, Azure Pipelines, Travis CI, Jenkins or Buildkite.
- Fail: none of these. We do not check what the pipeline runs.
The methodology has the formula that turns checks into a score.
How to pass
Add a workflow that runs the tests and the linter on every pull request. Use the same commands you document for local use, so an agent can run them before it pushes.
Which agents count it most
- OpenHands10% of its score
- GPT-5 Codex9% of its score
- Devin9% of its score
- Gemini CLI7% of its score
- Claude Code7% of its score
- Pi6% of its score
- Kimi CLI6% of its score
- Cursor5% of its score
- Aider4% of its score
Pass rate by language
Top-scoring repos that pass
- Rust · 33.1k ★
- apache/superset92.2Python · 75.1k ★
- axios/axios91.9JavaScript · 109.4k ★
- aio-libs/aiohttp91.9Python · 16.6k ★
- C++ · 131k ★
- Rust · 6.7k ★
- TypeScript · 115 ★
- TypeScript · 2 ★
- Python · 46k ★
- scrapy/scrapy90.6Python · 64.8k ★
- Clojure · 49.6k ★
- milvus-io/milvus89.6Go · 46.4k ★
Popular repos that fail it
- golang/go31.2Go · 139.2k ★
- Go · 128.3k ★
- swiftlang/swift49.9Swift · 70.5k ★
- karpathy/nanoGPT19.0Python · 63.7k ★
- Python · 58.6k ★
- ziglang/zig37.5Zig · 43.3k ★
- farzaa/clicky29.4Swift · 7.7k ★
- microsoft/AI24.2Python · 2.9k ★