BAAI turned 1,000 GitHub repositories into 5,000 agent skills for about $40 each
The DisCo framework reports an MLE-bench improvement from 31.1% to 72.9%, with the skill library published for inspection.
The Beijing Academy of Artificial Intelligence has published DisCo, a framework that converts existing GitHub repositories into reusable agent skills. It processed 1,000 repositories into roughly 5,000 skills at about $40 per repository, and reports MLE-bench performance rising from 31.1 percent to 72.9 percent.
The skill library is published and inspectable, which is unusual enough to be the most important sentence in the paper. A claimed benchmark gain that depends on an artefact nobody can examine is an assertion; one where the artefact is available is a result other people can attack.
What the framework actually does
The premise is that most of what an agent needs to do has already been implemented by somebody. A GitHub repository contains working code for a task, and the useful abstraction is not the repository but the capability inside it — extracted, documented and made callable.
DisCo automates that extraction. Forty dollars per repository, at 1,000 repositories, is $40,000 for a library of 5,000 skills. That is inexpensive relative to any hand-built alternative, and it is cheap enough that the approach scales to the long tail of software rather than to a curated set.
The MLE-bench gain — machine learning engineering tasks, from 31.1 percent to 72.9 percent — is more than a doubling. If it holds, the implication is that a substantial portion of what limits agents on technical work is not reasoning but the absence of well-formed tools.
Read against the week's other benchmark
τ^τ-Bench, published five days later, found that the best AI systems reach 23.9 percent on building customer-service agents where expert humans reach 82.2 percent.
Put the two results together and a consistent picture emerges. Given good reusable components, agents perform far better than without. Asked to make the design decisions that produce good components, they perform far worse than experts.
Which suggests the durable asset in agent engineering is the skill library, not the model — an argument that has commercial expression this month. Anthropic's alignment research treats reinforcement-learning environments as safety-critical infrastructure; SkillForge, published in late August, argues explicitly that agent skills should be maintained as software rather than accumulated as prompt fragments.
The licensing question the paper does not answer
Converting a GitHub repository into an agent skill is a derivative act, and the paper does not appear to address what licences the 1,000 source repositories carry.
GitHub hosts code under every licence from public domain to GPL to no licence at all — and code with no licence is not permissively available, it is simply unlicensed. A skill extracted from GPL code and distributed in a library raises obligations; one extracted from unlicensed code raises a different problem.
That matters more this month than it would have last year. The US Justice Department filed in support of OpenAI's fair use position on 2 September; the Seattle Times and Newsday sued three days later; authors are contesting the division of Anthropic's copyright settlement.
Whether transforming source code into an agent skill is fair use, a derivative work, or something else has not been litigated. A published library of 5,000 such skills is a considerably more concrete test case than a model whose training data cannot be inspected.
BAAI has not published the licence distribution of the source repositories.
Runs the newsroom. Rename this profile in the studio to your own byline.
Related
Every weekday, the AI stories that moved money or shipped code.
No cross-posting, unsubscribe anytime. See all newsletters