SkillForge v6
A skill creator that proves its skills work.
SkillForge routes any skill-related request (use, improve, create, compose), creates new skills through an evidence-driven pipeline, and maintains the health of your whole skill ecosystem. Its core principle: skill quality is a property of behavior, not documents - a skill is done when a fresh agent demonstrably does better with it than without it.
What v6 changed (and why)
v6 is a ground-up rework following a deep external audit (see SKILLFORGE_AUDIT.md on the repo, not shipped with the skill). The headline shifts:
| v5 | v6 |
|---|---|
| 4-agent "unanimous synthesis panel" reads the skill | Skills are executed: baseline (RED) runs before writing, with-skill (GREEN) runs after, behavioral delta is the gate |
| Self-scored "timelessness >= 7" approval | Falsifiable checks moved to lint (validate_skill.py); one adversarial reviewer refutes what lint can't catch |
| 5,049-word SKILL.md with ` | |
| ` "progressive disclosure" | 1,158-word SKILL.md; depth lives in references/ loaded on demand |
| Descriptions = "what this skill does" | Descriptions = trigger conditions only (workflow summaries make agents skip the body) |
launchd background advisor that analyzed / and queued into a file nothing read |
Advisor delivered through Claude Code hooks (SessionStart + UserPromptSubmit), caps enforced, opt-in |
| Personal-directory scanning on by default, hardcoded GitHub handles | Personal Context strictly opt-in with recorded consent; no shipped defaults |
| Hand-rolled YAML parser that failed SkillForge's own SKILL.md | One shared typed parser (scripts/frontmatter.py), 100+ unit tests, and a regression test that SkillForge validates itself |
| Index missed the Claude Code plugin cache entirely | Cross-runtime discovery: personal, Codex, Claude Code plugin cache; deduped; auto-refresh |
The pipeline
Phase 0 TRIAGE index + word-boundary matching -> USE | IMPROVE | CREATE | COMPOSE | CLARIFY
Phase 0b RED GATE fresh subagent attempts the task WITHOUT the skill; no failure = no skill
Phase 1 ANALYSIS load-bearing lenses (Inversion, Pareto, Root Cause), failure-form matching
Phase 2 SPEC tiered (minimal default / full for infrastructure), decisions + WHY
Phase 3 GENERATE fresh-context subagent receives ONLY the spec + baseline failures
Phase 4 GREEN GATE with-skill runs must clear the recorded baseline failures; trigger tests
Phase 5 REVIEW lint (validate_skill.py) + ONE adversarial reviewer charged to refute
Phase 6 SHIP with evals/ - a per-skill regression suite runnable forever after
What no other skill creator has
- Dedup before create (Phase 0): an index of every skill across runtimes answers "should this exist?" first.
- Skills ship with their tests:
evals/(trigger queries + behavioral scenarios) +run_skill_evals.py= regression testing for skills. - Ecosystem doctor:
skillforge_doctor.pyfinds trigger collisions between skills, duplicates, stale file references, budget violations, and pinned models across your entire roster. - Cross-runtime compile: author once,
compile_skill.py --target claude|codex|agentskills. - Friction mining (opt-in):
mine_skill_friction.py --consentfinds skill gaps in your own local session history. - Proactive advisor (opt-in): evidence-backed skill suggestions delivered via hooks, never auto-invoked.
Install (Claude Code)
git clone https://github.com/tripleyak/SkillForge.git /tmp/skillforge
cp -r /tmp/skillforge ~/.claude/skills/skillforge
cd ~/.claude/skills/skillforge && rm -rf README.md LICENSE CONTEXT.md docs .git .gitignore .skillignore index.html assets/images scripts/tests SKILLFORGE_AUDIT.md
cp /tmp/skillforge/commands/skillforge.md ~/.claude/commands/skillforge.md # optional /skillforge command
Optional advisor + hooks (interactive, everything opt-in):
python3 ~/.claude/skills/skillforge/scripts/install_skillforge.py
Requirements: Claude Code (or Codex CLI), Python 3.8+ (stdlib only; PyYAML used if present).
Toolbox
| Command | Purpose |
|---|---|
python3 scripts/discover_skills.py |
Build/refresh the skill index |
python3 scripts/triage_skill_request.py "" --json |
Route a request |
python3 scripts/validate_skill.py |
Full validation + lint |
python3 scripts/run_skill_evals.py [--live] |
Run a skill's regression evals |
python3 scripts/skillforge_doctor.py |
Ecosystem health report |
python3 scripts/init_skill.py --path |
Scaffold (includes evals/) |
python3 scripts/compile_skill.py --target |
Cross-runtime compile |
python3 scripts/package_skill.py ./dist |
Package as .skill |
python3 scripts/mine_skill_friction.py --consent |
Mine local transcripts for skill gaps |
CI: copy assets/templates/github-workflow-skill-ci.yml into .github/workflows/ of any skill repo.
Releases
Version history and detailed release notes live on the releases page. v6.0.0 is a ground-up rework; the audit that drove it ships in-repo as SKILLFORGE_AUDIT.md.
License
MIT - see LICENSE