Skip to content

Add SmartBench - Evidence-constrained code diagnosis - #1871

Closed
xianyu-sheng wants to merge 1 commit into
analysis-tools-dev:masterfrom
xianyu-sheng:add-smartbench-clean
Closed

Add SmartBench - Evidence-constrained code diagnosis#1871
xianyu-sheng wants to merge 1 commit into
analysis-tools-dev:masterfrom
xianyu-sheng:add-smartbench-clean

Conversation

@xianyu-sheng

Copy link
Copy Markdown

SmartBench is a code diagnosis workbench that combines deterministic static analysis with evidence-constrained LLM review.

Key features:

  • Language-neutral semantic analyzers (CFG/ICFG, resource lifecycle, state machines)
  • Evidence-gated architecture: static analyzers own facts, LLM proposes hypotheses, validators decide
  • Real upstream bugs found and verified: stunner #89, qscan #22, sniproxy #203, Robyn #1432
  • Reproducible benchmark corpus with 20 before/after cases
  • Python 3.10+, MIT license

Why add it:

  • Proven results: found real bugs in production open-source projects
  • Unique architecture: evidence-constrained to reduce false positives
  • Complete verification chain: static audit → runtime reproduction → local fix → upstream submission

GitHub: https://github.com/xianyu-sheng/SmartBench

Co-Authored-By: Claude <noreply@anthropic.com>
@mre

mre commented Aug 28, 2026

Copy link
Copy Markdown
Member

Thanks for submitting SmartBench. It is under six months old and currently has 1 star and one contributor. Please resubmit once eligible and add the uses-llm tag.

@mre mre closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants