Beyond the Leaderboard: Multi-Dimensional Evaluation of Dense and Mixture-Of-Experts Models for Automated Program Repair

Automated Program Repair APR with language models is usually evaluated by whether a generated patch passes the test suite, which can hide differences in maintainability, security, and computational cost. We propose a Weighted Quality Index QI, inspired by the ISO/IEC 25010 software quality model,...

From Silicon to Boot Code: Extending Automated Program Repair to Firmware-Layer Security Workarounds

Automated program repair APR research has been constrained to design time. Current techniques localize and fix bugs in RTL or HLS designs before a chip reaches production. Once a hardware vulnerability surfaces post-silicon, the patch must be manually generated: existing automation addresses patc...

Why LLMs Fail: A Failure Analysis and Partial Success Measurement for Automated Security Patch Generation

Large Language Models LLMs show promise for Automated Program Repair APR, yet their effectiveness on security vulnerabilities remains poorly characterized. This study analyzes 319 LLM-generated security patchesacross 64 Java vulnerabilities from the Vul4J benchmark. Using tri-axis evaluation...

Adversarial Bug Reports As a Security Risk in Language Model-Based Automated Program Repair

Large Language Model LLM - based Automated Program Repair APR systems are increasingly integrated into modern software development workflows, offering automated patches in response to natural language bug reports. However, this reliance on untrusted user input introduces a novel and underexplored...