Generating context aware mutants using hazard analysis!
multiplex was accepted to the ASE 2026 Tools and Datasets Track [1] and used to perform an empirical evaluation of LLM-based Hazard-guided mutation techniques at EASE 2026 [2].
Mutation testing introduces synthetic defects into a program under test to assess test suite adequacy. While recent advances in large language models (LLMs) have led to more realistic and hard-to-kill mutants, the field is currently dominated by single-purpose research prototypes that tightly couple the mutant generation technique with the LLM provider and build environment, thereby requiring mutation testing researchers to re-implement techniques for empirical comparison. To address this challenge, this paper introduces multiplex, a modular LLM-based mutation framework. multiplex enables researchers to implement their own LLM-based mutant generation (MG) modules to prototype or compare techniques, as well as configure and create language, LLM or framework-specific modules for extended study. In addition to providing an overview of the system design and implementation, this paper demonstrates how researchers can use multiplex to prototype and compare techniques, along with sharing experiments conducted with multiplex and outlining future extensions. The tool is available at https://github.com/LLM-Mutation/multiplex, and a video demonstration is available at https://youtu.be/l-hj7Pmn3oo.
Empirically Comparing Hazard-Guided LLM Mutation Techniques with Existing LLM- and Rule-Based Approaches
Mutation testing tools normally rely on rule-based operators that mechanically swap and change source code tokens without un- derstanding the code’s purpose. Recently, Large Language Models (LLMs) have enabled mutant generators to consider more of the code’s context and history. However, current LLM-based mutation testing methods have limited prompts that prevent them from un- leashing the full power and creativity of the LLM to produce a diverse set of aggressive, yet realistic and productive, mutants. This paper’s novel approach uses an LLM to interpret a method and re-implement it entirely with mutations guided by hazard analysis, analogous to a programmer misunderstanding a project’s require- ments or an aspect of an algorithm’s implementation. To enable the empirical comparison of the new hazard-guided techniques with both prior LLM-based and traditional mutation testing tools, this paper also presents and applies a framework that integrates repre- sentative LLM-based methods. Using this framework and 279 bugs in 15 projects from the Defects4J dataset, the results show that the hazard-guided techniques can harness both local and cloud-based LLMs to generate compilable, diverse, and powerful mutants that help test cases to detect unique defects not found by other methods.
