ChatGPT’s bug fixing performance is competitive to the common deep learning approaches.
ChatGPT can be used to solve coding problems isn’t new, but the researchers highlight that its unique capacity for dialogue with humans gives it a potential edge over other approaches and models.
ChatGPT solved 19 of the 40 Python bugs, putting it on par with CoCoNut (19) and Codex (21). But standard APR methods only solved seven of the issues.
The researchers found that ChatGPT’s success rate with follow-up interactions reached 77.5%.