Are AI Coding tools worth it - Part 2

Wed, 23 Oct 2024 16:04:46 +0000

In Part 1 , I started to examine the value of AI tools for developers, especially those that help them create or understand code. The tools selected are all free to use and are a mix of general-purpose ones and those that declare they are specifically for coding purposes

Tools being evaluated

1: ChatGPT https://chatgpt.com/

2: Claude https://claude.ai/

3: Amazon Q Developer https://aws.amazon.com/q/developer/

4: Codeium https://codeium.com/

5: JetBrains AI Assistant https://www.jetbrains.com/ai/

I asked each tool three questions

“Generate the main Javadoc to describe this class.”
“Assess the class for bugs and coding weaknesses and provide corrections or guidance.”
” Generate Junit test cases to provide around 90% code coverage -especially for edge conditions etc.”

Results so FAR

In Part 1, we saw that they could all generate Javadoc, but none could do it particularly well. ChatGPT came off best as providing the most comprehensive response, but all the tools were prone to hallucinations that made the result misleading.

Claude.io on JAVAPRO International

Are AI Coding tools worth it - Part 2

Tools being evaluated

I asked each tool three questions

Results so FAR