Javadoc on JAVAPRO International

Are AI Coding tools worth it - RESULTS

Wed, 30 Oct 2024 12:32:45 +0000

Background

In Part 1 , I examined the value of AI tools for developers, especially those that help them create or understand code. The tools selected are all free to use and a mix of general-purpose ones and those that declare they are specifically for coding purposes.

In Part 2, I examined these tools’ ability to find bugs and weaknesses. The results were spotty and not particularly useful or time-saving.

In this part, we’ll look at the results from asking these tools to create JUnit tests for the code and decide who wins (and why).

Are AI Coding tools worth it - Part 2

Wed, 23 Oct 2024 16:04:46 +0000

In Part 1 , I started to examine the value of AI tools for developers, especially those that help them create or understand code. The tools selected are all free to use and are a mix of general-purpose ones and those that declare they are specifically for coding purposes

Tools being evaluated

1: ChatGPT https://chatgpt.com/

2: Claude https://claude.ai/

3: Amazon Q Developer https://aws.amazon.com/q/developer/

4: Codeium https://codeium.com/

5: JetBrains AI Assistant https://www.jetbrains.com/ai/

I asked each tool three questions

“Generate the main Javadoc to describe this class.”
“Assess the class for bugs and coding weaknesses and provide corrections or guidance.”
” Generate Junit test cases to provide around 90% code coverage -especially for edge conditions etc.”

Results so FAR

In Part 1, we saw that they could all generate Javadoc, but none could do it particularly well. ChatGPT came off best as providing the most comprehensive response, but all the tools were prone to hallucinations that made the result misleading.

Are AI coding tools worth it? - part 1

Thu, 17 Oct 2024 10:00:00 +0000

This is the start of a few posts about AI coding tools. This one sets the scene and gives a taste of what I found. In later posts I’ll show more of what I got from using them as is. , explore what the sweet spots might be and end with a harder look at getting value out of them. If you want to skip to the end go here

A long time ago, in a company far, far away…

In the very early 90’s I had the, well, honour is not quite the word I want, the experience may be, to use a prototype of one of the first viable commercial speech-to-text and speech-to-control tools. I used it at work; I used it at home. It was a disaster in all ways. In the first go-around, the training for learning your voice was arduous, literally painful, and demanding. In the second prototype, it would often pick up the sound of the hard disk on the computer and interpret it as text, to much hilarity for all.