1 Billion RECORD CHallenge - AI Style

Thu, 07 Nov 2024 10:05:34 +0000

Previously …

I recently wrote about my experiences with getting AI coding tools to find bugs and write tests. I picked five different tools. The selection criteria were simple - free or trial with no credit card needed. I chose three tools designed to help developers and two general-purpose LLMs.

I gave them a 1000-line Java program and asked them to do three things.

Add Javadoc to explain what the code did
Analyse the code for bugs and design weaknesses
Create Junit testcases that would provide reasonable code coverage.

The results were disappointing but, in hindsight, predictable. Overall, the tools were better at creating documentation than finding bugs and, apart from one, were pretty poor at writing test cases.

Are AI Coding tools worth it - RESULTS

Wed, 30 Oct 2024 12:32:45 +0000

Background

In Part 1 , I examined the value of AI tools for developers, especially those that help them create or understand code. The tools selected are all free to use and a mix of general-purpose ones and those that declare they are specifically for coding purposes.

In Part 2, I examined these tools’ ability to find bugs and weaknesses. The results were spotty and not particularly useful or time-saving.

In this part, we’ll look at the results from asking these tools to create JUnit tests for the code and decide who wins (and why).

Claude.ai on JAVAPRO International

1 Billion RECORD CHallenge - AI Style

Are AI Coding tools worth it - RESULTS

Background