Chatgpt on JAVAPRO International

1BRC with AI - curious and curiouser said the LLM

Wed, 15 Jan 2025 11:48:56 +0000

Did you try turning it off and on first?

(I promise this is the last 1BRC-related post). Read on to discover how large language models (LLMs) performed in generating Java code, their strengths in context-aware development, and challenges such as hallucinations and nuanced errors. Learn what makes a good AI dev tool and what makes a great IDE integration.

A definition of insanity is repeating an action and expecting a different outcome. As developers, we’re used to doing that :-) It’s part of our operating manual.

1 Billion RECORD CHallenge - AI Style

Thu, 07 Nov 2024 10:05:34 +0000

Previously …

I recently wrote about my experiences with getting AI coding tools to find bugs and write tests. I picked five different tools. The selection criteria were simple - free or trial with no credit card needed. I chose three tools designed to help developers and two general-purpose LLMs.

I gave them a 1000-line Java program and asked them to do three things.

Add Javadoc to explain what the code did
Analyse the code for bugs and design weaknesses
Create Junit testcases that would provide reasonable code coverage.

The results were disappointing but, in hindsight, predictable. Overall, the tools were better at creating documentation than finding bugs and, apart from one, were pretty poor at writing test cases.

Are AI Coding tools worth it - RESULTS

Wed, 30 Oct 2024 12:32:45 +0000

Background

In Part 1 , I examined the value of AI tools for developers, especially those that help them create or understand code. The tools selected are all free to use and a mix of general-purpose ones and those that declare they are specifically for coding purposes.

In Part 2, I examined these tools’ ability to find bugs and weaknesses. The results were spotty and not particularly useful or time-saving.

In this part, we’ll look at the results from asking these tools to create JUnit tests for the code and decide who wins (and why).

Are AI Coding tools worth it - Part 2

Wed, 23 Oct 2024 16:04:46 +0000

In Part 1 , I started to examine the value of AI tools for developers, especially those that help them create or understand code. The tools selected are all free to use and are a mix of general-purpose ones and those that declare they are specifically for coding purposes

Tools being evaluated

1: ChatGPT https://chatgpt.com/

2: Claude https://claude.ai/

3: Amazon Q Developer https://aws.amazon.com/q/developer/

4: Codeium https://codeium.com/

5: JetBrains AI Assistant https://www.jetbrains.com/ai/

I asked each tool three questions

“Generate the main Javadoc to describe this class.”
“Assess the class for bugs and coding weaknesses and provide corrections or guidance.”
” Generate Junit test cases to provide around 90% code coverage -especially for edge conditions etc.”

Results so FAR

In Part 1, we saw that they could all generate Javadoc, but none could do it particularly well. ChatGPT came off best as providing the most comprehensive response, but all the tools were prone to hallucinations that made the result misleading.

Are AI coding tools worth it? - part 1

Thu, 17 Oct 2024 10:00:00 +0000

This is the start of a few posts about AI coding tools. This one sets the scene and gives a taste of what I found. In later posts I’ll show more of what I got from using them as is. , explore what the sweet spots might be and end with a harder look at getting value out of them. If you want to skip to the end go here

A long time ago, in a company far, far away…

In the very early 90’s I had the, well, honour is not quite the word I want, the experience may be, to use a prototype of one of the first viable commercial speech-to-text and speech-to-control tools. I used it at work; I used it at home. It was a disaster in all ways. In the first go-around, the training for learning your voice was arduous, literally painful, and demanding. In the second prototype, it would often pick up the sound of the hard disk on the computer and interpret it as text, to much hilarity for all.