1BRC with AI - curious and curiouser said the LLM

Wed, 15 Jan 2025 11:48:56 +0000

Did you try turning it off and on first?

(I promise this is the last 1BRC-related post). Read on to discover how large language models (LLMs) performed in generating Java code, their strengths in context-aware development, and challenges such as hallucinations and nuanced errors. Learn what makes a good AI dev tool and what makes a great IDE integration.

A definition of insanity is repeating an action and expecting a different outcome. As developers, we’re used to doing that :-) It’s part of our operating manual.

1 Billion RECORD CHallenge - AI Style

Thu, 07 Nov 2024 10:05:34 +0000

Previously …

I recently wrote about my experiences with getting AI coding tools to find bugs and write tests. I picked five different tools. The selection criteria were simple - free or trial with no credit card needed. I chose three tools designed to help developers and two general-purpose LLMs.

I gave them a 1000-line Java program and asked them to do three things.

Add Javadoc to explain what the code did
Analyse the code for bugs and design weaknesses
Create Junit testcases that would provide reasonable code coverage.

The results were disappointing but, in hindsight, predictable. Overall, the tools were better at creating documentation than finding bugs and, apart from one, were pretty poor at writing test cases.

Copilot on JAVAPRO International

1BRC with AI - curious and curiouser said the LLM

1 Billion RECORD CHallenge - AI Style