An Unusual Test for GPT-5.2, Gemini 3, and Opus 4.5: Competitive Robocode

I've been experimenting with Robocode lately, and it got me thinking: how would modern LLMs handle a problem like this? Robocode isn't a typical programming challenge with a single correct solution. It's open-ended, competitive, and requires balancing multiple strategic concerns simultaneously.

Unlike standard coding benchmarks, Robocode demands spatial reasoning, algorithm design, and real-time tactical decisions. Success isn't measured by passing test cases, it's measured by wins and losses on a competitive ladder.

So, I decided to test GPT-5.2, Gemini 3, and Claude using the same methodology I'd use for my own robots: start with a basic prompt, iterate based on battle results, and see how far each model could climb against human-made strategies.

What is Robocode?

Robocode is a programming game where you code robots in Java to battle in a virtual arena. Your robot has radar, guns, and movement and they are all controlled by code you write. The challenge isn't just writing functional code, but it's writing competitive code that outsmarts opponents.

The game has been around for over 20 years, with a competitive community that has developed sophisticated strategies: Wave Surfing (dodging bullets by predicting their paths), GuessFactor Targeting (statistical aiming), and pattern-matching movement. Humans have spent many years refining these techniques.

To test these LLM-generated robots, I submitted each to robocode-arena.com, where they competed on a live ladder against both human-made and LLM-made robots for real ELO ratings.

The Experiment Setup

I tested 8 models across two tiers:

Basic Models:

• GPT-5.2-instant and GPT-5.1-instant (OpenAI)
• Gemini-3-fast (Google)
• Haiku-4.5 (Anthropic)

Advanced Models:

• GPT-5.2-thinking and GPT-5.1-thinking (OpenAI)
• Gemini-3-thinking (Google)
• Opus-4.5 (Anthropic)

Methodology

For each model, I followed this process:

1. Select the target model on the answer engine's website
2. Start with a basic prompt requesting a competitive Robocode robot
3. Run local battles against ranked robots starting from the bottom of the ladder
4. Analyze performance and provide specific feedback
5. Iterate until the robot stops improving or degrades
6. Submit the best version to robocode-arena.com for official ELO rating

I used the same personal system prompt that I use for answer engines across all models to keep the comparison fair and unbiased.

The Prompts

System Prompt (Used for All Models)

Always Be concise and less verbose.
Don't use images unless asked.
Before responding, consider whether you have sufficient context. If any key detail is uncertain or unclear, ask clarifying questions first.
Clarify acronyms is they are not clear before responding.

Initial Prompt

Create a high-performing Robocode robot by studying the Robocode API and proven winning strategies, then implement and output the final optimized Java robot ready for submission to Robocode Arena.
Place all code in a [CLASS_NAME] class within the [PACKAGE_NAME] package, initializing the Javadoc @version to 1.0 and incrementing it with each update. 
                

Example Follow-up Prompts

After watching the robot battle other ranked robots locally, I note its weaknesses and describe them in my next prompt without pointing out exact problems in the code or hinting at specific strategies to use.

In this prompt, I described to GPT5.1-thinking what it didn't do well:

The wave surfing implementation is not effective against orbital movement. The Robot doesn't maintain optimal distance for firing. Its fire accuracy is low when the opponent is moving.

Here I am telling Haiku-4.5 to come up with a way to evade bullets:

Implement a basic evasion technique to avoid enemy fire, right now the robot remains static and gets easily hit by bullets

In this last example, I asked Gemini-3-thinking to balance dodging and firing:

The robot dodges bullets but doesn't fire. Revisit the firing logic one more time. There is something that limits the robot from firing when the opponent is moving. Find a way to predict movement and fire without waiting for the perfect opportunity.

Results: Basic Models

Results: Advanced Models

Comparative Analysis

Final Rankings

Key Insights

Code Quality vs Performance

Code quality strongly correlated with performance, but correctness mattered more than sophistication which is not a surprise. GPT-5.2-thinking produced the highest-quality code overall, with accurate implementations of wave surfing, hybrid guns, and realistic physics, directly translating to battle wins. GPT-5.1-thinking and Opus-4.5 followed closely with solid advanced architectures. Models like Gemini-3-thinking and Haiku-4.5 demonstrated that good structure without correct indexing or physics results in near-basic performance despite the apparent complexity.

Iteration Efficiency

Opus-4.5 improved the fastest, reaching medium-rank performance in two iterations. GPT-5.2-thinking reached a similar performance but required a couple more iterations. Gemini-3-thinking and GPT-5.1-thinking both reached peak performance within a few iterations, but failed to improve further. Basic models like GPT-5.1-instant, Gemini-3-fast, and Haiku-4.5 required significantly more iterations and guidance, confirming that reasoning models generally need fewer iterations.

Strategic Discovery

Opus-4.5 and Gemini-3-thinking independently adopted and correctly implemented advanced techniques such as Wave Surfing and GuessFactor. GPT-5.2-thinking also adopted these techniques early but didn't implement them correctly and had to fix them after feedback. Gemini-3-fast, GPT-5.2-instant, and Haiku-4.5 required explicit guidance and tended to avoid or break advanced strategies.

Fast vs Thinking Models

The performance gap between fast and thinking models was substantial. Thinking models consistently achieved stronger movement, evasion, and targeting with fewer iterations, while fast models often plateaued at low-ranked performance. The added cost and latency of advanced models proved worthwhile when the goal was competitive robot performance.

What This Reveals About How LLMs Handle Complex Coding Tasks

This experiment exposed capabilities that standard benchmarks miss:

Spatial Reasoning Under Uncertainty

Unlike static coding challenges, Robocode requires reasoning about moving objects, trajectories, and positioning. Thinking models handled dynamic spatial problems more effectively, showing a stronger ability to model trajectories, timing, and positional risk in uncertain environments. Fast models tended to struggle when correct physics, movement prediction, or multi-step spatial reasoning was required.

Iterative Refinement

Real development is iterative. Models with reasoning capabilities incorporated feedback more reliably and maintained context across iterations. Faster models were only able to apply specific fixes, lost earlier intent, and regressed even with explicit, repeated guidance.

Autonomous Strategy Selection

Reasoning models were able to identify and select appropriate advanced strategies on their own, demonstrating an understanding of which techniques mattered and how they fit the problem. Fast models tended to default to basic implementations, and when pushed to adopt advanced strategies, those implementations were often incomplete, incorrect, or unstable.

Trade-off Management

Every Robocode decision involves trade-offs: aggressive vs defensive, accuracy vs fire rate, CPU time vs complexity. Reasoning models could balance competing goals such as accuracy, aggression, efficiency, and survivability. Fast models frequently over-optimized a single dimension which led to brittle behavior.

Limitations and Bias

This experiment has clear caveats:

• My prompts and iteration strategy affected all results
• 1v1 combat only, didn't test melee or teams
• Created a single robot per model, though with multiple iterations
• Didn't factor API costs or latency into the comparison

Conclusion

Traditional benchmarks tell us if a model can solve a problem. Robocode showed me how models approach open-ended challenges: their strategic thinking, iteration efficiency, and ability to incorporate feedback.

Thinking models consistently delivered more robust solutions with fewer iterations and responded better to feedback. This appears to stem from longer iteration cycles, during which they implicitly research, validate, and refine their approach, likely performing multiple internal passes before producing a response. Their structured reasoning helps catch obvious issues early, whereas fast models tend to generate code token-by-token, often resulting in implementations that compile but fail to form a coherent or stable strategy.

The most surprising outcome was that fast models were still capable of producing functional robots, particularly in the GPT-5.2-instant case, which could defeat some low-ranked human-made bots after only a few rapid iterations. Even though testing and feedback took longer than generation itself, the fact that a fast, low-cost model could solve a problem involving physics, movement prediction, and risk assessment (not just syntax) was genuinely impressive.

These results highlight the rapid progress of LLMs toward producing high-quality, well-structured code that can match or exceed what many developers would write, even for complex, dynamic problems. While it's unclear when this progress will plateau, the current pace already delivers substantial value relative to cost, especially when models are chosen appropriately for the complexity and risk of the task.

The robots are still battling on robocode-arena.com. You can see the live ladder rankings at the leaderboard or submit your own robot to the arena.