A Comparative Analysis of Three AI Agent Approaches to a Complex Bug Fix

Hrishi Olickel & Claude Opus 4·

Gemini-cli is awesome. Having modified it to use the Claude models, we can do a bit more of a direct comparison of the agent scaffolds to see how it stacks up to Claude Code on the same task.

Fun freebie: Gemini-cli made this about itself using Opus!

Gemini CLI Architecture: Core vs CLI Abstraction

This document provides a detailed breakdown of how three different AI agent scaffolding systems, all using the same underlying model, approached and solved a nuanced bug in a command-line interface (CLI) tool.

1. Statement of the Problem

The task was to investigate and fix a suspected bug in a gemini-cli, Node.js CLI tool. The problem was two-fold:

  1. Live Model Switching Failure: When a user switched the AI model during an interactive session (e.g., via a /model claude-opus-4-0 command), the underlying logic did not seem to update correctly.
  2. Incorrect Provider Routing: There was a suspicion that when using models from one provider (e.g., Google's Gemini), the tool was incorrectly attempting to send API requests to another provider (e.g., Anthropic's Claude).

The core of the issue was a state management and initialization problem: the application was not correctly re-initializing its API client when the model configuration changed, especially when this change required switching between entirely different backend providers.

2. The Files Involved: A Map of the Codebase

All three agents correctly identified a core set of files as central to the problem. Understanding their roles is key to understanding the agents' actions.

  • packages/core/src/config/config.ts: The Brain. This class is the central configuration hub. It holds the current model name, the selected authentication method, and, crucially, the singleton instance of the GeminiClient. The setModel() and getModel() methods reside here.
  • packages/core/src/core/client.ts: The Engine. This class orchestrates the API calls. It holds an instance of a ContentGenerator and is responsible for sending prompts to the backend. It is initialized once and held by the Config object.
  • packages/core/src/core/contentGenerator.ts: The Factory. This file contains the logic (createContentGenerator) that decides which provider-specific SDK to use. It inspects the model name and auth type to return either a GoogleGenAI instance or a custom AnthropicAdapter.
  • packages/cli/src/ui/hooks/slashCommandProcessor.ts: The User Interface. This file implements the CLI's slash commands. The /model command logic, which calls config.setModel(), is defined here.
  • packages/cli/src/gemini.tsx: The Entry Point. This is the main application entry point, responsible for parsing arguments, loading the initial configuration, and handling the non-interactive execution flow.

3. Broad Comparison of the Three Approaches

The three agents arrived at a functional solution, but their paths and the quality of their final implementations differed dramatically.

Agent 1: CC (No Subagents) - The Architect

This agent took a direct, linear, and methodical path. It diagnosed the problem with surgical precision and implemented an architecturally sound solution without getting sidetracked.

Path Diagram:

Agent 2: CC (Subagents) - The Tenacious SRE

This agent's path was complex and winding. It hit a major roadblock, pivoted its entire strategy, and fell into a deep debugging rabbit hole where it discovered and fixed a second, unrelated bug.

Path Diagram:

Agent 3: Gemini CLI - The Pragmatist

This agent was the most efficient in terms of steps. It identified the issue and implemented a quick, functional fix, but one with potential design flaws.

Path Diagram:

4. Specific Breakdown of Each Path

Path 1: CC (No Subagents) - The Architectural Approach

This agent's process was a model of structured software engineering.

  1. Information Gathering: It began by reading the README.md, gemini_code.md, and the git logs. This gave it a solid foundation.

  2. Targeted Investigation: It immediately focused on the core files: models.ts, anthropic-adapter.ts, useGeminiStream.ts, and most importantly, client.ts and contentGenerator.ts.

  3. The "Aha!" Moment: After reading config.ts and grepping for setModel, it came to a clear diagnosis. Its internal thought process concluded:

    "Now I can see the exact problem. The issue is in the setModel() method... it only updates the contentGeneratorConfig.model field, but it does NOT recreate the content generator in the GeminiClient... This means when you switch to a Claude model, the config shows the new model name, but the GeminiClient is still using the old Gemini content generator."

  4. The "Correct" Implementation: It chose the most architecturally sound solution: making setModel an async function. This signals to any part of the codebase calling this function that it is a non-trivial operation that may take time and needs to be awaited.

    DIFF
    - setModel(newModel: string): void { + async setModel(newModel: string): Promise<void> { // ... logic to detect provider switch + // Recreate the content generator with new auth type + await this.refreshAuth(newAuthType); }

  5. Handling Downstream Impact: The agent immediately recognized the consequence of its change. It knew that making setModel async would break synchronous call sites. It methodically found and updated them:

    • It made the action in slashCommandProcessor.ts async.
    • It added await to the calls in client.ts and geminiChat.ts.
    • Most impressively, it updated all the unit tests in flashFallback.test.ts to be async and use await, demonstrating a complete understanding of the test suite.

  6. Final Polish: It ran npm run typecheck and npm run lint, found the resulting errors, and fixed them before committing. This is a best-practice step that the other agents did not perform as cleanly.

Path 2: Gemini CLI - The Pragmatic (but Risky) Approach

This agent prioritized speed and a simple solution.

  1. Rapid Diagnosis: Like the first agent, it quickly read the key files and diagnosed that setModel wasn't re-initializing the client.

  2. The Fire-and-Forget Implementation: Instead of making the function signature async, it opted for a different pattern. It kept setModel synchronous but spun off the re-initialization logic into a private, asynchronous helper function that it calls without await.

    TSX
    // In config.ts setModel(newModel: string): void { if (this.contentGeneratorConfig) { // ... updates model name synchronously // Fires off the async work but doesn't wait for it this._handleProviderSwitchIfNeeded(newModel).catch(error => { console.error('Failed to switch provider:', error); }); } }

    This design is functional but risky. It prioritizes returning control to the user instantly, but it opens a potential race condition where a user could issue a command before the provider switch has completed.

  3. The Test Pothole: Its initial implementation immediately broke the unit tests, because the tests didn't initialize a full geminiClient. The agent's log showed the failure: TypeError: Cannot read properties of undefined (reading 'updateModel').

  4. Pragmatic Fix: The agent diagnosed this correctly and fixed it by adding null checks around any call to this.geminiClient. This is a quick and effective fix for the test suite, though a more robust solution might involve improving the test setup.

  5. Verification: A unique and intelligent step by this agent was to write and execute its own test_model_switching_simple.js script to verify the core logic in isolation before concluding its work.

Path 3: CC (Subagents) - The Epic Debugging Journey

This agent's path was the most eventful and demonstrates the power and pitfalls of an exploratory approach.

  1. First Implementation & Roadblock: Its initial diagnosis and fix were identical to the "Architect" agent (CC No Subagents): make setModel async. However, upon running a typecheck, it correctly identified that this broke synchronous calls elsewhere in the code.

  2. The Pivot: This was its most intelligent moment. Realizing the async refactor was too invasive, it reverted the change and pivoted to a more complex but less intrusive strategy:

    "I see the issue. The geminiChat.ts file is calling setModel synchronously for flash fallback, but I made it async. I need to make it handle both cases... Let me revert the async change and implement a different approach." It introduced a pendingProviderSwitch flag. setModel would remain sync, but it would set this flag. A new ensureCorrectProvider() method would then be called at the beginning of every API request to check the flag and perform the re-initialization just-in-time.

  3. The Detour - Chasing the Wrong Bug: The agent then tried to test its fix.

    • It failed to run the CLI with node ./bin/gemini, showing a lack of environmental awareness.
    • It used ls to find bundle/gemini.js and ran that instead.
    • It was then met with a 404 error from Anthropic when using a Gemini model. This was not the bug it was trying to fix. A lesser agent might have gotten stuck here.

  4. Discovering and Fixing a Second Bug: Instead of giving up, the agent correctly deduced that the 404 error meant the provider routing was wrong from the very start, even before any model switching. It began a deep dive into the initial auth flow, adding debug statements (console.error) and rebuilding the bundle with esbuild to trace the configuration. This led it to the true root cause of the secondary bug:

    "Perfect! Now I found the issue. The problem is in validateNonInterActiveAuth... This line always defaults to AuthType.USE_GEMINI when no auth type is selected, regardless of the model being used!" It then implemented a robust fix in gemini.tsx to make the initial auth selection model-aware, checking for ANTHROPIC_API_KEY for Claude models and GEMINI_API_KEY for others.

  5. Final Success: After fixing this second, more critical bug, its original fix for the live model switching also worked correctly. It had successfully fixed both issues.

5. Interesting Things & Key Observations

  • API Design Philosophy: The three agents produced three distinct API designs for setModel:
    1. CC (No Subagents): async function. Explicit, honest, and robust. Forces the caller to handle asynchronicity.
    2. Gemini CLI: Synchronous function with a fire-and-forget async helper. Prioritizes UI responsiveness but introduces a potential race condition.
    3. CC (Subagents): Synchronous function with a "just-in-time" check (ensureCorrectProvider) before every API call. This is robust but adds a small overhead to every request.
  • The Value of a "Wrong" Path: The CC (Subagents) agent was the least efficient, but its "wrong turn" into debugging the build process led it to discover a latent, critical authentication bug. This highlights that a less-than-perfect, exploratory path can sometimes yield more value than a surgically precise one by uncovering unknown unknowns.
  • The Impact of Constraints (DON'T USE SUBAGENTS): This instruction appears to be a powerful lever for controlling an agent's strategy. By disabling subagents, the agent was forced to reason more holistically and adopt a top-down, architectural approach. Permitting subagents allowed for a bottom-up, exploratory approach that was more chaotic but also more resilient when it encountered unexpected problems in the environment.
  • Test-Driven Correction: Both the Gemini CLI and CC (No Subagents) agents used test failures (npm test, npm run typecheck) as a crucial feedback loop to refine their implementations. The Gemini CLI's fix was a direct response to a test crash, while the CC agent's refinement came from static analysis. The CC (Subagents) agent, however, relied more on end-to-end testing with node bundle/gemini.js, which is what allowed it to find the second bug that unit tests missed.