AI-Powered Code Review with LLMs 2026 — Automate Bug Detection

Sanjeev SharmaSanjeev Sharma
6 min read

Advertisement

Introduction

Why This Matters

The best code review is the one that happens instantly, every time, on every PR. AI code reviewers running on every pull request catch bugs, security vulnerabilities, and style problems before human reviewers see the code — compressing feedback cycles from hours to seconds.

Traditional linters catch syntax errors and style violations. AI reviewers understand intent, context, and logic. They catch SQL injection in dynamic queries, race conditions in async code, business logic bugs that only manifest with empty arrays, and O(n squared) algorithms with a clear O(n) alternative.

At $0.15 per PR review, AI code review costs less than 30 seconds of a senior engineer's time and catches at least one production bug per month in most teams. The ROI is unambiguous.

Core Review Engine

from openai import OpenAI
import json
 
client = OpenAI()
 
SYSTEM_PROMPT = """You are a senior software engineer performing code review.
Analyze the provided code diff and return a JSON array of review comments.
 
Each comment must have:
- line: line number (or null for general comments)
- severity: "critical" | "warning" | "suggestion" | "praise"
- category: "bug" | "security" | "performance" | "readability" | "logic"
- message: clear explanation of the issue
- suggestion: specific fix or improvement
 
Focus on:
1. CRITICAL: bugs, security vulnerabilities, data loss risks
2. WARNING: performance issues, error handling gaps, logic errors
3. SUGGESTION: improvements, better patterns, test coverage
4. PRAISE: well-written code worth acknowledging
 
Return ONLY valid JSON. No markdown, no explanation."""
 
def review_code(diff: str, filename: str, context: str = "") -> list[dict]:
    prompt = f"""File: {filename}
Context: {context or 'General code review'}
Code diff:
{diff}
Return JSON array of review comments."""
 
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": prompt},
        ],
        response_format={"type": "json_object"},
        temperature=0.2,
    )
 
    result = json.loads(response.choices[0].message.content)
    return result.get("comments", result) if isinstance(result, dict) else result

Security Vulnerability Scanner

SECURITY_PROMPT = """You are a security engineer performing code security audits.
Analyze the code for security vulnerabilities. Return JSON with:
- vulnerabilities: array of {type, severity, line, description, cwe_id, fix}
- risk_score: 0-10
- summary: brief security assessment
 
Check for: SQL injection, XSS, CSRF, path traversal, command injection,
hardcoded secrets, insecure deserialization, broken auth, sensitive data exposure."""
 
def security_scan(code: str, language: str = "python") -> dict:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": SECURITY_PROMPT},
            {"role": "user", "content": f"Language: {language}\n\nCode:\n```{language}\n{code}\n```"},
        ],
        response_format={"type": "json_object"},
        temperature=0,
    )
    return json.loads(response.choices[0].message.content)
 
# Test with vulnerable code
vulnerable_code = """
def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"  # SQL injection
    return db.execute(query)
"""
 
result = security_scan(vulnerable_code)
print(f"Risk score: {result['risk_score']}/10")
for vuln in result['vulnerabilities']:
    print(f"[{vuln['severity']}] {vuln['type']}: {vuln['description']}")

GitHub Actions Integration

# .github/workflows/ai-code-review.yml
name: AI Code Review
 
on:
  pull_request:
    types: [opened, synchronize]
 
jobs:
  review:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read
 
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
 
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
 
      - name: Install dependencies
        run: pip install openai PyGithub
 
      - name: Run AI Code Review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          REPO: ${{ github.repository }}
        run: python .github/scripts/ai_review.py
# .github/scripts/ai_review.py
import os
import subprocess
import json
from openai import OpenAI
from github import Github
 
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
gh = Github(os.environ["GITHUB_TOKEN"])
 
def get_pr_diff():
    result = subprocess.run(
        ["git", "diff", "origin/main...HEAD", "--unified=5"],
        capture_output=True, text=True
    )
    return result.stdout
 
def post_review_comment(repo_name, pr_number, comments):
    repo = gh.get_repo(repo_name)
    pr = repo.get_pull(int(pr_number))
 
    critical = [c for c in comments if c.get("severity") == "critical"]
    warnings = [c for c in comments if c.get("severity") == "warning"]
 
    summary = f"""## AI Code Review
 
**{len(critical)} critical issues** | **{len(warnings)} warnings** | **{len(comments)} total**
 
"""
    for comment in comments:
        icon = {"critical": "[CRITICAL]", "warning": "[WARNING]", "suggestion": "[TIP]", "praise": "[GOOD]"}.get(comment["severity"], "[NOTE]")
        summary += f"{icon} **[{comment['category']}]** {comment['message']}\n"
        if comment.get("suggestion"):
            summary += f"   Suggestion: {comment['suggestion']}\n\n"
 
    pr.create_issue_comment(summary)
 
diff = get_pr_diff()
if len(diff) > 100:
    comments = review_code(diff[:8000], "PR diff")
    post_review_comment(os.environ["REPO"], os.environ["PR_NUMBER"], comments)

Pre-commit Hook

#!/usr/bin/env python3
# .git/hooks/pre-commit (chmod +x)
import subprocess
import sys
import os
from openai import OpenAI
 
def main():
    result = subprocess.run(
        ["git", "diff", "--cached", "--unified=3"],
        capture_output=True, text=True
    )
    diff = result.stdout
 
    if len(diff) < 50:
        sys.exit(0)
 
    print("Running AI code review...")
    client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
 
    response = client.chat.completions.create(
        model="gpt-4o-mini",  # Fast + cheap for pre-commit
        messages=[{
            "role": "user",
            "content": f"""Quick review this diff. Only flag CRITICAL issues (bugs, security).
            Be concise. Format: ISSUE: [description] on line [N]
 
            {diff[:3000]}"""
        }],
        max_tokens=500,
        temperature=0,
    )
 
    review = response.choices[0].message.content
    if "ISSUE:" in review:
        print("\nAI Review found issues:\n")
        print(review)
        answer = input("\nCommit anyway? [y/N]: ")
        if answer.lower() != "y":
            sys.exit(1)
 
    print("AI review passed")
 
if __name__ == "__main__":
    main()

Custom Review Rules

CUSTOM_RULES = """
Additional rules for this codebase:
1. All database queries must use parameterized queries — never string formatting
2. All user inputs must be validated with Pydantic models
3. Async functions must have timeout handling
4. All new endpoints must include rate limiting
5. Secrets must come from environment variables, never hardcoded
"""
 
def review_with_custom_rules(diff: str) -> list[dict]:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT + "\n\n" + CUSTOM_RULES},
            {"role": "user", "content": f"Review this diff:\n{diff}"},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    result = json.loads(response.choices[0].message.content)
    return result.get("comments", [])

Common Mistakes / Pitfalls

  • Reviewing entire files instead of diffs — diffs are smaller, cheaper, and more focused on what changed
  • Using GPT-4o for pre-commit hooks — use GPT-4o-mini for speed; use GPT-4o for the full PR review
  • No token budget on the diff — large refactoring PRs can exceed context limits; slice to 8000 tokens
  • Not returning structured JSON — parsing unstructured review text in CI/CD is fragile and error-prone
  • Auto-blocking PRs on every AI finding — AI reviews should be informational, not mandatory gates, until tuned

Best Practices

  • Use response_format={"type": "json_object"} on every code review call to guarantee parseable output
  • Combine AI review with traditional linting — AI for logic/security, linters for style/formatting
  • Set temperature=0 or 0.1 for security scanning — you want reproducible, deterministic findings
  • Post AI review as an informational comment, not a required check, when first deploying
  • Add language-specific context to the system prompt — Python error handling differs from Go patterns

Key Takeaways

  • AI code review catches SQL injection, race conditions, and business logic bugs that static linters cannot detect
  • GPT-4o-mini at $0.01 per pre-commit check is the right model for fast, lightweight gate checks
  • GPT-4o at $0.15 per full PR review catches critical security vulnerabilities worth the cost difference
  • response_format={"type": "json_object"} guarantees structured output that CI/CD scripts can parse reliably
  • GitHub Actions integration posts AI review comments directly to the PR thread within 30 seconds of opening
  • Custom rule sets in the system prompt enforce codebase-specific conventions automatically on every PR
  • Pre-commit hooks provide immediate, local feedback before code is ever pushed to the remote
  • AI code review ROI is positive when it catches even one production bug per month at typical review costs

Advertisement

Sanjeev Sharma

Written by

Sanjeev Sharma

Full Stack Engineer · E-mopro

Related reading