Vision Service: Building an AI-Powered Vehicle Image Analysis Microservice

What Is vision-service?

vision-service is the AI brain behind the Kemotives vehicle marketplace. When a dealer uploads photos of a car, this microservice runs four parallel inspection pipelines in under two seconds:

1. Duplicate Detection — Has this vehicle been listed before?
2. License Plate Recognition & Masking — Read and anonymise the plate
3. Vehicle Spec Extraction — Make, model, year, colour from the images
4. Condition & Damage Assessment — What's the state of this vehicle?

Architecture

The service is built on FastAPI with a POST /analyse endpoint. To keep latency low, all four pipelines run concurrently via asyncio.gather with a ThreadPoolExecutor of 4 workers.

results = await asyncio.gather(
    detect_duplicates(images),
    recognise_plates(images),
    extract_vehicle_specs(images),
    assess_condition(images[0]),  # primary image only
)

---

The 4 AI Pipelines

1. Perceptual Hashing — Duplicate Detection

Rather than using heavy deep learning embeddings, I chose classical computer vision for speed and determinism. Each image is hashed using a combined Average Hash (ahash) + Difference Hash (dhash) at 16×16 resolution via Pillow and imagehash.

The hash string is stored in Supabase. On new submissions, Hamming distance is computed against all stored hashes. A distance ≤ 8 triggers a duplicate flag:

Confidence = 1 - (Distance / 8)

This approach is O(n) against stored listings but runs in milliseconds per image — far faster than neural embedding similarity search at our scale.

---

2. Hybrid License Plate Recognition

Plate detection uses a 3-stage hybrid pipeline:

Stage 1 — Google Cloud Vision OCR

• ImageAnnotatorClient.text_detection extracts text annotations and bounding boxes
• If the plate is missed, OpenCV applies CLAHE (Contrast Limited Adaptive Histogram Equalization) to boost contrast and retries

Stage 2 — Kenyan Plate Parsing

• Alphanumeric tokens are matched against Kenyan formats (KXX NNNX or KD dealer plates)
• OCR error correction handles common substitutions: O→0, I→1, S→5, Z→2

Stage 3 — Gemini Verification

• If OCR confidence is low, the image is passed to Gemini with a structured prompt for verification
• The plate region is then masked in the output image to protect privacy in public listings

---

3. Vehicle Specification Extraction

Using Gemini Vision, the service analyses all uploaded images collectively and returns structured vehicle data:

{
  "make": "Toyota",
  "model": "Land Cruiser",
  "year": 2019,
  "color": "White",
  "body_type": "SUV",
  "transmission": "Automatic"
}

This reduces listing time for dealers significantly — they upload photos and the specs are pre-filled.

---

4. Condition & Damage Assessment

The primary image is sent to Gemini with a damage-analysis prompt. The response classifies:

• Overall condition — Excellent / Good / Fair / Poor
• Detected damage — Dents, scratches, rust, broken glass (with location)
• Confidence score — How certain the model is

This feeds directly into the marketplace's verified listing badge system.

---

Tech Stack

Layer	Technology
API Framework	FastAPI + Uvicorn
AI / Vision LLM	Google Gemini (gemini-1.5-flash)
OCR	Google Cloud Vision API
Image Processing	OpenCV (cv2), Pillow
Hashing	imagehash (ahash + dhash)
Database	Supabase (hash storage)
Deployment	WSGI via passenger_wsgi.py

Key Engineering Decisions

Why not one LLM for everything? Perceptual hashing for duplicates is deterministic and 100× faster than asking an LLM to compare images. I used AI only where pattern matching couldn't do the job.

Why CLAHE? Standard OCR fails on low-contrast Kenyan plates — especially white plates in bright sunlight. Adaptive histogram equalization made the plate pop without blowing out the rest of the image.

Why FastAPI? The async-first design made asyncio.gather a natural fit. Running four I/O-bound AI calls in parallel instead of sequentially cut total analysis time from ~8 seconds to ~2 seconds.

View on GitHub →