Using LLM for Image Classification

We are going to build a command-line image classifier that sends product photos to a vision-capable LLM and returns structured JSON labels. It is useful for developers who need to categorize inventory or user-generated content without maintaining a custom computer-vision pipeline.

What you will need

• Python 3.10 or newer
• The OpenAI SDK: pip install openai
• An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Initialize the Oxlo.ai client

I start by importing the SDK and pointing it at Oxlo.ai. Because Oxlo.ai is fully OpenAI-compatible, the only difference is the base URL and API key.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY",
)

Step 2: Encode the image to base64

Vision LLMs expect images as base64 data URLs. I wrote a small helper that reads any local JPEG or PNG and returns the properly formatted string.

import base64

def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        encoded_string = base64.b64encode(image_file.read()).decode("utf-8")
    return f"data:image/jpeg;base64,{encoded_string}"

Step 3: Lock down the output with a system prompt

To keep the response machine-readable, I force the model to act as a strict classifier and return only JSON. This removes the need for regex gymnastics later.

SYSTEM_PROMPT = """You are a product-image classifier.
Analyze the image and return a single JSON object with exactly these keys:
- category: one of [electronics, clothing, food, furniture, other]
- confidence: an integer from 1 to 10
- description: a one-sentence description of what you see
Do not add markdown fences, explanations, or line breaks outside the JSON."""

Step 4: Build the classifier function

Now I wire the pieces together. The user message contains both the task text and the base64 image. I use kimi-k2.6 because it handles vision and reasoning well, and Oxlo.ai's request-based pricing means the cost stays flat even when I pass high-resolution images with long prompts.

import json

def classify_image(image_path):
    b64_image = encode_image(image_path)

    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Classify this product image."},
                    {
                        "type": "image_url",
                        "image_url": {"url": b64_image},
                    },
                ],
            },
        ],
    )

    raw = response.choices[0].message.content.strip()
    return json.loads(raw)

Step 5: Add a CLI wrapper

I wrap the classifier in a small script so I can run it from the terminal against any image file.

if __name__ == "__main__":
    import sys

    if len(sys.argv) < 2:
        print("Usage: python classify.py path/to/image")
        sys.exit(1)

    result = classify_image(sys.argv[1])
    print(json.dumps(result, indent=2))

Run it

Save everything as classify.py, export your key, and point it at a test image.

export OXLO_API_KEY="sk-oxlo.ai-..."
python classify.py photo.jpg

Example output:

{
  "category": "electronics",
  "confidence": 9,
  "description": "A wireless over-ear headphone on a white background"
}

Next steps

Pipe the JSON output into a Pandas DataFrame to batch-label a folder of images. You can also add a second LLM call on Oxlo.ai to generate alt-text for each classified product. Because Oxlo.ai charges per request rather than per token, running these sequential calls on large images stays predictable and cheap. See https://oxlo.ai/pricing for plan details.