YOLO Object Detection on Android for Robotics

Introduction

Object detection is an important capability for autonomous robots. A robot can use detections to identify people, vehicles, tools, obstacles, and other objects in its environment.

YOLO-family models are widely used for real-time object detection. In this tutorial, we will design an Android application that captures camera frames and performs YOLO inference locally.

System Architecture

CameraX
   |
Preprocessing
   |
YOLO Model
   |
Postprocessing
   |
Bounding Boxes
   |
Robot Perception Layer

The exact model format depends on the runtime you choose. For Android edge deployment, an exported model may be converted to a mobile-compatible format and executed using an appropriate inference runtime.

Android Project

Organize the project into separate layers:

vision/
├── CameraManager.kt
├── YoloDetector.kt
├── Detection.kt
└── DetectionOverlay.kt

This prevents camera handling, inference, and rendering from becoming tightly coupled.

Detection Model

Create a Kotlin model:

data class Detection(
    val classId: Int,
    val label: String,
    val confidence: Float,
    val boundingBox: RectF
)

Camera Pipeline

Use CameraX ImageAnalysis to obtain frames.

imageAnalysis.setAnalyzer(executor) { image ->
    detector.process(image)
    image.close()
}

For real-time robotics, use a backpressure strategy that drops stale frames rather than allowing an inference queue to grow indefinitely.

Preprocessing

Most object-detection models expect a fixed input size.

The preprocessing stage normally performs:

Rotation correction
Resize
Color conversion
Normalization
Tensor creation

For example:

Camera Frame
   ↓
Resize
   ↓
Normalize
   ↓
Tensor
   ↓
YOLO

The preprocessing code must match the model's training/export requirements.

Running Inference

Create a detector abstraction:

class YoloDetector {
    suspend fun detect(frame: ImageFrame): List<Detection> {
        // preprocess
        // inference
        // postprocess
        return emptyList()
    }
}

Run inference outside the Android main thread.

Postprocessing

Object detectors can return multiple candidate boxes. Postprocessing commonly includes:

Confidence filtering
Class filtering
Bounding-box conversion
Non-Maximum Suppression

For example:

Raw Predictions
      ↓
Confidence Filter
      ↓
NMS
      ↓
Final Detections

Drawing Bounding Boxes

The Android UI can display detection results over the live camera preview.

+--------------------------+
|                          |
|     +------------+       |
|     |   bottle   |       |
|     |    92%     |       |
|     +------------+       |
|                          |
+--------------------------+

Remember to map coordinates correctly when the preview and model input have different aspect ratios.

Using Detection for Robotics

Detection results can be passed to a robot control layer:

{
  "object": "person",
  "confidence": 0.92,
  "bbox": [120, 80, 350, 500]
}

The robotics layer can combine this information with depth, odometry, LiDAR, or other sensors.

Do not treat a 2D bounding box as a physical distance measurement unless the system has additional calibration or depth information.

Performance Optimization

For edge robotics:

Use a lightweight model where possible.
Reduce inference resolution when acceptable.
Reuse buffers.
Avoid unnecessary bitmap allocations.
Run inference off the UI thread.
Process only the latest frame.
Measure latency and thermal behavior.

A smaller model running consistently can be more useful for robotics than a larger model with unstable frame rates.

Testing

Test the detector with:

Indoor scenes
Outdoor scenes
Low light
Moving objects
Multiple objects
Camera rotation
Network disconnected

For robotics, also measure end-to-end latency:

Capture → Inference → Decision → Robot Command

Conclusion

YOLO-style object detection can turn an Android device into a useful edge-vision component for robotics. Kotlin, CameraX, and a mobile inference runtime provide the foundation for building perception prototypes that can later integrate with ROS 2 and autonomous navigation.