Object Detection on Android for Autonomous Robots

Introduction

Autonomous robots need to recognize objects in their environment. Object detection can identify people, vehicles, tools, signs, and obstacles from camera frames.

Android can perform edge inference locally, reducing dependency on network connectivity.

Architecture

CameraX
   |
Preprocessing
   |
Object Detection Model
   |
Postprocessing
   |
Detection Results
   |
Robot Perception Gateway

Detection Model

Define a reusable result type:

data class Detection(
    val label: String,
    val confidence: Float,
    val left: Float,
    val top: Float,
    val right: Float,
    val bottom: Float
)

This keeps the rest of the application independent from a particular model runtime.

CameraX Analysis

The camera analyzer should process frames asynchronously:

imageAnalysis.setAnalyzer(executor) { image ->
    detector.detect(image)
    image.close()
}

Use a latest-frame strategy when real-time responsiveness is more important than processing every frame.

Model Runtime

The detector can be implemented behind an interface:

interface ObjectDetector {
    suspend fun detect(frame: ImageFrame): List<Detection>
}

Possible mobile inference approaches include TensorFlow Lite or ONNX Runtime, depending on the model and deployment requirements.

Confidence Filtering

Not every prediction should be passed to the navigation system.

val valid = detections.filter {
    it.confidence >= 0.6f
}

The threshold should be evaluated against the target environment rather than chosen arbitrarily.

Non-Maximum Suppression

Detection models may produce overlapping predictions.

Prediction A  ───────
Prediction B    ───────
        ↓
       NMS
        ↓
Single Detection

Use the postprocessing method expected by your selected model.

Robot Integration

The Android device can send detections to the robot:

{
  "label": "person",
  "confidence": 0.94,
  "bbox": [120, 80, 350, 500]
}

For autonomous navigation, the robot should combine this with physical measurements such as depth or LiDAR when distance matters.

Tracking

Instead of detecting every object from scratch at every stage, an additional tracking layer can maintain object identities between frames.

Detection
   ↓
Tracking
   ↓
Object ID
   ↓
Navigation / Behavior

Tracking can reduce redundant processing and provide temporal context.

Performance Optimization

Important optimization techniques include:

Use an appropriately sized model
Reduce input resolution when acceptable
Reuse buffers
Avoid bitmap copies
Run inference off the main thread
Drop stale frames
Measure end-to-end latency

Autonomous Decision Making

Keep AI perception separate from robot control:

Camera
  ↓
Object Detection
  ↓
Perception State
  ↓
Navigation / Behavior
  ↓
Safety Controller
  ↓
Robot

This separation makes the system easier to test and safer to operate.

Testing

Evaluate the system using representative scenarios:

Static objects
Moving people
Multiple objects
Low light
Bright light
Partial occlusion
Camera vibration
Device thermal throttling

Measure both detection accuracy and real-time performance.

Conclusion

Object detection on Android can provide useful edge perception for autonomous robots. Kotlin, CameraX, and a mobile inference runtime create a flexible foundation that can later be connected to ROS 2, sensor fusion, navigation, and Physical AI agents.