YOLO27 Depth Estimation: From 2D Boxes to 3D Space

YOLO27 adds depth estimation. Learn visually how YOLO goes from flat 2D boxes to objects in 3D space — with an interactive demo and Python code.

By Amal V S, Backend & AI Systems Architect · Thu Oct 15 2026

Show YOLO a photo of a busy street and it answers instantly: car, 94%, right here — a neat rectangle around every object. Now ask it a simple follow-up: how far away is that car? For ten years of YOLO models, the honest answer was silence. YOLO saw the world the way a photograph does: flat.

YOLO27, announced by Ultralytics at YOLO Vision 2026, changes that with a new task: depth estimation. In this post we'll build the intuition from scratch — what a 2D detection really is, how a model can guess distance from a single image, and the one piece of geometry that turns a flat box into an object standing in real 3D space. There's an interactive demo halfway down. Play with it.

Key takeaways

YOLO27 was announced on 13 September 2026 and is not released yet — Ultralytics says launch is expected later this year.
It spans 4 sizes (N, S, M, L) and 7 tasks, adding depth estimation and semantic segmentation to the familiar detect / segment / pose / OBB / classify.
YOLO27 does not include a 3D object detection task. Its step into 3D is a per-pixel depth map.
Detection + depth + a camera's intrinsics = a 3D position for every detected object, using one simple formula.
All published numbers are preliminary. Ultralytics recommends YOLO26 for production until YOLO27 ships.

First, what is YOLO27 — exactly?

YOLO27 is the next generation of Ultralytics' YOLO family, unveiled on stage at YOLO Vision 2026 (Ultralytics event recap). The official docs are clear about its status: "The models are not yet available, and no launch date has been set" (Ultralytics docs).

Here's what's been announced:

28 models across four sizes — n, s, m, l — and seven tasks.
Tasks: detection, instance segmentation, semantic segmentation (-sem), depth estimation (-depth), classification, pose and oriented boxes (OBB).
Two detection designs: N and S use a streamlined CNN with dual-scale prediction for edge devices; M and L use a query-based, NMS-free transformer decoder. Every other task uses the CNN design.
Preliminary COCO detection: 42.3 mAP (YOLO27n) up to 60.4 mAP (YOLO27l) at 640 px, with 0.62–2.32 ms TensorRT FP16 latency — the first Ultralytics model past 60 mAP.

YOLO27 doesn't draw 3D boxes. It does something more fundamental: it estimates how far away every pixel is. That's the doorway from 2D to 3D.

Life in Flatland: what a 2D detection really tells you

A classic YOLO detection is just four numbers, a label and a confidence:

car   conf=0.94   box=[x1=36, y1=146, x2=164, y2=244]   # pixels, not metres

Those are image coordinates. They say where the car appears on the screen — not where it is in the world. And here's the trap: a small car close to the camera and a big truck far away can produce the exact same box. Size on screen depends on both how big something is and how far away it is. From one box, you can't untangle the two.

Three panels: YOLO's 2D boxes, a depth map coloring each pixel by distance, and a bird's-eye view placing each object at its real X and Z position

How can one image reveal depth?

Close one eye and look around the room. You still know the door is further than your desk. Your brain is using monocular cues — hints that exist in a single flat image:

Perspective: parallel lines (road edges, rails) converge toward the horizon.
Familiar size: cars are roughly the same size, so a tiny car must be far away.
Height in the image: on a ground plane, things further away sit closer to the horizon.
Occlusion: whatever blocks something else is in front of it.
Texture and haze: detail gets finer and contrast fades with distance.

A depth-estimation network learns these same cues from millions of images and outputs a depth map: an image the same size as the input where every pixel value is a distance from the camera. YOLO27's depth models ({c('yolo27n-depth')} … {c('yolo27l-depth')}) are evaluated on NYU Depth V2 (indoor rooms) and KITTI (driving).

Reading the score: what δ1 means

The headline depth metric is δ1: the share of pixels whose predicted depth is within 25% of the truth. Roboflow reports preliminary δ1 of 0.87 for YOLO27l-depth on NYU and 0.80 on KITTI, with the tiny YOLO27n-depth reaching 0.83 on both (Roboflow). In plain words: on indoor scenes, roughly 87 of every 100 pixels land within 25% of their real distance.

Relative vs metric depth — the detail that matters

Relative depth tells you the order: the chair is closer than the wall. Great for effects and layering, not for measuring.
Metric depth tells you metres: the chair is 2.4 m away. That's what robots and safety systems need.

It isn't public yet whether YOLO27-depth outputs metric or relative depth (Roboflow). Keep that question in mind when it ships.

The bridge: one formula from pixels to 3D

Every camera has four numbers called intrinsics: focal lengths fx, fy and the image centre cx, cy. A pixel tells you the direction of a ray leaving the camera. Depth tells you how far along that ray the object sits. Put them together:

Pinhole camera diagram: a ray from the camera through pixel (u, v) on the image plane reaches the 3D point (X, Y, Z) at depth Z; X = (u − cx)·Z/fx and Y = (v − cy)·Z/fy

X = (u - cx) * Z / fx      # left / right, metres
Y = (v - cy) * Z / fy      # up / down, metres
Z = depth(u, v)            # forward, metres (needs metric depth)

That's it. Three lines turn a flat detection into an object with a position in the world. Try it yourself:

Notice what happens when you slide the car away: the 2D box shrinks and slides toward the horizon, but the 3D position stays exactly where you put it. The box alone could never tell you that — the depth value is what pins the car in space.

In code: lifting YOLO detections into 3D

The geometry doesn't depend on any particular model. Here's a small, runnable function that takes boxes from any detector and a metric depth map, and returns a 3D position per object. It uses the median depth inside the box, which ignores background pixels that leak into the corners.

import numpy as np

def lift_boxes_to_3d(boxes_xyxy, depth_m, fx, fy, cx, cy):
    """boxes_xyxy: (N, 4) pixel boxes. depth_m: (H, W) metric depth map.
    Returns (N, 3) object positions [X, Y, Z] in metres (camera frame)."""
    points = []
    for x1, y1, x2, y2 in boxes_xyxy.astype(int):
        # use the central half of the box to avoid background at the edges
        w, h = x2 - x1, y2 - y1
        patch = depth_m[y1 + h // 4 : y2 - h // 4, x1 + w // 4 : x2 - w // 4]
        if patch.size == 0:                      # tiny box: fall back to all of it
            patch = depth_m[y1:y2 + 1, x1:x2 + 1]
        z = float(np.median(patch))
        u, v = (x1 + x2) / 2, (y1 + y2) / 2
        points.append([(u - cx) * z / fx, (v - cy) * z / fy, z])
    return np.array(points)

Today you can feed it boxes from YOLO26 — the model Ultralytics recommends for production right now — and a depth map from any metric depth model. When YOLO27 ships, swap in its detection and depth models and keep the same function:

from ultralytics import YOLO

detector = YOLO("yolo26n.pt")               # later: YOLO("yolo27n.pt")
result = detector("street.jpg")[0]
boxes = result.boxes.xyxy.cpu().numpy()      # (N, 4) pixel boxes
labels = [result.names[int(c)] for c in result.boxes.cls]

depth_m = ...  # (H, W) metric depth map — later: from a yolo27-depth model
positions = lift_boxes_to_3d(boxes, depth_m, fx=1000, fy=1000, cx=640, cy=360)
for name, (X, Y, Z) in zip(labels, positions):
    print(f"{name:>8}: {Z:5.1f} m ahead, {X:+5.1f} m to the side")

Want the whole scene instead of one point per object? Apply the same formula to every pixel and you get a 3D point cloud from a single photo — with numpy, it's a few lines:

H, W = depth_m.shape
u, v = np.meshgrid(np.arange(W), np.arange(H))
X = (u - cx) * depth_m / fx
Y = (v - cy) * depth_m / fy
cloud = np.stack([X, Y, depth_m], axis=-1).reshape(-1, 3)   # (H*W, 3) points

What a single camera with depth unlocks

Robotics: a robot arm knows not just that a box exists, but how far to reach for it.
Warehouse safety: alert when a person is within 2 m of a moving forklift — distance, not just presence.
Drones: keep a safe distance from obstacles using a light camera instead of heavy LiDAR.
AR: virtual objects can hide correctly behind real ones (occlusion) because the app knows what's in front.
Retail and smart spaces: measure queue lengths or shelf distances from existing CCTV cameras.

What it doesn't do (yet)

No 3D boxes. You get a position, not an object's 3D size or heading. Full 3D detection (as in autonomous-driving datasets) is a different task that YOLO27 doesn't list.
Scale needs care. One camera can't measure absolute scale on its own; metric depth relies on what the model learned — and the output type isn't confirmed yet.
Tricky surfaces. Mirrors, glass and very dark or shiny objects confuse every monocular depth model.
You need intrinsics. Without fx, fy, cx, cy you have depth, but not true X/Y in metres. Calibrate your camera once.
Everything is preliminary. No weights, final benchmarks or license have been published; earlier YOLO releases used AGPL-3.0.

The preliminary numbers at a glance

YOLO27n — 42.3 mAP · 0.62 ms (TensorRT FP16) · 3.0 M params
YOLO27s — 49.6 mAP · 0.79 ms · 11.8 M params
YOLO27m — 55.8 mAP · 1.39 ms · 22.8 M params
YOLO27l — 60.4 mAP · 2.32 ms · 72.3 M params (61.2 mAP at 800 px)
Depth (δ1, Roboflow-reported): YOLO27l-depth 0.87 NYU / 0.80 KITTI · YOLO27n-depth 0.83 NYU / 0.83 KITTI

Detection figures are COCO val at 640 px from the Ultralytics docs. Treat all of them as previews until the models are released and independently tested.

FAQ

Does YOLO27 do 3D object detection?

No. YOLO27's seven tasks are all image-based. Its new depth estimation task gives every pixel a distance, which you can combine with detections and camera geometry to place objects in 3D.

When will YOLO27 be released?

Ultralytics says later in 2026, with no date set. Until then, the official recommendation is YOLO26 for production.

What is monocular depth estimation?

Predicting distance for every pixel from a single image, using cues like perspective, familiar object sizes and occlusion — no stereo camera or LiDAR needed.

Can I get real distances in metres?

Only with a metric depth model and your camera's intrinsics. Then X = (u − cx)·Z/fx and Y = (v − cy)·Z/fy give each object's position in metres.

Prefer watching? An AI agent explains it in under a minute: YOLO27 learns depth — the 50-second explainer.

Sources: Ultralytics YOLO27 docs · YOLO Vision 2026 highlights · Roboflow: What is YOLO27? · arXiv: Ultralytics YOLO Evolution. Figures are preliminary and may change before release.