Load one RGB-D frame加载一帧 RGB-D

The following code loads LM-O^[1] scene 2, image 307. The next section runs the 2D detector on that RGB frame. BOP depth times depth_scale is millimeters, then divided by 1000. T-LESS^[2] and HB^[3] read test_primesense/. The others read test/. examples/05_bop_6d_detection.py uses the same reader on image 1. Image 1 has no row in the published detection file further down.

下面的代码加载 LM-O^[1] 场景 2 第 307 帧。下一节对该帧的 RGB 图像运行 2D 检测器。BOP 深度乘 depth_scale 是毫米，再除以 1000。T-LESS^[2] 和 HB^[3] 读 test_primesense/。其他读 test/。examples/05_bop_6d_detection.py 用同一个读取器读第 1 帧。第 1 帧在后面那份已公布的检测文件里没有记录。

In [2]

detector = WAPRDet2D(template, device=device, backend=det_backend)
estimator = WAPREstimator(device=device)
prepared = estimator.prepare_meshes([pair[0] for pair in meshes.values()])
mesh_setup_seconds = estimator.last_mesh_setup_seconds
pose_meshes = {obj_id: (mesh, pair[1]) for (obj_id, pair), mesh in zip(meshes.items(), prepared)}
rgb, depth_m, K = load_bop_rgbd(bop_path, dataset, scene_id, im_id)
print("rgb", tuple(rgb.shape), "depth_m", tuple(depth_m.shape), "K", tuple(K.shape), flush=True)

Source: examples/04_6d_localization.py, lines 114–120代码来源：examples/04_6d_localization.py，第 114–120 行

Out [2]

rgb (480, 640, 3) depth_m (480, 640) K (3, 3)

Captured stdout excerpt · 已保存标准输出摘录 ·

LM-O scene 2, image 307: RGB on the left and the corresponding depth on the right. The colorbar gives distance along the camera optical axis in meters (0.599–1.959 m in this frame); purple is nearer and yellow is farther. Black indicates missing or invalid depth. rgb is H×W×3, depth_m is H×W in meters, and K is 3×3 in pixels. The next section detects on this RGB and estimates poses using the same depth.LM-O 场景 2，第 307 帧：左侧为 RGB，右侧为对应深度。色条表示沿相机光轴方向的距离，单位为米（本帧有效范围 0.599–1.959 米）；紫色较近，黄色较远。黑色表示缺失或无效深度。rgb 是 H×W×3，depth_m 是 H×W、单位为米，K 是 3×3、单位为像素。下一节在这张 RGB 上检测，并结合相同深度估计位姿。

Independent RGB-D visualization code; it does not participate in inference.

以下为独立 RGB-D 可视化代码，不参与推理。

import numpy as np
import matplotlib.pyplot as plt

# Visualize metric depth without changing the inference input.
# 将米制深度可视化，不修改供推理使用的 depth_m。
valid_depth = np.isfinite(depth_m) & (depth_m > 0)
depth_display_m = np.ma.masked_where(~valid_depth, depth_m)
depth_cmap = plt.get_cmap("viridis").with_extremes(bad="#202020")
fig, axes = plt.subplots(1, 2, figsize=(12, 5), layout="constrained")
axes[0].imshow(rgb)
axes[0].set_title("RGB")
depth_image = axes[1].imshow(depth_display_m, cmap=depth_cmap)
axes[1].set_title("Depth (black = invalid)")
for axis in axes:
    axis.set_axis_off()
# The colorbar uses the full valid depth range, in meters.
# 色条覆盖全部有效深度的范围，单位为米。
fig.colorbar(depth_image, ax=axes, location="bottom", shrink=0.6, label="Depth (m)")
plt.show()