项目首页Project home
概览Overview
本页目录On this page

方法与流程

一个像素,两个表面点

沿相机射线,最近和最远的表面交点分别对应前、后表面。HccePose(BF) 预测两个表面的层次化连续坐标编码,并将解码后的三维点与图像像素配对。

HccePose(BF) 论文原图:表面预测、对应构建和位姿求解

论文中的 HccePose(BF) 网络结构与位姿求解流程。

从检测到位姿

YOLO 提供目标区域,HccePose 为对应物体预测掩码及表面坐标。结合匹配分辨率的相机内参,PnP 求解物体到相机的位姿。检测器可以识别多个类别;位姿权重按物体加载。

参阅论文和接口约定。

层次坐标编码与对应关系

HCCE 分别编码物体坐标的三个分量。网络同时预测掩码和前后表面编码,解码后得到物体坐标系下的三维点。一个前景像素沿相机射线对应最近与最远的两个表面交点,因而可以构建两组 2D–3D 对应。

阶段 数据流
检测与裁剪 YOLO 提供实例区域;图像缩放、补边及裁剪变换决定像素坐标的映射。
网络预测 逐物体网络预测掩码与前后表面编码。
建立对应 解码三维坐标,与掩码中的像素位置配对。
求解位姿 结合相机内参用 PnP 求解 X_camera = R @ X_model + t。

检测器可包含多个物体类别,每类可检测多个实例;HccePose 权重按物体分别训练和加载。换新物体的完整流程见物体准备、场景渲染、检测器、坐标标签与位姿训练。

深度与可选后端

MegaPose 从 HccePose 位姿继续精修,支持纯 RGB 或配准 RGB-D。WAPR 的当前公开接入从原图掩码、配准深度和米制网格估计位姿并评分,不依赖 HccePose 初值。它们不是同一个调用模式,输入与单位对照见精修接口。

本文解释已公开方法。论文图保留原貌,数组形状和实际调用以Tester、结果字典为准。

Method and pipeline

One pixel, two surface points

The nearest and farthest intersections along a camera ray correspond to the front and back surfaces. HccePose(BF) predicts hierarchical continuous coordinate codes for both surfaces and pairs the decoded 3D points with image pixels.

Original HccePose(BF) paper figure: surface prediction, correspondences and pose solving

HccePose(BF) network architecture and pose estimation pipeline, from the published paper.

From detection to pose

YOLO supplies object regions. HccePose predicts masks and surface coordinates for the corresponding object. PnP uses image-matched intrinsics to recover the object-to-camera pose. One detector can identify multiple categories; pose weights are loaded per object.

See the paper and API contract.

Hierarchical coordinates and correspondences

HCCE encodes the three object-coordinate components separately. The network predicts masks and front/back surface codes, which decode to 3D points in the object coordinate system. A foreground pixel corresponds to the nearest and farthest intersections along its camera ray, providing two sets of 2D–3D correspondences.

Stage Data flow
Detect and crop YOLO proposes instance regions; resizing, padding and crop transforms define pixel mappings.
Predict Per-object networks estimate masks and front/back codes.
Match Decode 3D coordinates and associate them with masked pixel locations.
Solve Use pixel intrinsics and PnP to estimate X_camera = R @ X_model + t.

A detector may recognize several categories and multiple instances per category. HccePose checkpoints are trained and loaded per object. For a new object, follow mesh preparation, rendering, detector training, coordinate labels and pose training.

Depth and optional backends

MegaPose refines HccePose poses using RGB or registered RGB-D. The current public WAPR integration estimates poses and scores from original-image masks, registered depth and meter meshes, without HccePose pose seeds. These are distinct entry points; see refinement interfaces for inputs and units.

This page explains the published method. The original paper figure is preserved. See Tester and results for actual calls and array shapes.

打开原图 ↗Open original ↗