项目首页Project home
快速入门Start here
本页目录On this page

模型与输入

配套资源

物体网格、模型信息、检测器类别顺序和逐物体位姿权重必须配套。公开资源见 HCCEPose 资源页。本包不附应用场景数据或权重。

dataset_root 保留公开加载器使用的目录结构:

TEXT
objects/
  models/models_info.json
  models/obj_000001.ply
  yolo11/train_obj_s/detection/obj_s/yolo11-detection-obj_s.pt
  HccePose/obj_01/best_score/

Tester 初始化时会加载 models_info.json 所列的全部物体,因此每个物体都需准备位姿权重,并保持表中的物体顺序与检测器类别顺序一致。传给 predict 的物体编号列表只筛选本次预测,不限制初始化时的模型加载。

在入口脚本中填写数据目录、物体编号和图像路径。检测器和逐物体位姿权重应来自可信且允许当前用途的来源,加载前按提供方的记录核对文件哈希。

下载一个 RGB-D 示例

完成环境安装后,将下面代码保存为项目根目录的 Python 脚本并运行。资源保存于 data/,保留 Hugging Face 的目录结构;其中 data/demo-bin-picking/ 是 dataset_root,无需改名为 objects/。

PYTHON
import hashlib
import json
import os
from huggingface_hub import snapshot_download

project_dir = os.path.dirname(os.path.abspath(__file__))
data_root = os.path.join(project_dir, "data")

# 固定公开版本中的文件大小;两个权重同时校验 LFS 公布的 SHA-256。
checkpoint_relative_path = "demo-bin-picking/HccePose/obj_01/best_score/0_9837step824500"
required_files = {
    "demo-bin-picking/models/models_info.json": (941, None),
    "demo-bin-picking/models/obj_000001.ply": (128911, None),
    checkpoint_relative_path: (
        349827430, "3308469a5082feb98181286026003e0b00ea8a349b8f0921006873a177dd4c70"),
    "demo-bin-picking/yolo11/train_obj_s/detection/obj_s/yolo11-detection-obj_s.pt": (
        114453888, "5a958cf1d90d3529f02ba1021aab164a369316f187f98fb01b151c484d3cfd13"),
    "test_imgs_RGBD/000003_rgb.png": (208215, None),
    "test_imgs_RGBD/000003_depth.png": (59239, None),
    "test_imgs_RGBD/000003_camK.json": (184, None),
}

# 下载与校验使用同一份七文件清单,不包含训练 ZIP 或对称性辅助文件。
snapshot_download(
    repo_id="SEU-WYL/HccePose",
    repo_type="dataset",
    revision="4e482e540314622981ccd4fce5beddac7f9d3e1d",
    local_dir=data_root,
    allow_patterns=list(required_files),
    endpoint="https://huggingface.co",
    token=False,
)

# 加载器会扫描该目录的全部普通文件;额外文件只报错,不自动删除。
checkpoint_path = os.path.join(data_root, *checkpoint_relative_path.split("/"))
checkpoint_dir = os.path.dirname(checkpoint_path)
extra_files = [os.path.join(checkpoint_dir, name) for name in os.listdir(checkpoint_dir)
               if os.path.isfile(os.path.join(checkpoint_dir, name))
               and name != os.path.basename(checkpoint_path)]
if extra_files:
    raise RuntimeError("Unexpected checkpoint files: " + ", ".join(sorted(extra_files)))

# 网络不可达时 SDK 可能返回已有目录;逐项校验后再转换相机文件。
for relative_path, (expected_size, expected_sha256) in required_files.items():
    path = os.path.join(data_root, *relative_path.split("/"))
    if not os.path.isfile(path):
        raise FileNotFoundError(path)
    actual_size = os.path.getsize(path)
    if actual_size != expected_size:
        raise RuntimeError(f"File size mismatch: {path}; expected {expected_size}, got {actual_size}")
    if expected_sha256 is not None:
        digest = hashlib.sha256()
        with open(path, "rb") as handle:
            while True:
                chunk = handle.read(1024 * 1024)
                if not chunk:
                    break
                digest.update(chunk)
        if digest.hexdigest() != expected_sha256:
            raise RuntimeError(f"SHA-256 mismatch: {path}")

# 使用该帧的内参;将字段式 JSON 转为入口需要的 3×3 数组。
capture_dir = os.path.join(data_root, "test_imgs_RGBD")
with open(os.path.join(capture_dir, "000003_camK.json"), encoding="utf-8") as handle:
    camera = json.load(handle)
cam_K = [[camera["fx"], 0, camera["cx"]],
         [0, camera["fy"], camera["cy"]], [0, 0, 1]]
camera_path = os.path.join(capture_dir, "000003_camera_matrix.json")
with open(camera_path, "w", encoding="utf-8") as handle:
    json.dump(cam_K, handle, indent=2)

该调用只读取公开资源,不需要登录。local_dir 决定资源实际位置,data/.cache/huggingface/ 存储下载元数据;再次运行可复用已有文件并重新校验。清单中的大小和权重摘要对应上述固定 HF 版本,共下载七个文件,不含训练 ZIP 或对称性辅助文件。校验失败时先检查报出的文件,再重新下载;示例不会自动删除现有文件。best_score/ 中只应有清单指定的无后缀 checkpoint,不要改名、解压或放入说明文件。

确认资源损坏后,可在该下载调用中加入 force_download=True 强制重新下载;额外 checkpoint 或说明文件需自行检查后移出 best_score/。

RGB 和深度分别位于 data/test_imgs_RGBD/000003_rgb.png、data/test_imgs_RGBD/000003_depth.png,相机数组是新生成的 data/test_imgs_RGBD/000003_camera_matrix.json。不要将 demo-bin-picking/camera.json 套用到这帧:它的分辨率和内参不同。运行方式见第一次预测与 WAPR 接入。

下载工具也提供最小示例预设:

BASH
python scripts/download_hf_assets.py --preset demo --dest data --endpoint hf

它使用同一固定七文件清单,校验后另存相机矩阵。其他预设未指定 --dest 时仍写入旧阶段脚本使用的项目根目录,wget 备用入口也使用该旧布局。选择较大的训练预设前,可先查看 python scripts/download_hf_assets.py --help。

离线准备 RGB 推理

默认网络先用 torchvision 权重初始化 ResNet-34,再加载各物体的 HccePose 权重。可先在联网机器上使用新的 Torch Hub 缓存,下载主干权重并查看缓存目录:

BASH
python -c "from torchvision.models import ResNet34_Weights; ResNet34_Weights.IMAGENET1K_V1.get_state_dict(check_hash=True)"
python -c "from pathlib import Path; import torch; print(Path(torch.hub.get_dir()) / 'checkpoints')"

在目标机执行第二条命令,将 resnet34-b627a593.pth 复制到输出的目录。torchvision 在下载时校验公开的 SHA-256 前缀 b627a593。复制后再比对两台机器上文件的完整 SHA-256。参阅 torchvision ResNet-34 权重说明。

图像和相机

彩色图使用 OpenCV 的 BGR 顺序。camera.json 为与原图分辨率对应的 3×3 数组,单位为像素。

深度图必须事先与彩色图完成像素配准。WAPR 接口接收米制深度;整数毫米图应显式乘以 1e-3。尺寸相同并不代表已经配准。

其他公开数据与训练资源

运行单图和 WAPR 示例只需上面的七文件清单。需要其他任务时按目录选择资源,不要无差别下载完整数据仓库。

任务 公开目录或文件
料箱模型、权重与训练数据 demo-bin-picking
纹理物体模型与权重 demo-tex-objs
图片输入 test_imgs
视频输入 test_videos
配准 RGB-D 帧 test_imgs_RGBD
材质库 cc0textures-512.zip
七个 BOP 数据集权重 BOP 评测与权重索引

Hugging Face 的 local_dir 保留上游层级。例如下载 demo-tex-objs/ 后,它本身才是对应数据集根目录;不要再额外包一层同名目录。训练 ZIP 与无后缀 checkpoint 是不同类型:ZIP 按训练教程解压,checkpoint 保留原文件名直接由加载器读取。

自己的数据应使用清晰的外部数据根目录,由该根构造模型、训练场景和标签路径。物体模型采用毫米,深度换算按对应采集元数据,而非根据文件扩展名猜测。制作新模型的完整流程见物体准备。

Models and inputs

Matching resources

Meshes, model metadata, detector class order and per-object pose weights must match. Published resources are available from the HCCEPose resource page. Application captures and weights are not bundled.

dataset_root follows the published loader layout:

TEXT
objects/
  models/models_info.json
  models/obj_000001.ply
  yolo11/train_obj_s/detection/obj_s/yolo11-detection-obj_s.pt
  HccePose/obj_01/best_score/

Tester loads every object listed in models_info.json during initialization, so prepare pose weights for each object and keep the object order in that file consistent with the detector’s class order. The object ID list passed to predict only selects objects for the current prediction; it does not limit which models are loaded at initialization.

Set the dataset path, object IDs and image paths in the entry script. Use detector and pose checkpoints from a trusted source that permits your intended use. Check their hashes against the provider’s records before loading them.

Download one RGB-D example

After installation, save the following code as a Python script at the project root and run it. Assets go under data/, preserving the Hugging Face layout. Use data/demo-bin-picking/ as dataset_root; there is no need to rename it to objects/.

PYTHON
import hashlib
import json
import os
from huggingface_hub import snapshot_download

project_dir = os.path.dirname(os.path.abspath(__file__))
data_root = os.path.join(project_dir, "data")

# Sizes from the fixed public revision; verify both weights against their LFS SHA-256.
checkpoint_relative_path = "demo-bin-picking/HccePose/obj_01/best_score/0_9837step824500"
required_files = {
    "demo-bin-picking/models/models_info.json": (941, None),
    "demo-bin-picking/models/obj_000001.ply": (128911, None),
    checkpoint_relative_path: (
        349827430, "3308469a5082feb98181286026003e0b00ea8a349b8f0921006873a177dd4c70"),
    "demo-bin-picking/yolo11/train_obj_s/detection/obj_s/yolo11-detection-obj_s.pt": (
        114453888, "5a958cf1d90d3529f02ba1021aab164a369316f187f98fb01b151c484d3cfd13"),
    "test_imgs_RGBD/000003_rgb.png": (208215, None),
    "test_imgs_RGBD/000003_depth.png": (59239, None),
    "test_imgs_RGBD/000003_camK.json": (184, None),
}

# Share one seven-file list between downloading and validation; omit training ZIPs and symmetry helpers.
snapshot_download(
    repo_id="SEU-WYL/HccePose",
    repo_type="dataset",
    revision="4e482e540314622981ccd4fce5beddac7f9d3e1d",
    local_dir=data_root,
    allow_patterns=list(required_files),
    endpoint="https://huggingface.co",
    token=False,
)

# The loader scans every regular file here; report extras without deleting them.
checkpoint_path = os.path.join(data_root, *checkpoint_relative_path.split("/"))
checkpoint_dir = os.path.dirname(checkpoint_path)
extra_files = [os.path.join(checkpoint_dir, name) for name in os.listdir(checkpoint_dir)
               if os.path.isfile(os.path.join(checkpoint_dir, name))
               and name != os.path.basename(checkpoint_path)]
if extra_files:
    raise RuntimeError("Unexpected checkpoint files: " + ", ".join(sorted(extra_files)))

# The SDK may return a local folder when offline; validate before converting the camera.
for relative_path, (expected_size, expected_sha256) in required_files.items():
    path = os.path.join(data_root, *relative_path.split("/"))
    if not os.path.isfile(path):
        raise FileNotFoundError(path)
    actual_size = os.path.getsize(path)
    if actual_size != expected_size:
        raise RuntimeError(f"File size mismatch: {path}; expected {expected_size}, got {actual_size}")
    if expected_sha256 is not None:
        digest = hashlib.sha256()
        with open(path, "rb") as handle:
            while True:
                chunk = handle.read(1024 * 1024)
                if not chunk:
                    break
                digest.update(chunk)
        if digest.hexdigest() != expected_sha256:
            raise RuntimeError(f"SHA-256 mismatch: {path}")

# Use this frame's intrinsics; convert the field-based JSON to the entry's 3×3 array.
capture_dir = os.path.join(data_root, "test_imgs_RGBD")
with open(os.path.join(capture_dir, "000003_camK.json"), encoding="utf-8") as handle:
    camera = json.load(handle)
cam_K = [[camera["fx"], 0, camera["cx"]],
         [0, camera["fy"], camera["cy"]], [0, 0, 1]]
camera_path = os.path.join(capture_dir, "000003_camera_matrix.json")
with open(camera_path, "w", encoding="utf-8") as handle:
    json.dump(cam_K, handle, indent=2)

This call accesses public resources without login. local_dir determines where files are stored; data/.cache/huggingface/ holds download metadata. Reruns reuse existing files and validate them again. The sizes and weight digests refer to the fixed HF revision above. Exactly seven files are selected, excluding training ZIPs and symmetry helpers. If validation fails, inspect the reported file before downloading it again; the example does not delete existing files. Keep only the specified extensionless checkpoint in best_score/: do not rename or extract it, or add documentation files there.

After confirming a file is damaged, add force_download=True to the download call to fetch it again. Inspect any extra checkpoints or documentation files before moving them out of best_score/ yourself.

The RGB and depth files are data/test_imgs_RGBD/000003_rgb.png and data/test_imgs_RGBD/000003_depth.png. The converted camera array is data/test_imgs_RGBD/000003_camera_matrix.json. Do not substitute demo-bin-picking/camera.json for this frame: it has different intrinsics and resolution. Continue with your first prediction or WAPR integration.

The existing download tool also provides a minimal preset:

BASH
python scripts/download_hf_assets.py --preset demo --dest data --endpoint hf

It uses the same fixed seven-file manifest, validates it and writes the separate camera matrix. Other presets retain the legacy project-root layout unless --dest is supplied; the wget fallback also writes that legacy root layout. See python scripts/download_hf_assets.py --help before selecting larger training presets.

Preparing RGB inference offline

The default network initializes a torchvision ResNet-34 before it loads each object's HccePose checkpoint. On a connected machine, preload the backbone into a fresh Torch Hub cache and print its checkpoint directory:

BASH
python -c "from torchvision.models import ResNet34_Weights; ResNet34_Weights.IMAGENET1K_V1.get_state_dict(check_hash=True)"
python -c "from pathlib import Path; import torch; print(Path(torch.hub.get_dir()) / 'checkpoints')"

Copy resnet34-b627a593.pth into the checkpoint directory printed by the second command on the target machine. Torchvision checks the published SHA-256 prefix b627a593 on download. Compare the full SHA-256 digest on both machines after copying. Torchvision ResNet-34 weights

Images and camera

Color images use OpenCV BGR order. camera.json is a 3×3 array of pixel intrinsics matching the original image resolution.

Depth must already be registered to the color image. WAPR expects meters; explicitly multiply integer millimeter depth by 1e-3. Identical dimensions alone do not establish registration.

Other public data and training resources

The seven-file manifest above is sufficient for the image and WAPR examples. Select additional resources by task rather than downloading the complete dataset repository.

Task Public directory or file
Bin-picking models, weights and training data demo-bin-picking
Textured-object models and weights demo-tex-objs
Image inputs test_imgs
Video inputs test_videos
Registered RGB-D frames test_imgs_RGBD
Textures cc0textures-512.zip
Seven BOP dataset weights BOP evaluation and weight index

Hugging Face local_dir retains upstream paths. After downloading demo-tex-objs/, that directory is the dataset root; do not nest another copy of its name. Training ZIP files and extensionless checkpoints are different: extract ZIPs following the training tutorial, while keeping checkpoint filenames intact for direct loading.

For your own data, expose one external root and derive mesh, scene and label paths from it. Models use millimeters. Convert depth according to capture metadata rather than guessing from the extension. See object preparation for the full new-model workflow.

打开原图 ↗Open original ↗