Detector usage and evaluation检测器使用与评估

This page covers model resources, detection calls, output inspection, and reported latency and accuracy for WAPRDet2D.

本页介绍 WAPRDet2D 的模型准备、检测调用、结果检查，以及已有延迟与精度结果。

Model and runtime configuration模型与运行配置

WAPRDet2D uses dino and grounding to select the visual encoder and box detector checkpoints. The published configuration is vitl14 with swinb; SAM 2.1^[3]-L and BERT^[6] are fixed dependencies. An unsupported model name raises ValueError before any download.

WAPRDet2D 通过 dino 与 grounding 分别选择视觉编码器和框检测器的权重。已公布的配置为 vitl14 与 swinb，SAM 2.1^[3]-L 和 BERT^[6] 为固定依赖。不支持的模型名称会在下载前触发 ValueError。

ViT-S, ViT-B, and ViT-L use embedding widths 384, 768, and 1024. The template bank has to be built with the same dino. A bank whose width does not match raises ValueError and asks for a rebuild. ViT-g and the register models are not in this table. The patch size stays 14.

ViT-S、ViT-B、ViT-L 的特征宽度是 384、768、1024。模板库必须用同一个 dino 来建。宽度不一致时抛出 ValueError，并要求重建这份库。表里没有 ViT-g，也没有带 register 的模型。图块大小仍是 14。

Constructing WAPRDet2D runs this schedule once. It is not repeated on the next frame.

构造 WAPRDet2D 时按这个顺序做一次。下一帧不会再做。

1. Check dino and grounding. An unknown name stops here.先核对 dino 和 grounding。名字不对就停在这里。
2. Place the files in assets/weights/det2d/. A file that is already there is kept. A missing DINOv2^[2] weight, GroundingDINO^[1] weight, sam2.1_l.pt, or BERT file is downloaded from the official URL into that directory. The GroundingDINO JSON is copied from third_party/GroundingDINO/groundingdino/config/ when assets/weights/det2d/ does not have it. vitl14 with swinb is then checked against wapr/det2d_assets.json. A known hash that does not match raises, and the file on disk is not overwritten.文件放在 assets/weights/det2d/。已存在的文件直接使用；缺失的 DINOv2^[2]、GroundingDINO^[1]、sam2.1_l.pt 或 BERT 权重从官方地址下载。若权重目录缺少 GroundingDINO JSON，则从 third_party/GroundingDINO/groundingdino/config/ 复制。vitl14 配 swinb 还会对照 wapr/det2d_assets.json；已知哈希不一致时报错，不覆盖磁盘文件。
3. backend="trt" builds the DINOv2 FP16 engine when this weight, this TensorRT^[5], and this GPU do not already have a matching one. vitl14 writes dino_patches_fp16.engine. vits14 and vitb14 write dino_patches_fp16_vits14.engine and dino_patches_fp16_vitb14.engine. A matching engine is loaded and not built again. backend="torch" skips the engine and runs NativeDino. GroundingDINO and SAM stay PyTorch^[4] FP32. They are not converted.backend="trt" 时，这份权重、这份 TensorRT^[5]、这块 GPU 还没有匹配的引擎，就构建 DINOv2 的 FP16 引擎。vitl14 写出 dino_patches_fp16.engine。vits14 和 vitb14 写出 dino_patches_fp16_vits14.engine 和 dino_patches_fp16_vitb14.engine。已经匹配的引擎直接加载，不再构建。backend="torch" 不建引擎，走 NativeDino。GroundingDINO 和 SAM 保持 PyTorch^[4] FP32，不做这步转换。
4. Load the template bank, then the encoder and GroundingDINO. The query crop uses the TensorRT engine when backend is trt. The bank itself was encoded by NativeDino in PyTorch.加载模板库，再加载编码器和 GroundingDINO。查询裁剪在 backend 为 trt 时用 TensorRT 引擎。库本身是用 PyTorch 的 NativeDino 编码的。

Compatible source trees belong in third_party/. See Detector resources for source and checkpoint files, and Detection setup for the GroundingDINO attention adaptation.

兼容源码放在 third_party/。源码与权重清单见检测器资源；GroundingDINO 注意力适配见检测环境配置。

Run detection检测调用

detect_many_categories_many_instances takes one uint8 RGB image and returns boxes and visible masks for multiple categories and instances. obj_ids=None searches all bank categories; one object id restricts the category while allowing multiple instances. confidence retains candidates whose best allowed-category score meets the threshold. Omitted or None, it uses score_threshold = 0.1. score_min is an alias; conflicting values raise ValueError.

detect_many_categories_many_instances 接收一张 uint8 RGB 图像，返回多类别、多实例的包围盒和可见区域掩码。obj_ids=None 搜索模板库的全部类别；指定单个物体编号时限定类别，仍允许多个实例。confidence 保留允许类别中最高分达到阈值的候选；省略或传入 None 时使用 score_threshold = 0.1。score_min 是同义参数，两者取值冲突时抛出 ValueError。

onboard_meshes takes {obj_id: mesh} with vertices in meters. Use the same dino for the template bank and detector. An empty cache_path keeps templates on the GPU; set a file path for reuse across runs. Example 01 runs this 2D call without a pose-estimation stage.

onboard_meshes 接收顶点以米计的 {obj_id: mesh}。模板库与检测器应使用同一个 dino。cache_path 留空时模板保存在显存，设置文件路径后可跨进程复用。示例 01 仅执行该 2D 检测调用。

from wapr.det2d import WAPRDet2D, onboard_meshes

dino = "vitl14"
grounding = "swinb"
confidence = 0.1

bank = onboard_meshes(meshes_m, cache_path="", device="cuda:0", dino=dino)
detector = WAPRDet2D(
    bank, device="cuda:0", backend="trt",
    dino=dino, grounding=grounding,
)
instances, timing = detector.detect_many_categories_many_instances(rgb, obj_ids=None, confidence=confidence)

Each instance has obj_id, score_2d, bbox as xywh pixels, and mask as a visible COCO RLE. score_2d is the detector's ranking score. On the pose path, estimate_frame_many_categories_many_instances retains this field and applies score_2d_min as its gate. examples/05_bop_6d_detection.py leaves template_path empty and then calls estimate_frame_many_categories_many_instances.

每条结果包含 obj_id、score_2d、像素 xywh 格式的 bbox，以及可见区域的 COCO RLE mask。score_2d 是检测器的排序分数。位姿流程保留这个字段，并以 score_2d_min 作为检测阈值。examples/05_bop_6d_detection.py 将 template_path 留空，再调用 estimate_frame_many_categories_many_instances。

Inspect and save results结果检查与保存

visualize_2d_detection in wapr/view.py draws the detection bbox in blue and mask with a tint per instance. Labels use names[obj_id] when supplied, otherwise obj N, followed by score_2d. Example 01 saves all detections to JSON and renders only the highest preview_top_k scores; this display limit does not change inference outputs.

wapr/view.py 中的 visualize_2d_detection 绘制蓝色检测框 bbox 和按实例着色的 mask。标签使用传入的 names[obj_id]，未提供名称时使用 obj N，随后显示 score_2d。示例 01 将完整检测结果保存为 JSON，仅绘制分数最高的 preview_top_k 条；显示数量不影响推理输出。

from wapr.view import visualize_2d_detection

image_out = visualize_2d_detection(
    rgb, instances,
    image="results/det2d/single_image.jpg",
    show=False,
    names=names,
)

image accepts a destination path, creating parent directories as needed, or True to use recipe.visualize_path (default outputs/vis) and filename (default det2d.jpg). The example uses show=False for a headless server. Set it to True for a local window that closes on a keypress; this requires DISPLAY. The call is independent of recipe.visualize.

image 为文件路径时写入该 JPEG 或 PNG，并创建所需目录；设为 True 时使用 recipe.visualize_path（默认 outputs/vis）与 filename（默认 det2d.jpg）。上例的 show=False 适用于无显示服务器。需要本地窗口时可设为 True，按键关闭窗口；无 DISPLAY 时会报错。该调用独立于 recipe.visualize。

For subsequent pose results, visualize_6d_pose uses pose_4x4, mesh and K to draw red 3D boxes and rendered contours. Optional gt_pose_4x4 is green for comparison. recipe.visualize = True enables pose-image output without opening a window; the 2D visualization call is independent of this switch.

后续位姿结果由 visualize_6d_pose 根据 pose_4x4、mesh 和 K 绘制红色 3D 包围盒与渲染轮廓；可选的 gt_pose_4x4 以绿色显示，用于对照。recipe.visualize = True 启用位姿图像输出，不打开窗口；2D 可视化调用独立于该开关。

Reported latency and accuracy延迟与精度结果

On one RTX 5090 image, the warmed-up GroundingDINO + SAM2 + DINOv2 CAD-matching front end measured 220.7 ms median. This is a single-image 2D measurement; it excludes 6D pose estimation and does not represent dataset-wide latency.

在 RTX 5090 上，GroundingDINO + SAM2 + DINOv2 CAD 匹配前端对一张图像预热后测得的耗时中位数为 220.7 ms。这是单图 2D 测量，不包含 6D 位姿估计，也不代表数据集级延迟。

Existing full-test BOP2D evaluations report the bbox / mask AP below (%). The evaluation uses a score threshold of 0.35; the detector default is 0.1. Full TensorRT AP has been measured for LM-O^[7], IC-BIN^[10] and TUD-L^[9] only.

已有完整测试集的 BOP2D 本地评测结果如下，数值为包围盒 / 掩码 AP（%）。评测采用分数阈值 0.35，检测器默认阈值为 0.1。TensorRT 完整 AP 仅测量了 LM-O^[7]、IC-BIN^[10] 和 TUD-L^[9]。