2D detection and segmentation2D 检测与分割
WAPRDet2D.detect_many_categories_many_instances performs CAD-conditioned detection and visible-region segmentation from one RGB image. Preparation builds a reusable template bank by rendering CAD meshes and encoding the views with DINOv2[2]. Per-frame inference then uses three model stages:
WAPRDet2D.detect_many_categories_many_instances 以单张 RGB 图像为输入,完成基于 CAD 的检测及可见区域分割。准备阶段渲染 CAD 网格,并用 DINOv2[2] 编码视图,建立可复用的模板库。逐帧推理包含三个模型阶段:
- GroundingDINO[1] generates candidate bounding boxes.GroundingDINO[1] 生成候选包围盒。
- SAM segments candidate regions into visible masks.SAM 分割候选区域,生成可见区域掩码。
- DINOv2 encodes the segmented regions and assigns CAD identities through template matching.DINOv2 编码分割区域,通过模板匹配确定 CAD 身份。
Each returned instance contains a detection box and a visible mask. The method does not use frame-level ground-truth object identities, instance counts, boxes, masks, or depth. Its score_2d ranks 2D detections and is distinct from the pose-level score_6d. The API calls its score gate confidence for compatibility; this fused matching score is not a calibrated probability.
每条返回结果均包含检测框和可见区域掩码。该方法不使用当前帧的真值物体类别、实例数量、包围盒、掩码或深度。返回的 score_2d 用于排序 2D 检测结果,与位姿层面的 score_6d 含义不同。接口沿用 confidence 作为分数阈值参数名;融合后的匹配分数并非校准概率。
- CAD renderingCAD 渲染
- Feature encoding特征编码
- Box proposals候选框
- Mask segmentation掩码分割
- Feature encoding特征编码
- CAD matchingCAD 匹配
- Candidate filtering候选筛选
References and licenses参考文献与许可
- GroundingDINO — Apache-2.0. Vendored detector source; original notices retained.随包检测源码;保留原始声明。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
- DINOv2 — Apache-2.0. Code and official DINOv2 weights; retain copyright and license.源码与官方 DINOv2 权重;保留版权和许可。 · GitHubGitHub · License/notice 1许可/声明 1
Model weights and dataset/task assets may have separate terms. WAPR's source license does not replace them.模型权重、数据集与任务资源可能有独立条款;WAPR 源码许可不替代这些许可。