Detection and pose setup检测与位姿环境

Start with the installation guide. Export bundled examples using export_examples; source-checkout users can run the repository's examples/ directly.

先按安装引导准备环境。随包示例可通过 export_examples 导出；源码用户可直接运行仓库中的 examples/。

Wheel resourceswheel 资源目录

Installed-wheel resources default to the user cache (~/.cache/wapr, or XDG_CACHE_HOME/wapr). Set WAPR_CACHE_DIR to an absolute directory before running to use another location. Required resources are obtained by the relevant package entry points. SAM3D weights default to ModelScope; existing local files are reused.

安装 wheel 后资源默认存入用户缓存（~/.cache/wapr 或 XDG_CACHE_HOME/wapr）。运行前设置绝对路径 WAPR_CACHE_DIR 可指定其他目录。所需资源由对应包入口获取；SAM3D 权重默认从魔搭下载，并复用已有本地文件。

python -m wapr.bootstrap --feature det2d --check
python -m wapr.bootstrap --feature det2d

2D detection and 6D pose estimation, localization and detection use the same setup. Pose refinement, scoring, GPU rendering and TensorRT^[6] deployment are installed together; no separate environment is needed for each task.

2D 检测及 6D 位姿估计、定位、检测使用同一套环境。位姿修正、评分、GPU 渲染和 TensorRT^[6] 推理部署一起准备，无需按任务重复安装。

Install CUDA-enabled PyTorch for your GPU first. Bootstrap prepares compatible pose dependencies, including OpenCV, ONNX and TensorRT. TensorRT engines are built for the target GPU and runtime environment.

先安装适合显卡的 CUDA 版 PyTorch。Bootstrap 准备兼容的位姿依赖，包括 OpenCV、ONNX 和 TensorRT。TensorRT 引擎在目标显卡及推理环境中构建。

Pose weights and inference engines位姿权重与推理引擎

The installer downloads the four pose checkpoints and LM-O^[8] sample packs and compiles the renderer. python -m wapr.export_engines builds the four standard engines and wbps_batch.engine on the inference GPU. Weights and engines go to cache/weights/; sample data goes to cache/samples/.

安装脚本下载四份位姿权重与 LM-O^[8] 样例包，并编译渲染器。python -m wapr.export_engines 在推理 GPU 上构建四份常规引擎及 wbps_batch.engine。权重与引擎保存在 cache/weights/，样例数据保存在 cache/samples/。

For pose-only use, run python -m wapr.bootstrap --feature core. To generate pose engines explicitly on the inference GPU, run python -m wapr.export_engines.

仅使用位姿估计时运行 python -m wapr.bootstrap --feature core。需要在推理显卡上显式生成位姿引擎时，运行 python -m wapr.export_engines。

python -m wapr.download_assets fetches all supported small sample packs and pose weights. Full evaluation datasets and tracking sequences are separate downloads; see Datasets and model resources.

python -m wapr.download_assets 下载全部支持的小样例包和位姿权重。完整评测数据集及跟踪序列另行下载，见数据集与模型资源。

Fetch获取

Pose weights are available from SEU-WYL/WAPR. Package entry points prepare missing resources and reuse complete local files. Weights are stored under cache/weights/; shared sample data is stored under cache/samples/.

位姿权重来自 SEU-WYL/WAPR。包内入口准备缺失资源并复用完整的本地文件；权重存于 cache/weights/，共用小样存于 cache/samples/。

These paths apply to source installation. Run the installer and python -m wapr.export_engines before creating WAPREstimator; source use requires prepared weights and engines. A ready resource pack causes no Hub request. Public downloads do not send a locally stored Hugging Face token to either endpoint.

上述路径适用于源码安装。构造 WAPREstimator 前，先运行安装脚本和 python -m wapr.export_engines，准备权重与引擎。资源包齐全时不会请求 Hub。公开资源下载不会向任一地址发送本机保存的 Hugging Face 令牌。

only = ("wapr_sapr_wbps",)

python -m wapr.download_assets

only is the tuple in wapr/download_assets.py. only = ("wapr_sapr_wbps",) downloads these four checkpoints. only = () also downloads the one-frame packs named on the BOP Challenge page.

only 是 wapr/download_assets.py 里的元组。only = ("wapr_sapr_wbps",) 下载这四份权重。only = () 还会下载BOP 挑战页列出的单帧示例。

2D detector sources2D 检测源码

Bootstrap prepares compatible GroundingDINO in cache/sources/GroundingDINO/, installs Ultralytics, and uses the official DINOv2 Torch Hub cache. Model weights are stored under cache/weights/det2d/. Existing compatible dependencies and cached files are reused.^[1]^[2]^[3]

默认准备会在 cache/sources/ 获取适配后的 GroundingDINO 源码。Ultralytics 通过 pip 安装，DINOv2 源码由官方 Torch Hub 管理。

The setup entry prepares adapted GroundingDINO source; DINOv2 uses official Torch Hub and Ultralytics is installed as a package.

Source preparation and offline resources源码准备与离线资源

GroundingDINO

Bootstrap prepares GroundingDINO source, JSON configs and its PyTorch attention adaptation automatically; no manual copy or editable installation is needed.

Bootstrap 自动准备 GroundingDINO 源码、两份 JSON 配置和 PyTorch 注意力适配；无需手工复制文件或执行 pip install -e。

python -m wapr.bootstrap --feature det2d

The supplied attention module routes both CPU and CUDA execution through multi_scale_deformable_attn_pytorch, avoiding GroundingDINO’s separately compiled _C extension. Copying it with the command above applies this adaptation; no further source edits are needed.

随附的注意力模块在 CPU 和 CUDA 上均调用 multi_scale_deformable_attn_pytorch，无需另行编译 GroundingDINO 的 _C 扩展。执行上方复制命令即可完成适配，不需要再次修改源码。

output = multi_scale_deformable_attn_pytorch(
    value, spatial_shapes, sampling_locations, attention_weights
)

SAM

SAM2.1-L uses the pip-installed Ultralytics package, which can be upgraded when dependencies are compatible. Its checkpoint is cache/weights/det2d/sam2.1_l.pt.

SAM2.1-L 通过 Ultralytics 提供点／框分割；权重缓存位置为 cache/weights/det2d/sam2.1_l.pt。

python -m wapr.bootstrap --feature sam2

DINOv2

DINOv2 uses the official torch.hub.load("facebookresearch/dinov2", ...) entry and reuses the existing Hub cache and TORCH_HOME. WAPR does not maintain a separate source checkout. xFormers is optional for this inference path.

DINOv2 使用官方 torch.hub.load("facebookresearch/dinov2", ...) 入口，复用现有 Hub 缓存及 TORCH_HOME。xFormers 对此推理路径非必需。

python -m wapr.bootstrap --feature dinov2 sam2 det2d

BERT

GroundingDINO’s text encoder is google-bert/bert-base-uncased, a model rather than a git checkout. The installer places its config, model, and tokenizer files in cache/weights/det2d/bert-base-uncased/. wapr/det2d.py points text_encoder_type to that local directory. For manual or offline setup, the five file links are in the file list below.

GroundingDINO 的文本编码器是 google-bert/bert-base-uncased 模型，不是 git 源码。安装脚本将配置、模型和 tokenizer 文件放在 cache/weights/det2d/bert-base-uncased/；wapr/det2d.py 将 text_encoder_type 指向该目录。手动或离线安装所需的五份文件链接见下方文件清单。

2D detector assets2D 检测器资源

GroundingDINO, SAM, DINOv2 and BERT^[7] provide the detector components. After preparing the source trees above, use the following file locations, direct download links and checksums for an offline installation.

检测器使用 GroundingDINO、SAM、DINOv2 与 BERT^[7]。完成上方源码配置后，离线安装可直接按以下文件位置、下载链接和校验值准备。

Weight files权重文件

Create cache/weights/det2d/. These weights are not in SEU-WYL/WAPR. You can place the published files yourself, or let WAPRDet2D download a missing one. Pass dino and grounding on that constructor. A missing file is written into cache/weights/det2d/, and backend="trt" builds the DINOv2 engine for that weight when none matches. The required weight files and destinations are listed below. The lists below are the published vitl14 and swinb pair.

先建 cache/weights/det2d/。这些权重不在 SEU-WYL/WAPR。可以自己放上已发表的文件，也可以让 WAPRDet2D 去下载缺的那一份。在构造时传入 dino 和 grounding。缺的文件写进 cache/weights/det2d/，backend="trt" 在没有匹配引擎时为这份 DINOv2 权重构建引擎。所需权重文件及目标位置列在下方。下面列出的是已发表的 vitl14 和 swinb。

GroundingDINO

The Swin-B config is copied automatically from cache/sources/GroundingDINO/groundingdino/config/groundingdino_swinb.json to cache/weights/det2d/ when the detector weights are prepared. A fresh upstream clone still calls a CUDA extension on GPU; the attention adaptation selects multi_scale_deformable_attn_pytorch instead.

准备检测器权重时，Swin-B 配置会自动从 cache/sources/GroundingDINO/groundingdino/config/groundingdino_swinb.json 复制到 cache/weights/det2d/。新克隆的上游源码在 GPU 上仍调用 CUDA 扩展；注意力适配使其改用 multi_scale_deformable_attn_pytorch。

Code代码: IDEA-Research/GroundingDINO → cache/sources/GroundingDINO/
Weight权重: groundingdino_swinb_cogcoor.pth → cache/weights/det2d/

SAM

SAM2.1-L uses the installed Ultralytics package and its sam2.1_l.pt checkpoint. No native Meta SAM2 package is installed.

SAM2.1-L 使用已安装的 Ultralytics 发行包与 sam2.1_l.pt 权重，不安装 Meta 原生 SAM2 包。

Package发行包: ultralytics>=8.3.70,<9; installed in the current interpreter安装在当前解释器
Weight权重: sam2.1_l.pt → cache/weights/det2d/

DINOv2

ViT-L/14. The checkout needs hubconf.py at Torch Hub cache/hubconf.py. The xFormers warning at startup can be ignored. Do not install xFormers for this package.

DINOv2 使用官方 torch.hub.load("facebookresearch/dinov2", ...) 入口，复用现有 Hub 缓存及 TORCH_HOME。xFormers 对此推理路径非必需。

Code代码: facebookresearch/dinov2 → Torch Hub cache/
Weight权重: dinov2_vitl14_pretrain.pth → cache/weights/det2d/

BERT

GroundingDINO uses Google's bert-base-uncased text encoder. Detection setup downloads the required text encoder files and loads them from the local cache.

GroundingDINO 使用 Google 发布的 bert-base-uncased 文本编码器。检测环境准备命令自动下载所需文本编码器文件，并从本地缓存加载。

The installer places these five files in cache/weights/det2d/bert-base-uncased/; for an offline installation, copy them there before starting the detector. wapr/det2d.py sets text_encoder_type to that directory. If an otherwise complete offline setup still attempts a Hub request, set HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. The loader permits the known missing bert.embeddings.position_ids key.

安装脚本会将这五份文件放在 cache/weights/det2d/bert-base-uncased/；离线安装时应在启动检测器前手动放齐。wapr/det2d.py 将 text_encoder_type 设为该目录。文件齐全但仍尝试访问 Hub 时，可设置 HF_HUB_OFFLINE=1 和 TRANSFORMERS_OFFLINE=1。加载器允许已知缺失的 bert.embeddings.position_ids 键。

Hashes哈希

Downloaded model files are verified with SHA256; reference checksums are listed below. TensorRT engines are generated on the target device for its GPU and runtime environment.

下载的模型文件通过 SHA256 校验，参考摘要如下。TensorRT 引擎在目标设备上生成，应在对应的 GPU 和运行环境中使用。

File文件	SHA256
`groundingdino_swinb_cogcoor.pth`	`46270f7a822e6906b655b729c90613e48929d0f2bb8b9b76fd10a856f3ac6ab7`
`sam2.1_l.pt`	`ab7e1ac9cb9f6eb3bcf197ece044f06a707ec49129361a2b47e93e1db6989efd`
`dinov2_vitl14_pretrain.pth`	`d5383ea8f4877b2472eb973e0fd72d557c7da5d3611bd527ceeb1d7162cbf428`
`bert-base-uncased/config.json`	`7160e1553ad2ca51d8c1cb066be533db31826e12d173824c1bb0cb1a4f187d20`
`bert-base-uncased/pytorch_model.bin`	`097417381d6c7230bd9e3557456d726de6e83245ec8b24f529f60198a67b203a`
`bert-base-uncased/tokenizer.json`	`ce64fce797c24f68df90b40a3f74f579b336a493db14bd583fd520ea0d8c9a98`
`bert-base-uncased/tokenizer_config.json`	`a025160ef0431f1a392f6f050c1310f4c5d9fb6f275932dbccba73c4d214bf10`
`bert-base-uncased/vocab.txt`	`07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3`
`groundingdino_swinb.json`	`65c95dabc0e6b02ad0edd506d21e2fbf535e422a56a870813e9b1806f4537d71`

The saved BOP2D accuracy results used MartinSmeyer/cocoapi v1.0. Its evaluation files are not required for detection or pose inference.

BOP2D 精度统计采用 MartinSmeyer/cocoapi v1.0 评测；检测与位姿推理无需安装其评测文件。

Troubleshooting故障排查

Common installation problems常见安装问题

Symptom现象	Check and action检查与处理
`nvcc`, `EGL`, or `OpenGL` missing	The PyTorch^[5] wheel does not supply the compiler or GL/EGL development files. For source compilation, install a CUDA toolkit supporting your GPU and your distribution's OpenGL/EGL development packages; set `CUDA_HOME` if the toolkit is outside `/usr/local/cuda`.PyTorch^[5] wheel 不提供编译器或 GL/EGL 开发文件。源码编译时安装支持当前 GPU 的 CUDA toolkit 和发行版对应的 OpenGL/EGL 开发包；若 toolkit 不在 `/usr/local/cuda`，设置 `CUDA_HOME`。
`torch.cuda.is_available() == False`	Check the NVIDIA driver and GPU access inside the container. A CUDA-enabled wheel alone cannot make a GPU visible.检查 NVIDIA 驱动和容器内 GPU 映射；仅安装带 CUDA 的 wheel 无法让容器获得 GPU。
2D source check fails2D 源码检查失败	Use the compatible `third_party/` trees. Fresh GroundingDINO^[1] clones need the PyTorch attention adaptation. For pose-only examples 02 and 03, set `prepare_2d_detector = False`.使用兼容的 `third_party/` 源码；新克隆的 GroundingDINO^[1] 需要纯 PyTorch 注意力适配。仅运行位姿示例 02、03 时可将 `prepare_2d_detector` 设为 `False`。
Asset download or hash fails资源下载或哈希失败	Check disk space and the URLs in the file lists above. Pose packs come from `SEU-WYL/WAPR`; detector files are listed above. Fetch incomplete files again and retain SHA-256 verification. Run `python -m wapr.download_assets` to retry the configured sample/pose packs.检查磁盘空间及上方文件清单中的下载地址。位姿资源来自 `SEU-WYL/WAPR`，检测文件见上方清单。文件不完整时重新获取，保留 SHA-256 校验；可运行 `python -m wapr.download_assets` 重试当前配置的小样例与位姿数据包。
Engine fails after moving machines换机器后引擎无法加载	Run `python -m wapr.export_engines` on the inference GPU to rebuild the four standard pose engines and the grouped scoring engine. Rerun `python -m wapr.bootstrap --feature core dinov2 det2d sam2` to prepare the 2D engine through the default installation. Build with the Python, TensorRT^[6] version, and GPU used for inference.在推理显卡上运行 `python -m wapr.export_engines`，重建四份常规位姿引擎及批量计算评分引擎。2D 引擎通过运行默认安装 `python -m wapr.bootstrap --feature core dinov2 det2d sam2` 准备。编译环境应与推理所用的 Python、TensorRT^[6] 版本和 GPU 对应。

Environment and resource preparation环境与资源准备

pip install wapr installs the package. python -m wapr.bootstrap --feature core dinov2 det2d sam2 prepares TensorRT, 2D detection, SAM2, DINOv2 and shared demo resources. Reconstruction and robotics are optional extensions. --feature core selects pose-only setup. Existing PyTorch is retained; replacing installed dependencies requires confirmation.

pip install wapr 安装项目包；python -m wapr.bootstrap --feature core dinov2 det2d sam2 在兼容环境准备 TensorRT、2D 检测、SAM2、DINOv2 和共用小样。重建与机器人扩展按需准备。--feature core 只准备位姿推理；保留现有 PyTorch，更换已安装依赖需确认。

Source and wheel share cache/weights/, cache/environments/ and dataset-specific cache/samples/ under the WAPR_CACHE_DIR resource root. Outputs are separate: outputs/<number_task>/. SAM2.1 uses the installed Ultralytics package. DINOv2 uses official Torch Hub and its existing cache. OpenGL is the default renderer; nvdiffrast is optional.

源码与 wheel 在 WAPR_CACHE_DIR 资源根下共用 cache/weights/、cache/environments/ 与按数据集组织的 cache/samples/。输出独立存于 outputs/<编号_任务>/。SAM2.1 通过 Ultralytics 调用；DINOv2 通过官方 Torch Hub 复用缓存。默认使用 OpenGL 渲染，nvdiffrast 为可选后端。

References and licenses参考文献与许可

GroundingDINO — Apache-2.0. Vendored detector source; original notices retained.随包检测源码；保留原始声明。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
Liu et al. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. ECCV 2024. · Paper论文 ↩ ↩ ↩
Ultralytics — AGPL-3.0. Installed Ultralytics loader; review AGPL obligations or obtain an upstream commercial license. Native Meta SAM 2 has separate terms.已安装的 Ultralytics 加载器；须遵守 AGPL 条款或取得上游商业授权。Meta 原生 SAM 2 另有许可。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2 ↩
DINOv2 — Apache-2.0. Code and official DINOv2 weights; retain copyright and license.源码与官方 DINOv2 权重；保留版权和许可。 · GitHubGitHub · License/notice 1许可/声明 1
Oquab et al. DINOv2: Learning Robust Visual Features without Supervision. arXiv:2304.07193, 2023. · Paper论文 ↩
SAM 2 — Apache-2.0; cctorch: BSD-3-Clause. Native source and official checkpoints; cctorch carries an additional BSD notice. Ultralytics-converted files require checking their distributor terms.原生源码与官方权重；cctorch 另附 BSD 声明。Ultralytics 转换文件还需核对分发方条款。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
Ravi et al. SAM 2: Segment Anything in Images and Videos. arXiv:2408.00714, 2024. · Paper论文
torch — BSD License. Runtime dependency; dependencies and model/data assets retain their own licenses.运行依赖；依赖及模型/数据资源保留各自许可。 · Original source原始来源 · License/notice 1许可/声明 1 · License/notice 2许可/声明 2 ↩ ↩ ↩ ↩
tensorrt-cu12 — Other/Proprietary License. Runtime dependency; dependencies and model/data assets retain their own licenses.运行依赖；依赖及模型/数据资源保留各自许可。 · GitHubGitHub · License/notice 1许可/声明 1 ↩ ↩ ↩ ↩
BERT base uncased — Apache-2.0. Official selected text model; preserve its model-repository terms.所选官方文本模型；保留其模型仓库条款。 · Original source原始来源 · License/notice 1许可/声明 1
Devlin et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT 2019. · Paper论文 ↩ ↩
LM-O data — CC-BY-SA-4.0. Data and model excerpt, not the loader code.数据与模型摘录，不是读取代码。 · Original source原始来源 · GitHub: BOP toolkitGitHub：BOP 工具集
Brachmann et al. Learning 6D Object Pose Estimation Using 3D Object Coordinates. ECCV 2014. · Paper论文 ↩ ↩

Model weights and dataset/task assets may have separate terms. WAPR's source license does not replace them.模型权重、数据集与任务资源可能有独立条款；WAPR 源码许可不替代这些许可。

SAM2 point/box segmentation: from wapr.sam2 import predict_mask. Detector setup selects compatible Transformers 4 and Hugging Face Hub versions; existing dependency replacements require approval.SAM2 点／框分割：from wapr.sam2 import predict_mask。检测环境自动选择兼容的 Transformers 4 与 Hugging Face Hub 版本；更换已有依赖须确认。