Windows supports examples 01–10. Examples 11–17 and the supplied SAM3D, robot and RoMa setups require Linux. Default Windows preparation skips RoMa; explicit requests report the platform restriction. GroundingDINO source preparation requires Git on PATH: install Git, reopen the terminal and verify git --version.
Windows 支持示例 01–10。示例 11–17 及当前 SAM3D、机器人、RoMa 安装方案需要 Linux。Windows 默认准备跳过 RoMa;显式请求时提示平台限制。准备 GroundingDINO 源码需要 PATH 中可用的 Git:请安装 Git,重开终端并用 git --version 确认。
Extension setup扩展环境
Start with the installation guide. Export bundled examples using export_examples; source-checkout users can run the repository's examples/ directly.
先按安装引导准备环境。随包示例可通过 export_examples 导出;源码用户可直接运行仓库中的 examples/。
Optional wheel featureswheel 可选功能
Basic installation does not install all optional frameworks. Prepare only the feature you need, or let its supported first-use entry prepare it. Extras such as wapr[det2d] provide helper dependencies; they do not replace feature preparation. Examples 01, 08, 09 and 12 prepare detection dependencies before use; importing these examples alone does not install optional features. Ultralytics is not a core pip dependency. If preparation is declined or incompatible, the feature stops before importing its optional framework. Inspect compatibility before installation:
基础安装不会安装全部可选框架。只准备需要的功能,或由支持首次使用准备的入口触发。wapr[det2d] 等 extras 提供辅助依赖,不代替功能准备。示例 01、08、09、12 会先准备检测依赖再使用,仅导入这些示例不会启动可选安装;Ultralytics 不属于 pip 基础依赖。准备被拒绝或环境不兼容时,对应功能会在导入可选框架前停止。可先检查兼容性:
--check inspects local dependencies without downloading or resolving an installation plan. Run preparation without --check to install missing dependencies; proposed replacements require confirmation.
--check 只检查本地依赖,不下载或联网解析安装计划。去掉 --check 可准备缺失依赖;更换已有依赖前会请求确认。
python -m wapr.bootstrap --feature det2d --check
python -m wapr.bootstrap --feature det2d
python -m wapr.bootstrap --feature sam3d
python -m wapr.bootstrap --feature robot
SAM3D uses an independent Python 3.11 environment under cache/environments/sam3d/. Managed Python interpreters are stored under cache/interpreters/; pose estimation retains its existing Python and PyTorch environment. SAM3D weights download from ModelScope by default, or reuse existing local weights. The upstream model license still applies. Examples 11 and 12 both default to cracker; mustard images elsewhere show recorded examples.
SAM3D 使用 cache/environments/sam3d/ 下的独立 Python 3.11 环境,托管解释器放在 cache/interpreters/;位姿估计保留原有 Python 与 PyTorch 环境。SAM3D 权重默认从魔搭下载,也可复用已有本地权重;使用仍须遵守上游模型许可。示例 11、12 默认均为 cracker;其他页面的 mustard 图片属于已录制展示。
Use --feature compatible to attempt the optional features supported by the current environment. Unsupported features are reported. Changes to existing dependencies require approval; inspect the proposed changes before accepting.
使用 --feature compatible 可尝试准备当前环境支持的可选功能;不支持的功能会报告原因。更换已有依赖需要确认,请先查看变更内容。
After installing the detection and pose environment, prepare only the extra dependencies needed by the application you use.
完成检测与位姿环境安装后,再按需准备以下应用的额外依赖。
Reconstruction物体重建
Choose reconstruction and robot extensions for the examples you want to run. SAM3D uses a compatible Linux environment and downloads weights from ModelScope by default. Upstream recommends about 32 GB GPU memory; actual requirements depend on the selected configuration.
根据需要运行的示例,准备重建或机器人扩展。SAM3D 使用兼容的 Linux 独立环境,默认从魔搭下载权重,上游建议约 32 GB 显存,实际容量以所选配置为准。
| Stage阶段 | Source and checkpoint expected by these scripts脚本需要的源码与权重 |
|---|---|
| Mask掩码 | SAM 2[2] in放在 Ultralytics; cache/weights/det2d/sam2.1_l.pt |
| Shape形状 | SAM 3D Objects[3] in放在 cache/sources/sam-3d-objects/; cache/weights/sam3d/checkpoints/pipeline.yaml and its sibling checkpoints及同目录权重 |
| Depth prior深度先验 | MoGe[4] v1 checkpointMoGe[4] v1 权重 cache/weights/moge-vitl/model.pt for the present SAM 3D pipeline供当前 SAM 3D[3] 流程使用 |
| Metric box米制包围盒 | UniPose9D[5] source plus its Git LFS源码及 Git LFS checkpoints/last.ckpt and与 config.yaml |
| Language and matching语言与匹配 | Local Qwen2.5-VL-3B-Instruct[7] for sentence prompts in 11; RoMa[6] for its shape stage, enabled by default示例 11 的句子提示使用本地 Qwen2.5-VL-3B-Instruct[7];其形状阶段默认启用 RoMa[6] |
UniPose9D[5] and Qwen[7] are prepared when requested. Reconstruction resources use the same WAPR_CACHE_DIR root as pose estimation: source dependencies in cache/sources/, model files in cache/weights/, and shared demonstration inputs in cache/samples/. Example 11 obtains a single RGB-D sample and its camera intrinsics; a complete sequence is not required.
UniPose9D[5] 与 Qwen[7] 按需准备。重建资源与位姿估计使用同一个 WAPR_CACHE_DIR 根目录:源码依赖位于 cache/sources/,模型文件位于 cache/weights/,共用演示输入位于 cache/samples/。示例 11 获取单帧 RGB-D 小样及相机内参,无需下载完整序列。
python -m wapr.bootstrap --feature unipose9d qwen
Prepare the selected features with the commands on this page before running reconstruction. Use upstream models under their respective licenses. Intermediate outputs are saved in outputs/11_reconstruct_object/stages/.
运行重建前,用本页命令准备所需功能,并遵守各上游模型许可。中间结果存于 outputs/11_reconstruct_object/stages/。
SAM2.1 uses the Ultralytics sam2.1_l.pt checkpoint. SAM3D weights are obtained from ModelScope by default under the upstream model license. MoGe checkpoints remain paired with the selected pipeline.
SAM2.1 使用 Ultralytics 的 sam2.1_l.pt。SAM3D 默认从魔搭获取权重,仍须遵守上游模型许可;MoGe 权重与当前流程匹配。
Install reconstruction features安装重建扩展
On Linux, prepare SAM3D and RoMa with the command below. WAPR uses a compatible reconstruction environment while retaining the PyTorch environment used for pose estimation. Model weights are cached for later runs.
在 Linux 上运行以下命令准备 SAM3D 和 RoMa。WAPR 使用兼容的重建环境,并保留位姿估计所用的 PyTorch 环境。模型权重下载后会缓存,供后续运行复用。
python -m wapr.bootstrap --feature sam3d roma
SAM3D weights download from ModelScope. Use is subject to the upstream model license. After exporting the bundled examples, run example 11 to reconstruct an object and example 12 to use the reconstructed model.
SAM3D 权重从魔搭下载,使用须遵守上游模型许可。导出随包示例后,运行示例 11 重建物体,再运行示例 12 使用重建模型。
Robot simulation机器人仿真
Use a compatible Linux environment with Python 3.10 or newer. Robotics uses ManiSkill[11] and SAPIEN[12]. Prepare robotics with the command below; then export the bundled examples using export_examples as shown in the installation guide.
使用兼容的 Linux 环境及 Python 3.10 或更高版本。机器人使用 ManiSkill[11] 与 SAPIEN[12]。先用以下命令准备机器人功能,再按安装指南使用 export_examples 导出随包示例。
python -m wapr.bootstrap --feature robot --check
python -m wapr.bootstrap --feature robotUse python -m wapr.bootstrap --feature robot --check to inspect the installed dependencies without starting a simulation. The first robot example then prepares its required task resources.
运行 python -m wapr.bootstrap --feature robot --check 可检查已有依赖,不启动仿真。首次运行机器人示例时会准备所需任务资源。
Example 14 needs ManiSkill's Bridge[16] assets. Example 14 downloads them automatically; the carrot and eggplant collision meshes are stored under cache/samples/maniskill/tasks/bridge_v2_real2sim_dataset/custom/models/. An explicit MS_ASSET_DIR overrides the asset root and the example derives BRIDGE_ROOT from it.
示例 14 需要 ManiSkill 的 Bridge[16] 资源。示例 14 会自动下载资源; cache/samples/maniskill/tasks/bridge_v2_real2sim_dataset/custom/models/ 下保存胡萝卜和茄子的碰撞网格。显式设置 MS_ASSET_DIR 时使用该资源根目录,示例会据此推导 BRIDGE_ROOT。
python -m wapr.bootstrap --feature robot --check
python -m wapr.bootstrap --feature robot
Optional performance experiments可选性能实验
The inference/renderer comparison additionally needs ONNX Runtime[14] GPU and nvdiffrast[8] in the same Python 3.10 environment. The recorded versions are ONNX Runtime 1.23.2 and nvdiffrast 0.3.3.1. Follow the official CUDA/cuDNN compatibility instructions and nvdiffrast installation instructions for the installed PyTorch/CUDA stack. These are optional benchmark dependencies, outside the default estimator installation.
推理与渲染后端对照还需在同一 Python 3.10 环境中准备 ONNX Runtime[14] GPU 与 nvdiffrast[8],记录版本分别为 1.23.2 与 0.3.3.1。按官方 CUDA/cuDNN 兼容说明与 nvdiffrast 安装说明匹配当前 PyTorch/CUDA 栈。这些仅用于可选性能实验,不属于默认估计器依赖。
Environment and resource preparation环境与资源准备
pip install wapr installs the package. python -m wapr.bootstrap --feature core dinov2 det2d sam2 prepares TensorRT, 2D detection, SAM2, DINOv2 and shared demo resources. Reconstruction and robotics are optional extensions. --feature core selects pose-only setup. Existing PyTorch is retained; replacing installed dependencies requires confirmation.
pip install wapr 安装项目包;python -m wapr.bootstrap --feature core dinov2 det2d sam2 在兼容环境准备 TensorRT、2D 检测、SAM2、DINOv2 和共用小样。重建与机器人扩展按需准备。--feature core 只准备位姿推理;保留现有 PyTorch,更换已安装依赖需确认。
Source and wheel share cache/weights/, cache/environments/ and dataset-specific cache/samples/ under the WAPR_CACHE_DIR resource root. Outputs are separate: outputs/<number_task>/. SAM2.1 uses the installed Ultralytics package. DINOv2 uses official Torch Hub and its existing cache. OpenGL is the default renderer; nvdiffrast is optional.
源码与 wheel 在 WAPR_CACHE_DIR 资源根下共用 cache/weights/、cache/environments/ 与按数据集组织的 cache/samples/。输出独立存于 outputs/<编号_任务>/。SAM2.1 通过已安装的 Ultralytics 调用;DINOv2 通过官方 Torch Hub 复用现有缓存。默认使用 OpenGL 渲染,nvdiffrast 为可选后端。
References and licenses参考文献与许可
- Ultralytics — AGPL-3.0. Installed Ultralytics loader; review AGPL obligations or obtain an upstream commercial license. Native Meta SAM 2 has separate terms.已安装的 Ultralytics 加载器;须遵守 AGPL 条款或取得上游商业授权。Meta 原生 SAM 2 另有许可。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
- SAM 2 — Apache-2.0; cctorch: BSD-3-Clause. Native source and official checkpoints; cctorch carries an additional BSD notice. Ultralytics-converted files require checking their distributor terms.原生源码与官方权重;cctorch 另附 BSD 声明。Ultralytics 转换文件还需核对分发方条款。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
- SAM 3D Objects — SAM License (custom). Source and model materials follow the upstream custom agreement, including redistribution, attribution and use restrictions.源码及模型材料遵循上游自定义协议,包括再分发、署名与使用限制。 · GitHubGitHub · License/notice 1许可/声明 1
- MoGe — MIT; bundled notices also include Apache-2.0. MoGe source and the selected moge-vitl model (model card: MIT); retain the complete combined LICENSE and dependency notices.MoGe 源码与所选 moge-vitl 权重(模型卡:MIT);保留完整组合 LICENSE 及依赖声明。 · GitHubGitHub · License/notice 1许可/声明 1
- UniPose9D — Apache-2.0. Inference repository and its release; helper models retain their own terms.推理仓库及其发布材料;辅助模型保留各自条款。 · GitHubGitHub · License/notice 1许可/声明 1
- RoMa — MIT. Matching source; separately obtained checkpoints and dependencies retain upstream terms.匹配源码;另行获取的权重与依赖保留上游条款。 · GitHubGitHub · License/notice 1许可/声明 1
- Qwen2.5-VL-3B-Instruct — Qwen Research License (custom). This exact 3B model permits non-commercial research/evaluation; commercial use requires permission from Alibaba Cloud. Do not infer its terms from other Qwen sizes.本项目选用的 3B 模型限定非商业研究/评估;商业使用须向 Alibaba Cloud 申请。不能套用其他 Qwen 规格的许可。 · Original source原始来源 · License/notice 1许可/声明 1
- nvdiffrast — NVIDIA Source Code License (custom). Optional renderer benchmark dependency; consult the upstream source license.可选渲染后端实验依赖;以其上游源码许可为准。 · GitHubGitHub · License/notice 1许可/声明 1
- torch — BSD License. Runtime dependency; dependencies and model/data assets retain their own licenses.运行依赖;依赖及模型/数据资源保留各自许可。 · Original source原始来源 · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
- numpy — BSD License. Runtime dependency; dependencies and model/data assets retain their own licenses.运行依赖;依赖及模型/数据资源保留各自许可。 · Original source原始来源 · License/notice 1许可/声明 1
- mani-skill — Apache-2.0. Runtime dependency; source license verified against the preserved LICENSE, with upstream copyright and third-party notices retained. Model/data assets keep separate terms.运行依赖;源码许可按已保留的 LICENSE 原文核实;保留上游版权及第三方声明,模型与数据资源另有条款。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
- sapien — MIT License. Runtime dependency; dependencies and model/data assets retain their own licenses.运行依赖;依赖及模型/数据资源保留各自许可。 · Original source原始来源 · GitHubGitHub · License/notice 1许可/声明 1
- pytorch3d — BSD-3-Clause. Runtime dependency; source license verified against the preserved LICENSE, with upstream copyright and third-party notices retained. Model/data assets keep separate terms.运行依赖;源码许可按已保留的 LICENSE 原文核实;保留上游版权及第三方声明,模型与数据资源另有条款。 · GitHubGitHub · License/notice 1许可/声明 1 · License/notice 2许可/声明 2
- onnxruntime-gpu — MIT. Optional runtime, optional inference runtime.可选推理后端,可选推理后端。 · Original source原始来源 · License/notice 1许可/声明 1 ↩ ↩
- xatlas — MIT License. Runtime dependency; dependencies and model/data assets retain their own licenses.运行依赖;依赖及模型/数据资源保留各自许可。 · GitHubGitHub · License/notice 1许可/声明 1
- BridgeData V2 · Original source原始来源 · GitHubGitHub
Model weights and dataset/task assets may have separate terms. WAPR's source license does not replace them.模型权重、数据集与任务资源可能有独立条款;WAPR 源码许可不替代这些许可。