目录

intro1 电脑配置2 环境配置2.1 资源下载2.2 conda环境

3 测试3.1 初始demo

4 补充4.1 cuda版本问题4.2 Build extensions报错4.3 run_demo.py报错

intro

论文我就不多说了

code: https://github.com/NVlabs/FoundationPose

paper: https://arxiv.org/abs/2312.08344

1 电脑配置

Linux 环境 ：ubuntu 22.04cuda 版本： 11.8显卡：GeForce RTX 4080IDE：pycharm

2 环境配置

2.1 资源下载

库

git clone https://github.com/NVlabs/FoundationPose.git

必须下载

权重

主目录下新建./weights/，放在该路径下

测试数据

主目录下新建./demo_data/，放在该路径下

可选下载

数据集

太大了，我没下载

model-free few-shot version

2.2 conda环境

这里参考blog

Eigen3 3.4.0安装

cd $HOME && wget -q https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz && \

tar -xzf eigen-3.4.0.tar.gz && \

cd eigen-3.4.0 && mkdir build && cd build

cmake .. -Wno-dev -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++14 ..

sudo make install

cd $HOME && rm -rf eigen-3.4.0 eigen-3.4.0.tar.gz

建议不要使用官方github上的，我用下面的路径一直配置不对

# Install Eigen3 3.4.0 under conda environment

conda install conda-forge::eigen=3.4.0

export CMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH:/eigen/path/under/conda"code>

创建虚拟环境

我这里都是在pycharm的终端中进行（个人习惯）

# create conda environment

conda create -n foundationpose python=3.9

# activate conda environment

conda activate foundationpose

python库

# install dependencies

python -m pip install -r requirements.txt

下面的貌似自动给安装好了

# Install NVDiffRast

python -m pip install --quiet --no-cache-dir git+https://github.com/NVlabs/nvdiffrast.git

# Kaolin (Optional, needed if running model-free setup)

python -m pip install --quiet --no-cache-dir kaolin==0.15.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.0.0_cu118.html

# PyTorch3D

python -m pip install --quiet --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py39_cu118_pyt200/download.html

这里可以检查下自己的torch、pytorch3d版本，终端输入python

Python 3.9.19 (main, May 6 2024, 19:43:03)

[GCC 11.2.0] :: Anaconda, Inc. on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import torch,torchvision,pytorch3d

>>> torch.__version__

'2.0.0+cu118'

>>> torchvision.__version__

'0.15.1+cu118'

>>> pytorch3d.__version__

'0.7.3'

>>> exit()

-构建C++扩展

# Build extensions

CMAKE_PREFIX_PATH=$CONDA_PREFIX/lib/python3.9/site-packages/pybind11/share/cmake/pybind11 bash build_all_conda.sh

这里会花费比较长时间，并且可能会有各种Warning，只要最后能到100%，无视

最后成功的输出

[100%] Linking CXX shared module mycpp.cpython-39-x86_64-linux-gnu.so

lto-wrapper: warning: using serial compilation of 2 LTRANS jobs

[100%] Built target mycpp

Obtaining file:///home/user/code/embodied_AI/FoundationPose/bundlesdf/mycuda

Preparing metadata (setup.py) ... done

Installing collected packages: common

Running setup.py develop for common

Successfully installed common-0.0.0

3 测试

<code>wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run

sudo sh cuda_11.8.0_520.61.05_linux.run

记得再下载cudnn

更换~/.bashrc里的cuda路径并且source

在虚拟环境终端下nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver

Copyright (c) 2005-2022 NVIDIA Corporation

Built on Wed_Sep_21_10:33:58_PDT_2022

Cuda compilation tools, release 11.8, V11.8.89

Build cuda_11.8.r11.8/compiler.31833905_

这里我还在conda中进行了下面操作

conda install cudatoolkit=11.8

成功

4.2 Build extensions报错

报错

bash: build_all_conda.sh: 没有那个文件或目录

原因：路径没有在FoundationPose下

遇到找不到Eigen3路径的

在终端使用下面代码重新安装

cd $HOME && wget -q https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz && \

tar -xzf eigen-3.4.0.tar.gz && \

cd eigen-3.4.0 && mkdir build && cd build

cmake .. -Wno-dev -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++14 ..

sudo make install

cd $HOME && rm -rf eigen-3.4.0 eigen-3.4.0.tar.gz

torch版本对应

如果cuda与torch会pytorch3d的版本不对应，也会报错，所以强烈建议按照官方的版本来，为自己11.6就是遇到各种问题换成11.8直接都解决

4.3 run_demo.py报错

packages/torch/nn/modules/module.py", line 1130, in _call_impl

return forward_call(*input, **kwargs)

File "/home/zqy/anaconda3/envs/foundationpose/lib/python3.9/site-packages/torch/nn/modules/transformer.py", line 437, in forward

return torch._transformer_encoder_layer_fwd(

RuntimeError: expected scalar type Half but found Float

原因：torch的版本太低，切换成2.0以上就不会遇到了