[Bug] 华为昇腾（Atlas 800T A2）使用lmdeploy

[holoodst]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

我们有华为昇腾的裸金属机，但是其网络是受限制的仅能访问国内的网络。按照指引通过dockerfile的方法安装lmdeploy总是报网络方面的错误。是否有其他的安装方法和更详细的文档或者是已经制作好的镜像。

Reproduction

DOCKER_BUILDKIT=1 docker build -t lmdeploy-aarch64-ascend:latest
-f docker/Dockerfile_aarch64_ascend .

Environment

无

Error traceback

No response

[lvhan028]

因为一些安全方面的原因，我们不好把镜像放在docker hub
关于其他方式，@jinminxi104 方便提供下么？

[CyCle1024]

能否提供一下网络报错的执行日志，Dockerfile中似乎没有访问github或者huggingface官网的命令，用的都是国内的源比如清华tuna镜像以及华为官网的python源码包。

[holoodst]

@CyCle1024 目前我们用的是华为的裸金属机运行的是欧拉系统，在这个上面安装的docker版本比较低，且高版本无法适配欧拉系统，使用dockerfile的时候是报的无法识别--mount这个标签。
我在本机x86的机子上运行虚拟机安装了高版本的docker运行dockerfile试图制作镜像 报错如下：

[holoodst]

@lvhan028 可否有其他地方下载制作完成的镜像，如果能提供其他方法安装也是更好的

[CyCle1024]

核心是--mount参数不支持，我们看看能否提供其他方式的Dockerfile安装。

[holoodst]

@CyCle1024 这个是我们的docker信息

[mikeswf]

我这边也是同样存在着因为无法连公司外的网无法将dockerfile运行成功的问题。能不能分享一个除了Drivers，Firmware 和 CANN之外，其他都配置好的镜像～

[holoodst]

@CyCle1024 我尝试改写dickerfile文件将--mount标签用--volume标签进行改写
`FROM ubuntu:20.04 as base_builder

WORKDIR /tmp

ARG http_proxy
ARG https_proxy
ARG DEBIAN_FRONTEND=noninteractive

RUN sed -i 's@http://..ubuntu.com@http://mirrors.tuna.tsinghua.edu.cn@g' /etc/apt/sources.list &&
apt update &&
apt install --no-install-recommends ca-certificates -y &&
apt install --no-install-recommends bc wget -y &&
apt install --no-install-recommends git curl gcc make g++ pkg-config unzip -y &&
apt install --no-install-recommends libsqlite3-dev libblas3 liblapack3 gfortran vim -y &&
apt install --no-install-recommends liblapack-dev libblas-dev libhdf5-dev libffi-dev -y &&
apt install --no-install-recommends libssl-dev zlib1g-dev xz-utils cython3 python3-h5py -y &&
apt install --no-install-recommends libopenblas-dev libgmpxx4ldbl liblzma-dev -y &&
apt install --no-install-recommends libicu66 libxml2 pciutils libgl1-mesa-glx libbz2-dev -y &&
apt install --no-install-recommends libreadline-dev libncurses5 libncurses5-dev libncursesw5 -y &&
sed -i 's@http://mirrors.tuna.tsinghua.edu.cn@https://mirrors.tuna.tsinghua.edu.cn@g' /etc/apt/sources.list &&
apt clean && rm -rf /var/lib/apt/lists/

ARG PYVERSION=3.10.5

ENV LD_LIBRARY_PATH=/usr/local/python${PYVERSION}/lib:
PATH=/usr/local/python${PYVERSION}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

RUN umask 0022 &&
wget https://repo.huaweicloud.com/python/${PYVERSION}/Python-${PYVERSION}.tar.xz &&
tar -xf Python-${PYVERSION}.tar.xz && cd Python-${PYVERSION} && ./configure --prefix=/usr/local/python${PYVERSION} --enable-shared &&
make -j 16 && make install &&
ln -sf /usr/local/python${PYVERSION}/bin/python3 /usr/bin/python3 &&
ln -sf /usr/local/python${PYVERSION}/bin/python3 /usr/bin/python &&
ln -sf /usr/local/python${PYVERSION}/bin/pip3 /usr/bin/pip3 &&
ln -sf /usr/local/python${PYVERSION}/bin/pip3 /usr/bin/pip &&
cd .. &&
rm -rf Python*

RUN -v /root/.cache/pip:/root/.cache/pip pip3 config set global.index-url http://mirrors.aliyun.com/pypi/simple &&
pip3 config set global.trusted-host mirrors.aliyun.com &&
pip3 install -U pip &&
pip3 install wheel==0.43.0 scikit-build==0.18.0 numpy==1.24 setuptools==69.5.1 &&
pip3 install decorator sympy cffi &&
pip3 install cmake ninja pyyaml &&
pip3 install pathlib2 protobuf attrs attr scipy &&
pip3 install requests psutil absl-py

ENV LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/hdf5/serial:$LD_LIBRARY_PATH
ENV LD_PRELOAD=/lib/aarch64-linux-gnu/libGLdispatch.so.0:$LD_PRELOAD

RUN if [ ! -d "/lib64" ];
then
mkdir /lib64 && ln -sf /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1;
fi

FROM ubuntu:20.04 as build_temp
COPY . /tmp

FROM base_builder as cann_builder

ARG ASCEND_BASE=/usr/local/Ascend
ARG TOOLKIT_PATH=$ASCEND_BASE/ascend-toolkit/latest

ENV LD_LIBRARY_PATH=
$ASCEND_BASE/driver/lib64:
$ASCEND_BASE/driver/lib64/common:
$ASCEND_BASE/driver/lib64/driver:
$ASCEND_BASE/driver/tools/hccn_tool/:
$TOOLKIT_PATH/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/:
$LD_LIBRARY_PATH

run files should be placed at the root dir of repo

ARG CHIP=all
ARG TOOLKIT_PKG=Ascend-cann-toolkit_.run
ARG KERNELS_PKG=Ascend-cann-kernels-.run

RUN -v /tmp:/tmp:cached
umask 0022 &&
mkdir -p $ASCEND_BASE/driver &&
if [ "$CHIP" != "all" ];
then
CHIPOPTION="--chip=$CHIP";
else
CHIPOPTION="";
fi &&
chmod +x $TOOLKIT_PKG $KERNELS_PKG &&
./$TOOLKIT_PKG --quiet --install --install-path=$ASCEND_BASE --install-for-all $CHIPOPTION &&
./$KERNELS_PKG --quiet --install --install-path=$ASCEND_BASE --install-for-all &&
rm -f $TOOLKIT_PKG $KERNELS_PKG

ENV GLOG_v=2
LD_LIBRARY_PATH=$TOOLKIT_PATH/lib64:$LD_LIBRARY_PATH
TBE_IMPL_PATH=$TOOLKIT_PATH/opp/op_impl/built-in/ai_core/tbe
PATH=$TOOLKIT_PATH/ccec_compiler/bin:$PATH
ASCEND_OPP_PATH=$TOOLKIT_PATH/opp
ASCEND_AICPU_PATH=$TOOLKIT_PATH

ENV PYTHONPATH=$TBE_IMPL_PATH:$PYTHONPATH

SHELL ["/bin/bash", "-c"]
RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> ~/.bashrc &&
. ~/.bashrc

dlinfer

transformers>=4.41.0 is required for internlm2 model

timm is required for internvl2 model

RUN -v pip-cache:/root/.cache/pip
pip3 install transformers>=4.41.0 timm &&
pip3 install dlinfer-ascend==0.1.0.post1

lmdeploy

FROM build_temp as copy_temp
RUN rm -rf /tmp/*.run

FROM cann_builder as final_builder
COPY --from=copy_temp /tmp /opt/lmdeploy
WORKDIR /opt/lmdeploy

RUN -v pip-cache:/root/.cache/pip
sed -i '/triton/d' requirements/runtime.txt &&
pip3 install -v --no-build-isolation -e .`
他不再报--mount无法识别但是有其他的报错如下图所示

[CyCle1024]

@holoodst 试试下面这个？看上去这个错误是dockerfile copy的语法不支持导致去拉取高版本的copy组件

FROM ubuntu:20.04 as base_builder

WORKDIR /opt/lmdeploy

ARG http_proxy
ARG https_proxy
ARG DEBIAN_FRONTEND=noninteractive

RUN sed -i 's@http://.*.ubuntu.com@http://mirrors.tuna.tsinghua.edu.cn@g' /etc/apt/sources.list && \
    apt update && \
    apt install --no-install-recommends ca-certificates -y && \
    apt install --no-install-recommends bc wget -y && \
    apt install --no-install-recommends git curl gcc make g++ pkg-config unzip -y && \
    apt install --no-install-recommends libsqlite3-dev libblas3 liblapack3 gfortran vim -y && \
    apt install --no-install-recommends liblapack-dev libblas-dev libhdf5-dev libffi-dev -y && \
    apt install --no-install-recommends libssl-dev zlib1g-dev xz-utils cython3 python3-h5py -y && \
    apt install --no-install-recommends libopenblas-dev libgmpxx4ldbl liblzma-dev -y && \
    apt install --no-install-recommends libicu66 libxml2 pciutils libgl1-mesa-glx libbz2-dev -y && \
    apt install --no-install-recommends libreadline-dev libncurses5 libncurses5-dev libncursesw5 -y && \
    sed -i 's@http://mirrors.tuna.tsinghua.edu.cn@https://mirrors.tuna.tsinghua.edu.cn@g' /etc/apt/sources.list && \
    apt clean && rm -rf /var/lib/apt/lists/*

ARG PYVERSION=3.10.5

ENV LD_LIBRARY_PATH=/usr/local/python${PYVERSION}/lib: \
    PATH=/usr/local/python${PYVERSION}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

RUN umask 0022  && \
    wget https://repo.huaweicloud.com/python/${PYVERSION}/Python-${PYVERSION}.tar.xz && \
    tar -xf Python-${PYVERSION}.tar.xz && cd Python-${PYVERSION} && ./configure --prefix=/usr/local/python${PYVERSION} --enable-shared && \
    make -j 16 && make install && \
    ln -sf /usr/local/python${PYVERSION}/bin/python3 /usr/bin/python3 && \
    ln -sf /usr/local/python${PYVERSION}/bin/python3 /usr/bin/python && \
    ln -sf /usr/local/python${PYVERSION}/bin/pip3 /usr/bin/pip3 && \
    ln -sf /usr/local/python${PYVERSION}/bin/pip3 /usr/bin/pip && \
    cd .. && \
    rm -rf Python*

RUN pip3 config set global.index-url http://mirrors.aliyun.com/pypi/simple && \
    pip3 config set global.trusted-host mirrors.aliyun.com && \
    pip3 install -U pip && \
    pip3 install wheel==0.43.0 scikit-build==0.18.0 numpy==1.24 setuptools==69.5.1 && \
    pip3 install decorator sympy cffi && \
    pip3 install cmake ninja pyyaml && \
    pip3 install pathlib2 protobuf attrs attr scipy && \
    pip3 install requests psutil absl-py

ENV LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/hdf5/serial:$LD_LIBRARY_PATH
ENV LD_PRELOAD=/lib/aarch64-linux-gnu/libGLdispatch.so.0:$LD_PRELOAD

RUN if [ ! -d "/lib64" ]; \
    then \
        mkdir /lib64 && ln -sf /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1; \
    fi

COPY . /opt/lmdeploy/

ARG ASCEND_BASE=/usr/local/Ascend
ARG TOOLKIT_PATH=$ASCEND_BASE/ascend-toolkit/latest

ENV LD_LIBRARY_PATH=\
$ASCEND_BASE/driver/lib64:\
$ASCEND_BASE/driver/lib64/common:\
$ASCEND_BASE/driver/lib64/driver:\
$ASCEND_BASE/driver/tools/hccn_tool/:\
$TOOLKIT_PATH/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/:\
$LD_LIBRARY_PATH

# run files should be placed at the root dir of repo
ARG CHIP=all
ARG TOOLKIT_PKG=Ascend-cann-toolkit_*.run
ARG KERNELS_PKG=Ascend-cann-kernels-*.run

RUN umask 0022 && \
    mkdir -p $ASCEND_BASE/driver && \
    if [ "$CHIP" != "all" ]; \
    then \
        CHIPOPTION="--chip=$CHIP"; \
    else \
        CHIPOPTION=""; \
    fi && \
    chmod +x $TOOLKIT_PKG $KERNELS_PKG && \
    ./$TOOLKIT_PKG --quiet --install --install-path=$ASCEND_BASE --install-for-all $CHIPOPTION && \
    ./$KERNELS_PKG --quiet --install --install-path=$ASCEND_BASE --install-for-all && \
    rm -f $TOOLKIT_PKG $KERNELS_PKG

ENV GLOG_v=2 \
    LD_LIBRARY_PATH=$TOOLKIT_PATH/lib64:$LD_LIBRARY_PATH \
    TBE_IMPL_PATH=$TOOLKIT_PATH/opp/op_impl/built-in/ai_core/tbe \
    PATH=$TOOLKIT_PATH/ccec_compiler/bin:$PATH \
    ASCEND_OPP_PATH=$TOOLKIT_PATH/opp \
    ASCEND_AICPU_PATH=$TOOLKIT_PATH

ENV PYTHONPATH=$TBE_IMPL_PATH:$PYTHONPATH

SHELL ["/bin/bash", "-c"]
RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> ~/.bashrc && \
    . ~/.bashrc

# dlinfer
# transformers>=4.41.0 is required for internlm2 model
# timm is required for internvl2 model
RUN pip3 install transformers>=4.41.0 timm && \
    pip3 install dlinfer-ascend==0.1.0.post1

RUN sed -i '/triton/d' requirements/runtime.txt && \
    pip3 install -v --no-build-isolation -e .

[holoodst]

@CyCle1024 刚试了一下还是不得行

是否还需要对docker源之类的做特别的设置

[CyCle1024]

@CyCle1024 刚试了一下还是不得行 是否还需要对docker源之类的做特别的设置

可以考虑源代理，但是看描述不是源问题而是去拉取了dockerfile-copy模块，这个是不应该的。我这边尝试一下能否在一些x86服务器上做跨平台构建，你之前用个人PC的构建方式不太对，需要指定--platform，具体我会尝试一下再反馈。

[holoodst]

@CyCle1024 非常感谢

[CyCle1024]

@CyCle1024 刚试了一下还是不得行 是否还需要对docker源之类的做特别的设置

可以考虑源代理，但是看描述不是源问题而是去拉取了dockerfile-copy模块，这个是不应该的。我这边尝试一下能否在一些x86服务器上做跨平台构建，你之前用个人PC的构建方式不太对，需要指定--platform，具体我会尝试一下再反馈。

@holoodst @mokeyish
我这边已经在 x86_64 机器上构建 docker 镜像成功，并在 arm64 的 Atlas 800T A2 服务器上测试运行过 llama 7B 模型成功，步骤如下：

环境准备

1. 操作系统为 Ubuntu 20.04，架构为 amd64 (x86_64)
2. Docker，已验证过的版本信息如下：

docker version

Client:
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.21.1
 Git commit:        24.0.7-0ubuntu2~20.04.1
 Built:             Wed Mar 13 20:29:24 2024
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.21.1
  Git commit:       24.0.7-0ubuntu2~20.04.1
  Built:            Wed Mar 13 20:29:24 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.12
  GitCommit:
 runc:
  Version:          1.1.12-0ubuntu2~20.04.1
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:

3. Docker buildx，请先运行docker buildx version确认 Docker buildx 版本，已验证的插件版本：github.com/docker/buildx v0.17.1 257815a。如果显示 docker: 'buildx' is not a docker command. 则需要按照以下步骤安装 Docker buildx:

• (a) 添加 docker GPG 密钥: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
• (b) 添加 docker apt 仓库: sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
• (c) 更新软件源: sudo apt update
• (d) 安装 docker-buildx-plugin: sudo apt install docker-buildx-plugin
• (e) 确认安装版本: docker buildx version

创建跨平台的 docker builder

docker buildx create --name lmdeploy_builder
docker buildx inspect lmdeploy_builder --bootstrap

利用 builder 构建 docker 镜像

cd 至 lmdeploy 项目根目录并确认将 Ascend-cann-kernels-910b_*_linux.run 和
Ascend-cann-toolkit_*_linux-aarch64.run 以及 Ascend-cann-nnal_*_linux-aarch64.run放在项目根目录。

DOCKER_BUILDKIT=1 docker buildx build \
    -t lmdeploy-aarch64-dlinfer:v0.6.1 \
    --builder lmdeploy_builder \
    --platform linux/arm64 \
    --load \
    -f docker/Dockerfile_aarch64_ascend .

由于是在 x86_64 架构上模拟 arm64 执行，运行效果非常慢，整个构建在网络通畅的情况下也要 1-2 个小时 (CPU Intel 10代i7 16线程)。构建完成后会，确认镜像架构：

docker inspect lmdeploy-aarch64-dlinfer:v0.6.1 | grep Architecture

看到返回 arm64 架构说明构建正确。

[holoodst]

@CyCle1024 佬我依据最新提供的方法成功的制作出了镜像，验证：
docker inspect lmdeploy-aarch64-910b-dlinfer:v0.6.1 | grep Architecture
然后将镜像重新load进昇腾的docker中
执行验证语句：
docker run -e ASCEND_VISIBLE_DEVICES=0,1,2,3 --rm --name lmdeploy_v -t lmdeploy-aarch64-910b-dlinfer:v0.6.1 lmdeploy check_env
会出现报错

(base) [root@bms-hyzy images]# docker run -e ASCEND_VISIBLE_DEVICES=0,1,2,3 --rm --name lmdeploy_v -t lmdeploy-aarch64-910b-dlinfer:v0.6.1 lmdeploy check_env
WARNING: IPv4 forwarding is disabled. Networking will not work.
DrvMngGetConsoleLogLevel failed. (g_conLogLevel=3)
[EVENT] PROFILING(1,python3.10):2024-10-14-07:24:12.183.589 [msprof_callback_impl.cpp:336] >>> (tid:1) Started to register profiling ctrl callback.
[EVENT] PROFILING(1,python3.10):2024-10-14-07:24:12.184.109 [msprof_callback_impl.cpp:343] >>> (tid:1) Started to register profiling hash id callback.
[INFO] PROFILING(1,python3.10):2024-10-14-07:24:12.184.254 [prof_atls_plugin.cpp:83] >>> (tid:1) RegisterProfileCallback, callback type is 7
[EVENT] PROFILING(1,python3.10):2024-10-14-07:24:12.184.357 [msprof_callback_impl.cpp:350] >>> (tid:1) Started to register profiling enable host freq callback.
[INFO] PROFILING(1,python3.10):2024-10-14-07:24:12.184.455 [prof_atls_plugin.cpp:83] >>> (tid:1) RegisterProfileCallback, callback type is 8
[ERROR] ATRACE(1,python3.10):2024-10-14-07:24:12.200.254 [trace_driver_api.c:56](tid:1) get platform info failed, drvErr=87.
[INFO] HCCL(1,python3.10):2024-10-14-07:24:12.248.380 [adapter_rts.cc:2389][1][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1]
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:12.426.231 [runtime.cc:1749]1 CheckHaveDevice:Call halGetDeviceInfo failed: drvRet=87, module type=0, info type=1.
[INFO] PROFILING(1,python3.10):2024-10-14-07:24:12.426.573 [prof_atls_plugin.cpp:160] >>> (tid:1) Module[7] register callback of ctrl handle.
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:12.427.506 [driver.cc:65]1 GetDeviceCount:Call drvGetDevNum, drvRetCode=87.
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:12.427.638 [api_c_device.cc:21]1 rtGetDeviceCount:ErrCode=507899, desc=[driver error:internal error], InnerCode=0x7020010
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:12.427.745 [error_message_manage.cc:53]1 FuncErrorReason:report error module_type=3, module_name=EE8888
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:12.427.850 [error_message_manage.cc:53]1 FuncErrorReason:rtGetDeviceCount execute failed, reason=[driver error:internal error]
[ERROR] ASCENDCL(1,python3.10):2024-10-14-07:24:12.428.001 [device.cpp:366]1 aclrtGetDeviceCount: get device count failed, runtime result = 507899.
[ERROR] APP(1,python3.10):2024-10-14-07:24:12.428.151 [log_inner.cpp:76]1 compiler_depend.ts:device_count:24: "[PTA]:"get device count of NPU failed""
Warning : ASCEND_HOME_PATH environment variable is not set.
[INFO] ASCENDCL(1,python3.10):2024-10-14-07:24:12.585.503 [acl_resource_manager.cpp:91]1 GetRuntimeV2Env: runtime v2 flag : model flag = 1, singleOp flag = 1
[ERROR] IDEDD(1,python3.10):2024-10-14-07:24:12.596.258 [adx_dsmi.cpp:174][tid:1]>>> Failed to drvGetPlatformInfo, ret=87
[INFO] IDEDD(1,python3.10):2024-10-14-07:24:12.596.581 [adx_dump_record.cpp:221][tid:200]>>> start dump thread, remote dump record temp path : /opt/lmdeploy/.
[INFO] PROFILING(1,python3.10):2024-10-14-07:24:12.596.710 [prof_atls_plugin.cpp:160] >>> (tid:1) Module[48] register callback of ctrl handle.
[INFO] IDEDD(1,python3.10):2024-10-14-07:24:12.596.752 [adx_server_manager.cpp:205][tid:201]>>> Run Server(0) Process
[INFO] PROFILING(1,python3.10):2024-10-14-07:24:12.597.082 [prof_atls_plugin.cpp:160] >>> (tid:1) Module[45] register callback of ctrl handle.
[ERROR] IDEDD(1,python3.10):2024-10-14-07:24:12.597.435 [adx_dsmi.cpp:54][tid:201]>>> Get physical device number failed, err: 87
[INFO] GE(1,python3.10):2024-10-14-07:24:12.716.583 [op_tiling_manager.cc:102][EVENT]1 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [119080] micro second.
[INFO] PROFILING(1,python3.10):2024-10-14-07:24:12.727.807 [prof_atls_plugin.cpp:160] >>> (tid:1) Module[6] register callback of ctrl handle.
[ERROR] DVPP(1,python3.10):2024-10-14-07:24:12.728.870 [PlatformInfo.cpp:29][WRAP] [InitPara:29] [tid:1] get device num fail. ret = 87
[ERROR] DVPP(1,python3.10):2024-10-14-07:24:12.729.003 [dvpp_wrapper_manager.cpp:104][WRAP] [InitDvppWrapper:104] [tid:1] Init Platform Info fail.
[EVENT] PROFILING(1,python3.10):2024-10-14-07:24:13.019.737 [msprof_callback_impl.cpp:89] >>> (tid:1) MsprofCtrlCallback called, type: 255
[EVENT] PROFILING(1,python3.10):2024-10-14-07:24:13.021.087 [ai_drv_dev_api.cpp:333] >>> (tid:1) Succeeded to DrvGetApiVersion version: 0x72317
[ERROR] PROFILING(1,python3.10):2024-10-14-07:24:13.022.459 [ai_drv_dev_api.cpp:78] >>> (tid:1) Failed to drvGetPlatformInfo, ret=87
[ERROR] PROFILING(1,python3.10):2024-10-14-07:24:13.023.320 [config_manager.cpp:60] >>> (tid:1) halGetDeviceInfo get device type version failed , ret:87
[ERROR] ASCENDCL(1,python3.10):2024-10-14-07:24:13.023.488 [acl.cpp:278]1 aclInit: [INIT][DEFAULT][Init][Version]init soc version failed, ret = 507008
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.024.402 [api_impl.cc:4916]1 GetDevErrMsg:report error module_type=3, module_name=EE8888
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.024.564 [api_impl.cc:4916]1 GetDevErrMsg:ctx is NULL!
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.024.692 [api_impl.cc:4972]1 GetDevMsg:Failed to GetDeviceErrMsg, retCode=0x7070001.
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.024.807 [logger.cc:1567]1 GetDevMsg:GetDeviceMsg failed, getMsgType=0.
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.024.943 [api_c_device.cc:427]1 rtGetDevMsg:ErrCode=107002, desc=[context pointer null], InnerCode=0x7070001
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.025.050 [error_message_manage.cc:48]1 FuncErrorReason:report error module_name=EE1001
[ERROR] RUNTIME(1,python3.10):2024-10-14-07:24:13.025.157 [error_message_manage.cc:48]1 FuncErrorReason:rtGetDevMsg execute failed, reason=[context pointer null]
[ERROR] IDEDD(1,python3.10):2024-10-14-07:24:13.097.930 [adx_dsmi.cpp:54][tid:201]>>> Get physical device number failed, err: 87
[ERROR] IDEDD(1,python3.10):2024-10-14-07:24:13.598.383 [adx_dsmi.cpp:54][tid:201]>>> Get physical device number failed, err: 87
[ERROR] IDEDD(1,python3.10):2024-10-14-07:24:14.098.785 [adx_dsmi.cpp:54][tid:201]>>> Get physical device number failed, err: 87
sys.platform: linux
Python: 3.10.5 (main, Oct 12 2024, 11:01:46) [GCC 9.4.0]
CUDA available: False
MUSA available: False
numpy_random_seed: 2147483648
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
PyTorch: 2.1.0
PyTorch compiling details: PyTorch built with:
  - GCC 10.2
  - C++ Version: 201703
  - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - Build settings: BLAS_INFO=open, BUILD_TYPE=Release, CXX_COMPILER=/opt/rh/devtoolset-10/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-unused-private-field -Wno-aligned-allocation-unavailable -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=open, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.1.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

TorchVision: 0.16.0
LMDeploy: 0.6.1+9b52f8d
transformers: 4.45.2
gradio: Not Found
fastapi: 0.115.2
pydantic: 2.9.2
triton: Not Found
[ERROR] IDEDD(1,python3.10):2024-10-14-07:24:14.599.185 [adx_dsmi.cpp:54][tid:201]>>> Get physical device number failed, err: 87
[INFO] IDEDD(1,python3.10):2024-10-14-07:24:15.099.411 [adx_server_manager.cpp:238][tid:201]>>> server manager stop
[INFO] RUNTIME(1,python3.10):2024-10-14-07:24:15.423.075 [runtime.cc:1898] 1 ~Runtime: deconstruct runtime
[INFO] RUNTIME(1,python3.10):2024-10-14-07:24:15.427.244 [runtime.cc:1905] 1 ~Runtime: wait monitor success, use=0.