• Members 3 posts
    2026年6月3日 10:28

    使用沐曦官方镜像:sglang:0.5.9-maca.ai3.5.3.208-torch2.8-py312-ubuntu22.04-amd64 部署模型 Qwen3.5-9B 时,能够成功部署,但是报了下面的错误,/bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found, 为什么沐曦的sglang会找nvidia的nvcc

    [2026-06-03 10:24:38] INFO: Started server process [12085]
    [2026-06-03 10:24:38] INFO: Waiting for application startup.
    [2026-06-03 10:24:38] INFO: Application startup complete.
    [2026-06-03 10:24:38] INFO: Uvicorn running on http://0.0.0.0:9100 (Press CTRL+C to quit)
    [2026-06-03 10:24:39] INFO: 127.0.0.1:58588 - "GET /model_info HTTP/1.1" 200 OK
    [2026-06-03 10:24:47 TP1] Failed to load JIT KV-Cache kernel with row_bytes=1024: ninja exited with status 1
    stdout:
    [1/2] /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    FAILED: cuda_0.o
    /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    /bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found
    ninja: build stopped: subcommand failed.

    [2026-06-03 10:24:47 TP0] Failed to load JIT KV-Cache kernel with row_bytes=1024: ninja exited with status 1
    stdout:
    [1/2] /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    FAILED: cuda_0.o
    /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    /bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found
    ninja: build stopped: subcommand failed.

    [2026-06-03 10:24:47 TP0] Prefill batch, #new-seq: 1, #new-token: 80, #cached-token: 0, full token usage: 0.00, mamba usage: 0.00, #running-req: 0, #queue-req: 0, input throughput (token/s): 0.00, cuda graph: False
    [2026-06-03 10:24:48] INFO: 127.0.0.1:58590 - "POST /v1/chat/completions HTTP/1.1" 200 OK
    [2026-06-03 10:24:48] The server is fired up and ready to roll!

  • Members 3 posts
    2026年6月3日 10:41

    1.服务器厂家:
    2.沐曦GPU型号:c500
    3.操作系统内核版本:沐曦官网提供的 docker 运行的sglang与用户本地os内核版本没啥关系
    4.是否开启CPU虚拟化: 否
    5.mx-smi回显:
    图片.png
    6.docker info回显:
    图片.png
    7.镜像版本:
    8.启动容器命令:
    docker run -itd \
    --privileged --network=host --ipc=host --ulimit memlock=-1 \
    --security-opt seccomp=unconfined \
    --security-opt apparmor=unconfined \
    --name Qwen3.5-9B \
    -v /data/models/:/models/ \
    cr.metax-tech.com/public-ai-release/maca/sglang:0.5.9-maca.ai3.5.3.208-torch2.8-py312-ubuntu22.04-amd64 bash

    9.容器内执行命令:
    python3 -m sglang.launch_server \
    --host 0.0.0.0 --port 9100 \
    --model-path /models/Qwen3.5-9B --served-model-name qwen3.5-9b \
    --tensor-parallel-size 2 --mem-fraction-static 0.85 --context-length 40960 \
    --trust-remote-code --enable-metrics --disable-cuda-graph

  • arrow_forward

    Thread has been moved from 公共.

  • Members 521 posts
    2026年6月3日 16:59

    尊敬的开发者您好,请在容器内执行

    ln -s /opt/maca/tools/cu-bridge/bin/cucc  /opt/maca/tools/cu-bridge/bin/nvcc