MetaX-Tech Developer Forum 论坛首页
  • 沐曦开发者
search
Sign in

lionel

  • Members
  • Joined 2026年6月2日
  • message 帖子
  • forum 主题
  • favorite 关注者
  • favorite_border Follows
  • person_outline 详细信息

lionel has posted 3 messages.

  • See post chevron_right
    lionel
    Members
    /bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found 解决中 2026年6月3日 10:41

    1.服务器厂家:
    2.沐曦GPU型号:c500
    3.操作系统内核版本:沐曦官网提供的 docker 运行的sglang与用户本地os内核版本没啥关系
    4.是否开启CPU虚拟化: 否
    5.mx-smi回显:
    图片.png
    6.docker info回显:
    图片.png
    7.镜像版本:
    8.启动容器命令:
    docker run -itd \
    --privileged --network=host --ipc=host --ulimit memlock=-1 \
    --security-opt seccomp=unconfined \
    --security-opt apparmor=unconfined \
    --name Qwen3.5-9B \
    -v /data/models/:/models/ \
    cr.metax-tech.com/public-ai-release/maca/sglang:0.5.9-maca.ai3.5.3.208-torch2.8-py312-ubuntu22.04-amd64 bash

    9.容器内执行命令:
    python3 -m sglang.launch_server \
    --host 0.0.0.0 --port 9100 \
    --model-path /models/Qwen3.5-9B --served-model-name qwen3.5-9b \
    --tensor-parallel-size 2 --mem-fraction-static 0.85 --context-length 40960 \
    --trust-remote-code --enable-metrics --disable-cuda-graph

  • See post chevron_right
    lionel
    Members
    /bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found 解决中 2026年6月3日 10:28

    使用沐曦官方镜像:sglang:0.5.9-maca.ai3.5.3.208-torch2.8-py312-ubuntu22.04-amd64 部署模型 Qwen3.5-9B 时,能够成功部署,但是报了下面的错误,/bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found, 为什么沐曦的sglang会找nvidia的nvcc

    [2026-06-03 10:24:38] INFO: Started server process [12085]
    [2026-06-03 10:24:38] INFO: Waiting for application startup.
    [2026-06-03 10:24:38] INFO: Application startup complete.
    [2026-06-03 10:24:38] INFO: Uvicorn running on http://0.0.0.0:9100 (Press CTRL+C to quit)
    [2026-06-03 10:24:39] INFO: 127.0.0.1:58588 - "GET /model_info HTTP/1.1" 200 OK
    [2026-06-03 10:24:47 TP1] Failed to load JIT KV-Cache kernel with row_bytes=1024: ninja exited with status 1
    stdout:
    [1/2] /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    FAILED: cuda_0.o
    /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    /bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found
    ninja: build stopped: subcommand failed.

    [2026-06-03 10:24:47 TP0] Failed to load JIT KV-Cache kernel with row_bytes=1024: ninja exited with status 1
    stdout:
    [1/2] /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    FAILED: cuda_0.o
    /opt/maca/tools/cu-bridge/bin/nvcc --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_80,code=sm_80 -std=c++20 -O3 --expt-relaxed-constexpr -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/tvm_ffi/include -I/opt/conda/lib/python3.12/site-packages/sglang/jit_kernel/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_kvcache_1024_false_cb53493e06f7ab59/cuda.cu -o cuda_0.o
    /bin/sh: 1: /opt/maca/tools/cu-bridge/bin/nvcc: not found
    ninja: build stopped: subcommand failed.

    [2026-06-03 10:24:47 TP0] Prefill batch, #new-seq: 1, #new-token: 80, #cached-token: 0, full token usage: 0.00, mamba usage: 0.00, #running-req: 0, #queue-req: 0, input throughput (token/s): 0.00, cuda graph: False
    [2026-06-03 10:24:48] INFO: 127.0.0.1:58590 - "POST /v1/chat/completions HTTP/1.1" 200 OK
    [2026-06-03 10:24:48] The server is fired up and ready to roll!

  • See post chevron_right
    lionel
    Members
    MetaX C500 使用 vllm-metax:0.19.0-maca.ai3.5.3.502-torch2.8-py312镜像部署 Qwen3.6-35B-A3B-FP8 失败 解决中 2026年6月2日 15:17

    图片.png
    使用 vllm-metax:0.19.0-maca.ai3.5.3.502-torch2.8-py312-ubuntu22.04-amd64 镜像部署 Qwen3.6-35B-A3B-FP8
    报错:Value error, fp8 quantization is currently not supported in maca.
    请问:原因是c500 还不支持fp8模型吗

  • 沐曦开发者论坛
powered by misago