deepseek v4 flash sglang启动参数？
e411
- Members
- Joined 2026年5月8日
-
我看到2026-08-21重新发布了一个使用新版本的maca的sglang 0.5.12镜像，这个镜像和之前07-22的sglang 0.5.12有何区别？和08-07发布的0.5.13版本，支持的模型有何区别？
名称版本
sglang:0.5.12-maca.ai3.8.2.7-torch2.10-py310-kylinv11-amd64
推荐安装MACA Driver 3.8.2.xsglang 0.5.12Pytorch 2.10Python 3.10
更新时间
2026-08-21 14:31:17
基本描述
提供了面向曦云C500系列的 sglang 推理运行环境，适配多个 sglang 版本，支持 DeepSeek、Kimi-K2、Qwen 等多个系列模型，可用于提供 LLM 推理和性能验证服务。名称版本
sglang:0.5.13-maca.ai3.8.1.3-torch2.10-py312-kylinv11-amd64
推荐安装MACA Driver 3.8.1.xsglang 0.5.13Pytorch 2.10Python 3.12
更新时间
2026-08-07 19:05:52
基本描述
提供了面向曦云C500系列的 sglang 推理运行环境，适配多个 sglang 版本，支持 DeepSeek、Kimi-K2、Qwen 等多个系列模型，可用于提供 LLM 推理和性能验证服务。名称版本
sglang:0.5.12-maca.ai3.8.0.3-torch2.8-py310-kylinv11-amd64
推荐安装MACA Driver 3.8.0.xsglang 0.5.12Pytorch 2.8Python 3.10
更新时间
2026-07-22 17:51:33
基本描述
提供了面向曦云C500系列的 sglang 推理运行环境，适配多个 sglang 版本，支持 DeepSeek、Kimi-K2、Qwen 等多个系列模型，可用于提供 LLM 推理和性能验证服务。 -
硬件 C500 * 8
我使用的命令是：
#!/bin/bash export MACA_SMALL_PAGESIZE_ENABLE=1 export MACA_VLLM_ENABLE_MCTLASS_FUSED_MOE=0 export MACA_VLLM_ENABLE_MCTLASS_PYTHON_API=1 vllm serve /llm_models/DeepSeek-V4-Flash-FlexSMQ-AWQ-W8A8 --served-model-name DeepSeek-V4-Flash --trust-remote-code \ --kv-cache-dtype bfloat16 --block-size 256 --port 8001 --max-model-len 1048576 --gpu-memory-utilization 0.9 \ --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 \ --enable-auto-tool-choice --reasoning-parser deepseek_v4 --default-chat-template-kwargs '{"enable_thinking": true}' \ --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE", "custom_ops":["all"]}' \ --max-num-seq 32 -tp 8 --speculative_config '{"method": "mtp", "num_speculative_tokens": 1}'然后在加载完模型之后产生巨量的tilelang的编译报错，然后失败退出
-
sudo sh metax-driver-3.8.0.10-deb-x86_64.run [sudo] password for cast503: Verifying archive integrity... 100% MD5 checksums are OK. All good. Uncompressing MetaX driver installer 100% uninstalling metax-linux-dkms (Reading database ... 142019 files and directories currently installed.) Removing metax-linux-dkms (3.8.30) ... Module metax-linux-3.8.30 for kernel 6.8.0-110-generic (x86_64). Before uninstall, this module version was ACTIVE on this kernel. metax.ko.zst: - Uninstallation - Deleting from: /lib/modules/6.8.0-110-generic/updates/dkms/ - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. Running the post_remove script: depmod.... Module metax-linux-3.8.30 for kernel 6.8.0-124-generic (x86_64). Before uninstall, this module version was ACTIVE on this kernel. metax.ko.zst: - Uninstallation - Deleting from: /lib/modules/6.8.0-124-generic/updates/dkms/ - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. Running the post_remove script: depmod.... Deleting module metax-linux-3.8.30 completely from the DKMS tree. uninstalling mxsmt (Reading database ... 141807 files and directories currently installed.) Removing mxsmt (3.7.2.30) ... mxsmt package uninstalled successfully Purging configuration files for mxsmt (3.7.2.30) ... mxsmt package uninstalled successfully dpkg: warning: while removing mxsmt, directory '/opt/mxdriver' not empty so not removed uninstalling mxfw (Reading database ... 141795 files and directories currently installed.) Removing mxfw (3.7.2) ... mxfw package uninstalled successfully Purging configuration files for mxfw (3.7.2) ... mxfw package uninstalled successfully deb/mxfw_*.deb deb/mxsmt_*.deb deb/metax-linux[-_]*.deb installing packages under /opt/mxdriver installing deb/mxfw_3.8.0-1.all.deb Selecting previously unselected package mxfw. (Reading database ... 141709 files and directories currently installed.) Preparing to unpack deb/mxfw_3.8.0-1.all.deb ... Unpacking mxfw (3.8.0) ... Setting up mxfw (3.8.0) ... mxfw package installed successfully installing deb/mxsmt_3.8.0.10.amd64.deb Selecting previously unselected package mxsmt. (Reading database ... 141795 files and directories currently installed.) Preparing to unpack deb/mxsmt_3.8.0.10.amd64.deb ... Unpacking mxsmt (3.8.0.10) ... Setting up mxsmt (3.8.0.10) ... mxsmt package installed successfully Processing triggers for libc-bin (2.39-0ubuntu8.7) ... installing deb/metax-linux_3.9.10-1_amd64.deb Selecting previously unselected package metax-linux. (Reading database ... 141807 files and directories currently installed.) Preparing to unpack .../metax-linux_3.9.10-1_amd64.deb ... Unpacking metax-linux (3.9.10) ... Setting up metax-linux (3.9.10) ... Notice: The current kernel version (6.8.0-124-generic) is not supported. Please upgrade your kernel to one of the following versions: 5.10.0-0.deb10.28-amd64 5.10.0-29-amd64 5.10.0-30-amd64 5.10.200-4-velinux1u1-amd64 5.15.0-100-generic 5.15.0-105-generic 5.15.0-112-generic 5.15.0-119-generic 5.15.0-124-generic 5.15.0-130-generic 5.15.0-138-generic 5.15.0-25-generic 5.15.0-46-generic 5.15.0-58-generic 5.15.0-60-generic 5.15.0-72-generic 5.15.0-78-generic 5.15.0-88-generic 5.15.119-rt65-home-ear1.1 5.19.0-46-generic 5.19.17 5.4.0-131-generic 5.4.0-144-generic 5.4.0-148-generic 5.4.0-176-generic 5.4.0-26-generic 5.4.0-42-generic 5.4.18-142-generic 6.1.141-1-generic 6.1.152-1-generic 6.12.34-1-generic 6.12.47-1-generic 6.14.0-1003-oem 6.14.0-27-generic 6.8.0-31-generic 6.8.0-49-generic 6.8.0-53-generic And then reinstall the metax-linux package. dpkg: error processing package metax-linux (--install): installed metax-linux package post-installation script subprocess returned error exit status 1 Errors were encountered while processing: metax-linux uninstalling metax-linux (Reading database ... 141924 files and directories currently installed.) Removing metax-linux (3.9.10) ... metax-linux package uninstalled successfully Purging configuration files for metax-linux (3.9.10) ... metax-linux package uninstalled successfully Reading package lists... Done Building dependency tree... Done Reading state information... Done dkms is already the newest version (3.0.11-1ubuntu13). autoconf is already the newest version (2.71-3). automake is already the newest version (1:1.16.5-1.3ubuntu1). The following packages were automatically installed and are no longer required: libdebuginfod-common libdebuginfod1t64 libfwupd2 libgusb2 libmspack0t64 libxmlsec1t64 libxmlsec1t64-openssl libxslt1.1 zerofree Use 'sudo apt autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Selecting previously unselected package metax-linux-dkms. (Reading database ... 141807 files and directories currently installed.) Preparing to unpack .../metax-linux-dkms_3.9.10-1_amd64.deb ... Unpacking metax-linux-dkms (3.9.10) ... Setting up metax-linux-dkms (3.9.10) ... Loading new metax-linux-3.9.10 DKMS files... Building for 6.8.0-124-generic Building for architecture x86_64 Building initial module for 6.8.0-124-generic autoreconf: export WARNINGS= autoreconf: Entering directory '.' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal --force autoreconf: configure.ac: tracing autoreconf: configure.ac: not using Libtool autoreconf: configure.ac: not using Intltool autoreconf: configure.ac: not using Gtkdoc autoreconf: running: /usr/bin/autoconf --force configure.ac:23: warning: The macro `AC_HELP_STRING' is obsolete. configure.ac:23: You should run autoupdate. ./lib/autoconf/general.m4:204: AC_HELP_STRING is expanded from... ./lib/autoconf/general.m4:1534: AC_ARG_ENABLE is expanded from... m4/config.m4:14: AC_METAX_CONFIG is expanded from... configure.ac:23: the top level autoreconf: running: /usr/bin/autoheader --force autoreconf: configure.ac: not using Automake autoreconf: 'config/install-sh' is updated autoreconf: Leaving directory '.' ERROR (dkms apport): unable to determine source package for metax-linux-dkms Error! Bad return status for module build on kernel: 6.8.0-124-generic (x86_64) Consult /var/lib/dkms/metax-linux/3.9.10/build/make.log for more information. dpkg: error processing package metax-linux-dkms (--install): installed metax-linux-dkms package post-installation script subprocess returned error exit status 10 Errors were encountered while processing: metax-linux-dkms ERROR: failed to install deb/metax-linux_3.9.10-1_amd64.deb错误原因：metax-linux-3.9.10/common/mx_devinfo_c500.h 中代码有问题：
#define MX_DEV_PCI_C500_LIST \ MX_DEV_PCI_C500(0x9999, 0x4001, 0x4019, c500, MXC500, mx_devinfo_c500) \ MX_DEV_PCI_C500(0x9999, 0x4000, 0x4018, c550, MXC550, mx_devinfo_c550) \ MX_DEV_PCI_C500(0x9999, 0x4083, 0x409b, n260, MXN260, mx_devinfo_n260) \ MX_DEV_PCI_C500(0x9999, 0x4040, 0x4058, c500x, MXC500X, mx_devinfo_c500x) \ MX_DEV_PCI_C500(0x9999, 0x4020, 0x4038, c588, MXC588, mx_devinfo_c588)正确的代码应该是：
#define MX_DEV_PCI_C500_LIST \ MX_DEV_PCI_C500(0x9999, 0x4001, 0x4019, c500, "MXC500", mx_devinfo_c500) \ MX_DEV_PCI_C500(0x9999, 0x4000, 0x4018, c550, "MXC550", mx_devinfo_c550) \ MX_DEV_PCI_C500(0x9999, 0x4083, 0x409b, n260, "MXN260", mx_devinfo_n260) \ MX_DEV_PCI_C500(0x9999, 0x4040, 0x4058, c500x, "MXC500X", mx_devinfo_c500x) \ MX_DEV_PCI_C500(0x9999, 0x4020, 0x4038, c588, "MXC588", mx_devinfo_c588)也就是说倒数第二个参数本应是字符串，双引号都丢失了
-
请问下 modelscope.cn/models/metax-tech/DeepSeek-V4-Flash-FlexSMQ-AWQ-W8A8 这个量化模型在8卡沐曦C500上是否能够部署，如果要部署应使用哪个镜像，使用什么参数启动？