huanglu2518

您好，使用vllm-metax:0.26.0镜像部署Deepseek-V4-Flash-W8A8，任务未完成时，会偶发性输出终止，出现终止end符，这个怎么解决？

新发布的vllm-metax:0.26.0镜像是否支持Deepseek-V4-Flash-W8A8的工具调用？之前的0.25.0版本不支持。

按照您给的命令升级完了，启动报错了信息类似下面的：由于内网不能拍照和下载下来，因此找了一个报错信息基本一样的
ERROR 02-25 14:51:52 [multiproc_executor.py:772] WorkerProc failed to start.
ERROR 02-25 14:51:52 [multiproc_executor.py:772] Traceback (most recent call last):
ERROR 02-25 14:51:52 [multiproc_executor.py:772] File "/home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 743, in worker_main
ERROR 02-25 14:51:52 [multiproc_executor.py:772] worker = WorkerProc(args, *kwargs)
ERROR 02-25 14:51:52 [multiproc_executor.py:772] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-25 14:51:52 [multiproc_executor.py:772] File "/home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/vllm/v1/executor/multiproc_executor.py", line 569, in init
ERROR 02-25 14:51:52 [multiproc_executor.py:772] self.worker.init_device()
ERROR 02-25 14:51:52 [multiproc_executor.py:772] File "/home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/vllm/v1/worker/worker_base.py", line 326, in init_device
ERROR 02-25 14:51:52 [multiproc_executor.py:772] self.worker.init_device() # type: ignore
ERROR 02-25 14:51:52 [multiproc_executor.py:772] ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-25 14:51:52 [multiproc_executor.py:772] File "/home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 210, in init_device
ERROR 02-25 14:51:52 [multiproc_executor.py:772] current_platform.set_device(self.device)
ERROR 02-25 14:51:52 [multiproc_executor.py:772] File "/home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/vllm/platforms/cuda.py", line 123, in set_device
ERROR 02-25 14:51:52 [multiproc_executor.py:772] torch.cuda.set_device(device)
ERROR 02-25 14:51:52 [multiproc_executor.py:772] File "/home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/cuda/init.py", line 567, in set_device
ERROR 02-25 14:51:52 [multiproc_executor.py:772] torch._C._cuda_setDevice(device)
ERROR 02-25 14:51:52 [multiproc_executor.py:772] torch.AcceleratorError: CUDA error: CUDA-capable device(s) is/are busy or unavailable
ERROR 02-25 14:51:52 [multiproc_executor.py:772] Search for cudaErrorDevicesUnavailable' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information. ERROR 02-25 14:51:52 [multiproc_executor.py:772] CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. ERROR 02-25 14:51:52 [multiproc_executor.py:772] For debugging consider passing CUDA_LAUNCH_BLOCKING=1 ERROR 02-25 14:51:52 [multiproc_executor.py:772] Compile withTORCH_USE_CUDA_DSA` to enable device-side assertions.
ERROR 02-25 14:51:52 [multiproc_executor.py:772]
INFO 02-25 14:51:52 [multiproc_executor.py:730] Parent process exited, terminating worker
INFO 02-25 14:51:52 [multiproc_executor.py:730] Parent process exited, terminating worker
INFO 02-25 14:51:53 [parallel_state.py:1212] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:38723 backend=nccl
[W225 14:51:56.592294033 TCPStore.cpp:340] [c10d] TCP client failed to connect/validate to host 127.0.0.1:38723 - retrying (try=0, timeout=600000ms, delay=2716ms): Interrupted system call
Exception raised from delay at /pytorch/torch/csrc/distributed/c10d/socket.cpp:115 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x80 (0x7f401375fb80 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x5ffc5d1 (0x7f4070afd5d1 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #2: <unknown function> + 0x14a15e8 (0x7f406bfa25e8 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x607828b (0x7f4070b7928b in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0x6078624 (0x7f4070b79624 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #5: <unknown function> + 0x5ff4ea3 (0x7f4070af5ea3 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #6: c10d::TCPStore::TCPStore(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, c10d::TCPStoreOptions const&) + 0x41d (0x7f4070afc8cd in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0xd71465 (0x7f40801c8465 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
frame #8: <unknown function> + 0xdadda6 (0x7f4080204da6 in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
frame #9: <unknown function> + 0x3cc7ad (0x7f407f8237ad in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
frame #10: <unknown function> + 0x1fd496 (0x55ba06b99496 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #11: _PyObject_MakeTpCall + 0x24b (0x55ba06b7584b in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #12: <unknown function> + 0x22d1de (0x55ba06bc91de in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #13: _PyObject_Call + 0x12b (0x55ba06bb3dab in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #14: <unknown function> + 0x2155df (0x55ba06bb15df in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #15: <unknown function> + 0x1d9b63 (0x55ba06b75b63 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #16: <unknown function> + 0x4c9bb (0x7f40969ff9bb in /home/amax/miniconda/envs/llama_factory/lib/python3.11/site-packages/scipy/spatial/_distance_pybind.cpython-311-x86_64-linux-gnu.so)
frame #17: _PyObject_MakeTpCall + 0x24b (0x55ba06b7584b in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #18: _PyEval_EvalFrameDefault + 0x665 (0x55ba06b83b75 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #19: <unknown function> + 0x27b5c0 (0x55ba06c175c0 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #20: <unknown function> + 0x204083 (0x55ba06ba0083 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #21: PyObject_Vectorcall + 0x2c (0x55ba06b8facc in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #22: _PyEval_EvalFrameDefault + 0x665 (0x55ba06b83b75 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #23: _PyFunction_Vectorcall + 0x165 (0x55ba06ba97e5 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #24: PyObject_Call + 0x136 (0x55ba06bb3b46 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #25: _PyEval_EvalFrameDefault + 0x43bb (0x55ba06b878cb in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #26: _PyFunction_Vectorcall + 0x165 (0x55ba06ba97e5 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #27: PyObject_Call + 0x136 (0x55ba06bb3b46 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #28: _PyEval_EvalFrameDefault + 0x43bb (0x55ba06b878cb in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #29: _PyFunction_Vectorcall + 0x165 (0x55ba06ba97e5 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #30: <unknown function> + 0x2152b3 (0x55ba06bb12b3 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #31: <unknown function> + 0x1d9b63 (0x55ba06b75b63 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #32: PyObject_Call + 0xbe (0x55ba06bb3ace in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #33: _PyEval_EvalFrameDefault + 0x43bb (0x55ba06b878cb in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #34: _PyFunction_Vectorcall + 0x165 (0x55ba06ba97e5 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #35: PyObject_Call + 0x136 (0x55ba06bb3b46 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #36: _PyEval_EvalFrameDefault + 0x43bb (0x55ba06b878cb in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #37: <unknown function> + 0x2a4435 (0x55ba06c40435 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #38: PyEval_EvalCode + 0x9d (0x55ba06c3fb7d in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #39: <unknown function> + 0x2c164a (0x55ba06c5d64a in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #40: <unknown function> + 0x2bd343 (0x55ba06c59343 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #41: PyRun_StringFlags + 0x62 (0x55ba06c4ebb2 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #42: PyRun_SimpleStringFlags + 0x3c (0x55ba06c4e96c in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #43: Py_RunMain + 0x30f (0x55ba06c6873f in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #44: Py_BytesMain + 0x37 (0x55ba06c2fa67 in /home/amax/miniconda/envs/llama_factory/bin/python)
frame #45: __libc_start_main + 0xf3 (0x7f40f929d083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #46: <unknown function> + 0x2938d9 (0x55ba06c2f8d9 in /home/amax/miniconda/envs/llama_factory/bin/python)

(EngineCore_DP0 pid=4097130) ERROR 02-25 14:52:00 [core.py:946] EngineCore failed to start.

在沐曦C500单卡使用vllm-metax:0.19.0-maca.ai3.5.3.502-torch2.8-py310-ubuntu22.04-amd64镜像部署qwen3.5-35b-a3b-w8a8报错，显示错误信息为ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.在modelscope上查看到阿里发布的qwen3.5-35b-a3b的"tokenizer_class": "Qwen2Tokenizer",沐曦量化过后的显示为 "tokenizer_class": "TokenizersBackend",应该选择哪个版本的镜像进行部署？

应该下载哪个版本的驱动？KylinV10SP2还是KylinV11?

应该下载哪个版本的驱动？KylinV10SP2还是KylinV11?

目前操作系统使用的是麒麟v10国防版，CPU使用的是海光7490，驱动版本为3.5.x.x，目前为了适配最新的DSV4模型，怎么升级驱动来适配最新的VLLM以及SGLang推理框架？

这个个人主题在哪里开通，能否私信您。

这个跟咱们的技术支持沟通后，让发在这里找下问题解决办法。这个是属于C500和海光7490的适配问题，不属于GPUStack的问题。

部署环境为：沐曦C500 8卡服务器*2、CPU-海光7490
运行环境：GPUStack V2.1.2 docker
模型为：Qwen3.5-235B-a8w8
模型运行镜像为：gpustack/runner:maca3.5-vllm0.14.0
在使用gpustack控制台手动设置运行节点的显卡为第2张卡时，报错了。
信息为如图所示，怀疑是在获取GPU Index时报错了。