INFO 07-20 17:29:07 [__init__.py:44] Available plugins for group vllm.platform_plugins:
INFO 07-20 17:29:07 [__init__.py:46] - metax -> vllm_metax:register
INFO 07-20 17:29:07 [__init__.py:49] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 07-20 17:29:07 [__init__.py:238] Platform plugin metax is activated
INFO 07-20 17:29:07 [envs.py:123] Plugin sets VLLM_USE_FLASHINFER_SAMPLER to False. Reason: flashinfer sampler are not supported on maca
INFO 07-20 17:29:07 [envs.py:123] Plugin sets VLLM_ENGINE_READY_TIMEOUT_S to 3600. Reason: set timeout to 3600s for model loading
INFO 07-20 17:29:07 [envs.py:123] Plugin sets VLLM_FLOAT32_MATMUL_PRECISION to high. Reason: set float32 matmul precision to high for better performance on Maca platform
INFO 07-20 17:29:07 [envs.py:123] Plugin sets VLLM_USE_V2_MODEL_RUNNER to False. Reason: v2 model runner is still under development and not fully tested on Maca platform, disable it by default
WARNING 07-20 17:29:09 [config.py:70] Support for Transformers v4 is deprecated. The Transformers v4 codepath will become unmaintained in vLLM v0.22.0 and will be removed in vLLM v0.24.0. Please upgrade to Transformers v5: pip install --upgrade transformers
INFO Print the version information of mcoplib during compilation.
Version info:Mcoplib_Version = '0.4.7'
Build_Maca_Version = '3.8.0.23'
GIT_BRANCH = 'HEAD'
GIT_COMMIT = '3c71b39'
Vllm Op Version = 0.22.0
SGlang Op Version = 0.5.12
INFO Staring Check the current MACA version of the operating environment.
INFO: Release major.minor matching, successful:3.8.
WARNING 07-20 17:29:28 [__init__.py:87] The quantization method 'awq' already exists and will be overwritten by the quantization config <class 'vllm_metax.quant_config.awq.MacaAWQConfig'>.
WARNING 07-20 17:29:28 [__init__.py:87] The quantization method 'awq_marlin' already exists and will be overwritten by the quantization config <class 'vllm_metax.quant_config.awq_marlin.MacaAWQMarlinConfig'>.
WARNING 07-20 17:29:28 [__init__.py:87] The quantization method 'compressed-tensors' already exists and will be overwritten by the quantization config <class 'vllm_metax.quant_config.compressed_tensors.MacaCompressedTensorsConfig'>.
WARNING 07-20 17:29:28 [__init__.py:87] The quantization method 'gptq' already exists and will be overwritten by the quantization config <class 'vllm_metax.quant_config.auto_gptq.MacaAutoGPTQConfig'>.
WARNING 07-20 17:29:28 [__init__.py:87] The quantization method 'auto_gptq' already exists and will be overwritten by the quantization config <class 'vllm_metax.quant_config.auto_gptq.MacaAutoGPTQConfig'>.
WARNING 07-20 17:29:28 [__init__.py:87] The quantization method 'moe_wna16' already exists and will be overwritten by the quantization config <class 'vllm_metax.quant_config.moe_wna16.MacaMoeWNA16Config'>.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture DeepSeekMTPModel is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_mtp:DeepSeekMTP.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture DeepseekV2ForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_v2:DeepseekV2ForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture DeepseekV3ForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_v2:DeepseekV3ForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture DeepseekV32ForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_v2:DeepseekV3ForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture GlmMoeDsaForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_v2:GlmMoeDsaForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture DeepseekV4ForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_v4:DeepseekV4ForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture DeepSeekV4MTPModel is already registered, and will be overwritten by the new model class vllm_metax.models.deepseek_v4_mtp:DeepSeekV4MTP.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture Step3p5MTP is already registered, and will be overwritten by the new model class vllm_metax.models.step3p5_mtp:Step3p5MTP.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture Qwen3OmniMoeForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_metax.models.qwen3_omni_moe_thinker:Qwen3OmniMoeThinkerForConditionalGeneration.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture MiMoV2ForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.mimo_v2:MiMoV2ForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture MiMoV2FlashForCausalLM is already registered, and will be overwritten by the new model class vllm_metax.models.mimo_v2:MiMoV2FlashForCausalLM.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture MiMoV2MTPModel is already registered, and will be overwritten by the new model class vllm_metax.models.mimo_v2_mtp:MiMoV2MTP.
WARNING 07-20 17:29:28 [registry.py:983] Model architecture MiMoV2OmniMTPModel is already registered, and will be overwritten by the new model class vllm_metax.models.mimo_v2_mtp:MiMoV2OmniMTP.
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:344]
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:344] █ █ █▄ ▄█
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:344] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.22.0
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:344] █▄█▀ █ █ █ █ model /llm_models/DeepSeek-V4-Flash-FlexSMQ-AWQ-W8A8
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:344] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:344]
(APIServer pid=13) INFO 07-20 17:29:28 [utils.py:278] non-default args: {'model_tag': '/llm_models/DeepSeek-V4-Flash-FlexSMQ-AWQ-W8A8', 'default_chat_template_kwargs': {'enable_thinking': True}, 'enable_auto_tool_choice': True, 'tool_call_parser': 'deepseek_v4', 'port': 8001, 'model': '/llm_models/DeepSeek-V4-Flash-FlexSMQ-AWQ-W8A8', 'tokenizer_mode': 'deepseek_v4', 'trust_remote_code': True, 'max_model_len': 512000, 'served_model_name': ['deepseek-v4-flash'], 'reasoning_parser': 'deepseek_v4', 'tensor_parallel_size': 8, 'block_size': 256, 'gpu_memory_utilization': 0.8, 'kv_cache_dtype': 'bfloat16', 'max_num_seqs': 2, 'enable_chunked_prefill': True, 'async_scheduling': True, 'compilation_config': {'mode': None, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'ir_enable_torch_wrap': None, 'splitting_ops': None, 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': None, 'compile_ranges_endpoints': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': None, 'static_all_moe_layers': []}}
(APIServer pid=13) INFO 07-20 17:29:28 [config.py:431] Replacing legacy 'type' key with 'rope_type'
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] Error in inspecting model architecture 'DeepseekV4ForCausalLM'
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] Traceback (most recent call last):
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 1385, in _run_in_subprocess
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] returned.check_returncode()
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/subprocess.py", line 502, in check_returncode
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] raise CalledProcessError(self.returncode, self.args, self.stdout,
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] subprocess.CalledProcessError: Command '['/opt/conda/bin/python3.12', '-m', 'vllm.model_executor.models.registry']' returned non-zero exit status 1.
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950]
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] The above exception was the direct cause of the following exception:
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950]
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] Traceback (most recent call last):
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 948, in _try_inspect_model_cls
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] return model.inspect_model_cls()
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/logging_utils/log_time.py", line 21, in _wrapper
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] result = func(*args, **kwargs)
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 909, in inspect_model_cls
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] mi = _run_in_subprocess(
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 1388, in _run_in_subprocess
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] raise RuntimeError(
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] RuntimeError: Error raised in subprocess:
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] <frozen runpy>:128: RuntimeWarning: 'vllm.model_executor.models.registry' found in sys.modules after import of package 'vllm.model_executor.models', but prior to execution of 'vllm.model_executor.models.registry'; this may result in unpredictable behaviour
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] Traceback (most recent call last):
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen runpy>", line 198, in _run_module_as_main
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen runpy>", line 88, in _run_code
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 1411, in <module>
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] _run()
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 1404, in _run
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] result = fn()
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 910, in <lambda>
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] lambda: _ModelInfo.from_model_cls(self.load_model_cls())
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 923, in load_model_cls
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] mod = importlib.import_module(self.module_name)
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/importlib/__init__.py", line 90, in import_module
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] return _bootstrap._gcd_import(name[level:], package, level)
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm_metax/models/deepseek_v4.py", line 23, in <module>
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] from vllm_metax.customized.pluggable_layer.deepseek_v4_attention import (
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] File "/opt/conda/lib/python3.12/site-packages/vllm_metax/customized/pluggable_layer/deepseek_v4_attention.py", line 9, in <module>
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] from vllm.model_executor.layers.deepseek_v4_attention import (
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950] ModuleNotFoundError: No module named 'vllm.model_executor.layers.deepseek_v4_attention'
(APIServer pid=13) ERROR 07-20 17:30:00 [registry.py:950]
(APIServer pid=13) Traceback (most recent call last):
(APIServer pid=13) File "/opt/conda/bin/vllm", line 8, in <module>
(APIServer pid=13) sys.exit(main())
(APIServer pid=13) ^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 92, in main
(APIServer pid=13) args.dispatch_function(args)
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 148, in cmd
(APIServer pid=13) uvloop.run(run_server(args))
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=13) return __asyncio.run(
(APIServer pid=13) ^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=13) return runner.run(main)
(APIServer pid=13) ^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=13) return self._loop.run_until_complete(task)
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=13) return await main
(APIServer pid=13) ^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 678, in run_server
(APIServer pid=13) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 692, in run_server_worker
(APIServer pid=13) async with build_async_engine_client(
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=13) return await anext(self.gen)
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=13) async with build_async_engine_client_from_engine_args(
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=13) return await anext(self.gen)
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 124, in build_async_engine_client_from_engine_args
(APIServer pid=13) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1730, in create_engine_config
(APIServer pid=13) model_config = self.create_model_config()
(APIServer pid=13) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1560, in create_model_config
(APIServer pid=13) return ModelConfig(
(APIServer pid=13) ^^^^^^^^^^^^
(APIServer pid=13) File "/opt/conda/lib/python3.12/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=13) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=13) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=13) Value error, Model architectures ['DeepseekV4ForCausalLM'] failed to be inspected. Please check the logs for more details. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]
(APIServer pid=13) For further information visit https://errors.pydantic.dev/2.13/v/value_error
您好，部署按照之前命令无法成功，请你再确认一下
