沐曦官方什么时候出qwen3.8 27B W8A8 ?
沐曦官方什么时候出qwen3.8 27B W8A8 ?
尊敬的开发者您好,等待近期更新
同问,qwen3.8-27b-FB8 部署提示不支持fp8
(APIServer pid=9768) Traceback (most recent call last):
(APIServer pid=9768) File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
(APIServer pid=9768) return _run_code(code, main_globals, None,
(APIServer pid=9768) File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
(APIServer pid=9768) exec(code, run_globals)
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 710, in <module>
(APIServer pid=9768) uvloop.run(run_server(args))
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/uvloop/__init__.py", line 69, in run
(APIServer pid=9768) return loop.run_until_complete(wrapper())
(APIServer pid=9768) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=9768) return await main
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 670, in run_server
(APIServer pid=9768) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 684, in run_server_worker
(APIServer pid=9768) async with build_async_engine_client(
(APIServer pid=9768) File "/opt/conda/lib/python3.10/contextlib.py", line 199, in __aenter__
(APIServer pid=9768) return await anext(self.gen)
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=9768) async with build_async_engine_client_from_engine_args(
(APIServer pid=9768) File "/opt/conda/lib/python3.10/contextlib.py", line 199, in __aenter__
(APIServer pid=9768) return await anext(self.gen)
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 124, in build_async_engine_client_from_engine_args
(APIServer pid=9768) vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/engine/arg_utils.py", line 1549, in create_engine_config
(APIServer pid=9768) model_config = self.create_model_config()
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/vllm/engine/arg_utils.py", line 1398, in create_model_config
(APIServer pid=9768) return ModelConfig(
(APIServer pid=9768) File "/opt/conda/lib/python3.10/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=9768) s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=9768) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=9768) Value error, fp8 quantization is currently not supported in maca. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]
(APIServer pid=9768) For further information visit https://errors.pydantic.dev/2.13/v/value_error
尊敬的开发者您好,C500不支持FP8