N260部署Qwen3.8-27B-W8A8-INT8模型，提示cannot import name 'normalize_tool_choice' from 'xgrammar'

15287246311
2026年8月26日
chat_bubble_outline 1
  • link
    Members 11 posts
    2026年8月26日 23:49
    link

    服务器镜像版本：cr.metax-tech.com/public-ai-release/maca/vllm-metax:0.24.0-maca.ai3.8.2.5-torch2.10-py312-ubuntu22.04-amd64

    (APIServer pid=35) INFO: 127.0.0.1:54738 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
    (APIServer pid=35) ERROR: Exception in ASGI application
    (APIServer pid=35) Traceback (most recent call last):
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 422, in run_asgi
    (APIServer pid=35) result = await app( # type: ignore[func-returns-value]
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 63, in call
    (APIServer pid=35) return await self.app(scope, receive, send)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/applications.py", line 1159, in call
    (APIServer pid=35) await super().call(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/applications.py", line 96, in call
    (APIServer pid=35) await self.middleware_stack(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in call
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in call
    (APIServer pid=35) await self.app(scope, receive, _send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/cors.py", line 88, in call
    (APIServer pid=35) await self.app(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 180, in call
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 178, in call
    (APIServer pid=35) await self.app(scope, receive, send_wrapper)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in call
    (APIServer pid=35) await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    (APIServer pid=35) await app(scope, receive, sender)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
    (APIServer pid=35) await self.app(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 670, in call
    (APIServer pid=35) await self.middleware_stack(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 690, in app
    (APIServer pid=35) await route.handle(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 280, in handle
    (APIServer pid=35) await self.app(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 134, in app
    (APIServer pid=35) await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    (APIServer pid=35) await app(scope, receive, sender)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
    (APIServer pid=35) response = await f(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 674, in app
    (APIServer pid=35) raw_response = await run_endpoint_function(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 328, in run_endpoint_function
    (APIServer pid=35) return await dependant.call(values)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/utils/api_utils.py", line 91, in wrapper
    (APIServer pid=35) return handler_task.result()
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/utils/api_utils.py", line 112, in wrapper
    (APIServer pid=35) return await func(*args,     kwargs)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/api_router.py", line 61, in create_chat_completion
    (APIServer pid=35) generator = await handler.create_chat_completion(request, raw_request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 247, in create_chat_completion
    (APIServer pid=35) return await self._with_kv_transfer_rejection_cleanup(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 176, in _with_kv_transfer_rejection_cleanup
    (APIServer pid=35) return await awaitable
    (APIServer pid=35) ^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 267, in _create_chat_completion
    (APIServer pid=35) result = await self.render_chat_request(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 233, in render_chat_request
    (APIServer pid=35) return await self.openai_serving_render.render_chat(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/render/serving.py", line 387, in render_chat
    (APIServer pid=35) conversation, engine_inputs = await self.preprocess_chat(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/render/serving.py", line 964, in preprocess_chat
    (APIServer pid=35) ).adjust_request(
    (APIServer pid=35) ^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/parser/abstract_parser.py", line 461, in adjust_request
    (APIServer pid=35) request = self._apply_structural_tag(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/parser/abstract_parser.py", line 487, in _apply_structural_tag
    (APIServer pid=35) structure_tag = self._tool_parser.get_structural_tag(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/tool_parsers/abstract_tool_parser.py", line 178, in get_structural_tag
    (APIServer pid=35) from vllm.tool_parsers.structural_tag_registry import get_model_structural_tag
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/tool_parsers/structural_tag_registry.py", line 12, in <module>
    (APIServer pid=35) from xgrammar import StructuralTag, normalize_tool_choice
    (APIServer pid=35) ImportError: cannot import name 'normalize_tool_choice' from 'xgrammar' (/opt/conda/lib/python3.12/site-packages/xgrammar/init.py)
    (APIServer pid=35) INFO: 127.0.0.1:34610 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
    (APIServer pid=35) ERROR: Exception in ASGI application
    (APIServer pid=35) Traceback (most recent call last):
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 422, in run_asgi
    (APIServer pid=35) result = await app( # type: ignore[func-returns-value]
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 63, in call
    (APIServer pid=35) return await self.app(scope, receive, send)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/applications.py", line 1159, in call
    (APIServer pid=35) await super().call(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/applications.py", line 96, in call
    (APIServer pid=35) await self.middleware_stack(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in call
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in call
    (APIServer pid=35) await self.app(scope, receive, _send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/cors.py", line 88, in call
    (APIServer pid=35) await self.app(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 180, in call
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 178, in call
    (APIServer pid=35) await self.app(scope, receive, send_wrapper)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in call
    (APIServer pid=35) await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    (APIServer pid=35) await app(scope, receive, sender)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
    (APIServer pid=35) await self.app(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 670, in call
    (APIServer pid=35) await self.middleware_stack(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 690, in app
    (APIServer pid=35) await route.handle(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/routing.py", line 280, in handle
    (APIServer pid=35) await self.app(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 134, in app
    (APIServer pid=35) await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    (APIServer pid=35) raise exc
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    (APIServer pid=35) await app(scope, receive, sender)
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
    (APIServer pid=35) response = await f(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 674, in app
    (APIServer pid=35) raw_response = await run_endpoint_function(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/fastapi/routing.py", line 328, in run_endpoint_function
    (APIServer pid=35) return await dependant.call(values)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/utils/api_utils.py", line 91, in wrapper
    (APIServer pid=35) return handler_task.result()
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/utils/api_utils.py", line 112, in wrapper
    (APIServer pid=35) return await func(*args,     kwargs)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/api_router.py", line 61, in create_chat_completion
    (APIServer pid=35) generator = await handler.create_chat_completion(request, raw_request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 247, in create_chat_completion
    (APIServer pid=35) return await self._with_kv_transfer_rejection_cleanup(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 176, in _with_kv_transfer_rejection_cleanup
    (APIServer pid=35) return await awaitable
    (APIServer pid=35) ^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 267, in _create_chat_completion
    (APIServer pid=35) result = await self.render_chat_request(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 233, in render_chat_request
    (APIServer pid=35) return await self.openai_serving_render.render_chat(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/render/serving.py", line 387, in render_chat
    (APIServer pid=35) conversation, engine_inputs = await self.preprocess_chat(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/entrypoints/serve/render/serving.py", line 964, in preprocess_chat
    (APIServer pid=35) ).adjust_request(
    (APIServer pid=35) ^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/parser/abstract_parser.py", line 461, in adjust_request
    (APIServer pid=35) request = self._apply_structural_tag(request)
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/parser/abstract_parser.py", line 487, in _apply_structural_tag
    (APIServer pid=35) structure_tag = self._tool_parser.get_structural_tag(
    (APIServer pid=35) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/tool_parsers/abstract_tool_parser.py", line 178, in get_structural_tag
    (APIServer pid=35) from vllm.tool_parsers.structural_tag_registry import get_model_structural_tag
    (APIServer pid=35) File "/opt/conda/lib/python3.12/site-packages/vllm/tool_parsers/structural_tag_registry.py", line 12, in <module>
    (APIServer pid=35) from xgrammar import StructuralTag, normalize_tool_choice
    (APIServer pid=35) ImportError: cannot import name 'normalize_tool_choice' from 'xgrammar' (/opt/conda/lib/python3.12/site-packages/xgrammar/init.py)

  • arrow_forward

    Thread has been moved from 产品&运维.

  • link
    Members 949 posts
    2026年8月27日 10:17
    link

    尊敬的开发者您好，此镜像不支持qwen3.8，等待后续镜像更新

  • arrow_forward

    Thread has been moved from 解决中.