LM Cache

我想问下，沐曦适配的vllm，内置有添加LM Cache吗，可以直接
vllm serve Qwen/Qwen3-8B \
--port 8000 --kv-transfer-config \
'{"kv_connector":"LMCacheConnectorV1", "kv_role":"kv_both"}'来设置LM Cache吗

尊敬的开发者您好，镜像内置LM Cache，参考社区命令使用即可

可以提供一下社区命令哪个页面连接吗

(Worker_TP1 pid=560) WARNING 07-13 14:12:40 [kv_cache_utils.py:1329] Hybrid KV cache manager is disabled for this hybrid model, This means we do not enable any optimizations for saving KV cache memory (e.g., dropping the KV cache outside the sliding window). The compute of layers like sliding window is still saved.
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] WorkerProc hit an exception.
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] Traceback (most recent call last):
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 957, in worker_busy_loop
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] output = func(args, kwargs)
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] return func(args, kwargs)
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 385, in determine_available_memory
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] cudagraph_memory_estimate = self.model_runner.profile_cudagraph_memory()
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] return func(*args, kwargs)
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5951, in profile_cudagraph_memory
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] self._init_minimal_kv_cache_for_profiling()
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5870, in _init_minimal_kv_cache_for_profiling
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] kv_cache_groups = get_kv_cache_groups(self.vllm_config, kv_cache_spec)
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/core/kv_cache_utils.py", line 1620, in get_kv_cache_groups
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] unify_hybrid_kv_cache_specs(kv_cache_spec)
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/core/kv_cache_utils.py", line 1402, in unify_hybrid_kv_cache_specs
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] raise ValueError(
(Worker_TP1 pid=560) ERROR 07-13 14:12:40 [multiproc_executor.py:962] ValueError: Hybrid KV cache manager is disabled but failed to convert the KV cache specs to one unified type.
(Worker_TP0 pid=559) WARNING 07-13 14:12:41 [kv_cache_utils.py:1329] Hybrid KV cache manager is disabled for this hybrid model, This means we do not enable any optimizations for saving KV cache memory (e.g., dropping the KV cache outside the sliding window). The compute of layers like sliding window is still saved.
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] WorkerProc hit an exception.
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] Traceback (most recent call last):
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 957, in worker_busy_loop
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] output = func(args, kwargs)
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] return func(args, kwargs)
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 385, in determine_available_memory
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] cudagraph_memory_estimate = self.model_runner.profile_cudagraph_memory()
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] return func(*args, kwargs)
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5951, in profile_cudagraph_memory
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] self._init_minimal_kv_cache_for_profiling()
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5870, in _init_minimal_kv_cache_for_profiling
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] kv_cache_groups = get_kv_cache_groups(self.vllm_config, kv_cache_spec)
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/core/kv_cache_utils.py", line 1620, in get_kv_cache_groups
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] unify_hybrid_kv_cache_specs(kv_cache_spec)
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/core/kv_cache_utils.py", line 1402, in unify_hybrid_kv_cache_specs
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] raise ValueError(
(Worker_TP0 pid=559) ERROR 07-13 14:12:41 [multiproc_executor.py:962] ValueError: Hybrid KV cache manager is disabled but failed to convert the KV cache specs to one unified type.
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] EngineCore failed to start.
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] Traceback (most recent call last):
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 1110, in run_engine_core
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] engine_core = EngineCoreProc(args, engine_index=dp_rank, kwargs)
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] return func(args, kwargs)
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 876, in init
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] super().init(
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 128, in init
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] kv_cache_config = self._initialize_kv_caches(vllm_config)
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] return func(*args, kwargs)
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 250, in _initialize_kv_caches
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] available_gpu_memory = self.model_executor.determine_available_memory()
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/abstract.py", line 147, in determine_available_memory
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] return self.collective_rpc("determine_available_memory")
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 403, in collective_rpc
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] return future if non_block else future.result()
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 90, in result
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] return super().result()
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 451, in result
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] return self.get_result()
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] raise self._exception
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 94, in _wait_for_response
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] response = self.aggregate(self.get_response())
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 390, in get_response
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] raise RuntimeError(
(EngineCore pid=419) ERROR 07-13 14:12:41 [core.py:1136] RuntimeError: Worker failed with error 'Hybrid KV cache manager is disabled but failed to convert the KV cache specs to one unified type.', please check the stack trace above for the root cause
(Worker_TP0 pid=559) WARNING 07-13 14:12:41 [multiproc_executor.py:884] WorkerProc was terminated
(Worker_TP1 pid=560) WARNING 07-13 14:12:41 [multiproc_executor.py:884] WorkerProc was terminated
(EngineCore pid=419) ERROR 07-13 14:13:00 [multiproc_executor.py:283] Worker proc VllmWorker-1 died unexpectedly, shutting down executor.
(EngineCore pid=419) Process EngineCore:
(EngineCore pid=419) Traceback (most recent call last):
(EngineCore pid=419) File "/opt/conda/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=419) self.run()
(EngineCore pid=419) File "/opt/conda/lib/python3.10/multiprocessing/process.py", line 108, in run
(EngineCore pid=419) self._target(self._args, self._kwargs)
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 1140, in run_engine_core
(EngineCore pid=419) raise e
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 1110, in run_engine_core
(EngineCore pid=419) engine_core = EngineCoreProc(args, engine_index=dp_rank, kwargs)
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=419) return func(*args, kwargs)
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 876, in __init
(EngineCore pid=419) super().init(
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 128, in init
(EngineCore pid=419) kv_cache_config = self._initialize_kv_caches(vllm_config)
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(EngineCore pid=419) return func(args, kwargs)
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 250, in _initialize_kv_caches
(EngineCore pid=419) available_gpu_memory = self.model_executor.determine_available_memory()
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/abstract.py", line 147, in determine_available_memory
(EngineCore pid=419) return self.collective_rpc("determine_available_memory")
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 403, in collective_rpc
(EngineCore pid=419) return future if non_block else future.result()
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 90, in result
(EngineCore pid=419) return super().result()
(EngineCore pid=419) File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 451, in result
(EngineCore pid=419) return self.get_result()
(EngineCore pid=419) File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
(EngineCore pid=419) raise self._exception
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 94, in _wait_for_response
(EngineCore pid=419) response = self.aggregate(self.get_response())
(EngineCore pid=419) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 390, in get_response
(EngineCore pid=419) raise RuntimeError(
(EngineCore pid=419) RuntimeError: Worker failed with error 'Hybrid KV cache manager is disabled but failed to convert the KV cache specs to one unified type.', please check the stack trace above for the root cause
(APIServer pid=1) Traceback (most recent call last):
(APIServer pid=1) File "/opt/conda/bin/vllm", line 8, in <module>
(APIServer pid=1) sys.exit(main())
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/cli/main.py", line 92, in main
(APIServer pid=1) args.dispatch_function(args)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/cli/serve.py", line 122, in cmd
(APIServer pid=1) uvloop.run(run_server(args))
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/uvloop/__init.py", line 69, in run
(APIServer pid=1) return loop.run_until_complete(wrapper())
(APIServer pid=1) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=1) return await main
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 678, in run_server
(APIServer pid=1) await run_server_worker(listen_address, sock, args, uvicorn_kwargs)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 692, in run_server_worker
(APIServer pid=1) async with build_async_engine_client(
(APIServer pid=1) File "/opt/conda/lib/python3.10/contextlib.py", line 199, in aenter
(APIServer pid=1) return await anext(self.gen)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 100, in build_async_engine_client
(APIServer pid=1) async with build_async_engine_client_from_engine_args(
(APIServer pid=1) File "/opt/conda/lib/python3.10/contextlib.py", line 199, in aenter
(APIServer pid=1) return await anext(self.gen)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 136, in build_async_engine_client_from_engine_args
(APIServer pid=1) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 217, in from_vllm_config
(APIServer pid=1) return cls(
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 146, in init
(APIServer pid=1) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=1) return func(args, kwargs)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 130, in make_async_mp_client
(APIServer pid=1) return AsyncMPClient(client_args)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(APIServer pid=1) return func(args, kwargs)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 900, in init
(APIServer pid=1) super().init(
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 535, in init
(APIServer pid=1) with launch_core_engines(
(APIServer pid=1) File "/opt/conda/lib/python3.10/contextlib.py", line 142, in exit
(APIServer pid=1) next(self.gen)
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/utils.py", line 1119, in launch_core_engines
(APIServer pid=1) wait_for_engine_startup(
(APIServer pid=1) File "/opt/conda/lib/python3.10/site-packages/vllm/v1/engine/utils.py", line 1178, in wait_for_engine_startup
(APIServer pid=1) raise RuntimeError(
(APIServer pid=1) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

我按照LM Cache启动模型设置了--kv-transfer-config \
'{"kv_connector":"LMCacheConnectorV1", "kv_role":"kv_both"}'，上面是启动报错，使用的vllm20.0版本，N260*2

尊敬的开发者您好，参考docs.vllm.ai/en/v0.20.0/examples/others/lmcache/?h=lmca

没有看到vllm serve启动怎么用

如图中，社区连接给的github已经失效了，所以我想问下有没有直接的命令提供

尊敬的开发者您好，请参考社区相关命令。若失效，请在vLLM以及LM Cache社区提相关issue