TEMP: no shared tokenizer from PR-3512

Gotta get conflicts with the main line figured resolved first. Just don't use the shared tokenizer for now. Signed-off-by: Travis Johnson <[email protected]>
IBM · May 20, 2024 · 64b18a7 · 64b18a7
1 parent 79cc739
commit 64b18a7
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/vllm/entrypoints/grpc/grpc_server.py b/vllm/entrypoints/grpc/grpc_server.py
@@ -103,7 +103,8 @@ def __init__(self, engine: AsyncLLMEngine, args: argparse.Namespace):
 
     async def _post_init(self):
         self.config = await self.engine.get_model_config()
-        self.tokenizer_group = await self.engine.get_tokenizer_group()
+         # self.tokenizer_group = await self.engine.get_tokenizer_group()
+        self.tokenizer_group = self.engine.engine.tokenizer
         self.tokenizer = await self.engine.get_tokenizer()
 
         # Swap in the special TGIS stats logger