Optimizing high-scale LLM inference @ TBank.
- ℹ️ Labeled PR #6 in riZZZhik/Mooncake
- ℹ️ Unlabeled PR #6 in riZZZhik/Mooncake
- 💪 Opened PR #7 in riZZZhik/Mooncake
- ℹ️ Labeled PR #7 in riZZZhik/Mooncake
- ❌ Closed PR #6 in riZZZhik/Mooncake
Optimizing high-scale LLM inference @ TBank.
SGLang is a high-performance serving framework for large language models and multimodal models.
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Optimize and Inference TensorRT Machine Learning models in Python
Python 1