Skip to main content
Back to all repositories

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

8,820stars1,020forks84watchers/subscribers213issues
ai-inferencedeep-learninggenerative-aiinference-platformllmllm-inferencellm-servingllmopsmachine-learningml-engineeringmlopsmodel-inference-servicemodel-servingmultimodalpython
Language
Python
License
Apache License 2.0
Size
101.4 MB
Created
Apr 2, 2019
Last Updated
Sep 3, 2026
Last Pushed
Aug 28, 2026

Available Plugins

Loading plugins...

Evaluate before installing

  1. Review the source repository, recent maintenance, and license on GitHub.
  2. Read the marketplace manifest and plugin source files before running commands.
  3. Start with the smallest required permission set and validate behavior in a safe environment.