Models / Vector Databases 🧠

Models / Vector Databases 🧠#

Clients mentioned earlier take as input a model or vector database (DB) specification. These are in most cases generic wrappers around a class of models/dbs (e.g. transformers based LLMs) defined as attrs classes and include initialization parameters such as quantization schemes, inference options, embedding model (in case of vector DBs) etc. These specifications aim to standardize model initialization across diverse deployment platforms.

Available Models#

Model Name

Description

OllamaModel

A LLM/MLLM model loaded from an Ollama checkpoint. Supports configurable generation and deployment options available in Ollama API. Complete list of Ollama models [here].(https://ollama.com/library). This wrapper can be used with the OllamaClient.

TransformersLLM

LLM models from HuggingFace/ModelScope based checkpoints. Supports quantization (“4bit”, “8bit”) specification. This model wrapper can be used with the GenericHTTPClient or any of the RoboML clients.

TransformersMLLM

Multimodal LLM models from HuggingFace/ModelScope checkpoints for image-text inputs. Supports quantization. This model wrapper can be used with the GenericHTTPClient or any of the RoboML clients.

Whisper

OpenAI’s automatic speech recognition (ASR) model with various sizes (e.g., "small", "large-v3", etc.). These models are available on the RoboML platform and can be used with any RoboML client. Recommended, RoboMLWSClient.

SpeechT5

Microsoft’s model for TTS synthesis. Configurable voice selection. This model is available on the RoboML platform and can be used with any RoboML client. Recommended, RoboMLWSClient.

Bark

SunoAI’s Bark TTS model. Allows a selection voices. This model is available on the RoboML platform and can be used with any RoboML client. Recommended, RoboMLWSClient.

MeloTTS

MyShell’s multilingual TTS model. Configure via language (e.g., "JP") and speaker_id (e.g., "JP-1"). This model is available on the RoboML platform and can be used with any RoboML client. Recommended, RoboMLWSClient.

VisionModel

A generic wrapper for object detection and tracking models available on MMDetection framework. Supports optional tracking, configurable thresholds, and deployment with TensorRT. This model is available on the RoboML platform and can be used with any RoboML client. Recommended, RoboMLRESPClient.

Available Vector Databases#

Vector DB

Description

ChromaDB

Chroma is an open-source AI application database with support for vector search, full-text search, and multi-modal retrieval. Supports “ollama” and “sentence-transformers” embedding backends. Can be used with the ChomaClient.

Note

For ChromaDB, make sure you install required packages:

pip install ollama  # For Ollama backend (requires Ollama runtime)
pip install sentence-transformers  # For Sentence-Transformers backend

To use Ollama embedding models (available models), ensure the Ollama server is running and accessible via specified host and port.

Contents