ROS Agents 🤖

ROS Agents is a fully-loaded framework for creating interactive embodied agents that can understand, remember, and act upon contextual information from their environment.

• Agents in the real world: Designed to be used with autonomous robot systems that operate in dynamic environments, specifically AMRs.

• Intuitive API: Simple pythonic API to utilize local or cloud based ML models (specifically Multimodal LLMs and other Transformer Architectures) on robots.

• Semantic Memory: Integrates vector databases, semantic routing and other supporting components to quickly build arbitrarily complex graphs for agentic information flow. No need to utilize bloated “GenAI” frameworks on your robot.

• Made in ROS2: Utilizes ROS2 as the underlying distributed communications backbone. Theoretically, all devices that provide a ROS2 package can be utilized to send data to ML models, as long as the datatype callback has been implemented.

Checkout Installation Instructions 🛠️

Get started with the Quickstart Guide 🚀

Get familiar with Basic Concepts 📚

Dive right in with Examples ✨

Contributions

ROS Agents has been developed in collaboration betweeen Automatika Robotics and Inria. Contributions from the community are most welcome.

Table of Contents

• ROS Agents 🤖
  • Contributions
• Installation 🛠️
  • Pre-Requisits
  • Install ROS Agents (Ubuntu)
  • Install ROS Agents from source
• Quick Start 🚀
• Basic Concepts 📚
  • Component
  • Topic
  • Model/DB Client
  • Models/DBs
• Examples ✨
  • Create a conversational agent with audio
  • Prompt engineering for LLMs/MLLMs using vision models
  • Create a spatio-temporal semantic map
  • Create a Go-to-X component using map data
  • Use Tool Calling in Go-to-X
  • Create a semantic router to route text queries between different components
  • Bringing it all together 🤖
  • Making the System Robust And Production Ready
• API Reference
  • agents