agents.components.vision

agents.components.vision#

Module Contents#

Classes#

Vision

This component performs object detection and tracking on input images and outputs a list of detected objects, along with their bounding boxes and confidence scores.

API#

class agents.components.vision.Vision(*, inputs: List[Union[agents.ros.Topic, agents.ros.FixedInput]], outputs: List[agents.ros.Topic], model_client: Optional[agents.clients.model_base.ModelClient] = None, config: Optional[agents.config.VisionConfig] = None, trigger: Union[agents.ros.Topic, List[agents.ros.Topic], float] = 1.0, component_name: str, **kwargs)#

Bases: agents.components.model_component.ModelComponent

This component performs object detection and tracking on input images and outputs a list of detected objects, along with their bounding boxes and confidence scores.

Parameters:

  • inputs (list[Union[Topic, FixedInput]]) – The input topics for the object detection. This should be a list of Topic objects or FixedInput objects, limited to Image (or RGBD) type.

  • outputs (list[Topic]) – The output topics for the object detection. This should be a list of Topic objects, Detection and Tracking types are handled automatically.

  • model_client (Optional[ModelClient]) – Optional model client for the vision component to access remote vision models. If not provided, enable_local_classifier should be set to True in VisionConfig This should be an instance of ModelClient. Defaults to None.

  • config (VisionConfig) – The configuration for the vision component. This should be an instance of VisionConfig. If not provided, defaults to VisionConfig().

  • trigger (Union[Topic, list[Topic], float]) – The trigger value or topic for the vision component. This can be a single Topic object, a list of Topic objects, or a float value for timed components.

  • component_name (str) – The name of the vision component. This should be a string and defaults to “vision_component”.

Example usage:

image_topic = Topic(name="image", msg_type="Image")
detections_topic = Topic(name="detections", msg_type="Detections")
config = VisionConfig()
model_client = ModelClient(model=DetectionModel(name='yolov5'))
vision_component = Vision(
    inputs=[image_topic],
    outputs=[detections_topic],
    model_client=model_client
    config=config,
    component_name = "vision_component"
)
custom_on_configure()#

Create model client if provided and initialize model.

custom_on_deactivate()#

Destroy model client if it exists

property warmup: bool#

Enable warmup of the model.

custom_on_activate()#

Custom configuration for creating triggers.

create_all_subscribers()#

Override to handle trigger topics and fixed inputs. Called by parent BaseComponent

activate_all_triggers() → None#

Activates component triggers by attaching execution step to callbacks

destroy_all_subscribers() → None#

Destroys all node subscribers

trigger(trigger: Union[agents.ros.Topic, List[agents.ros.Topic], float]) → None#

Set component trigger

validate_topics(topics: Sequence[Union[agents.ros.Topic, agents.ros.FixedInput]], allowed_topic_types: Optional[Dict[str, List[Union[Type[agents.ros.SupportedType], List[Type[agents.ros.SupportedType]]]]]] = None, topics_direction: str = 'Topics')#

Verify component specific inputs or outputs using allowed topics if provided

Contents