multimodal

5 Articles
New multimodal AI tool supports ecological applications
Tech

New multimodal AI tool supports ecological applications

The TaxaBind framework creates a unified database by distilling information from five different modalities into one binding modality. In TaxaBind’s case, the binding...

Psychology-based tasks assess multi-modal LLM visual cognition limits
Tech

Psychology-based tasks assess multi-modal LLM visual cognition limits

The help or hinder task; one of the tasks used to test the visual cognition of multimodal LLMs. Credit: MIT. Over the past...

A Minecraft-based benchmark to train and test multi-modal multi-agent systems
Tech

A Minecraft-based benchmark to train and test multi-modal multi-agent systems

More than 30 target objects or resources are used in TeamCraft tasks. Credit: UCLA. Researchers at the University of California- Los Angeles (UCLA)...

Open-source framework goes beyond language to enhance multimodal AI training capabilities
Tech

Open-source framework goes beyond language to enhance multimodal AI training capabilities

A couple of oranges seen through the lens of multiple modalities, with each slice showing a different way one might perceive and understand...

Integrated multi-modal sensing and learning system could give robots new capabilities
Tech

Integrated multi-modal sensing and learning system could give robots new capabilities

Soft robot fingers equipped with tactile sensors grasping an egg. The bottom-right images show the tactile sensing results. Credit: Binghao Huang. To assist...