High-capacity vision-language models (VLMs) are trained on large web datasets, enabling them to effectively recognize visual and language patterns and function across multiple languages. However, for robots to reach a similar level of proficiency, they would need to gather firsthand data across various objects, environments, tasks, and situations. In this context, researchers have introduced Robotic Transformer 2 (RT-2), a vision-language-action (VLA) model that learns from both web and robotics

2.3K views · 61 reactions | Vision Language Models (VLMs) understand natural language prompts and perform visual question answering. ➡️ https://nvda.ws/4cTW5Ox Learn how you can build VLM-powered visual AI agents for a wide range of apps. #SIGGRAPH2024 | NVIDIA AI | Facebook