|
Title
|
|
|
|
A multi-modal AI approach for intuitively instructable autonomous systems
| |
|
Author
|
|
|
|
| |
|
Abstract
|
|
|
|
| We present a multi-modal AI framework to intuitively instruct and control Automated Guided Vehicles. We define a general multi-modal AI architecture, which has a loose coupling between three different AI modules, including spoken language understanding, visual perception and Reinforcement Learning navigation. We use the same multi-modal architecture for two different use cases implemented in two different platforms: an off-road vehicle, which can pick objects, and an indoor forklift that performs automated warehouse inventory. We show how the proposed architecture can be used for a wide range of tasks and can be implemented in different hardware, demonstrating a high degree of modularity. |
| |
|
Language
|
|
|
|
English
| |
|
Source (journal)
|
|
|
|
International journal on advances in systems and measurements
| |
|
Publication
|
|
|
|
2023
| |
|
Volume/pages
|
|
|
|
16
:1&2
(2023)
, p. 1-13
| |
|
Full text (open access)
|
|
|
|
| |
|
Full text (publisher's version - intranet only)
|
|
|
|
| |
|