ODYSSEY icon

Open-World Quadrupeds Exploration and
Manipulation for Long-Horizon Tasks

Kaijun Wang1*, Liqin Lu2*, Mingyu Liu1, Jianuo Jiang3, Zeju Li1, Bolin Zhang1, Wancai Zheng2,
Xinyi Yu2, Hao Chen1, Chunhua Shen1
1Zhejiang University.
2Zhejiang University of Technology.
3The Chinese University of Hong Kong, Shenzhen.
*equal contribution

Abstract

Language-guided long-horizon mobile manipulation has long been a grand challenge in semantic reasoning, generalizable manipulation, and adaptive locomotion. Three fundamental limitations hinder progress: First, although large language models have shown promise in enhancing spatial reasoning and task planning through learned semantic priors, existing implementations remain confined to tabletop scenarios, failing to address the constrained perception and limited actuation ranges characteristic of mobile platforms. Second, current manipulation strategies exhibit insufficient generalization when confronted with the diverse object configurations encountered in open-world environments. Third, while crucial for practical deployment, the dual requirement of maintaining high platform maneuverability alongside precise end-effector control in unstructured settings remains understudied in the literature.
In this work, we present ODYSSEY, a unified mobile manipulation framework for agile quadruped robots equipped with manipulators, which seamlessly integrates high-level task planning with low-level whole-body control. To address the challenge of egocentric perception in language-conditioned tasks, we introduce a hierarchical planner powered by a vision-language model, enabling long-horizon instruction decomposition and precise action execution. At the control level, our novel whole-body policy achieves robust coordination of locomotion and manipulation across challenging terrains. We further present the first comprehensive benchmark for long-horizon mobile manipulation, evaluating diverse indoor and outdoor scenarios. Through successful sim-to-real transfer, we demonstrate the system's generalization and robustness in real-world deployments, underscoring the practicality of legged manipulators in unstructured environments. Our work advances the feasibility of generalized robotic assistants capable of complex, dynamic tasks.

ODYSSEY Preview DEMO

Our proposed ODYSSEY system empowers quadruped robots 🐾 to excel in diverse environments — from household chores 🏠 and services 🍽️ to outdoor maintenance 🌿. Acting as butlers, waiters, gardeners, and more, these versatile robot assistants bring intelligence and agility to every task 🤖✨.

ODYSSEY Mobile Planner

ODYSSEY pipeline spans the entire process of a long-horizon task, including multi-modal semantic perception, map-aware global planning, geometry-constrained action grounding, and step-wise execution by a reinforced learned low-level whole-body policy.

ODYSSEY Benchmark

ODYSSEY Benchmark is the first simulation platform designed for long-horizon mobile manipulation which highlights the following major points: (1) The benchmark includes 8 multi-stage tasks designed to evaluate the full pipeline of embodied mobile manipulation, covering perception, planning, navigation, and manipulation. (2) Each task consists of 2-3 subgoals with diverse object types, spatial layouts, and physical interaction modes to reflect realistic and complex scenarios. (3) ODYSSEY spans both indoor and outdoor environments, featuring over 300 task variations and numerous unique assets to ensure robust generalization. (4) Detailed task configurations enable comprehensive benchmarking of system performance across a wide range of practical challenges.

Versatile object configurations in ODYSSEY benchmark.

Variation counts, and asset usage per task.

Real-world Deployment

ODYSSEY robot in action
ODYSSEY robot system

Real-world experiments.

BibTeX

BibTex Code Here