BiCap is a novel bimodal dataset designed to advance the development of learning-from-demonstration (LfD) algorithms for dual-arm robotic manipulation. It comprises 4,026 demonstrations collected from fifteen participants performing pouring, opening, and passing tasks with household objects. Each demonstration includes an RGB video recording, upper-limb kinematics, and a symbolic task plan annotated using a bio-inspired action context-free grammar (BACFG). BiCap overcomes fundamental limitations of existing LfD datasets, which often lack formally structured symbolic annotations and exhaustive capture of dual-arm manipulation skills, by coupling fine-grained symbolic representations with trajectory data. It provides a rigorous foundation for developing and evaluating algorithms capable of learning, planning, and generalising complex dual-arm manipulation behaviours.
A unique dataset collected using an RGB camera and an advanced motion capture system. The paper presenting BiCap is in the International Journal of Robotics Research (IJRR). Nevertheless, several slight improvements have been made to the dataset since its publication in IJRR. This website provides the latest information on BiCap.
@article{carmona_bicap_2025,
author = {Carmona, David and Yu, Haoyong},
title = {BiCap: A novel bi-modal dataset of daily living dual-arm manipulation actions},
journal = {The International Journal of Robotics Research},
volume = {44},
number = {6},
year = {2025},
pages = {891--907},
issn = {0278-3649},
url = {https://doi.org/10.1177/02783649241290836},
doi = {10.1177/02783649241290836}
}
BiCap can be downloaded from Hugging Face.
BiCap
├── Motion Capture # Folder containing all the subjects' motion data.
│ ├── BBM01 # Data of the subject number BBM01.
│ │ ├── pouring_bowl_cracker_01.csv # The motion data for the first trial of the subject pouring box of crackers into a bowl.
│ │ ├── pouring_bowl_cracker_02.csv # The motion data for the second trial of the subject pouring box of crackers into a bowl.
│ │ ├── passing_masterchef_04.csv # The motion data for the fourth trial of the subject passing a masterchef can.
│ │ └── ...
│ └── ...
├── Videos # Folder containing all the subjects' videos taken with the RGB camera and their corresponding task plans.
│ ├── BBM01 # Data of the subject number BBM01.
│ │ ├── Expt 1 (Pouring) # Data of the first experiment (the pouring task).
│ │ │ ├── Bowl and Cracker box # RGB Video and task plan data for pouring a box of crackers into a bowl.
│ │ │ │ ├── Trial 1.mp4 # RGB Video data from the camera for the first trial.
│ │ │ │ ├── Trial 1.xml # Task plan for the first trial.
│ │ │ │ ├── Trial 2.mp4 # RGB Video data from the camera for the second trial.
│ │ │ │ ├── Trial 2.xml # Task plan for the second trial.
│ │ │ │ └── ...
│ │ │ └── ...
│ │ └── ...
│ └── ...
├── participants.xlsx # The subjects' data.
├── readme.pdf
└── readme.md
The RGB videos were recorded with a Stereolabs ZED2 camera. A human expert annotated the symbolic task plans using the RGB videos and the BACFG.
The participants’ motion was captured with four Vicon Vantage 16 and four Vicon MX T160 cameras positioned across the workspace.
The subjects’ identifiers inside the XML files differ from the folders’ names (e.g., BBM01, BBM02, or BBM03). The mapping is:
The joint angles present in the csv files are:
The markers’ positions in the csv are:
The following data is unavailable in the dataset:
David Carmona: dcmoreno@nus.edu.sg
This dataset is made available under the Creative Commons Attribution 4.0 International License - see the LICENSE.md file for details