File size: 1,509 Bytes
5cb0391 117e02f 5cb0391 c8ef7a8 5cb0391 117e02f |
1 2 3 4 5 6 7 8 9 10 11 |
# In-Context Imitation Learning via Next-Token Prediction
by <a href="https://max-fu.github.io">Max (Letian) Fu*</a>, <a href="https://qingh097.github.io/">Huang Huang*</a>, <a href="https://www.linkedin.com/in/gaurav-datta/">Gaurav Datta*</a>, <a href="https://yunliangchen.github.io/">Lawrence Yunliang Chen</a>, <a href="https://autolab.berkeley.edu/people">William Chung-Ho Panitch</a>, <a href="https://fangchenliu.github.io/">Fangchen Liu</a>, <a href="https://www.research.autodesk.com/people/hui-li/">Hui Li</a>, and <a href="https://goldberg.berkeley.edu">Ken Goldberg</a> at UC Berkeley and Autodesk (*equal contribution).
[[Paper]()] | [[Project Page](https://in-context-robot-transformer.github.io/)] | [[Checkpoints](https://huggingface.co/mlfu7/ICRT)] | [[Dataset](https://huggingface.co/datasets/Ravenh97/ICRT-MT)]
This repo contains the checkpoints for *In-Context Imitation Learning via Next-Token Prediction*. We investigate how to bring few-shot, in-context learning capability that exists in next-token prediction models (i.e. GPT) into real-robot imitation learning policies.
In particular, we store the pre-trained vision encoder and ICRT model separately. Please find them in [encoder](crossmae_rtx/cross-mae-rtx-vitb.pth), [ICRT](icrt_vitb_droid_pretrained/icrt_vitb_droid_pretrained.pth), and [ICRT-Llama7B](icrt_llama7b_lora/icrt_llama7b_lora.pth).
Please refer to the [project page](https://github.com/Max-Fu/icrt) on installing the repo, training and inferencing the model.
|