Robotics
English
yunfanj commited on
Commit
f949589
0 Parent(s):
.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: MIT License
3
+ spdx-id: MIT
4
+ featured: true
5
+ hidden: false
6
+
7
+ description: A short and simple permissive license with conditions only requiring preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code.
8
+
9
+ how: Create a text file (typically named LICENSE or LICENSE.txt) in the root of your source code and copy the text of the license into the file. Replace [year] with the current year and [fullname] with the name (or names) of the copyright holders.
10
+
11
+ using:
12
+ Babel: https://github.com/babel/babel/blob/master/LICENSE
13
+ .NET: https://github.com/dotnet/runtime/blob/main/LICENSE.TXT
14
+ Rails: https://github.com/rails/rails/blob/master/MIT-LICENSE
15
+
16
+ permissions:
17
+ - commercial-use
18
+ - modifications
19
+ - distribution
20
+ - private-use
21
+
22
+ conditions:
23
+ - include-copyright
24
+
25
+ limitations:
26
+ - liability
27
+ - warranty
28
+
29
+ ---
30
+
31
+ MIT License
32
+
33
+ Copyright (c) 2024 Yunfan Jiang
34
+
35
+ Permission is hereby granted, free of charge, to any person obtaining a copy
36
+ of this software and associated documentation files (the "Software"), to deal
37
+ in the Software without restriction, including without limitation the rights
38
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
39
+ copies of the Software, and to permit persons to whom the Software is
40
+ furnished to do so, subject to the following conditions:
41
+
42
+ The above copyright notice and this permission notice shall be included in all
43
+ copies or substantial portions of the Software.
44
+
45
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
46
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
47
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
48
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
49
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
50
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
51
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ pipeline_tag: robotics
6
+ ---
7
+ # Model Card for TRANSIC Policies
8
+
9
+ <!-- Provide a quick summary of what the model is/does. -->
10
+
11
+ This modelcard is accompanied with the [paper](https://arxiv.org/abs/2405.10315) titled TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction.
12
+ It includes robot policies trained in the simulation and transferred to the real world for complex and contact-rich manipulation tasks.
13
+
14
+ ## Model Details
15
+
16
+ ### Model Description
17
+
18
+ <!-- Provide a longer summary of what this model is. -->
19
+ This model repository includes three parts, 1) teacher policies trained in simulation with reinforcement learning; 2) student policies distilled from successful trajectories generated by teacher policies; and 3) residual policies learned in the real world to augment simulation policies.
20
+
21
+ The first part can be found in the `rl` directory. We provide RL teacher policies for 8 different tasks. The second part can be found in the `student` directory. We provide 5 student policies corresponding to 5 skills used in assembling the square table from [FurnitureBench](https://github.com/clvrai/furniture-bench). The third part can be found in the `residual` directory. They augment those 5 simulation base policies (student policies).
22
+
23
+
24
+ - **Developed by:** [Yunfan Jiang](https://yunfanj.com/)
25
+ - **Model type:** [PyTorch Checkpoints](https://pytorch.org/tutorials/recipes/recipes/saving_and_loading_a_general_checkpoint.html)
26
+ - **License:** [MIT](LICENSE)
27
+
28
+ ### Model Sources
29
+
30
+ - **Repositories:** [TRANSIC](https://github.com/transic-robot/transic), [TRANSIC-Envs](https://github.com/transic-robot/transic-envs)
31
+ - **Paper:** [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://arxiv.org/abs/2405.10315)
32
+ - **Demo:** Provided on Our [Website](https://transic-robot.github.io/)
33
+
34
+ ## Uses & How to Get Started with the Model
35
+
36
+ Please see our [codebase](https://github.com/transic-robot/transic) for detailed usage.
37
+
38
+ ## Training Details
39
+
40
+ ### Training Data
41
+
42
+ We provide training data in our [🤗Hugging Face data repository](https://huggingface.co/datasets/transic-robot/data).
43
+
44
+ ### Training Procedure
45
+
46
+ 1. Teacher policies are first trained with reinforcement learning from scratch in simulation;
47
+ 2. We then rollout teacher policies to generate successful trajectories;
48
+ 3. These generated data are used to train student policies through behavior cloning. Student policies take point-cloud and proprioceptive observations and output joint actions.
49
+ 4. We then deploy student policies on the real robot. A human operator monitors the execution, intervenes when necessary, and provides online correction through teleoperation. Such teleoperation data are collected.
50
+ 5. We use collected correction data to learn residual policies, which then augment simulation policies for successful sim-to-real transfer.
51
+
52
+ ## Evaluation
53
+
54
+ Policies are evaluated in simulation and the real world. We use task success rate as the metric.
55
+
56
+ ## Citation
57
+
58
+ **BibTeX:**
59
+
60
+ ```
61
+ @article{jiang2024transic,
62
+ title = {TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction},
63
+ author = {Yunfan Jiang and Chen Wang and Ruohan Zhang and Jiajun Wu and Li Fei-Fei},
64
+ year = {2024},
65
+ journal = {arXiv preprint arXiv: Arxiv-2405.10315}
66
+ }
67
+ ```
68
+
69
+ ## Model Card Contact
70
+
71
+ [Yunfan Jiang](https://yunfanj.com/), email: `yunfanj[at]cs[dot]stanford[dot]edu`
residual/insert.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4e3a91843429063ca73174460cdbd1d3747836e92426997f9126f2247cc3cd5
3
+ size 5958913
residual/lift_leaned_leg.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6760ced28c86b54a0c5d064b29f4fea62560402a53b3811bb742967090be8bb
3
+ size 10302447
residual/reach_and_grasp.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2457159aec58a319b29ad2e9203489b72533371ce945b2c2ef87d670848e77e5
3
+ size 10300856
residual/screw.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cdf6af4b51c6bbd53bb40e2293bee0732bb055a5834d4794fec3222af6759ff8
3
+ size 10302511
residual/stabilize.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6a3b581f29366f77ac00dcab6d5733dd998050d43adf8429bdf9b56bdbed991
3
+ size 6096062
rl/insert_full.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8cd5f2ca60e23281d53d9d4dd025695b00705e017a1808042f7e6994bb9c479
3
+ size 2596017
rl/insert_single.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5629bf09973db5ef20e10010333c0f893d8e2fc8f291344da7720d8d69d3b2d1
3
+ size 2429723
rl/lift_leaned_leg.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4102726c52187de8df2655aab607aa45f128664ccb18829190fcf6dd89a9078
3
+ size 2614080
rl/one_leg_grasp.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f358e7d76a9122e6e5b53fe2554c092b4f0d4a5867d43a969a757e76de63835
3
+ size 2374100
rl/reach_and_grasp_full.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0592f67c122e3a883db02cab924d6c59adebddd1921adc351b57651dfc12c169
3
+ size 2373610
rl/reach_and_grasp_single.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f358e7d76a9122e6e5b53fe2554c092b4f0d4a5867d43a969a757e76de63835
3
+ size 2374100
rl/screw.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:651cdaab6a3215ce73173bf928c54b2f9172193a320bca510296666fee84a75a
3
+ size 2423437
rl/stabilize.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d422517857f064e19d66c8e6d149bcaf5130f07823c4eb335146230c5770e666
3
+ size 2663843
student/insert.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e988d50948e3d9fe86e7f11f97aa260cc181c13959f7870486b5559f09da759
3
+ size 10629471
student/lift_leaned_leg.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f892954c5d2e5b573ad506538527ac96539a852bb1808bc9e04ab5d4a22c39b4
3
+ size 10629407
student/reach_and_grasp.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c278b9746c3dba753a72a8403ce79aeb1d632e044f18add77c065873900b23cd
3
+ size 10629407
student/screw.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01f59765b01cafdc3192245435b4140ad336a21b4019b1584616142bfbc2d556
3
+ size 61067373
student/stabilize.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe767c5028ada8f1a27b028c572544123fa10c942aacd0d02e3e42f9bfce4e41
3
+ size 61066035