phython96 commited on
Commit
3686a97
·
verified ·
1 Parent(s): c61bb99

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -1
README.md CHANGED
@@ -7,4 +7,29 @@ tags:
7
 
8
  This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
9
  - Library: https://huggingface.co/phython96/ROCKET-1
10
- - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
 
8
  This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
9
  - Library: https://huggingface.co/phython96/ROCKET-1
10
+ - Docs: [More Information Needed]
11
+ - Paper: https://huggingface.co/papers/2410.17856
12
+ - Github: https://github.com/CraftJarvis/ROCKET-1
13
+ - Project: https://craftjarvis.github.io/ROCKET-1
14
+
15
+ ## Usage
16
+ ```python
17
+ from rocket.arm.models import ROCKET1
18
+ from rocket.stark_tech.env_interface import MinecraftWrapper
19
+
20
+ model = ROCKET1.from_pretrained("phython96/ROCKET-1").to("cuda")
21
+ memory = None
22
+ input = {
23
+ "img": torch.rand(224, 224, 3, dtype=torch.uint8),
24
+ 'segment': {
25
+ 'obj_id': torch.tensor(6), # specify the interaction type
26
+ 'obj_mask': torch.zeros(224, 224, dtype=torch.uint8), # highlight the regions of interest
27
+ }
28
+ }
29
+ agent_action, memory = self.agent.get_action(input, memory, first=None, input_shape="*")
30
+ env_action = MinecraftWrapper.agent_action_to_env(agent_action)
31
+
32
+ # --------------------- the output --------------------- #
33
+ # agent_action = {'buttons': tensor([1], device='cuda:0'), 'camera': tensor([54], device='cuda:0')}
34
+ # env_action = {'attack': array(0), 'back': array(0), 'forward': array(0), 'jump': array(0), 'left': array(0), 'right': array(0), 'sneak': array(0), 'sprint': array(0), 'use': array(0), 'drop': array(0), 'inventory': array(0), 'hotbar.1': array(0), 'hotbar.2': array(0), 'hotbar.3': array(0), 'hotbar.4': array(0), 'hotbar.5': array(0), 'hotbar.6': array(0), 'hotbar.7': array(0), 'hotbar.8': array(0), 'hotbar.9': array(0), 'camera': array([-0.61539427, 10. ])}
35
+ ```