Hugging Face's logo

hyzhou
/
MedVersa_Internal

Transformers
PyTorch
medomni
Inference Endpoints
Model card Files Community
1
hyzhou commited on
Commit
faad385
·
1 Parent(s): e1c7564

update

Browse files
Files changed (36) hide show
  1. __pycache__/utils.cpython-39.pyc +0 -0
  2. medomni/__pycache__/__init__.cpython-39.pyc +0 -0
  3. medomni/common/__pycache__/__init__.cpython-39.pyc +0 -0
  4. medomni/common/__pycache__/config.cpython-39.pyc +0 -0
  5. medomni/common/__pycache__/dist_utils.cpython-39.pyc +0 -0
  6. medomni/common/__pycache__/logger.cpython-39.pyc +0 -0
  7. medomni/common/__pycache__/registry.cpython-39.pyc +0 -0
  8. medomni/common/__pycache__/utils.cpython-39.pyc +0 -0
  9. medomni/datasets/__pycache__/__init__.cpython-39.pyc +0 -0
  10. medomni/datasets/__pycache__/data_utils.cpython-39.pyc +0 -0
  11. medomni/datasets/builders/__pycache__/__init__.cpython-39.pyc +0 -0
  12. medomni/datasets/builders/__pycache__/base_dataset_builder.cpython-39.pyc +0 -0
  13. medomni/datasets/builders/__pycache__/image_text_pair_builder.cpython-39.pyc +0 -0
  14. medomni/datasets/datasets/__pycache__/__init__.cpython-39.pyc +0 -0
  15. medomni/datasets/datasets/__pycache__/base_dataset.cpython-39.pyc +0 -0
  16. medomni/datasets/datasets/__pycache__/caption_datasets.cpython-39.pyc +0 -0
  17. medomni/datasets/datasets/__pycache__/cc_sbu_dataset.cpython-39.pyc +0 -0
  18. medomni/datasets/datasets/__pycache__/laion_dataset.cpython-39.pyc +0 -0
  19. medomni/datasets/datasets/__pycache__/med_dataset.cpython-39.pyc +0 -0
  20. medomni/datasets/datasets/__pycache__/medcaption_datasets.cpython-39.pyc +0 -0
  21. medomni/models/__pycache__/Qformer.cpython-39.pyc +0 -0
  22. medomni/models/__pycache__/UNet.cpython-39.pyc +0 -0
  23. medomni/models/__pycache__/__init__.cpython-39.pyc +0 -0
  24. medomni/models/__pycache__/base_model.cpython-39.pyc +0 -0
  25. medomni/models/__pycache__/blip2.cpython-39.pyc +0 -0
  26. medomni/models/__pycache__/eva_vit.cpython-39.pyc +0 -0
  27. medomni/models/__pycache__/medomni.cpython-39.pyc +0 -0
  28. medomni/models/__pycache__/modeling_llama.cpython-39.pyc +0 -0
  29. medomni/models/medomni.py +10 -2
  30. medomni/processors/__pycache__/__init__.cpython-39.pyc +0 -0
  31. medomni/processors/__pycache__/base_processor.cpython-39.pyc +0 -0
  32. medomni/processors/__pycache__/blip_processors.cpython-39.pyc +0 -0
  33. medomni/processors/__pycache__/randaugment.cpython-39.pyc +0 -0
  34. medomni/tasks/__pycache__/__init__.cpython-39.pyc +0 -0
  35. medomni/tasks/__pycache__/base_task.cpython-39.pyc +0 -0
  36. medomni/tasks/__pycache__/image_text_pretrain.cpython-39.pyc +0 -0
__pycache__/utils.cpython-39.pyc CHANGED
Binary files a/__pycache__/utils.cpython-39.pyc and b/__pycache__/utils.cpython-39.pyc differ
 
medomni/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/__pycache__/__init__.cpython-39.pyc and b/medomni/__pycache__/__init__.cpython-39.pyc differ
 
medomni/common/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/common/__pycache__/__init__.cpython-39.pyc and b/medomni/common/__pycache__/__init__.cpython-39.pyc differ
 
medomni/common/__pycache__/config.cpython-39.pyc CHANGED
Binary files a/medomni/common/__pycache__/config.cpython-39.pyc and b/medomni/common/__pycache__/config.cpython-39.pyc differ
 
medomni/common/__pycache__/dist_utils.cpython-39.pyc CHANGED
Binary files a/medomni/common/__pycache__/dist_utils.cpython-39.pyc and b/medomni/common/__pycache__/dist_utils.cpython-39.pyc differ
 
medomni/common/__pycache__/logger.cpython-39.pyc CHANGED
Binary files a/medomni/common/__pycache__/logger.cpython-39.pyc and b/medomni/common/__pycache__/logger.cpython-39.pyc differ
 
medomni/common/__pycache__/registry.cpython-39.pyc CHANGED
Binary files a/medomni/common/__pycache__/registry.cpython-39.pyc and b/medomni/common/__pycache__/registry.cpython-39.pyc differ
 
medomni/common/__pycache__/utils.cpython-39.pyc CHANGED
Binary files a/medomni/common/__pycache__/utils.cpython-39.pyc and b/medomni/common/__pycache__/utils.cpython-39.pyc differ
 
medomni/datasets/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/__pycache__/__init__.cpython-39.pyc and b/medomni/datasets/__pycache__/__init__.cpython-39.pyc differ
 
medomni/datasets/__pycache__/data_utils.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/__pycache__/data_utils.cpython-39.pyc and b/medomni/datasets/__pycache__/data_utils.cpython-39.pyc differ
 
medomni/datasets/builders/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/builders/__pycache__/__init__.cpython-39.pyc and b/medomni/datasets/builders/__pycache__/__init__.cpython-39.pyc differ
 
medomni/datasets/builders/__pycache__/base_dataset_builder.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/builders/__pycache__/base_dataset_builder.cpython-39.pyc and b/medomni/datasets/builders/__pycache__/base_dataset_builder.cpython-39.pyc differ
 
medomni/datasets/builders/__pycache__/image_text_pair_builder.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/builders/__pycache__/image_text_pair_builder.cpython-39.pyc and b/medomni/datasets/builders/__pycache__/image_text_pair_builder.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/__init__.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/__init__.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/base_dataset.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/base_dataset.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/base_dataset.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/caption_datasets.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/caption_datasets.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/caption_datasets.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/cc_sbu_dataset.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/cc_sbu_dataset.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/cc_sbu_dataset.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/laion_dataset.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/laion_dataset.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/laion_dataset.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/med_dataset.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/med_dataset.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/med_dataset.cpython-39.pyc differ
 
medomni/datasets/datasets/__pycache__/medcaption_datasets.cpython-39.pyc CHANGED
Binary files a/medomni/datasets/datasets/__pycache__/medcaption_datasets.cpython-39.pyc and b/medomni/datasets/datasets/__pycache__/medcaption_datasets.cpython-39.pyc differ
 
medomni/models/__pycache__/Qformer.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/Qformer.cpython-39.pyc and b/medomni/models/__pycache__/Qformer.cpython-39.pyc differ
 
medomni/models/__pycache__/UNet.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/UNet.cpython-39.pyc and b/medomni/models/__pycache__/UNet.cpython-39.pyc differ
 
medomni/models/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/__init__.cpython-39.pyc and b/medomni/models/__pycache__/__init__.cpython-39.pyc differ
 
medomni/models/__pycache__/base_model.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/base_model.cpython-39.pyc and b/medomni/models/__pycache__/base_model.cpython-39.pyc differ
 
medomni/models/__pycache__/blip2.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/blip2.cpython-39.pyc and b/medomni/models/__pycache__/blip2.cpython-39.pyc differ
 
medomni/models/__pycache__/eva_vit.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/eva_vit.cpython-39.pyc and b/medomni/models/__pycache__/eva_vit.cpython-39.pyc differ
 
medomni/models/__pycache__/medomni.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/medomni.cpython-39.pyc and b/medomni/models/__pycache__/medomni.cpython-39.pyc differ
 
medomni/models/__pycache__/modeling_llama.cpython-39.pyc CHANGED
Binary files a/medomni/models/__pycache__/modeling_llama.cpython-39.pyc and b/medomni/models/__pycache__/modeling_llama.cpython-39.pyc differ
 
medomni/models/medomni.py CHANGED
@@ -186,12 +186,20 @@ class MedOmni(Blip2Base, PyTorchModelHubMixin):
186
  self.llama_tokenizer.add_tokens("<3DSEG>")
187
  # self.llama_tokenizer.add_tokens("<2DPOINT>")
188
  self.llama_tokenizer.add_tokens("<N/A>")
 
 
189
  self.det_token_idx = self.llama_tokenizer("<DET>", add_special_tokens=False).input_ids[0]
190
  self.seg_token_idx_2d = self.llama_tokenizer("<2DSEG>", add_special_tokens=False).input_ids[0]
191
  self.seg_token_idx_3d = self.llama_tokenizer("<3DSEG>", add_special_tokens=False).input_ids[0]
192
- # self.point_token_idx_2d = self.llama_tokenizer("<2DPOINT>", add_special_tokens=False).input_ids[0]
193
  self.na_token_idx = self.llama_tokenizer("<N/A>", add_special_tokens=False).input_ids[0]
194
  self.llama_tokenizer.pad_token = 0
 
 
 
 
 
 
 
195
 
196
  if self.low_resource:
197
  self.llama_model = LlamaForCausalLM.from_pretrained(
@@ -391,7 +399,7 @@ class MedOmni(Blip2Base, PyTorchModelHubMixin):
391
  max_length=self.max_txt_len,
392
  add_special_tokens=False
393
  ).to(image.device)
394
-
395
  targets = to_regress_tokens.input_ids.masked_fill(
396
  to_regress_tokens.input_ids == self.llama_tokenizer.pad_token_id, -100
397
  )
 
186
  self.llama_tokenizer.add_tokens("<3DSEG>")
187
  # self.llama_tokenizer.add_tokens("<2DPOINT>")
188
  self.llama_tokenizer.add_tokens("<N/A>")
189
+
190
+ ### transformers == 4.28
191
  self.det_token_idx = self.llama_tokenizer("<DET>", add_special_tokens=False).input_ids[0]
192
  self.seg_token_idx_2d = self.llama_tokenizer("<2DSEG>", add_special_tokens=False).input_ids[0]
193
  self.seg_token_idx_3d = self.llama_tokenizer("<3DSEG>", add_special_tokens=False).input_ids[0]
 
194
  self.na_token_idx = self.llama_tokenizer("<N/A>", add_special_tokens=False).input_ids[0]
195
  self.llama_tokenizer.pad_token = 0
196
+
197
+ ### transformers == 4.41 (not recommended)
198
+ # self.det_token_idx = 32001
199
+ # self.seg_token_idx_2d = 32002
200
+ # self.seg_token_idx_3d = 32003
201
+ # self.na_token_idx = 32004
202
+ # self.llama_tokenizer.pad_token_id = 29900
203
 
204
  if self.low_resource:
205
  self.llama_model = LlamaForCausalLM.from_pretrained(
 
399
  max_length=self.max_txt_len,
400
  add_special_tokens=False
401
  ).to(image.device)
402
+
403
  targets = to_regress_tokens.input_ids.masked_fill(
404
  to_regress_tokens.input_ids == self.llama_tokenizer.pad_token_id, -100
405
  )
medomni/processors/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/processors/__pycache__/__init__.cpython-39.pyc and b/medomni/processors/__pycache__/__init__.cpython-39.pyc differ
 
medomni/processors/__pycache__/base_processor.cpython-39.pyc CHANGED
Binary files a/medomni/processors/__pycache__/base_processor.cpython-39.pyc and b/medomni/processors/__pycache__/base_processor.cpython-39.pyc differ
 
medomni/processors/__pycache__/blip_processors.cpython-39.pyc CHANGED
Binary files a/medomni/processors/__pycache__/blip_processors.cpython-39.pyc and b/medomni/processors/__pycache__/blip_processors.cpython-39.pyc differ
 
medomni/processors/__pycache__/randaugment.cpython-39.pyc CHANGED
Binary files a/medomni/processors/__pycache__/randaugment.cpython-39.pyc and b/medomni/processors/__pycache__/randaugment.cpython-39.pyc differ
 
medomni/tasks/__pycache__/__init__.cpython-39.pyc CHANGED
Binary files a/medomni/tasks/__pycache__/__init__.cpython-39.pyc and b/medomni/tasks/__pycache__/__init__.cpython-39.pyc differ
 
medomni/tasks/__pycache__/base_task.cpython-39.pyc CHANGED
Binary files a/medomni/tasks/__pycache__/base_task.cpython-39.pyc and b/medomni/tasks/__pycache__/base_task.cpython-39.pyc differ
 
medomni/tasks/__pycache__/image_text_pretrain.cpython-39.pyc CHANGED
Binary files a/medomni/tasks/__pycache__/image_text_pretrain.cpython-39.pyc and b/medomni/tasks/__pycache__/image_text_pretrain.cpython-39.pyc differ