Fix for using local RadPhi2

#1
by MoritzLangenberg - opened

Hello,

when I try to run the code demo of GREEN with GREEN-RadPhi2 saved in a local folder:

from green_score import GREEN

model = GREEN(
    model_id_or_path="/path/to/GREEN-RadPhi2",
    do_sample=False,  # should be always False
    batch_size=16,
    return_0_if_no_green_score=True,
    cuda=True
)

refs = [
    "Interstitial opacities without changes."
]
hyps = [
    "Interstitial opacities at bases without changes."
]

mean_green, greens, explanations = model(refs=refs, hyps=hyps)
print("Mean reward for the given examples is: ", mean_green)
print("Array of rewards for the given examples is: ", greens)
print(explanations[0]) # LLM output for first pair

I receive the following error message:

Could not locate the configuration_phi.py inside StanfordAIMI/RadPhi-2.
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/home/user/.local/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/StanfordAIMI/RadPhi-2/resolve/main/configuration_phi.py

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 402, in cached_file
    resolved_file = hf_hub_download(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1325, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1823, in _raise_on_head_call_error
    raise head_call_error
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata
    r = _request_wrapper(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper
    response = _request_wrapper(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper
    hf_raise_for_status(response)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 352, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-667179ca-4fbfd09a57c5c8f7747353ee;d2972de1-2308-4818-9c5f-4afb0728feed)

Repository Not Found for url: https://huggingface.co/StanfordAIMI/RadPhi-2/resolve/main/configuration_phi.py.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/path/M3D/green_phi2_script.py", line 6, in <module>
    model = GREEN(
  File "/home/user/.local/lib/python3.10/site-packages/green_score/green.py", line 217, in __init__
    self.model = GREENModel(cuda, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/green_score/green.py", line 42, in __init__
    self.model = AutoModelForCausalLM.from_pretrained(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 524, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 963, in from_pretrained
    config_class = get_class_from_dynamic_module(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 502, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 306, in get_cached_module_file
    resolved_module_file = cached_file(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 425, in cached_file
    raise EnvironmentError(
OSError: StanfordAIMI/RadPhi-2 is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`  

A fix for this is commenting out line 44 trust_remote_code=True in /home/user/.local/lib/python3.10/site-packages/green_score/green.py. Still, we are wondering why this OSError occurs. We have model_id_or_path="/path/to/GREEN-RadPhi2", i.e. have RadPhi2 saved in a local folder, but somehow huggingface tries to get it from the web.

Best regards,
Moritz

Stanford AIMI org
edited Jun 18

Hi Moritz,

thanks for posting. I am trying to reproduce your error.
The following code works for me:
transformers==4.41.2
torch==2.3.0+cu121

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("StanfordAIMI/GREEN-RadPhi2")
model.save_pretrained('green')

tokenizer = AutoTokenizer.from_pretrained("StanfordAIMI/GREEN-RadPhi2")
tokenizer.save_pretrained('green')


model = GREEN(
    model_id_or_path="green",
    do_sample=False,  # should be always False
    batch_size=16,
    return_0_if_no_green_score=True,
    cuda=True
)

refs = [
    "Interstitial opacities without changes."
]
hyps = [
    "Interstitial opacities at bases without changes."
]

mean_green, greens, explanations = model(refs=refs, hyps=hyps)
print("Mean reward for the given examples is: ", mean_green)
print("Array of rewards for the given examples is: ", greens)
print(explanations[0]) # LLM output for first pair

Please let me know if you still encounter the issue.

Best,
Sophie

Thank you for your help. I tried your code with transformers==4.41.2 and torch==2.3.0+cu121 and also using the setup.py but am facing the same error as above.

Stanford AIMI org

The error might be related to your huggingface setup. Would you mind try running huggingface-cli login and make sure to pass a token again?

Stanford AIMI org

Hello,

The error was due because our GREEN-RadPhi2 model had a dependency on our RadPhi2 model that is private.
I change the config in that push:
https://huggingface.co/StanfordAIMI/GREEN-RadPhi2/commit/5ed02c254d8278b0e64cfea0e74ccc89a0b9bff0

can you please try again? If there is a new error, please report it here.

JB

This works, thank you very much!

Sign up or log in to comment