Text Generation
Transformers
Safetensors
llama
code
granite
conversational
Eval Results
text-generation-inference

Response is not good as expected

#3
by skumarai - opened

I have tried both 3b and 8b models and getting below response.
I copied same code from model card and tested it. can anyone help why i am getting bad response .

3b Model response

  • This IS expected if you are initializing LlamaForCausalLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  • This IS NOT expected if you are initializing LlamaForCausalLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
    Question:
    Write a code to find the maximum value in a list of numbers.

Answer:

def find_largest_even_odd_elements_list(arr):
    even_sum = even_list = odd_sum =0
        for i in range(0n2):
            if(arr[i)%2 ==0 and i<3:\
                even_list.append(arr[i])
            elif(arr[i)%2!=0 and odd_list.count(1)>=1)::\:\s.rindex(max_

8b model response
---------------------------
- This IS expected if you are initializing LlamaForCausalLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing LlamaForCausalLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Question:
Write a code to find the maximum value in a list of numbers.

Answer:
[jButton,jButton,njButton24[0jButton,0 of <<reement<,,,,,0 of
0":,,0,NuDreement,,,0 of
njButton,0 <<, <<,":": I3:":":000 of1 <<<<EFlags <<, <<, <<jButton",,<<,0 <<",1, of1 <<":":"<<_3 << <<,18.2 <<,1,<<

IBM Granite org

you need to install HF transformers from source for it to work correctly @skumarai .
Some changes required to run our models are currently not in the release and with the next release, a pip install should work.

Relevant PR: https://github.com/huggingface/transformers/pull/30031

git clone https://github.com/huggingface/transformers
cd transformers/
pip install ./
cd ..

Thank you for quick response @mayank-mishra .

sure, I will try to update transformers to reflect above mentioned PR changes and test the model.

IBM Granite org

@skumarai were you able to get good outputs?
can we close this issue?

Hi @mayank-mishra
yes, I got the good results.
we can close this.
Thanks.

skumarai changed discussion status to closed
IBM Granite org

awesome.

Sign up or log in to comment