the model loves to generate: // Your code here

#3
by Meital - opened

Hi, thanks for training the model!

Have you noticed that the model really likes to complete code with:
// Your code here
or its python version of
# Your code here

I try to run some evals on it, and in many cases, it just completes my code with this placeholder.
I know it is a chat model, and it works excellent for conversions, but I wanted to take it for humaneval spin to check its programming abilities, and I get for python much less than your 70 because it doesn't want to generate my the code in many cases :(

any idea?

It works good for code or text completion.

Here is my system prompt:
You are a text completion machine.
Finish the following text starting from the last symbol:

Test case:
In JavaScript, the .find() function is used on arrays to locate the first element in the array that satisfies the provided testing function.
In Python, a similar functionality can be achieved using a list comprehension or the filter() function.
Here's how you can achieve the equivalent of .find() in JavaScript using Python:
Using list comprehen

Answer:
sion:

                def find_in_python(lst, func):
                    for element in lst:
                        if func(element):
                            return element
                    return None  # If no matching element is found
                    # model answer....

....

Sign up or log in to comment