Impressive. Solved the 24 Game And Good At Logic.

#4
by deleted - opened
deleted
β€’
edited Mar 26

Other Mixtrals, including Mixtral Instruct, keep breaking the rules, don't even get 24, yet still often claim success, and so on, such as using numbers that weren't even provided. However, this one made 3 failed attempts, returned the correct result, admitted failure, then came to a correct solution equating to 24. It also did very well on logic problems.

It still can't be used as a general purpose LLM because it has some linguistic blind spots. For example, when asked multiple times with different wording to change the words in a poem without changing its meaning it just keeps repeating the poem word for word. Then apologizes, clarifies what task is being asked of it, claims it's about to do the task, then just repeats the poem word for word again.

Sign up or log in to comment