Temporal fix for "DefaultCPUAIIocator: not enough memory: you tried to allocate 13107200 bytes" error
As many other people I got this error too. Im runnig an rtx 3060 12gb + Ryzen 9 12 cores + 16gb ram
I tried every single parameter in webui.bat but nothing helped. Today I figured out that just setting the virtual memory of the HDD where you have oobabooga installed fixes the issue. You need to check the "System managed size" box and thats all.
It fixed the error for me and now Im able to start the webui. It takes around 127-200 seconds to load the model itself.
The only issue now is that, even tho the responses are very fast (it takes around 4-6 seconds to generate a whole phrase) they dont make any sense.
For example if I ask the AI to give me information about Coca-Cola brand, it says stuff like "we are sorry that you are having issues, please call this number (800)XXX-XXX-XXX -Alex"
It doesent matter what GUI model you choose, it will most likely ignore what you asked and say stuff on its own.
I managed to fix that too by using --pre_layer 35 in the webui.bat but that only makes the responses extremely slow and they dont really make any sense either
If someone manages to make this thing work it will be higly appreciated
Also running a 3060 12gb here with less cpu spec. Thanks to your HDD Virtual memory tip I got it to load ( although it sort of happened unexpectedly, poss after a reboot ).
It's however just spitting out complete gibberish at me and cant move any further forwards.
Also running a 3060 12gb here with less cpu spec. Thanks to your HDD Virtual memory tip I got it to load ( although it sort of happened unexpectedly, poss after a reboot ).
It's however just spitting out complete gibberish at me and cant move any further forwards.
You can try the --pre_layer parameter, but as I mentioned earlier, this will only cause the responses to be slow and it looks like the AI dont always listen to what you say and star saying stuff on its own.
I have seen videos of people just launching the model in a couple seconds without messing with the virtual memory stuff and the AI give fast and coherent responeses...
Today I upgraded to 32gb ram and it still doesent work as it should, Im starting to think the problem could be my AMD CPU, I have no idea how to make this thing work as it should
Also running a 3060 12gb here with less cpu spec. Thanks to your HDD Virtual memory tip I got it to load ( although it sort of happened unexpectedly, poss after a reboot ).
It's however just spitting out complete gibberish at me and cant move any further forwards.You can try the --pre_layer parameter, but as I mentioned earlier, this will only cause the responses to be slow and it looks like the AI dont always listen to what you say and star saying stuff on its own.
I have seen videos of people just launching the model in a couple seconds without messing with the virtual memory stuff and the AI give fast and coherent responeses...
Today I upgraded to 32gb ram and it still doesent work as it should, Im starting to think the problem could be my AMD CPU, I have no idea how to make this thing work as it should
Ended up deleting the folder and reinstalling using this tutorial - except I transferred the model over: IT WORKS!!
( also add the recommended ARGs to the batch file )
Also running a 3060 12gb here with less cpu spec. Thanks to your HDD Virtual memory tip I got it to load ( although it sort of happened unexpectedly, poss after a reboot ).
It's however just spitting out complete gibberish at me and cant move any further forwards.You can try the --pre_layer parameter, but as I mentioned earlier, this will only cause the responses to be slow and it looks like the AI dont always listen to what you say and star saying stuff on its own.
I have seen videos of people just launching the model in a couple seconds without messing with the virtual memory stuff and the AI give fast and coherent responeses...
Today I upgraded to 32gb ram and it still doesent work as it should, Im starting to think the problem could be my AMD CPU, I have no idea how to make this thing work as it shouldEnded up deleting the folder and reinstalling using this tutorial - except I transferred the model over: IT WORKS!!
( also add the recommended ARGs to the batch file )
wow man It really works now, I still have no idea what happened before... and I think I upgraded to 32gb ram for nothing but it is what it is I guess