Windows coversion almost successful
I saw the note about Windows so I thought I should give some details.
I followed all the steps in an Anaconda environment and it's almost successful except the .json files. Those are all broken and copying them from the HF folder does not help.
MD5 checks are OK on the other files.
Hey,
As mentioned here: https://huggingface.co/OpenAssistant/oasst-sft-6-llama-30b-xor/discussions/1#64444613c63001ae6355dda7, just convert the line ending CRLF to LF using a notepad and you should be good with the checksum :)
Thanks
I actually took the ones from the HF version before the XOR and modified a few values to match the new model. Don't ask me what as I didn't take notes :)
Off to add some more VRAM as it OOMs with 72GB.
Thanks
I actually took the ones from the HF version before the XOR and modified a few values to match the new model. Don't ask me what as I didn't take notes :)
Off to add some more VRAM as it OOMs with 72GB.
Before you add more VRAM, try loading the final model in 8-bit, should take no more than 35GB VRAM.