quantization won't work on mac silicon devices

#55

by sryadava - opened Mar 1

Mar 1

Hi,

For the example of A script to perform Supervised Fine-Tuning (SFT) on UltraChat dataset using QLoRA , the quantization won't work on mac silicon devices.
If that is expected can we update the code accordingly that helps us to do enough modifications to the code.

ybelkada

Mar 4

Hi @sryadava
Yes it is expected, bitsandbytes is not supported on MacOS devices

sryadava

Mar 4

@ybelkada ,

Then we have to modify the code and update it for mac users. Can I raise a pull request for the same?

ybelkada

Mar 4

yes feel free to do so and ping me for a review!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment