uukuguy

/

speechless-llama2-13b

like 4

We use ties merge(https://github.com/cg123/ties-merge.git). The only problem we encountered was the inability to use the installed protobuf package. According to system prompts, we set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python and resolved this issue.
Here is the merge script we are using:

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python \
        python ties_merge.py \
                ${BASE_MODEL} \
                ${TARGET_MODEL} \
                --merge ${SUB_MODEL_1} \
                --merge ${SUB_MODEL_2} \
                --cuda

thank you