zackli4ai
/

llama3.2-1b-train-aware-quantization

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

llama3.2-1b-train-aware-quantization

1 contributor

History: 7 commits

zackli4ai's picture

Upload model-8da4w-qat-module-swap.pt with huggingface_hub

d79bca3 verified 10 days ago

.gitattributes

1.52 kB

initial commit 10 days ago
LICENSE.txt

7.71 kB

Upload LICENSE.txt with huggingface_hub 10 days ago
README.md

41.7 kB

Upload README.md with huggingface_hub 10 days ago
USE_POLICY.md

6.02 kB

Upload USE_POLICY.md with huggingface_hub 10 days ago
config.json

877 Bytes

Upload config.json with huggingface_hub 10 days ago
generation_config.json

189 Bytes

Upload generation_config.json with huggingface_hub 10 days ago
model-8da4w-qat-module-swap.pt
Detected Pickle imports (16)
- "torchao.quantization.quant_api._int8_asymm_per_token_quant",
- "torch.int8",
- "torch._tensor._rebuild_from_type_v2",
- "torchao.dtypes.affine_quantized_tensor.PlainAQTLayout",
- "torch.CharStorage",
- "torchao.quantization.linear_activation_quantized_tensor.LinearActivationQuantizedTensor",
- "torch._utils._rebuild_tensor_v2",
- "torchao.quantization.quant_primitives.ZeroPointDomain",
- "torch.device",
- "torchao.dtypes.affine_quantized_tensor.AffineQuantizedTensor",
- "torchao.dtypes.utils.PlainLayoutType",
- "collections.OrderedDict",
- "torch.serialization._get_layout",
- "torch.bfloat16",
- "torch.BFloat16Storage",
- "torch._utils._rebuild_wrapper_subclass"
How to fix it?
1.51 GB
LFS

Upload model-8da4w-qat-module-swap.pt with huggingface_hub 10 days ago