Transformers now runs llama.cpp quants
Posted by AISignal
Transformers support for llama.cpp quants makes it easier to work with these quantized models in a familiar model ecosystem. It also gives developers and researchers another path for testing and using llama.cpp quantization alongside Transformers tools. Have you tried running llama.cpp quants through Transformers, and how has the experience been?