Effective AI-assisted development using only local resources
Could you elaborate on what benchmarks you meant when you said "it performs higher on some benchmarks we care about"?
Sure. In this example, like quite a few NVFP4 quantization examples for small / micro / nano language models, scores for LiveCodeBench, SCICODE, and AIME 2024 *went up* after quantization.
Could you elaborate on what benchmarks you meant when you said "it performs higher on some benchmarks we care about"?
Sure. In this example, like quite a few NVFP4 quantization examples for small / micro / nano language models, scores for LiveCodeBench, SCICODE, and AIME 2024 *went up* after quantization.