noisedestroyers/localgenai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

progress 235b

Browse Source
This commit is contained in:
noisedestroyers
2026-06-08 15:31:50 +01:00
parent a29793032d
commit de1635872f
25 changed files with 1598 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
pyinfra/framework/compose/kimi-linear.yml
View File

@@ -17,7 +17,7 @@
# Weights. Despite their HF name, cyankiwi's "AWQ" Kimi-Linear weights
# are actually `compressed-tensors` int4 group-quantized — see config.json.
# Download with:
# huggingface-cli download cyankiwi/Kimi-Linear-48B-A3B-Instruct-AWQ-4bit \
# hf download cyankiwi/Kimi-Linear-48B-A3B-Instruct-AWQ-4bit \
# --local-dir /models/moonshotai/Kimi-Linear-48B-A3B-Instruct-AWQ-4bit
# Size: ~35 GB on disk (4-bit). 8-bit variant is ~54 GB if quality drives
# us up later; both fit 128 GB unified comfortably.