4·
2 months agoIs that quantized? 4 bit Qwen 3.6 can get 22tps on a 1060.
C++ Programmer
- 0 Posts
- 5 Comments
Joined 2 years ago
Cake day: March 9th, 2025
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
2·4 months agoSaved. Awesome stuff 🥕
This is a fantastic post. I am very happy to have seen it tonight.
3·1 year agoCanada will add bike lines Not in Ontario while Doug Ford is in power (at least 4 more years).
What are you using to run the model? Llama.cpp will automatically split the model between your system ram and graphics card’s vram.
Qwen 3.6 is a mixture of experts model with only 3B parameters active at a time. Even without quantization your card could easily run that.