BENCHMARKED: TensorFold made Qwen3.8 27B more than 3x faster on my 64GB M5 Pro Mac mini, on two versions in one day.
In the morning I tested 1.2.1. On its own, the model wrote about 17 tokens a second. I then switched on the DFlash2 drafter, a small helper that guesses the next few tokens so the big model only has to check them and it jumped to 56.
And later in the afternoon
@ashxhart
shipped 1.2.2. His release notes focused on other hardware, so I reran everything on my Mac mini.
On mine it runs within a few percent of 1.2.1, and the drafter still makes it 3x faster. With randomness off, the answer came out word for word the same as without the drafter.
- Chat, 3.26x faster on 1.2.1 and 3.35x on 1.2.2
- A 31K token prompt, 2.94x and 2.77x
- Code, 5.4x and 5.5x, up to 91.90 tokens a second
- 28.5 GiB peak memory on both
- New in 1.2.2, the first request doesn’t wait for the drafter to load
A massive thanks to
@ashxhart
and all the contributors behind 1.2.2.
The TensorFold challenge from
@YukonResearch
is open until Nov 6th if you want to try pushing it further on your own Mac.