The Intel Arc Pro B70’s 32 GB of VRAM makes it an interesting local-inference card. I tested it with vLLM’s XPU backend and a quantized Qwen3.8-27B derivative. The result: roughly 70 output tokens per second for prose, 94 for code, and a working 262,144-token context window—with some important caveats.
The setup
The model was noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound, an OrcaRouter derivative, served under the API alias qwen3.8:27b. These results describe that specific checkpoint and configuration.
The stack used Linux, a pinned vLLM XPU development image, and matching Intel compute runtimes on the host and inside the container. The fast path also required MTP compatibility patches and INT4 draft-weight overlays. This wasn’t a stock-container, one-command installation.
The important settings:
- INT4 model weights, FP16 compute, and FP8 KV cache.
- Multi-token prediction (MTP) with four speculative tokens.
- A fixed 10 GiB KV-cache budget and 4,096-token prefill batches.
- One active request, with additional requests queued.
- Prefix caching and asynchronous scheduling disabled.
MTP made the biggest difference. Without it, a lone request managed about 33 tok/s. With the optimized draft path, short prose/code tests reached 70/94 tok/s. Those are median end-to-end output rates over three repetitions with 512-token outputs and thinking disabled—not isolated decode speed or a model-quality score.
The context window actually fits
Rather than trusting the configured limit, I sent a 261,972-token prompt containing three values distributed through the document. The model retrieved all three and stopped normally.
That request took eight minutes and seventeen seconds. Sampled free VRAM bottomed out around 1.49 GiB during the native-context tests.
So yes, the context fits. No, a quarter-million-token prompt is not interactive. Input, output, and chat/tool overhead also share the same total budget. This demonstrates narrow retrieval capability, not reliable reasoning over arbitrary documents of that size.
Concurrency is the catch
In this installed XPU build, enabling two or four active requests crashed the engine when speculative decoding overlapped another request’s prefill. Disabling MTP made parallel processing work:
- Two short streams: about 32 tok/s each, 64 total.
- Four short streams: about 30 tok/s each, 120 total.
Four 512-token replies finished in roughly 17 seconds, versus 29 seconds for prose or 22 for code when queued with MTP. But two parallel replies were slower overall than simply queueing them on the fast configuration. With 8K-token prompts, the four-request advantage shrank to roughly 29 versus 26 seconds.
These were controlled, identical-prompt bursts—not a general multi-user capacity test.
My takeaway
For one local assistant, I would keep MTP enabled and concurrency at one. The B70 can deliver useful 27B-model performance and generous context on a single card, but the software stack still needs care.
The best configuration here wasn’t the one handling the most simultaneous requests. It was the one making the usual request fast.