Local AI hardware offers a compelling alternative to cloud-based solutions for running LLMs. By leveraging local edge computing, users can process and store their data on-site, ensuring that sensitive information remains secure and out of reach from third-party surveillance or data breaches.
An overview of different hardware, reviews and resources I put together while figuring out what to buy.
All with a focus on high quality and getting the best price-to-performance ratio.
Maybe it will help you too - if you like it, give it a thumbs up!
Also, make sure to support the sources - they did a great job!
| Name | Processor | Memory | Bandwidth | PCIe (v/l/m) | Ethernet | Fan Noise (idle/load) |
|---|---|---|---|---|---|---|
| Apple | ||||||
| Apple Mac Mini | M5 Pro | ? | ~348GB/s | - | 10G | 0db / ? |
| Apple Mac Studio | M4 Max | ≤128GB | 546GB/s | - | 10G | 0db / 44db |
| AMD | ||||||
| Beelink AI Mini | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | ? | ? | ? |
| Beelink GTR9 Pro | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | - | 10G (2x) | ? |
| Corsair AI Workstation 300 | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | ? | 2.5G | ? |
| Framework Desktop | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | 4.0 / x4 / x4 | 5G | 0dB / 48dB |
| GMKtec EVO-X2 | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | - | 2.5G | 27dB / 55dB |
| HP Z2 Mini G1a Workstation | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | - | 2.5G | 32dB / 67dB |
| Minisforum MS-S1 Max | Ryzen AI Max+ 395 | ≤128GB | 256GB/s | 4.0 / x4 / x16 | 10G (2x) | ? |
| NVIDIA | ||||||
| Dell Pro Max | Grace Blackell GB10 | ≤128GB | 273GB/s | - | 200G (2x) | ? |
| NVIDIA DGX Spark | Grace Blackell GB10 | ≤128GB | 273GB/s | - | 200G (2x) | ? |
unreleased
- Github / ggerganov (LLM, Benchmark)
- YouTube / Alex Ziskind (LLM, Benchmark)
- YouTube / Alex Ziskind (LLM, Benchmark)
- ServeTheHome (Review, Disassembling, Benchmark)
- YouTube / ServeTheHome (Review, Disassembling, LLM, Benchmark)
- YouTube / ETA Prime (Review, Disassembling, Benchmark)
- YouTube / Alex Ziskind (LLM, Benchmark)
unreleased
- YouTube / Alex Ziskind (LLM, Benchmark)
- YouTube / Jeff Geerling (LLM, Cluster, Benchmark)
- Notebookcheck (Review, Disassembling, Benchmark)
- ServeTheHome (Review, Disassembling, LLM, Benchmark)
- YouTube / Alex Ziskind (LLM, Benchmark)
- YouTube / ServeTheHome (Review, LLM, Benchmark)
- Notebookcheck (Review, Disassembling, Benchmark)
- YouTube / Donato Capitella (LLM, Benchmark)
- YouTube / @minisforumofficial (LLM, Benchmark)
- YouTube / ETA Prime (Review, Disassembling, Benchmark)
- YouTube / Jim's Garage (Review, Disassembling, LLM, Benchmark)
- YouTube / Level1Techs (Review, Disassembling, LLM, Benchmark)
- YouTube / NASCompares (Review, Disassembling, Benchmark)
- YouTube / mydealz (Review, Benchmark, Power)
- ServeTheHome (Review, Disassembling, LLM, Benchmark)
- YouTube / Digital Spaceport (LLM, Benchmark)
- YouTube / ServeTheHome (Review, LLM, Benchmark)