{"id":426,"date":"2026-09-05T22:11:32","date_gmt":"2026-09-06T01:11:32","guid":{"rendered":"https:\/\/www.linuxpro.com.br\/2026\/09\/servidor-ia-recondicionado-64gb-vram-v100-p100-mi25\/"},"modified":"2026-09-08T05:23:38","modified_gmt":"2026-09-08T08:23:38","slug":"servidor-ia-recondicionado-64gb-vram-v100-p100-mi25","status":"publish","type":"post","link":"https:\/\/www.linuxpro.com.br\/en\/2026\/09\/servidor-ia-recondicionado-64gb-vram-v100-p100-mi25\/","title":{"rendered":"Refurbished AI server with 64 GB of VRAM"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" alt=\"Mascote do LinuxPro instalando GPU no servidor de IA\" src=\"\/wp-content\/uploads\/2026\/09\/servidor-ia-v3.webp\" width=\"1052\" height=\"652\" \/><\/p>\n<p>VRAM determines the model. Dense Qwen 27B\u201332B, quantized Llama 70B, Mixtral 8\u00d77B \u2014 these need 32 to 64 GB, not the 24 GB of a 4090. Engineer <strong>Donato Capitella<\/strong>, with <strong>Bargain Hardware<\/strong>, built a refurbished Supermicro with four data-center GPUs (64 GB total) for about \u00a32.000 and benchmarked V100, P100, and MI25 in the same chassis. This article follows the video <a href=\"https:\/\/www.youtube.com\/watch?v=zp8j4vO-wz0\">Refurbished 64GB VRAM AI Server for Local AI<\/a>.<\/p>\n<p><!-- more --><\/p>\n<div style=\"position:relative;padding-bottom:56.25%;height:0;overflow:hidden;margin:1.5em 0\">\n<iframe src=\"https:\/\/www.youtube.com\/embed\/zp8j4vO-wz0\" title=\"Refurbished 64GB VRAM AI Server for Local AI \u2014 Donato Capitella\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen style=\"position:absolute;top:0;left:0;width:100%;height:100%;border:0\"><\/iframe>\n<\/div>\n<p>Anyone already running <a href=\"\/en\/2026\/09\/rodando-ia-local-no-linux-com-ollama\/\">Ollama<\/a> on a desktop hits the consumer card's ceiling. Here the bet is different: server chassis, passive card, llama.cpp compiled for the right architecture. Why the V100's Tensor Cores beat the P100 is covered in the post <a href=\"\/en\/2026\/09\/como-funcionam-placas-de-video\/\">How graphics cards work<\/a> and in the <a href=\"\/en\/2026\/09\/gpu-vs-tpu-como-funcionam-chips-ia\/\">GPU vs TPU<\/a>.<\/p>\n<h2>The chassis: 12\u00aa generation Supermicro<\/h2>\n<p>The video's foundation is an enterprise Supermicro server (4U\u20135U tower\/rack, CSE-747 family) designed for HPC, not to sit under a desk:<\/p>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">2\u00d7 Intel Xeon Gold 6150     36 n\u00facleos \/ 72 threads\n64 GB DDR4 ECC Registered   4\u00d7 16 GB @ 2666 MHz\n2\u00d7 SSD SATA 480 GB + NVMe U.2\n4\u00d7 PCIe 3.0 x16             full-height, dual-slot\n2\u00d7 10GbE na placa-m\u00e3e\nturbinas frontais           ar for\u00e7ado nas aletas\nfontes redundantes          1600\u20132000 W\n<\/code><\/pre>\n<p>Data-center Tesla and Instinct cards are <strong>passive<\/strong>: they have no onboard fan. The chassis blowers push air through the copper fins at thousands of RPM. A gaming desktop with 4\u00d7 V100 cooks the cards \u2014 and a home PSU cannot handle the peak.<\/p>\n<h2>V100 vs P100 vs MI25<\/h2>\n<p>Three sets of four 16 GB cards (64 GB total):<\/p>\n<table>\n<thead>\n<tr>\n<th><\/th>\n<th>Tesla V100 16 GB<\/th>\n<th>Tesla P100 16 GB<\/th>\n<th>Instinct MI25 16 GB<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Architecture<\/td>\n<td>Volta 12 nm (sm_70)<\/td>\n<td>Pascal 16 nm (sm_60)<\/td>\n<td>Vega 10 14 nm (gfx900)<\/td>\n<\/tr>\n<tr>\n<td>Memory<\/td>\n<td>16 GB HBM2, 900 GB\/s<\/td>\n<td>16 GB HBM2, 720\u2013900 GB\/s<\/td>\n<td>16 GB HBM2, 484 GB\/s<\/td>\n<\/tr>\n<tr>\n<td>Tensor Cores<\/td>\n<td>yes (1\u00aa generation, 640)<\/td>\n<td>no<\/td>\n<td>no<\/td>\n<\/tr>\n<tr>\n<td>Precisions<\/td>\n<td>FP16\/32\/64, INT8<\/td>\n<td>FP16\/32\/64<\/td>\n<td>FP16\/32<\/td>\n<\/tr>\n<tr>\n<td>BF16 \/ FP8<\/td>\n<td>not native<\/td>\n<td>not native<\/td>\n<td>not native<\/td>\n<\/tr>\n<tr>\n<td>Stack<\/td>\n<td>CUDA \/ llama.cpp<\/td>\n<td>CUDA \/ llama.cpp<\/td>\n<td>ROCm \/ Vulkan \/ llama.cpp<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The dividing line is the Tensor Core. LLMs are MatMul; the V100 performs batched matrix operations. P100 and MI25 take the same path through CUDA cores \/ Vega SIMD \u2014 slower, cheaper. None of the three supports native BF16: the runtime converts to FP16.<\/p>\n<p>A 2026 detail the video already anticipates in practice: <strong>CUDA 13 dropped sm_70<\/strong>. V100 is the last architecture with first-class support in CUDA 12.x. Donato's containers pin <strong>CUDA 12.6.3<\/strong> and compile llama.cpp with <code data-no-translation=\"\">-DCMAKE_CUDA_ARCHITECTURES=70<\/code>. Toolkit 13 on the host = a binary that generates no code for the card.<\/p>\n<h2>Software: llama.cpp, not vLLM<\/h2>\n<p>Donato publishes toolboxes (Toolbx\/Podman) with llama.cpp already compiled for each die:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/kyuz0\/nvidia-v100-ai-toolboxes\">nvidia-v100-ai-toolboxes<\/a> \u2014 CUDA 12.6.3, sm_70<\/li>\n<li><a href=\"https:\/\/github.com\/kyuz0\/nvidia-p100-ai-toolboxes\">nvidia-p100-ai-toolboxes<\/a> \u2014 sm_60<\/li>\n<li><a href=\"https:\/\/github.com\/kyuz0\/mi25-gfx900-ai-toolboxes\">mi25-gfx900-ai-toolboxes<\/a> \u2014 gfx900, ROCm\/Vulkan<\/li>\n<\/ul>\n<pre data-no-translation=\"\"><code class=\"language-bash\" data-no-translation=\"\"># V100: toolbox com llama.cpp CUDA sm_70\ntoolbox create -c llama-v100-cuda \n  --image docker.io\/kyuz0\/nvidia-v100-ai-toolboxes:latest\ntoolbox enter llama-v100-cuda\nllama-cli --list-devices   # precisa ver as 4\u00d7 16 GB\n<\/code><\/pre>\n<p>For serving, llama.cpp wins: mature GGUF (Q4_K_M, Q4_K_XL, Q8_0), automatic layer splitting across the 4 GPUs (<code data-no-translation=\"\">-ngl 99<\/code>), low overhead. vLLM is the production standard on Hopper\/Blackwell; on Volta\/Pascal it falls back to PyTorch SDPA, AWQ barely works, and loading over PCIe 3.0 hurts.<\/p>\n<pre data-no-translation=\"\"><code class=\"language-bash\" data-no-translation=\"\">llama-server \n  -m \/models\/qwen-27b-instruct-q4_k_xl.gguf \n  -ngl 99 \n  -c 32768 \n  -fa on \n  --host 0.0.0.0 --port 8080\n<\/code><\/pre>\n<p>Anyone already using the GGUF workflow without Ollama's daemon has <a href=\"\/en\/2026\/09\/llama-model-cli\/\">llama-model<\/a> in the same spirit: one binary, one server, no extra layer. The <a href=\"\/en\/2017\/08\/curso-de-docker-gratis\/\">Docker course<\/a> covers Podman\/Toolbx if the host does not have it yet.<\/p>\n<h2>Numbers: Qwen 27B Q4<\/h2>\n<p>Dense model, 4-bit quantization, the setup the video uses as a coding assistant. Donato's own numbers (llama.cpp, NGL=99, flash attention):<\/p>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">gera\u00e7\u00e3o (tokens\/s)     Qwen 27B Q4\n4\u00d7 V100 16 GB          ~31\u201334   (ainda ~28 com contexto 32k)\n4\u00d7 P100 16 GB          ~18\u201322\n4\u00d7 MI25 16 GB          ~14\u201317\n\nprompt processing V100\ncurto:  ~852 tok\/s\n32k:    ~620 tok\/s\n<\/code><\/pre>\n<p>31 tok\/s is interactive conversation. The P100's 20 tok\/s serves background agents. MI25 loads the model at the lowest entry price \u2014 bandwidth of 484 GB\/s and no matrix accelerator take their toll. Large MoE (Qwen 122B-A10B Q3 in the toolboxes README) fits in 64 GB; throughput drops, VRAM is what matters.<\/p>\n<h2>Everyday pain points<\/h2>\n<ul>\n<li><strong>Noise: 60\u201375 dB.<\/strong> Four passive cards at 250\u2013300 W each. Not for a bedroom or under a desk. Rack, basement, garage, server room.<\/li>\n<li><strong>PCIe 3.0.<\/strong> ~16 GB\/s per x16 slot. Loading a 30\u201350 GB GGUF: 1 to 3 minutes. V100 NVLink exists, but only in specific chassis\/SXM setups \u2014 the typical Bargain Hardware tower uses separate PCIe cards.<\/li>\n<li><strong>BF16 becomes FP16.<\/strong> A modern model pretrained in bfloat16 loses some exponent range. Well-calibrated quantization (Unsloth Q4_K_XL etc.) reduces the pain.<\/li>\n<li><strong>Power.<\/strong> 4\u00d7 GPU + 2\u00d7 Xeon 150 W + blowers: 800\u20131.200 W continuously. A suitable circuit and redundant PSU are not optional.<\/li>\n<li><strong>CUDA 13.<\/strong> Do not just \u201cupdate the toolkit and call it done\u201d. Stay on the toolboxes' 12.6, or the binary ignores the V100.<\/li>\n<\/ul>\n<h2>Is it worth it in 2026?<\/h2>\n<p>If the goal is <strong>64 GB of local VRAM<\/strong> for dense 27B\u201332B or MoE, and the budget is \u00a32.000\u20133.000, <strong>4\u00d7 V100<\/strong> still delivers the best return of the group: real Tensor Cores, stable llama.cpp, a ready-built machine with an industrial power supply. P100 is the middle ground. MI25 is the AMD entry point, with bandwidth limitations in mind.<\/p>\n<p>It is not quiet, not PCIe 5.0, not native BF16. It is data sovereignty: the model never leaves the rack. A new 4090\/5090 still wins in tok\/s per watt and software (vLLM, FP8) \u2014 and still lacks 64 GB.<\/p>\n<h2>References<\/h2>\n<ul>\n<li><a href=\"https:\/\/www.youtube.com\/watch?v=zp8j4vO-wz0\">Donato Capitella \u2014 Refurbished 64GB VRAM AI Server<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/kyuz0\/nvidia-v100-ai-toolboxes\">V100 toolboxes (CUDA 12.6.3 \/ sm_70)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/kyuz0\/nvidia-p100-ai-toolboxes\">P100 toolboxes (sm_60)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/kyuz0\/mi25-gfx900-ai-toolboxes\">MI25 toolboxes (gfx900)<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/rodando-ia-local-no-linux-com-ollama\/\">Local AI on Linux with Ollama<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/llama-model-cli\/\">llama-model: GGUF and llama-server without the daemon<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/como-funcionam-placas-de-video\/\">How graphics cards work<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/gpu-vs-tpu-como-funcionam-chips-ia\/\">GPU vs TPU<\/a><\/li>\n<\/ul>\n<p>Four V100s in a noisy chassis do not become an H100. They become 64 GB of VRAM that a consumer card still does not offer \u2014 and a llama.cpp that, compiled for sm_70, chats at 30 tokens per second without sending the prompt to the cloud.<\/p>","protected":false},"excerpt":{"rendered":"<p>VRAM determines the model. Dense Qwen 27B\u201332B, quantized Llama 70B, Mixtral 8\u00d77B \u2014 these require 32 to 64 GB, not the 24 GB of a 4090. Engineer Donato Capitella, with Bargain Hardware, built a refurbished Supermicro with four data center GPUs (64 GB total) for around \u00a32,000 and benchmarked V100, P100 \u2026 <a title=\"Refurbished AI server with 64 GB of VRAM\" class=\"read-more\" href=\"https:\/\/www.linuxpro.com.br\/en\/2026\/09\/servidor-ia-recondicionado-64gb-vram-v100-p100-mi25\/\" aria-label=\"Read more about Servidor de IA recondicionado com 64 GB de VRAM\">Read more<\/a><\/p>","protected":false},"author":0,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[115,2,120],"tags":[153,134,116,128,152],"class_list":["post-426","post","type-post","status-publish","format-standard","hentry","category-ia","category-linux","category-servidores","tag-cuda","tag-gpu","tag-ia","tag-llm","tag-nvidia"],"_links":{"self":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts\/426","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/comments?post=426"}],"version-history":[{"count":2,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts\/426\/revisions"}],"predecessor-version":[{"id":1136,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts\/426\/revisions\/1136"}],"wp:attachment":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/media?parent=426"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/categories?post=426"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/tags?post=426"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}