Understand the NVIDIA B300, the HGX B300 platform, and the BIZON X9000 G5. We analyze the video featuring a Kimi K3 and vLLM benchmark for up to 128 users, along with memory, networking, power, and evaluation criteria.
There is a project almost everyone depends on without knowing it. When you run a local model with Ollama, LM Studio, Jan, or that AI app on your phone, chances are that underneath it all, llama.cpp is doing the calculations — an inference engine written in pure C++, … Read more
If you have run a local LLM with Ollama or llama.cpp, you know it works very well — for one or two people. When demand becomes an API that must handle dozens of simultaneous requests, the situation changes: the GPU sits idle waiting, memory becomes fragmented, and throughput plummets. That is precisely the problem … Read more
AI agents that live in your terminal have become a commodity — but an agent that learns from its own experience is still rare. Hermes Agent, from Nous Research, is exactly that: an open source agent (MIT, written in Python) with a built-in learning loop — it creates skills from the tasks it … Read more
Around 2013, Google ran the calculation that alarmed Mountain View: if every Android user used Search’s voice commands for three minutes a day on x86 CPUs, the company would have to double its data centers just for inference. Consumer GPUs were still adapted gaming cards. The answer was … Read more
In January 2025, DeepSeek released R1 (and, weeks earlier, V3), competing with o1 and GPT-4o in reasoning — at a fraction of the training and inference cost. The weights were released openly. The trick is not a dataset trick: it is a rewrite of the Transformer’s core, Multi-Head Latent Attention … Read more
VRAM determines the model. Dense Qwen 27B–32B, quantized Llama 70B, Mixtral 8×7B — these require 32 to 64 GB, not the 24 GB of a 4090. Engineer Donato Capitella, with Bargain Hardware, built a refurbished Supermicro with four data center GPUs (64 GB total) for around £2,000 and benchmarked V100, P100 … Read more
Programming is: data + rules → answer. Pricing a house, recognizing a face, understanding Portuguese — manually written rules cannot handle it all. Machine Learning reverses the equation: data + example answers → the model (the rules). This article follows KodeKloud’s course How Machine Learning Works, End to End: from spreadsheets to ChatGPT, the cycle … Read more
Running AI models locally is no longer just a laboratory activity: with Ollama, you download and run LLMs on your own Linux machine without sending data to the cloud or paying per token. This post explains where the project came from — its origin explains much of its design — and shows how … Read more
