{"id":415,"date":"2026-09-05T21:40:50","date_gmt":"2026-09-06T00:40:50","guid":{"rendered":"https:\/\/www.linuxpro.com.br\/2026\/09\/machine-learning-ponta-a-ponta\/"},"modified":"2026-09-08T05:23:38","modified_gmt":"2026-09-08T08:23:38","slug":"machine-learning-ponta-a-ponta","status":"publish","type":"post","link":"https:\/\/www.linuxpro.com.br\/en\/2026\/09\/machine-learning-ponta-a-ponta\/","title":{"rendered":"Machine Learning from end to end"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" alt=\"Mascote do LinuxPro alimentando dados na pipeline ML\" src=\"\/wp-content\/uploads\/2026\/09\/machine-learning-v3.webp\" width=\"1052\" height=\"652\" \/><\/p>\n<p class=\"translation-block\">Programming is: data + rules \u2192 answer. Pricing houses, recognizing faces, understanding Portuguese \u2014 manual rules cannot handle it. The <strong>Machine Learning<\/strong> approach reverses the equation: data + example answers \u2192 the model (the rules). This text follows the course <a href=\"https:\/\/www.youtube.com\/watch?v=du1B6633-hU\">How Machine Learning Works, End to End<\/a> by <strong>KodeKloud<\/strong>: from spreadsheets to ChatGPT, the cycle is the same.<\/p>\n<p><!-- more --><\/p>\n<div style=\"position:relative;padding-bottom:56.25%;height:0;overflow:hidden;margin:1.5em 0\">\n<iframe src=\"https:\/\/www.youtube.com\/embed\/du1B6633-hU\" title=\"How Machine Learning Works, End to End \u2014 KodeKloud\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen style=\"position:absolute;top:0;left:0;width:100%;height:100%;border:0\"><\/iframe>\n<\/div>\n<p>The neuron is covered in the <a href=\"\/en\/2026\/09\/o-que-e-uma-rede-neural\/\">3Blue1Brown post<\/a>; the DL map in the <a href=\"\/en\/2026\/09\/deep-learning-para-iniciantes\/\">freeCodeCamp crash course<\/a>. Here the thread is the entire pipeline.<\/p>\n<h2>Rules vs. examples<\/h2>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">cl\u00e1ssico:     Dados + Regras     \u2192  Respostas\nML:           Dados + Respostas  \u2192  Regras (o modelo)\n<\/code><\/pre>\n<p>CRUD and invoices fit in <code data-no-translation=\"\">if<\/code>. Real estate with 40 subjective variables does not. You feed the algorithm historical data; it finds the relationship and predicts what it has not seen yet.<\/p>\n<h2>Spreadsheet: features, labels, dirty data<\/h2>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">\u00e1rea m\u00b2 | quartos | bairro   | pre\u00e7o (y)\n120     | 3       | Centro   | 650.000\n 85     | 2       | Jardins  | 720.000\n 45     | 1       | Sub\u00farbio | 280.000\n<\/code><\/pre>\n<ul>\n<li><strong>X (features):<\/strong> what goes in \u2014 area, bedrooms, neighborhood, age.<\/li>\n<li><strong>y (label):<\/strong> what you want to predict \u2014 the price.<\/li>\n<li><strong>Sample:<\/strong> each row.<\/li>\n<\/ul>\n<p>Before training: nulls (mean\/median or drop), outliers, text \u2192 numbers (one-hot for the neighborhood), scaling (m\u00b2 must not overwhelm \u201cnumber of bedrooms\u201d just because the number is larger).<\/p>\n<h2>Training vs. inference<\/h2>\n<p><strong>Training:<\/strong> the model looks at thousands of examples and adjusts weights\/biases until the error falls. It consumes GPUs, time, clusters. <strong>Inference:<\/strong> frozen weights; new data goes in and the answer comes out in milliseconds.<\/p>\n<p>The simplest case \u2014 one variable:<\/p>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">\u0177 = w \u00b7 x + b\n<\/code><\/pre>\n<p><code data-no-translation=\"\">w<\/code> is how much the price rises per m\u00b2; <code data-no-translation=\"\">b<\/code> is the baseline (\u201czero-area\u201d land). A deep network is that line stacked up, with nonlinearity in between \u2014 the GPU in the <a href=\"\/en\/2026\/09\/gpu-vs-tpu-como-funcionam-chips-ia\/\">GPU vs TPU post<\/a> does nothing else.<\/p>\n<h2>Loss and the valley<\/h2>\n<p>Error for one house: <code data-no-translation=\"\">y \u2212 \u0177<\/code>. MSE takes the average of the squares:<\/p>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">MSE = (1\/N) \u00b7 \u03a3 (y \u2212 \u0177)\u00b2\n<\/code><\/pre>\n<p>Squaring makes everything positive ( +100 and \u2212100 do not cancel out) and penalizes outliers: being off by R$ 50 thousand hurts four times as much as being off by 25 thousand.<\/p>\n<p class=\"translation-block\"><strong>Gradient descent:<\/strong> start with a random weight, calculate the slope of the error curve, take a step in the <em>opposite<\/em> direction. The step size is the learning rate <code>\u03b1<\/code>:<\/p>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">w_novo = w \u2212 \u03b1 \u00b7 \u2202Loss\/\u2202w\n<\/code><\/pre>\n<p>High \u03b1: the model jumps over the bowl and never lands. Low \u03b1: it crawls, and may still get stuck in a local hole.<\/p>\n<h2>Generalizing \u2014 without leaking the test set<\/h2>\n<p>A score of 10 in training means nothing. What matters is data that is <em>new<\/em>.<\/p>\n<ul>\n<li>Training ~80% \u2014 adjusts weights.<\/li>\n<li>Validation ~10% \u2014 chooses the algorithm and hyperparameters.<\/li>\n<li>Test ~10% \u2014 the final exam, untouched.<\/li>\n<\/ul>\n<p><strong>Underfit:<\/strong> a straight line on a sine wave. Gets both training and test wrong. <strong>Overfit:<\/strong> a deep tree that memorizes noise. 99% in training, 50% with the real customer.<\/p>\n<p><strong>Data leakage:<\/strong> test information enters training. Classic example: normalizing with the <em>global<\/em> mean before the split. In real estate: the \u201cagent's commission\u201d column, which exists only because the house already sold for that price. The lab looks beautiful; production breaks.<\/p>\n<h2>The map of problems<\/h2>\n<pre data-no-translation=\"\"><code class=\"language-text\" data-no-translation=\"\">Machine Learning\n \u251c\u2500 supervisionado (tem y)\n \u2502    \u251c\u2500 regress\u00e3o      (pre\u00e7o, temperatura)\n \u2502    \u2514\u2500 classifica\u00e7\u00e3o  (spam, churn, benigno\/maligno)\n \u2514\u2500 n\u00e3o-supervisionado (sem y)\n      \u251c\u2500 clustering     (K-Means, perfil de cliente)\n      \u2514\u2500 dimensionalidade (PCA)\n<\/code><\/pre>\n<h2>From \u0177 = wx+b to ChatGPT<\/h2>\n<p>The cycle does not change: forward \u2192 loss \u2192 backprop \u2192 descent. An LLM just stacks billions of parameters on top of text.<\/p>\n<ol>\n<li><strong>Token + embedding:<\/strong> text becomes pieces, pieces become vectors. \u201cLinux\u201d and \u201cUnix\u201d end up close together.<\/li>\n<li><strong>Pretraining:<\/strong> predicting the next token across trillions of words. It becomes the universe's autocomplete \u2014 not an assistant yet, just continuing text in the style of the web.<\/li>\n<li><strong>SFT:<\/strong> thousands of question\/answer pairs. It learns to follow instructions.<\/li>\n<li><strong>RLHF \/ DPO:<\/strong> humans rank answers; a reward model + reinforcement nudges it toward being helpful, honest, and refusing harmful requests.<\/li>\n<\/ol>\n<p>In 2026 the extras are attention (Transformer), multimodality (text+image+audio), and agents (tool calling, terminal scripts, ReAct loop). On your own Linux machine, the same cycle fits in <a href=\"\/en\/2026\/09\/rodando-ia-local-no-linux-com-ollama\/\">Ollama<\/a>.<\/p>\n<h2>References<\/h2>\n<ul>\n<li>Course: <a href=\"https:\/\/www.youtube.com\/watch?v=du1B6633-hU\">KodeKloud \u2014 How Machine Learning Works, End to End<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/o-que-e-uma-rede-neural\/\">What is a neural network?<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/deep-learning-para-iniciantes\/\">Deep Learning for beginners<\/a><\/li>\n<li><a href=\"\/en\/2026\/09\/gpu-vs-tpu-como-funcionam-chips-ia\/\">GPU vs TPU<\/a><\/li>\n<\/ul>\n<p>Spreadsheet, loss, gradient, split. ChatGPT is that with more zeros in the equation. No magic \u2014 and with leakage, the lab lies.<\/p>","protected":false},"excerpt":{"rendered":"<p>Programming is: data + rules \u2192 answer. Pricing a house, recognizing a face, understanding Portuguese \u2014 manually written rules cannot handle it all. Machine Learning reverses the equation: data + example answers \u2192 the model (the rules). This article follows KodeKloud\u2019s course How Machine Learning Works, End to End: from spreadsheets to ChatGPT, the cycle \u2026 <a title=\"Machine Learning from end to end\" class=\"read-more\" href=\"https:\/\/www.linuxpro.com.br\/en\/2026\/09\/machine-learning-ponta-a-ponta\/\" aria-label=\"Read more about Machine Learning de ponta a ponta\">Read more<\/a><\/p>","protected":false},"author":0,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13,115,2],"tags":[116,168,128,167],"class_list":["post-415","post","type-post","status-publish","format-standard","hentry","category-cursos","category-ia","category-linux","tag-ia","tag-kodekloud","tag-llm","tag-machine-learning"],"_links":{"self":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts\/415","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/comments?post=415"}],"version-history":[{"count":2,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts\/415\/revisions"}],"predecessor-version":[{"id":1135,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/posts\/415\/revisions\/1135"}],"wp:attachment":[{"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/media?parent=415"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/categories?post=415"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.linuxpro.com.br\/en\/wp-json\/wp\/v2\/tags?post=415"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}