Skip to content

Add computer vision lesson to the ML track, and verify the Supabase layer #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samanyugoyal2010 wants to merge 1 commit into
base: main
Choose a base branch
from
+546 −2
Open

Add computer vision lesson to the ML track, and verify the Supabase layer #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/app/learn/ml/(chapters)/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import { FeatureScalingLesson } from "@/components/ml/FeatureScalingLesson";
import { GradientDescentLesson } from "@/components/ml/GradientDescentLesson";
import { RegularisationLesson } from "@/components/ml/RegularisationLesson";
import { NeuralNetworksLesson } from "@/components/ml/NeuralNetworksLesson";
import { ComputerVisionLesson } from "@/components/ml/ComputerVisionLesson";
import { FromNotebookToProductionLesson } from "@/components/ml/FromNotebookToProductionLesson";
import { LEARN_VIBECODING_HREF } from "@/lib/links";
import { getAdjacent, getChapter, getChapters, getPositionLabel } from "@/lib/learn-nav";
Expand Down Expand Up @@ -60,6 +61,7 @@ const ML_LESSON_BODIES: Record<string, () => React.ReactElement> = {
"gradient-descent": GradientDescentLesson,
regularisation: RegularisationLesson,
"neural-networks": NeuralNetworksLesson,
"computer-vision": ComputerVisionLesson,
"from-notebook-to-production": FromNotebookToProductionLesson,
};

Expand Down
145 changes: 145 additions & 0 deletions src/components/ml/ComputerVisionLesson.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
import { Callout } from "@/components/learn/primitives/Callout";
import { CompareGrid, TakeawayCard } from "@/components/learn/primitives/Cards";
import { Lead, LessonSection, P, Strong } from "@/components/learn/primitives/LessonSection";
import { ConvolutionExplorer } from "@/components/ml/ConvolutionExplorer";

export function ComputerVisionLesson() {
return (
<div>
<Lead>
You have heard that a neural network can recognise a cat. It cannot see one. What arrives at
the network is a grid of brightness values, and what it learns is a stack of tiny filters
that each react to one kind of edge. This lesson is that filter, made concrete: nine numbers
and a little arithmetic, slid over a picture until something appears.
</Lead>

<LessonSection id="an-image-is-a-grid-of-numbers" title="An image is a grid of numbers">
<P>
Open any photo far enough and it stops being a photo. It is a rectangle of pixels, and
each pixel is a number: how bright that spot is, from 0 for black to 1 for white. A small
grey square is <Strong>0.4</Strong>. Its neighbour is <Strong>0.41</Strong>. There is no
&ldquo;cat&rdquo; anywhere in the file — only a few hundred thousand of these values laid
out in a grid.
</P>
<P>
Colour changes nothing important. A colour image is three of these grids stacked — one
for red, one for green, one for blue — and everything below works the same on each. The
picture on the right is a single 14&times;14 grid, drawn so you can read the numbers off
it: the disc is a patch of high values, the dark background is low ones.
</P>
<P>
So the question of vision is not &ldquo;what is in the picture&rdquo;. It is a narrower,
answerable one: <Strong>what can you compute from a grid of numbers</Strong> that tells
you a corner is here, or a stroke of fur runs there?
</P>
</LessonSection>

<LessonSection
id="a-filter-is-nine-numbers-and-some-arithmetic"
title="A filter is nine numbers and some arithmetic"
>
<P>
Here is the whole operation. Take a small square of weights — three by three, so nine
numbers. Lay it over one pixel and its eight neighbours. Multiply each pixel by the weight
sitting on top of it, add the nine products up, and write the total into that pixel&rsquo;s
spot in a new grid. That is a <Strong>convolution</Strong>, and it is the only arithmetic
in this entire field that you cannot skip.
</P>
<P>
The nine weights are the filter, and their pattern is everything. Make them all{" "}
<Strong>one-ninth</Strong> and each pixel becomes the average of its neighbourhood: the
picture blurs. Make the centre large and its neighbours negative and you get the opposite,
a sharpen. Make them sum to <Strong>zero</Strong> and something stranger happens — a flat
region cancels itself out to black, and only the places where brightness changes survive.
That last one is an edge detector, and it is doing arithmetic, not magic.
</P>
<P>
Drag the strength below from nothing to full. At zero you see the original untouched; on
the way up, the filter&rsquo;s effect fades in, so you can watch exactly what those nine
numbers add and take away.
</P>
</LessonSection>

<ConvolutionExplorer />

<LessonSection id="the-same-window-slid-everywhere" title="The same window, slid everywhere">
<P>
Notice what the filter is not. It is not a rule about the disc, or about the right-hand
bar. It is nine numbers that know nothing about where they are, and they are applied at{" "}
<Strong>every position in the grid</Strong>, unchanged. The edge filter finds the edge of
the disc and the edge of the bar with the same nine weights, because an edge is an edge
wherever it sits.
</P>
<P>
That reuse is the point, and it buys two things at once. An edge detector learned in the
top-left corner works in the bottom-right for free — the network does not have to see a
cat in every position to recognise one that has moved. And it is cheap: a filter is nine
weights whether the image is 14 pixels wide or 4,000. A fully-connected layer over a
megapixel image would need a million weights per unit; a convolution needs nine, slid.
</P>
<CompareGrid
items={[
{
title: "What sliding one small filter buys",
tone: "positive",
children: (
<>
A pattern learned in one place is recognised everywhere. Nine weights cover an
image of any size. The same edge detector serves the whole picture.
</>
),
},
{
title: "What it assumes",
tone: "caution",
children: (
<>
That what matters is <em>local</em> and the same everywhere — true for edges and
textures, less true when a pixel&rsquo;s meaning depends on the far side of the
image. That is what stacking layers is for.
</>
),
},
]}
/>
</LessonSection>

<LessonSection
id="what-the-network-learns-for-itself"
title="What the network learns for itself"
>
<P>
Every filter you just tried was chosen by hand — someone knew that{" "}
<Strong>-1 -1 -1 / -1 8 -1 / -1 -1 -1</Strong> finds edges. A convolutional network throws
that knowledge away. It starts with random weights in each filter and lets gradient descent
adjust them, exactly as in the previous lesson, until the filters that help it name the
picture are the ones that survive. Nobody tells it to look for edges. It discovers that
edges are worth looking for.
</P>
<P>
And it does not stop at one layer. Feed the edge grid into another convolution and its
filters combine edges into corners and curves; feed that forward again and later filters
respond to eyes, wheels, letters. This is the neural network you already met, with one
structural idea added: <Strong>the same small filter, slid everywhere, stacked</Strong>.
The depth builds meaning; the sliding makes it affordable.
</P>
<Callout tone="note" title="A filter has one more number than it looks">
The nine weights are usually joined by a tenth, a <em>bias</em> added to every sum, and
the result is passed through the same bend — the activation — that gave a plain network
its curve. The convolution is what makes it a network <em>for images</em>; everything else
is the machinery you have already seen.
</Callout>
</LessonSection>

<TakeawayCard
items={[
"An image is a grid of brightness numbers. Colour is three such grids. There is no picture in there, only values.",
"A convolution is nine weights laid over a pixel and its neighbours, multiplied and summed into a new grid. The pattern of the weights is the whole filter.",
"Weights that sum to zero detect edges; all-equal weights blur; a heavy centre sharpens. Same arithmetic, different nine numbers.",
"One small filter is slid over every position unchanged, so a pattern learned once is found anywhere — and nine weights cover an image of any size.",
"A convolutional network does not use hand-picked filters. Gradient descent learns the nine numbers, and stacking layers builds edges into objects.",
]}
/>
</div>
);
}
155 changes: 155 additions & 0 deletions src/components/ml/ConvolutionExplorer.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
"use client";

import { useId, useState } from "react";
import { SegmentedControl } from "@/components/learn/primitives/SegmentedControl";
import { formatPercent } from "@/lib/ml/format";
import { GRID, IMAGE, KERNELS, filteredImage } from "@/lib/ml/computer-vision-data";

const CELL = 18;
const DIM = GRID * CELL;

/** Brightness 0..1 to a warm ink-on-paper ramp. Deterministic by construction. */
function pixelFill(value: number): string {
const dark = [35, 32, 27];
const light = [251, 246, 236];
const r = Math.round(dark[0] + (light[0] - dark[0]) * value);
const g = Math.round(dark[1] + (light[1] - dark[1]) * value);
const b = Math.round(dark[2] + (light[2] - dark[2]) * value);
return `rgb(${r} ${g} ${b})`;
}

/** Integers print bare; fractions (the blur's 1/9) to two places. No locale. */
function formatWeight(value: number): string {
return Number.isInteger(value) ? String(value) : value.toFixed(2);
}

function PixelGrid({ pixels, label }: { pixels: readonly (readonly number[])[]; label: string }) {
return (
<svg
viewBox={`0 0 ${DIM} ${DIM}`}
className="w-full rounded-learn-md"
role="img"
aria-label={label}
>
{pixels.map((row, y) =>
row.map((value, x) => (
<rect
key={`${x}-${y}`}
x={x * CELL}
y={y * CELL}
width={CELL}
height={CELL}
fill={pixelFill(value)}
/>
)),
)}
<rect
x={0.5}
y={0.5}
width={DIM - 1}
height={DIM - 1}
fill="none"
stroke="var(--learn-line)"
strokeWidth={1}
/>
</svg>
);
}

export function ConvolutionExplorer() {
const [kernelId, setKernelId] = useState("edges");
const [strength, setStrength] = useState(1);
const sliderId = useId();

const kernel = KERNELS.find((k) => k.id === kernelId) ?? KERNELS[0];
const output = filteredImage(kernelId, strength);

return (
<figure className="learn-card mt-8 overflow-hidden rounded-learn-xl p-5 md:p-7">
<figcaption className="text-[13px] uppercase tracking-[0.08em] text-learn-muted">
Slide a filter across the image
</figcaption>

<p className="mt-2 text-[15px] leading-[1.6] text-learn-strong">
The picture on the left is a grid of brightness values — nothing more.{" "}
<strong className="font-semibold">Pick a filter</strong>, then{" "}
<strong className="font-semibold">drag the strength</strong> from nothing to full and watch
the same nine numbers, slid over every pixel, redraw it on the right.
</p>

<div className="mt-5 grid gap-5 sm:grid-cols-2">
<div>
<h3 className="text-[12px] uppercase tracking-[0.08em] text-learn-subtle">Original</h3>
<div className="mt-2">
<PixelGrid pixels={IMAGE} label="The original image as a grid of brightness values." />
</div>
</div>
<div>
<h3 className="text-[12px] uppercase tracking-[0.08em] text-learn-subtle">
{kernel.label === "None" ? "Filtered (no filter)" : `${kernel.label} filter`}
</h3>
<div className="mt-2">
<PixelGrid
pixels={output}
label={`The image after the ${kernel.label} filter at ${formatPercent(strength)} strength.`}
/>
</div>
</div>
</div>

<div className="mt-6 grid items-start gap-5 md:grid-cols-[auto_minmax(0,1fr)]">
{/* The kernel itself: nine numbers, centre pixel picked out. */}
<div>
<h3 className="text-[12px] uppercase tracking-[0.08em] text-learn-subtle">
The filter (3&times;3)
</h3>
<div className="mt-2 grid w-max grid-cols-3 gap-1">
{kernel.weights.map((weight, index) => (
<span
key={index}
className={`flex h-11 w-11 items-center justify-center rounded-learn-sm border-[0.5px] font-mono text-[13px] tabular-nums ${
index === 4
? "border-learn-accent bg-learn-quiet text-learn-strong"
: "border-learn-line bg-white text-learn-muted"
}`}
>
{formatWeight(weight)}
</span>
))}
</div>
</div>

<div>
<SegmentedControl
label="Filter"
variant="chips"
options={KERNELS.map((k) => ({ value: k.id, label: k.label }))}
value={kernelId}
onValueChange={setKernelId}
/>

<div className="mt-4 flex items-center justify-between gap-3">
<label htmlFor={sliderId} className="text-[13px] font-medium text-learn-strong">
Strength
</label>
<span className="font-mono text-[14px] text-learn-strong tabular-nums">
{formatPercent(strength)}
</span>
</div>
<input
id={sliderId}
type="range"
min={0}
max={1}
step={0.01}
value={strength}
onChange={(event) => setStrength(Number(event.target.value))}
className="mt-1 w-full accent-learn-accent"
/>

<p className="mt-4 text-[14px] leading-[1.5] text-learn-muted">{kernel.note}</p>
</div>
</div>
</figure>
);
}
65 changes: 65 additions & 0 deletions src/components/ml/MlLessonCover.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -802,6 +802,70 @@ function DriftCover() {
);
}

function VisionCover() {
/* Left: a coarse pixel grid with a bright disc, and the 3x3 filter window
laid over its edge. Right: what an edge filter leaves — only the boundary.
Brightness is carried by opacity on a single ink, so it reads at 160px. */
const N = 5;
const cell = 10.5;
const gap = 1.2;
const leftX = 12;
const rightX = 96;
const top = 20;
const centre = 2;
const cells = Array.from({ length: N }, (_, j) =>
Array.from({ length: N }, (_, i) => ({ i, j })),
).flat();
const dist = (i: number, j: number) => Math.hypot(i - centre, j - centre);
return (
<Frame>
{cells.map(({ i, j }) => (
<rect
key={`o${i}-${j}`}
x={leftX + i * (cell + gap)}
y={top + j * (cell + gap)}
width={cell}
height={cell}
rx={1}
fill="var(--learn-ink-strong)"
opacity={dist(i, j) < 1.9 ? 0.85 : 0.14}
/>
))}
<rect
x={leftX + (cell + gap) - 1.5}
y={top - 1.5}
width={3 * cell + 2 * gap + 3}
height={3 * cell + 2 * gap + 3}
rx={2}
fill="none"
stroke="var(--learn-accent)"
strokeWidth={1.6}
/>
<path
d="M78 45 L90 45 M86 41 L90 45 L86 49"
fill="none"
stroke="var(--learn-chart-axis)"
strokeWidth={1.4}
/>
{cells.map(({ i, j }) => {
const d = dist(i, j);
return (
<rect
key={`e${i}-${j}`}
x={rightX + i * (cell + gap)}
y={top + j * (cell + gap)}
width={cell}
height={cell}
rx={1}
fill="var(--learn-series-1)"
opacity={d >= 1.2 && d < 2.4 ? 0.95 : 0.08}
/>
);
})}
</Frame>
);
}

const COVERS: Record<string, () => React.ReactElement> = {
"what-is-ml": RulesCover,
"features-and-labels": FeaturesCover,
Expand All @@ -824,6 +888,7 @@ const COVERS: Record<string, () => React.ReactElement> = {
"gradient-descent": DescentCover,
regularisation: PenaltyCover,
"neural-networks": NetworkCover,
"computer-vision": VisionCover,
"from-notebook-to-production": DriftCover,
};

Expand Down
Loading
Loading