Paint by Numbers

A readable Python + PyTorch implementation of Paul Haeberli's "Paint By Numbers: Abstract Image Representations" (SIGGRAPH '90).

Source photograph	Painted

Every color on the right was point-sampled from the photograph on the left. Nothing was picked by hand.

The paper's idea: a painting is an ordered list of brush strokes, each with five attributes.

Attribute	Meaning
Location	where it sits, normalized to [0, 1]
Color	RGBA, sampled from the source at that point
Size	radius, in units of the canvas's short side
Direction	angle of the long axis
Shape	what the mark looks like

Rendering is stamping them onto a canvas in order. Every effect in the paper — impressionist portraits, pointillism, Voronoi mosaics, animation between paintings — is a different way of choosing those five numbers.

Two things follow from the representation:

• Resolution independence. Positions and sizes are normalized, so one stroke list renders at any size. A bigger output is a higher-resolution painting, not an upscaled bitmap.
• Order matters. Later strokes cover earlier ones, so re-sorting the list changes the image without touching a single stroke.

The code favors clarity over speed: plain loops instead of clever vectorization, one module per idea, docstrings citing the section each piece comes from.

Structure

The painter looks at the photograph and decides every stroke; the renderer turns those strokes into pixels.

stylize.py      command-line entry point: a photograph in, a painting out

painter/        decides what each stroke should be
    layout.py       where strokes go       -- jittered grid
    color.py        what color each is     -- point sampling the source
    direction.py    which way each points  -- luminance gradient
    paint.py        runs the above, coarse to fine, into a finished painting
    enhance.py      seasons the photograph before painting starts
    operations.py   transforms a painting after it is finished

render/         what a painting is, and how it becomes pixels
    stroke.py       the five attributes
    brushes.py      brush shapes
    canvas.py       stamping strokes onto pixels
    painting.py     the ordered list, plus rendering and archiving

utils/          image I/O and Gaussian blur

Dependencies run one way, painter → render: nothing in render ever looks at a photograph, and only canvas.py touches pixels.

Installation

Needs Python 3.10 or newer (developed on 3.12), plus PyTorch, NumPy, and Pillow.

conda create -n paint_by_num python=3.12 -y
conda activate paint_by_num
pip install -r requirements.txt

No GPU required. PyTorch is used here only as a tensor library, and the whole pipeline runs on the CPU — the sample image paints in about a second. The default PyTorch wheel is all you need on any machine.

Usage

Paint a photograph:

python stylize.py assets/pushu.jpg

The result lands in output/<name>_painting.png. With no arguments it paints the bundled sample.

The painter lays a coarse layer over the whole canvas, then adds finer strokes only where the painting still disagrees with the photograph. Detail collects on faces, hands, and edges while flat areas keep their broad strokes — the paper's "amount of detail across the painting was modulated to direct the viewer." Stroke directions follow the image's own contours.

Options

Option	Default	What it does
source	assets/pushu.jpg	the photograph to paint
-o, --output	output/<name>_painting.png	where to write the result
--shape	rect	brush shape — rect, circle, line, triangle, pentagon, scatter, bristle, or cone
--sizes	0.05,0.028,0.016	stroke radii for each pass, coarse to fine, as fractions of the canvas's short side
--refine	0.06	how wrong an area must look before finer strokes are spent on it; higher is looser
--direction	auto	auto follows the image's contours, or give a fixed angle in degrees
--smooth	3.0	blur radius in pixels before measuring the gradient (auto only)
--scale	1.0	output size relative to the source
--seed	0	random seed for stroke placement
--save-strokes	off	also write the stroke list as text
--sharpen	0	season: push edges apart, making the light side lighter and the dark side darker
--blur	6.0	season: how far from an edge --sharpen reaches, in pixels
--saturate	0	season: enrich color while holding brightness constant
--noise	0	season: roughen flat areas so their strokes vary in color
--palette	0	season: reduce to this many colors, as an impressionist's limited palette would

The five season: options enhance the photograph before painting starts and are off by default; the rest control the painting itself. The sections below show what each group does.

Brush shapes

Seven shapes composite normally, and each gives the painting a different character.

rect · circle · line · triangle · pentagon · scatter · bristle

python stylize.py --shape circle

Mosaics

--shape cone works differently. Instead of compositing color, it z-buffers cones standing tip-first at each stroke's center. Distance from the center is the cone's depth, so the depth test leaves every pixel showing its nearest stroke — carving the canvas into Dirichlet domains, a Voronoi diagram of the positions. Since depth decides the outcome, cone strokes are order-independent: reversing the list gives a pixel-identical image.

python stylize.py --shape cone

Looser or tighter

--refine decides how much of the coarse underpainting survives. Raising it leaves broad strokes standing over more of the canvas; 0 refines everywhere, repainting the picture at every stroke size instead of adding to it.

python stylize.py --refine 0.14

The other lever is --sizes, whose finest value sets how abstract the result reads: below about 0.015 strokes stop looking like strokes and the painting collapses into a blurred photograph; above 0.02 faces stop being legible.

Seasoning the photograph

The paper's Spice for images section enhances the source before any paint touches it. Much of the oil-painted richness comes from this step.

python stylize.py --sharpen 0.6 --saturate 0.4 --noise 0.04 --palette 16

--sharpen and --saturate are one operation underneath: interpolate between the image and a reference, and let the parameter run past the end into extrapolation. Away from a blurred copy sharpens edges; away from a grayscale copy deepens color at constant luminance.

--noise barely shows on the photograph itself, but gives every stroke in a flat region a slightly different color, which keeps large areas alive. It also hides the banding --palette would otherwise cause, so use the two together.

Rendering at any size

A painting is geometry, not pixels, so --scale re-renders the same strokes at any resolution — a larger output resolves more of each stroke rather than enlarging blocks.

python stylize.py --scale 4

Saving the stroke list

--save-strokes writes the painting as text, in the layout of the paper's Figure 5:

painting with 5648 strokes
position              RGBA color        attr:    siz      dir  brush
0.171874 0.889075    39  91 104 255  attr: 0.050000  179.926  rect

Positions and sizes are normalized and directions are in degrees, so the file describes the painting independently of any resolution. Reloading and re-rendering reproduces the original to within one colour level.

Reference

Paul Haeberli, "Paint By Numbers: Abstract Image Representations", Computer Graphics 24(4), SIGGRAPH '90, pp. 207–214.