DEEPFIN: how an AI sorts clownfish
DEEPFIN is an AI that sorts photos of fish without ever being told what is in them. Pick where to start:





From the study DEEPFIN, iScience (2026).
DEEPFIN is an AI that sorts photos of fish without ever being told what is in them. Pick where to start:





From the study DEEPFIN, iScience (2026).
Nemo comes in many shapes and colors, A. Ocellaris and A. Percula are two sister clownfish species that can be difficult to identify. Given the two wild type examples can you name the species?
DEEPFIN was never told which fish is which. It looked at photos of 68 clownfish, one per fish, and sorted them into groups on its own. Below is its map: each photo sits where DEEPFIN placed it, and similar-looking fish end up close together. The coloured borders show the experts' labels, which we only added afterwards.
DEEPFIN can read the photos with different neural networks, and each draws its own map. Pick one:
■ Ocellaris ■ Percula · outlined: the 10 quiz fish · hover or tap a photo to enlarge
The tell? Experts count the spines on the back fin and look at how thick the black outlines are. These two sister species split about 5 million years ago and can still interbreed, so even trained eyes hesitate.
DEEPFIN also sorts baby clownfish into their growth stages, with no labels needed, which could save scientists weeks of work on thousands of images. Read the paper in iScience · Code on GitHub
These are the developmental stages of the clownfish Amphiprion ocellaris. As the babies grow they change shape and colour, and biologists sort them into seven stages (Roux et al., 2019), from a newly hatched, see-through larva (stage 1) to a juvenile with white bars (stage 7). Each mystery fish is somewhere in between. Can you tell which stage it is in?


The seven stages were defined by Roux and colleagues in Staging and normal table of postembryonic development of the clownfish (Amphiprion ocellaris), Developmental Dynamics (2019). Each fish is measured, and checked for features that appear as it grows:
| Measurements | |
|---|---|
| Standard length | Snout to the tip of the notochord in young larvae; snout to the middle of the tail base once the tail has formed |
| Snout–anus length | Snout to the anus |
| Head length | Snout to the edge of the gill cover |
| Body depth | From the top of the head, where the dorsal fin starts, straight down to where the pelvic fin attaches |
| Eye diameter | Across the longest axis of the eye |
| Snout length | Snout to the front edge of the eye |
| Head depth | At right angles to the head length, from the anus to the base of the dorsal fin's soft rays |
| Features that appear | |
| Anal fin | When its soft rays and its spines appear |
| Dorsal fin | When its soft rays and its spines appear |
| Pelvic fin | How far it has developed |
| White stripes | When each of the three white body stripes appears |
| Notochord flexion | How far the end of the notochord (the early backbone) has bent upward to form the tail |
This is just 144 fish. A single experiment can produce thousands of photos like these.
This is what DEEPFIN does. Without being shown a single labelled example, it sorted photos of young A. ocellaris into their developmental stages (as defined by Roux et al., 2019), getting about 87% of them right in our study. Instead of staging every fish by hand, a scientist only needs to check each group and fix the few strays.
Groups above use the experts' stage labels, to show what a finished sort looks like.
A neural network that learned to see from over a million everyday photos (none of them baby clownfish) turns each picture into hundreds of numbers (512 to 1,280, depending on the network) describing its shapes, textures and colours. A method called UMAP then squeezes those numbers onto a flat map, so similar-looking fish land close together. Finally, a clustering step draws the boundaries between groups.
DEEPFIN can use different neural networks, and each draws its own map. Pick one:
Each photo sits exactly where DEEPFIN placed it. DEEPFIN was never told any stages: the coloured borders show the experts' labels, added only to check its work. Hover or tap a photo to enlarge it.
How does an AI see a fish?
Computers don't see pictures. They see numbers. Follow one clownfish photo through a real neural network that has never seen these two species, and find out how it can still tell them apart, for example by checking… how much each fish looks like a bicycle?
Two lab photos of each species, and one wild photo of each from iNaturalist.
To a computer your photo is a grid of 224 × 224 dots, each with three colour numbers: 150,528 numbers. The network (MobileNetV2) passes them through dozens of layers. Each layer looks for patterns in the layer before. Here are a few of the patterns it found in your fish, from bright (strong) to dark (absent):
Early layers see small details on a fine grid. Deeper layers see bigger patterns on a coarser grid, until the whole fish is summed up in 1,280 numbers, and then scored against 1,000 everyday things.
The network learned from over a million photos of 1,000 everyday things: dogs, cars, pizza… and “anemone fish”, which is another name for clownfish.
Both wild fish get the same answer:
The final answer is no help: both species are just “anemone fish”. But just before it answers, the network sums up the photo in 1,280 numbers, a kind of fingerprint. It never learned our two species, yet fingerprints of the same species look alike.
Each thin stripe is one number, from low (dark) to high (bright). We lined the numbers up so the ones that differ most between the two species sit at opposite ends.
This is what DEEPFIN uses. It puts the fingerprints of all the fish on a map without being told which species is which, and the two species land in separate groups.
Each photo sits where DEEPFIN placed it, using only the 1,280-number fingerprints (MobileNetV2). The coloured borders show the species, which we added afterwards to check its work.
The network also gives every photo a score for each of the 1,000 things it knows: how much it looks like a bicycle, a penguin, a wig… Pick two of these scores. Every fish moves to a spot: how much it looks like the first (left → right) and the second (bottom → top).