Segmentation: a label for every pixel

A segmenter looks at a cropped region and answers a richer question: not just what is here, but where and how much. Where exactly is the scratch? How big is the stain? How many pills sit in the tray? It trades classifier speed for a pixel-by-pixel mask, the spatial detail you need for measuring, counting, or localizing defects.

The setup is built on the same hierarchy as a classifier: Types own Classes, and Types are stamped onto images as ROIs.

Think of it like a paint-by-numbers kit. The Inspection Type is the kit (palette plus shape outlines). The Classes are the paint colors you choose from. The ROIs are the canvases you paint each capture onto, and every painted pixel ends up assigned to exactly one class, including the implicit background.

OV80i recipes can mix segmenters and classifiers

The OV80i is the only camera in the lineup that supports multi-model recipes. Pair a segmenter (for masks, counts, and measurements) with a classifier (for verdicts on known categories) inside one inspection. See Understanding Classifier for the other half.

The three concepts

Before you can train an AI model, the camera needs to know where to look, what the possible outcomes are, and how each crop should be evaluated. Those three questions map to three concepts, and they nest in a very specific way.

01 / Concept

Inspection Type

A named bucket. Holds one shared dataset, one shared list of classes, and one trained AI model.

02 / Concept

Classes

The vocabulary of possible outcomes for this inspection. Defined once on the type.

03 / Concept

Region of Interest (ROI)

A rectangle drawn on the image. Many ROIs can share one type and, therefore, one dataset and one model.

The Mental Model: a type owns the classes and the ROIs

An Inspection Type is a bucket. It holds one shared dataset, one shared list of classes, and one trained AI model. You then stamp that bucket onto the image in multiple places, those stamps are the ROIs. Every ROI of the same type inherits the same classes and is evaluated by the same model.

• Inspection Type is the bucket.
• Classes are the outcome vocabulary.
• ROI is a location on the image.

For a segmenter: a label for every pixel, painted, not picked

A segmenter takes each ROI crop and does something richer than picking a class: it labels every pixel. Instead of choosing a dropdown answer, you brush defects onto the training images, and the model learns the shape of each class. The output per ROI is a mask, a pixel-by-pixel map you can measure, localize, and count.

Live example: surface defect detection

Imagine a brushed metal surface with two scratches, one dent, and a stain. You'd configure it like this:

• Type: Surface Quality (1 ROI, segmenter)
• Classes: background (implicit), scratch, dent, stain
• ROI: Surface_Top (covers the inspectable area)

Instead of picking one label for the whole region, you paint each defect with its class color. The model output is a mask, broken down per class:

Class	Coverage
background	93.5%
scratch	3.1%
dent	1.4%
stain	2.0%

Three things to internalize from this example:

1. Paint, don't pick. Instead of picking one label for the whole ROI, you paint every defect pixel by pixel. The model learns the shape of each class, not just whether it's present.
2. The ROI is pixel territory. The segmenter looks inside the ROI and labels every pixel. More surface area = more pixels to evaluate. Still respect the 512 × 512 rule, grid-tile if the area is huge.
3. Classes are defect types. Not pass/fail, each class is a distinct thing you want to localize: scratch, dent, stain. Plus an implicit background.

Deep dive: how classes behave

Each class is a color on the palette

In a segmenter, classes aren't labels you pick from a dropdown, they're paint you apply. Select a class, and your brush starts writing pixels in that class's color. Every pixel in the ROI ends up assigned to exactly one class, including the default background.

1. Define once on the type. Add a class (e.g. burn_mark) and give it a color. Every ROI of this type now has that color available on its palette.
2. Background is free. You don't paint the "nothing wrong here" pixels. Any unpainted pixel is automatically background.
3. One class per pixel. Painting with dent over a scratch replaces the label, classes can't stack on a single pixel.
4. Start simple. Two or three defect classes usually outperform six vague ones. Merge similar defects until the model struggles to distinguish them, then split.

Deep dive: how ROIs behave

The Golden Rule of ROIs

Smaller regions win. Make each ROI just big enough to contain the feature. Smaller ROIs mean less training data, faster iteration, and more accurate AI decisions, the feature dominates the crop instead of getting lost in background, and nothing gets downscaled.

Still small, still specific, even for segmenters

It's tempting to draw one giant ROI over an entire surface, but the Golden Rule still applies. A segmenter's ROI gets downscaled to fit the model input just like a classifier's does, and a defect that's only 20 pixels wide disappears when you shrink a 2000-pixel-wide ROI down to 512.

1. Keep each ROI under 512 × 512 px. Beyond that, detail is permanently lost to downscaling. Small defects become invisible.
2. Tight crops mean tight masks. A small ROI keeps each pixel at full resolution, so the mask lines up with real defect boundaries.
3. Big surface? Grid-tile it. Don't stretch one ROI across a whole panel. Lay a grid of smaller ROIs so every tile stays full-res and the model can see fine defects.
4. Masks persist per capture. Every training image stores its own painted mask for the ROI; the model learns from all of them.
5. Rich outputs. Per ROI you get pixel counts per class, bounding boxes for each defect region, and measurable areas, enough for size-based pass/fail rules.

Data flow: every ROI comes back with a mask, not a label

At runtime the camera crops the ROI out of the full image, feeds it to the trained segmenter, and gets back a per-pixel prediction, a mask of the same size where every pixel carries a class. From that mask the camera derives pixel counts, defect regions, and bounding boxes.

1. Capture the full frame with the ROI marked.
2. Crop the ROI down to a single image, kept at or under 512 × 512 px.
3. Segmenter model predicts a class for every pixel using an encoder-decoder architecture.
4. Mask output is a colored pixel map plus pixel counts per class, plus bounding boxes for each connected region. Pass/fail rules run on top of those numbers in the IO Block.

Get creative: a mask is a geometric object

Once the segmenter returns a mask, you're not stuck with pass/fail. You have the exact shape and location of every class, which means you can derive measurements, counts, distances, and spatial relationships. That unlocks a whole category of inspections that look nothing like "find the scratch", and the same segmenter can serve all of them.

01 / Measure

Area & coverage

Pixel counts become mm² on a calibrated camera. Set pass/fail in real-world units, "reject if stain > 3 mm²", instead of fuzzy percentages.

02 / Count

Distinct blobs

Four small pits tell a different story than one big one. Count connected regions per class to flag defect frequency, not just total coverage.

03 / Measure

Dimensions & shape

Length, width, aspect ratio, orientation, all fall out of the mask. Flag long scratches, ignore micro ones. Or flag round dents and ignore elongated ones.

04 / Relate

Blob-to-blob distance

Euclidean gap between two painted regions, centroid-to-centroid or nearest-edge. Useful for component spacing, pin pitch, weld-to-weld gaps, or detecting defect clusters.

05 / Relate

Distance to a reference

Paint a second class for your reference (edge, fiducial, adjacent component) then measure how far the defect sits from it. A defect 8 mm inside might be fine; 0.5 mm from the edge might be a reject.

06 / Expand

Not just defects

Paint whatever you want to find: pills in a blister, screws in a tray, weld beads, gaps, fiducials, cells on a slide. Segmentation is a general-purpose scalpel, not a defect-only tool.

Setup recap

A quick checklist before you train. If each of these is true, your segmenter will have a solid foundation.

• One Inspection Type per surface. Inspecting two different materials or defect families? Give each its own type so they get their own classes, dataset, and model.
• Classes represent defect types, not severity. scratch, dent, stain, each a distinct visual thing. Severity and pass/fail come from the mask's pixel counts downstream.
• Background is implicit. You never paint or label "good" pixels. Anything unpainted belongs to background.
• ROIs sized thoughtfully. Cover the area of interest, but stay under 512 × 512 px per ROI. Grid-tile if the surface is larger.
• Paint every visible defect. Missed defects in training data teach the model they're background, the worst possible lesson for a quality check.
• Start with 10 to 15 images per class. Train, run live preview, add targeted images where the mask is wrong, retrain. Two to four iterations is typical.

What's next

• Understanding Classifier, the sibling page on dropdown-pick models.
• Regions of Interest (ROIs), the practical guide to drawing ROIs.
• Training Your AI, the labeling and training workflow.