Each control type reads something different from your picture: its edges, its depth, a pose, its colours.
Edges and outlines
Detect outlines that can guide shapes while allowing changes to colour, materials, and style.
No example published yet. canny Finds hard outlines a pixel wide wherever light meets dark. It makes no distinction between the edge of an object and the edge of a texture, so a detailed photograph gives a busy map.
No example published yet. hed Reads soft edges, following the shapes a person would trace rather than every hard boundary, which suits a photograph better than canny does.
No example published yet. pidinet Reads soft edges like hed, thinner and cleaner, and runs quicker.
No example published yet. teed The newest of the soft edge readers, with thin continuous lines and less speckle than hed or pidinet.
No example published yet. pyracanny Runs canny over several sizes of your picture at once, so the large shapes survive and the fine texture drops out.
No example published yet. binary Turns everything darker than a threshold black and the rest white, with no edge finding at all, so it wants a stark drawing to start from.
No example published yet. scribble Traces the dark parts of your picture coarsely, on purpose, and expects a drawing on a pale ground; on a photograph use fakescribbles.
No example published yet. fakescribbles Thins the soft edge map down to a loose hand-drawn scribble, which is the one to use on a photograph.
No example published yet. scribble_xdog Draws a scribble from the difference between two blurs, with finer strokes than the plain scribble and more of the faint detail kept.
No example published yet. scribble_pidinet Thins pidinet's edges to a scribble, cleaner and more continuous than the other scribbles.
Line art and straight lines
Create line maps. mlsd detects straight segments, which can be useful for buildings.
No example published yet. standard_lineart Calculates a plain ink drawing directly from the pixels, with no model involved. It is fast, and it loses faint detail.
No example published yet. lineart Traces a photograph the way an illustrator would ink it, with weighted strokes and the faintest detail left out.
No example published yet. lineart_anime Draws the lines in the anime idiom, with even strokes and closed shapes ready to be coloured in.
No example published yet. lineart_anime_denoise Draws anime lines after cleaning out paper grain and screentone, which suits scans and heavily compressed pictures.
No example published yet. mlsd Keeps straight line segments only and discards everything curved, which usually leaves the architecture. hough is the older name for this detector. The horde still accepts it, and older workers only know that name.
Depth
Estimate relative depth to guide the arrangement of objects.
No example published yet. depth Reads how far away each part of your picture is. It is reliable across a whole scene, though it loses small objects.
No example published yet. midas_depth The older and faster depth map, coarser than the rest and steady on an ordinary photograph.
No example published yet. zoe_depth Reads depth in real distances rather than relative ones, so near and far stay in proportion across the picture.
No example published yet. depth_anything A newer depth map that holds fine detail: hair, railings, leaves.
No example published yet. depth_anything_v2 The sharpest depth map on offer, with cleaner boundaries between objects and less smearing than depth_anything.
Surface direction
Estimate which direction surfaces face. This can guide shape; it does not preserve the original lighting.
No example published yet. normal Works out which way each surface faces from a depth map, rougher than normal_bae and tolerant of an odd picture.
No example published yet. normal_bae Reads which way each surface faces with a model trained for it, the cleaner of the two and the usual pick.
Pose
Detect body positions to guide a figure's pose. Detection can miss people or body parts.
No example published yet. openpose Draws the people as skeletons, with body, hands and face as coloured sticks and dots on black, and discards everything else.
Regions and objects
Divide the image into labelled regions that can guide the layout of a new image.
No example published yet. seg Splits your picture into flat coloured fields, one colour for each kind of thing, in the quick and coarse version.
No example published yet. oneformer_ade20k The same split done by a stronger model over a scenery vocabulary: walls, sky, floor, furniture, plants.
No example published yet. oneformer_coco The same split over an object vocabulary, which names people, animals and things carried or sat on rather than the parts of a room.
Colour and texture
Use colour or image detail as guidance. Tile controls can help with refinement, though they may also change features.
No example published yet. color Reduces your picture to a grid of flat colour blocks, keeping the palette and where each colour sits, and nothing of the drawing.
No example published yet. shuffle Distorts your picture into a swirl. The colours and textures are kept, and the composition is destroyed. Use it to pass on a picture's palette and feel without its layout.
No example published yet. recolor_luminance Makes a greyscale copy, weighted the way the human eye judges brightness. Use it as the map when you want to change a picture's colours but keep its shading.
No example published yet. recolor_intensity Makes a grey copy from the strongest colour channel, which leaves saturated colours brighter than the luminance version does.
No example published yet. tile Blurs your picture to its broad shapes, which is what a tile ControlNet needs to add detail without inventing a different picture.
No example published yet. tile_ttplanet_guided A gentler tile map that blurs the flat areas and leaves the edges crisp, and the steadier one on a photograph.
No example published yet. tile_ttplanet_simple The plain tile map, an even blur with no edges kept, quicker and enough when your picture is already clean.