Selections & Masks in PHotoshop-- Part I

Article and Photography by Ron Bigelow

www.ronbigelow.com

Photoshop CS or Photoshop CS2 Used in this Tutorial

Layers is one of the most powerful capabilities of Photoshop. This capability allows photographers to apply edits to specific layers rather than to the original image data. However, the full potential of Layers is not realized until it is combined with the use of masks. Masks allow a photographer to restrict the effects of layers to specific areas of an image; for instance, one may darken the sky without darkening the rest of the image. In most cases, in order to create a mask, a selection must first be created. Selections divide images into three areas: the areas where the pixels are completely selected, the areas where the pixels are partially selected, and the areas where the pixels are not selected. Once a selection has been turned into a mask, the mask will determine how the associated layer influences the image based on the selection of the pixels.

This all appears very straightforward. Yet, as many photographers will testify, it is much harder to carry out in practice that it sounds. This is due to two issues. First, it is not always easy to select the correct pixels. Often, a method of selection will ignore some of the pixels that need to be selected; at the same time, it may select pixels that should not be selected. Second, it is often difficult to create a selection so that the resulting mask blends the selected and non-selected pixels in such a manner that the use of the mask is invisible.

Yet, photographers accomplished in the use of selections routinely create complicated selections that gracefully blend layers in a seamless manner. So, what do these photographers know that some others do not? Well, for starters, they know how to choose the correct tool for the job. These photographers know a multitude of selection tools and techniques, and they know which tools and techniques work best in which situations. On the other hand, photographers less adept in selections often know only a few selection tools and techniques. As with a mechanic who only has a pair of pliers when he really needs a torque wrench, the results can be less than desired.

With this in mind, this series of articles reviews several selection and masking tools and techniques. The series starts with the simplest approaches and gradually progresses to more sophisticated ones.

Selection: Marquee Tools

Figure 1: Tools Palette

The Marquee tools are probably the simplest of all the selection tools. The Marquee tools can be found on the Tools palette (see Figure 1). There are four Marquee tools:

Figure 2 shows the Marquee Options bar. On the left side of the bar are four selection options:

The Feather box option allows the selection to be feathered by a specified number of pixels. Feathering softens the edges of the selection by creating a transition between the selected and the non-selected areas. The anti-alias checkbox also softens the selection by softening the transition of color from the selected to the non-selected areas. The purpose of these two options is to soften the selection so that there will not be an abrupt transition from those areas that are selected to those that are not. Thus, when a mask is created from the selection, the transition between these areas will not be discernable.
Figure 2: Options Bar for Marquee Tools

The question quickly comes up, "Which is best: feathering, anti-aliasing, both, or neither one?" In general, the answer is: neither one. Different selections require different amounts of softening of the edges in order to get a good transition from the selected to non-selected areas. For example, an image from a thirteen megapixel camera will usually require more softening (i.e., a wider transition region) than an image from a six megapixel camera (all else being equal) because any given amount of detail is spread across a larger number of pixels in the image taken with the thirteen megapixel camera. Another case in point, an image of a baby's face would usually require a softer edged selection than an image of cactus needles. The problem is, at the time of the selection, it is generally not known exactly how much softening of the selection will work best. Some experimentation will likely be required. However, when a selection is made with feathering or anti-aliasing, the softening is built into the selection. While the edges can be made softer, it is more difficult to make the edges harder. Thus, if a photographer uses feathering or anti-aliasing and later decides that the selection has an edge that is too soft, he will probably have to redo the selection -- probably a time consuming task. Luckily, there is a better option. If the feathering and anti-aliasing options are not used, a Gaussian Blur can be used on the completed selection to soften the edges (this approach will be covered later in this article). If the photographer later decides that the edges need a different amount of softening, the photographer can go back to the original selection (saved before any Gaussian Blur was used) and try a different amount of Gaussian Blur. Thus, it is recommended that Gaussian Blur be used to soften the edges of the selection rather than the feathering or anti-aliasing options.

The style option (enabled with the Rectangular and Elliptical Marquee tools) allows for three options (see Figure 3):

If the Fixed Aspect Ratio or Fixed Size options are chosen, the width and height options must be entered. If the Fixed Aspect Ratio option is chosen, the width and height are entered as a ratio (e.g., 1.5:1). If the Fixed Size option is entered, the width and height are entered in pixels.

Figure 3: Style Option
Figure 4: Image that Needs the Edges Burned
Once all of the Marquee tool options have been set, the tool is ready to be used. The use of the Marquee tools is simplicity itself. A selection is simply drawn on the image with the Marquee tool. As an example, Figure 4 shows an image that will be worked with the Elliptical Marquee tool. This image needs the edges burned (for more information, see Burning the Edges). The procedure for this is fairly simple. A solid black layer is added to the image, an elliptical selection is created, the selection is blurred, the blurred selection is added to the black layer as a mask, and the Opacity of the layer is adjusted. In the following example, it is assumed that the solid black layer has already been added. The example starts with the creation of the selection.
Figure 5: Selection Created using the Elliptical Marquee Tool

Figure 5 shows the image after the Elliptical Marquee tool was dragged across the image to create the selection. At this point, the selection has a hard edge. This selection needs to be softened, so the image is move into the Quick Mask Mode (click on the Edit in Quick Mask Mode on the Tools palette).

Figure 6 shows the selection in the Quick Mask Mode. The area that has not been selected is shown in red. To soften the edge, Gaussian Blur is launched (choose Filter/Blur/Gaussian Blur). Figure 7 shows the Gaussian Blur dialogue box.

Figure 6: Selection in Quick Mask Mode before Blurring

 

Figure 7: Gaussian Blur Dialog Box
For burning the edges, a very large radius is needed (for most other masks a much smaller radius will be used). Once the blur has been added, the mask is as shown in Figure 8.
Figure 8: Selection in Quick Mask Mode after Blurring
Moving out of the Quick Mask Mode (click on the Edit in Standard Mode on the Tools palette), the selection is now added to the Black layer (with the black layer selected, choose Layer/Layer Mask/Hide Selection). If the layer already has a blank mask, it must be deleted first (right click the mask and choose Delete Layer Mask).
Figure 9: Layers Palette

Figure 9 shows the Layers palette. Figure 10 shows the final mask.

Figure 10: Final Mask

 

The primary benefit of the Marquee tools is their ease of use. On the down side, the Marquee tools have a very limited use for photographers. It is not very often that photographers need selections that are perfect rectangles, ellipses, or rows/columns of pixels. On the other hand, while I only occasionally use the Marquee tools with my images, I use the tools extensively when preparing graphics for my articles.

Lasso Tools

Figure 11: Tools Palette

The Lasso tools provide somewhat more flexibility than the Marquee tools. The Lasso tools can be found on the Tools palette (see Figure 11). There are three Lasso tools:

 

Selection: Lasso Tool

The Lasso tool is the first of this group of tools. This tool allows a photographer to create a freehand selection.

Figure 12: Options Bar for Lasso Tool

Figure 12 shows the Options bar for the Lasso tool. The Lasso Options bar has the same Selection, Feather, and Anti-alias options as covered in the Marquee tools section.

The Lasso tool is great when an irregular shaped selection that does not require straight edges needs to be created. Figures 13 and 14 show an example where the Lasso Tool is ideal. These two images are of the same scene except the first image was exposed to keep detail in the shadows, and the second image was exposed to keep detail in the sky. The two images are put into the same file and a mask is used to blend them.

Figure 13: Image with Exposure Set for Foreground
Figure 14: Image with Exposure Set for Sky
Figure 15: Image with Lasso Tool Selection

 

The selection is simply drawn freehand on the image (see Figure 15).

Figure 16: Layers Palette
Figure 16 shows the Layers palette after the selection was blurred and added, as a mask, to the layer that was exposed for the sky (the blurring and creation of the mask are done as in the previous example). Figure 17 shows the final mask.
Figure 17: Final Mask
The main benefit of the Lasso tool is its flexibility. It is a great tool when an odd shaped, hand drawn selection is required.

Selection: Polygonal Lasso Tool

The second Lasso tool is the Polygonal Lasso. This tool allows a photographer to create a hand drawn selection that has straight edges. 

Figure 18: Options Bar for Polygonal Lasso Tool

Figure 18 shows the Options bar for the Polygonal Lasso tool. The Polygonal Lasso Options bar has the same Selection, Feather, and Anti-alias options as covered in the Marquee tools section.

Figure 19: Image with that Needs Curves Work on the Stone
The Polygonal Lasso tool is a good choice when an irregular shaped selection that does require straight edges needs to be created. Figure 19 shows an example where the Polygonal Lasso Tool is a good choice. This image shows the ruins of a building in an old ghost town. It is desired to use Curves to make tonal edits on the stone without affecting the rest of the image. A selection is needed that can be added to a Curves layer.
Figure 20: Image with Polygonal Lasso Tool Selection
The selection is drawn by clicking the Polygonal Lasso Tool at each point where the end of a line segment should be (see Figure 20). In this image, the outer borders of the stone were selected; then, the inner borders where subtracted from the selection.
Figure 21: Layers Palette
Figure 21 shows the Layers palette after the selection was blurred and added to the Curves layer. Figure 22 shows the final mask.
Figure 22: Final Mask
As with the previous tool, the main benefit of the Polygonal Lasso tool is its flexibility. It is a good tool when an odd shaped, hand drawn selection that requires straight edges is required.

Selection: Magnetic Lasso Tool

The last Lasso tool is the Magnetic Lasso. This is a great tool when a selection is needed that borders high contrast edges.

Figure 23 shows the Options bar for the Magnetic Lasso tool. The Magnetic Lasso Options bar has the same Selection, Feather, and Anti-alias options as covered in the Marquee tools section. However, there are additional options that do not exist with the other Lasso tools. The width option determines the width (in pixels) over which the Magnetic Lasso tool will detect an edge. The larger the width, the farther from the curser the Magnetic Lasso tool will be able to detect an edge. The Edge Contrast determines how much contrast is needed before the Magnetic Lasso tool can detect an edge. The larger the Edge Contrast setting (in percent), the greater the contrast that will be needed for the Magnetic Lasso tool to detect an edge. The Frequency determines how often the Magnetic Lasso tool anchors the selection to the edge. When enabled, the Stylus Pressure icon varies the width over which the Magnetic Lasso will detect an edge. 

Figure 23: Options Bar for Magnetic Lasso Tool
Figure 24: Image with that Needs Curves Work on the Sky
Figures 24 shows an example where the magnetic Lasso Tool works well. The image shows the clouds of a storm front that contrast with the much darker mountain range. It is desired to use Curves to work on the cloud contrast without affecting the rest of the image. A selection is needed that can be added to a Curves layer.
Figure 25: Image with Magnetic Lasso Tool Selection

The selection is drawn by clicking the Magnetic Lasso Tool at the starting point and dragging it across the edge (see Figure 25). The Magnetic Lasso tool will add anchor points allow the edge. Clicking along the edge will manually add an anchor point. Hitting the Delete key will delete the last anchor point.

Figure 26: Layers Palette
Figure 26 shows the Layers palette after the selection was blurred and added to the Curves layer. Figure 27 shows the final mask.
Figure 27: Final Mask
The Magnetic Lasso tool works well for selections on images with high contrast edges as long as there is not too much fine detail along the edges. For instance, if the mountain range in this image had a large number of pine trees along the ridge, the Magnetic Lasso tool might not have been the best selection tool.

Selection: Magic Wand Tool

Figure 28: Tools Palette

One of the most popular selection tools is the Magic Wand. The Magic Wand tool works best on images where the selection is based on color. The Magic Wand can be found on the Tools palette (see Figure 28).

Figure 29 shows the Options bar for the Magic Wand tool. The Magic Wand Options bar has the same Selection and Anti-alias options as covered in the Marquee tools section. However, there are additional options that are unique to the Magic Wand tool. The Tolerance option determines the range of colors that will be selected. A low Tolerance setting will cause the Magic Wand to select only colors that are very similar to the color that was clicked. A high Tolerance setting will cause the Magic Wand to select more colors. When the contiguous option is checked, the Magic Wand selects only areas that are bordering the pixels that are clicked. Otherwise, all colors within the selected color range will be selected no matter where they are located in the image. When the Sample All Layers option is checked, the Magic Wand will base the selection on all of the layers. Otherwise, the selection will be based on the colors in the active layer only.

Figure 29: Options Bar for Magic Wand Tool
Figure 30: Image that Needs Work on the Leaf
Figures 30 shows an image with a red leaf that needs some hue/saturation work. A selection is needed that can be added to a Hue/Saturation layer.
Figure 31: Image with Magic Wand Tool Selection
The selection is made by clicking on the red leaf (see Figure 31). Depending on the image, it may be necessary to click several times, on different areas, with the Selection option set to Add to Selection in order to add colors to the selection. If undesired areas get selected, it may be necessary to click on those areas with the selection option set to Subtract from Selection to subtract colors.
Figure 32: Layers Palette
Figure 32 shows the Layers palette after the selection was blurred and added to the Hue/Saturation layer. Figure 33 shows the final mask.
Figure 33: Final Mask
While the Magic Wand works well with images, such as this one, where the color of the object to be selected contrasts with the surrounding colors. If this is not to case, another selection tool should be utilized.

Articles

Selections -- Part II