Learn Photoshop CS4

Rotoscoping using After Effects

and Photoshop
Both Adobe® Photoshop® and Adobe After Effects® allow you to rotoscope footage. Rotoscoping is the art of
altering footage frame-by-frame. In this tutorial, you will learn the art of rotoscoping using animated paint tools in
Photoshop and animated masks in After Effects.

REQUIREMENTS
To complete the task(s) demonstrated in this tutorial, you will need the following software and files:
• Products – Adobe Photoshop CS4
• Sample file – lrvid4107_xp.zip
• Prerequisite knowledge – Intermediate knowledge of painting and masking inside Photoshop and After Effects.

Rotoscoping in Photoshop using the Clone Stamp tool

The Clone Stamp tool in Photoshop allows you to paint over unwanted elements in your footage by copying or
cloning from other parts of the frame. To use the Clone Stamp tool:
1. Open a movie file in Photoshop. Choose Window > Animation. You will see a layered timeline in the Animation
(Timeline) panel at the bottom of the Photoshop window.
2. To focus on a specific portion of the animation, move the current-time indicator to a point in the Timeline.

3. Move the Work Area Start slider to the beginning of the focus area and the Word Area End slider to the end.

4. To view a looped preview of the selected work area, click the Play button at the bottom of the Animation panel.

Figure 1: Playing a looped preview

© 2008 Adobe Systems Incorporated and its licensors. All rights reserved. Adobe, the Adobe logo and Adobe trademarks used in this tutorial are
either registered trademarks or trademarks of Adobe Systems Incorporated and/or other countries. For more information on Adobe trademarks,
please go to http://www.adobe.com/misc/agreement.html. All other trademarks are the property of their respective owner.
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

5. Use the Zoom tool to enlarge the portion of footage you are working on.

6. Choose the Clone Stamp tool from the toolbox.

Figure 2: The Clone Stamp tool

7. Choose Window > Clone Stamp to open the Clone Source panel. Select Show Overlay to display a transparent
image of the source point while you are painting with the Clone Stamp tool.
8. Look for an area of your footage to sample for the source point. Use an area of the footage that does not contain
the element you want to remove, but that will match the area you will be painting over. Press Option/Alt-click to
set your source point.
9. Line up the source point with the footage you want to remove. Click to set the position relative to the source point.

10. Once you have set your position, paint over the areas of the frame that you want to remove. Move the
current-time indicator backwards and forwards through the frames and continue painting to remove all traces
of the element.

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

Tip: Use the Select Next/Previous Frame buttons at the bottom of the Animation panel to ensure that you have
reviewed every frame and painted over every aspect of the element being removed from the footage.

Figure 3: Reviewing footage

11. Change the brush size and brush type as needed to complete the cloning process. You can change the brush
diameter and hardness using the Brush Preset picker in the Options bar when the Clone Stamp tool is selected.

Figure 4: Changing the brush diameter and hardness

12. You can review your changes in the Clone Source panel. Click the Toggle The Clone Source Panel icon in the
upper-right corner of the Options bar to open and close the panel.

Figure 5: The Clone Source panel

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

Using animated masks in After Effects

Creating animated masks from imported Photoshop documents is another way to disguise unwanted elements
in your footage. The advantage of animated masks is that you don’t have to paint frame-by-frame to remove an
element from a scene. To create an animated mask:

Importing a document from Photoshop

To create a mask, you first need to import a Photoshop document into After Effects. To import the document:
1. In Photoshop, choose File > Save As to save the movie you were working with in the previous section as a .psd
file. When the Save As dialog box opens, select Photoshop as the format and click Save.
2. In After Effects, choose File > Import. When the Import File dialog box opens, select All Acceptable File Types
from the Enable menu, select the .psd file you saved in Step 1 and click OK. Note that the Import File dialog box
resembles the native OS window, either Finder (Mac OS) or Windows Explorer (Windows). Metadata about the file
you selected is displayed in Column or Detail view, including file size, kind, and created and modified dates.

Figure 6: The Layer Options dialog box

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

3. Once you click OK, the Layer Options dialog box opens. Choose Footage from the Import Kind menu and make
sure the Merged Layers radio button is selected under Layer Options. Click OK. The file appears in the Project
panel of After Effects.

Figure 7: Import File dialog box

4. Drag the imported file to the New Composition icon in the Project panel. You have successfully created a new
After Effects composition that has the same length, frame rate, and size of the source file.

Figure 8: Creating a new composition

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

Creating a mask
Creating an animated mask in After Effects is an excellent way to rotoscope footage. To create a mask:
1. Press Command/Ctrl+D to duplicate the composition on a new layer. You’ll make your adjustments to this layer,
so you don’t alter the original footage.
2. Move the current-time indicator to the point at which the element that you would like to mask appears. Use the
Zoom tool as needed to get a precise view of the element.
3. Trim the layer so that only those frames containing the element are included, by dragging the In and Out points
to the correct start and end points in the Timeline.

Figure 9: Expanding to full-screen view

4. Press the tilde key to view the frame in full-screen view. Select Fit Up To 100% from the Magnification Ratio
menu in the lower-left corner of the After Effects window to get the maximum screen size possible.
5. Choose the Pen tool from the toolbox.

Figure 10: Selecting the Pen tool

6. Begin drawing around the part of the footage that contains the element you want to mask. Note that your
drawing here does not have to be precise. A simple outline around the element will do.

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

7. Add a second mask (Mask 2) that separates the set from the elements you want to keep in your footage. Make
sure you do not include any points from the first mask when drawing the second mask.

Figure 11: Drawing animated masks

8. Use the tilde key to get out of the full-screen view so you start to add keyframes and animate the second mask layer.

Tip: You can use the arrow-shaped Pointer tool from the toolbox to drag the anchor points of the second mask’s outline
and adjust its position.

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

Animating the mask

Because you are working with moving footage, the masks you create in your composition also have to move. To
animate a mask:
1. Click the colored square icon next to one of the masks. A Mask Color dialog box opens.

Figure 12: Changing the color of a mask

2. Select a color that differs from that of the other mask and click OK. You will now see the two mask outlines in
two different colors in the Composition panel.
3. Click the triangle next to Mask 2, the mask you are going to use to hide another footage element, to reveal the
mask properties. The property you will be animating is the Mask Path.
4. Move the current-time indicator to the beginning of the trimmed piece of footage and click to add a keyframe to
the layer in the Timeline representing the Mask 2 Mask Path property. Then move the current-time indicator to the
Out point of the trimmed layer and click again to set a second keyframe.
5. Make sure the second Mask Path keyframe is selected, zoom out and double-click the mask. Drag it to move it
to the right, outside the footage area into pasteboard. You have just animated the mask. Scrub the current-time
indicator to see Mask 2 move.

Tutorial
 Learn Photoshop CS4 
Rotoscoping using After Effects and Photoshop

6. To ensure that this mask moves at the same rate as the rest of the elements in the footage, you can add an ease.
To add an ease, choose Animation > Keyframe Assistant > Easy Ease Out.

Figure 13: Adding an Ease

7. Adjust the starting and ending keyframes as needed to ensure that Mask 2 stays in line with the elements it
contains.
8. Set the mask mode of Mask 2 to Subtract by clicking the menu next to the mask’s name in the Timeline panel.

9. Collapse the Timeline so that only the layer names are showing. Select the layer containing the masks.

10. To remove the element in Mask 1 from the footage, hover the cursor over the layer outside of the trimmed
portion. A double-ended arrow allows you to slide the timing of the layer. Slide the layer such that the element in
Mask 1 is entirely removed from the trimmed section of the footage.
11. Scrub the Timeline to ensure that the element is no longer included in your footage. You may need to nudge
some keyframes to ensure the correct position of the masks.
12. To reduce the harshness of the masking effect even further, reveal the Mask Feather property for Mask 2 by
clicking the arrow next to the mask name.

Figure 14: Revealing Mask properties

Tutorial
 Learn Photoshop CS4 10
Rotoscoping using After Effects and Photoshop

13. Adjust the Feather value to 25 pixels.

Figure 15: Adjusting the feather value

Exporting an After Effects composition to Photoshop

In some cases, you may need to export an After Effects file back to Photoshop for further modifications. To export
your composition:
1. Choose Composition > Make Movie.

2. Choose a lossless render option that does not add any compression and that is compatible with Photoshop. The
default Lossless setting meets these criteria.
3. You can now transfer compositions seamlessly between After Effects and Photoshop as many times as necessary.

Where to go from here

Photoshop CS4 Help and Support Center
Learn Photoshop CS4
Introducing Photoshop CS4, by Deke McClelland
Introducing After Effects CS4, by Chad Perkins
Exporting from After Effects to Flash via XFL, by Todd Perkins
Lynda.com

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Tutorial