Metahumans - Facial Animation

Record mocap data and use it to create Metahuman facial animation

Capture Your Performance with Live Link Face

  1. Download the Live Link Face app to your iPhone. iPhone 11 and later are supported.*

  2. Open Live Link Face and select "Metahuman Animator" mode.

  3. Calibrate the app to the performer

  4. IMPORTANT: For each performer, capture one recording of them moving between the following poses (and holding each pose for a moment). These poses will be used later on to create a Metahuman ID (MHID):

  5. Record your performances.

  6. Transfer your performances to your computer. I do so, by uploading to Microsoft OneDrive, but any method that preserves the file quality should work.

*Note about iPhone 15*

Metahuman Animator does not yet officially support iPhone 15, so you will see a warning when you open the LiveLink app. However, this process will still work with iPhone 15. I have not been able to determine yet whether working with an iPhone 15 negatively impacts the quality of the final facial animations.

Ingest Footage in Unreal and Create an MHID

  1. In Unreal, create a Capture Source:

    1. Right click in the Content Browser and select: [Metahuman Animator --> Capture Source]
    2. Select "LiveLinkFace Archives".
    3. For "Storage Path", select the folder containing your Live Link Face footage. If you have multiple recordings, this will be the folder containing the recording subfolders, NOT the subfolders themselves.

  2. Ingest your footage:

    1. In the top menu bar, select: [Tools --> Capture Manager]
    2. In the capture manager window that opens, select your device on the left  side. Your device is the Capture Source you created earlier.
    3. You should see your footage. Select the footage you would like to ingest, select the target folder, click "Add to Queue", and then in the queue panel on the right, click "Import".

  3. Set up a Metahuman Identity (MHID) for each performer using the pose performance captured above. An MHID helps Unreal properly map the performer's face to the Metahuman facial rig. To learn how to properly set up a MHID, I recommend watching this video from Unreal:

Create Metahuman Facial Animation

  1. Right click in the Content Browser and select: [Metahuman Animator --> Metahuman Performance Asset]

  2. Open the Metahuman Performance Asset.

  3. Open the Metahuman Performance Asset and select the desired performance footage along with the MHID for the performer in that footage.

  4. Set In/Out points on the timeline. If you want the option of exporting neck movement, be sure that this range contains at least one frame with the performer facing the camera directly.

  5. Click "Process" in the upper left toolbar. Processing may take awhile as the engine will make multiple passes of the footage.

  6. Once processing is complete, export the animation:

    1. Select "Export Animation".
    2. For "Target Skeleton or Skeletal Mesh" select [YourMetahumanName]_FaceMesh
    3. Export

    Tip

    You can export the same animation sequence to different Metahuman face meshes in order to use the same animation on different Metahumans.

Contact: nolan@frankensteinlearning.com
Copyright © 2023 Frankenstein Learning. All rights reserved.