224 lines
4.7 KiB
Markdown
224 lines
4.7 KiB
Markdown
# Facial Capture - MetaHuman Animator
|
|
|
|
## Overview
|
|
|
|
This document describes the facial capture workflow using MetaHuman Animator for the Dubai Metaverse project.
|
|
|
|
## MetaHuman Animator
|
|
|
|
### Overview
|
|
|
|
MetaHuman Animator is a tool for capturing facial animation from video or live performance and applying it to MetaHumans in Unreal Engine.
|
|
|
|
### Requirements
|
|
|
|
- **MetaHuman Animator Plugin**: Enable in Unreal Engine
|
|
- **Camera**: High-quality camera (webcam, DSLR, or phone)
|
|
- **Lighting**: Good lighting conditions
|
|
- **Calibration**: Facial calibration process
|
|
|
|
---
|
|
|
|
## Setup
|
|
|
|
### Plugin Installation
|
|
|
|
1. **Enable Plugin**:
|
|
- Edit > Plugins
|
|
- Search for "MetaHuman Animator"
|
|
- Enable plugin
|
|
- Restart editor
|
|
|
|
2. **Verify Installation**:
|
|
- Check plugin is active
|
|
- Verify MetaHuman Animator tools are available
|
|
|
|
### Camera Setup
|
|
|
|
1. **Camera Selection**:
|
|
- **Webcam**: Basic setup
|
|
- **DSLR**: Higher quality
|
|
- **Phone**: Convenient option
|
|
|
|
2. **Lighting**:
|
|
- **Even Lighting**: Avoid harsh shadows
|
|
- **Front Lighting**: Light from front
|
|
- **Avoid Backlighting**: Avoid strong backlighting
|
|
|
|
3. **Positioning**:
|
|
- **Distance**: 2-3 feet from camera
|
|
- **Angle**: Face camera directly
|
|
- **Stability**: Keep head still
|
|
|
|
---
|
|
|
|
## Calibration
|
|
|
|
### Calibration Process
|
|
|
|
1. **Start Calibration**:
|
|
- Open MetaHuman Animator
|
|
- Start calibration process
|
|
- Follow on-screen instructions
|
|
|
|
2. **Calibration Steps**:
|
|
- **Neutral Expression**: Maintain neutral expression
|
|
- **Head Movement**: Follow head movement instructions
|
|
- **Facial Expressions**: Perform requested expressions
|
|
- **Completion**: Complete calibration
|
|
|
|
3. **Calibration Validation**:
|
|
- Review calibration quality
|
|
- Re-calibrate if needed
|
|
- Save calibration
|
|
|
|
---
|
|
|
|
## Recording
|
|
|
|
### Live Performance
|
|
|
|
1. **Start Recording**:
|
|
- Begin live performance
|
|
- Speak dialogue
|
|
- Perform expressions
|
|
|
|
2. **Performance Tips**:
|
|
- **Clear Speech**: Speak clearly
|
|
- **Natural Expressions**: Use natural facial expressions
|
|
- **Head Movement**: Natural head movement
|
|
- **Eye Contact**: Look at camera
|
|
|
|
3. **Stop Recording**:
|
|
- Stop recording when complete
|
|
- Review performance
|
|
- Process if satisfied
|
|
|
|
### Video Recording
|
|
|
|
1. **Record Video**:
|
|
- Record video of performance
|
|
- Ensure good quality
|
|
- Follow recording guidelines
|
|
|
|
2. **Import Video**:
|
|
- Import video to MetaHuman Animator
|
|
- Process video
|
|
- Extract facial animation
|
|
|
|
---
|
|
|
|
## Processing
|
|
|
|
### Animation Processing
|
|
|
|
1. **Process Performance**:
|
|
- MetaHuman Animator processes performance
|
|
- Extracts facial animation
|
|
- Generates animation data
|
|
|
|
2. **Review Animation**:
|
|
- Review processed animation
|
|
- Check quality
|
|
- Refine if needed
|
|
|
|
3. **Export Animation**:
|
|
- Export animation to Unreal Engine
|
|
- Apply to MetaHuman
|
|
- Test in level
|
|
|
|
---
|
|
|
|
## Application
|
|
|
|
### Apply to MetaHuman
|
|
|
|
1. **Select MetaHuman**:
|
|
- Select MetaHuman in level
|
|
- Open MetaHuman Animator
|
|
|
|
2. **Apply Animation**:
|
|
- Apply captured animation
|
|
- Preview animation
|
|
- Adjust if needed
|
|
|
|
3. **Integration**:
|
|
- Integrate with dialogue system
|
|
- Sync with audio
|
|
- Test in context
|
|
|
|
---
|
|
|
|
## Best Practices
|
|
|
|
### Recording
|
|
|
|
1. **Quality**: Ensure good recording quality
|
|
2. **Lighting**: Use good lighting
|
|
3. **Stability**: Keep head stable
|
|
4. **Natural**: Use natural expressions
|
|
|
|
### Processing
|
|
|
|
1. **Calibration**: Ensure good calibration
|
|
2. **Review**: Review processed animation
|
|
3. **Refinement**: Refine if needed
|
|
4. **Testing**: Test in context
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
**Issue**: Calibration fails
|
|
- **Solution**: Check lighting conditions
|
|
- **Solution**: Ensure face is clearly visible
|
|
- **Solution**: Follow calibration instructions carefully
|
|
|
|
**Issue**: Animation quality poor
|
|
- **Solution**: Improve recording quality
|
|
- **Solution**: Re-calibrate
|
|
- **Solution**: Check lighting
|
|
|
|
**Issue**: Animation not applying
|
|
- **Solution**: Check MetaHuman Animator plugin
|
|
- **Solution**: Verify animation export
|
|
- **Solution**: Check MetaHuman setup
|
|
|
|
---
|
|
|
|
## Use Cases
|
|
|
|
### NPC Dialogue
|
|
|
|
- **Purpose**: Animate NPCs during dialogue
|
|
- **Implementation**: Capture dialogue performance
|
|
- **Application**: Apply to MetaHuman NPCs
|
|
|
|
### Cinematic Sequences
|
|
|
|
- **Purpose**: Animate characters in cinematics
|
|
- **Implementation**: Capture performance for cinematics
|
|
- **Application**: Apply to cinematic MetaHumans
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
### Performance Documentation
|
|
|
|
Document each performance:
|
|
- **Purpose**: What the performance is for
|
|
- **MetaHuman**: Which MetaHuman it's applied to
|
|
- **Context**: Where it's used
|
|
- **Quality**: Animation quality notes
|
|
|
|
---
|
|
|
|
**Version**: 1.0
|
|
**Last Updated**: [Current Date]
|
|
|
|
**Note**: Facial capture is optional. NPCs can function without facial capture using standard animation.
|
|
|