Skip to main content

Add support for multiple character images to improve rendering of teeth

Feature Request

My initial experiments suggest that when supplying photo character images the most realistic lip-syncing results come if the character has their mouth in a neutral, closed position. However, this leads to a problem in that then the model doesn’t have any knowledge of the appearance of the character’s teeth or mouth interior and so it essentially has to guess, leading to unrealistic results.

My proposal is that you should make it possible to supply additional reference images for a character, which should then be taken into account by the model when creating the final video sequence. Users could be encouraged to supply at least one photo of the character with their teeth visible. Overall, this should lead to more realistic results.

Posted by Chris Thompson ·
Status: Rejected3 comments

Log in to comment and vote

Comments3

  • Michael Lingelbach

    Team•

    Nov 9, 2024

    I’m going to close this out, as we know why the teeth look the way they do, and there’s no benefit to adding reference images.

    • Chris Thompson

      •

      Nov 9, 2024

      OK, although I’m a little surprised you think a reference image showing the character’s teeth wouldn’t help. Without one, the model can surely only predict the teeth based on typical dentition whereas if it used a reference image it should be able take individual differences into account.

  • Sunil Ratnayake

    •

    Nov 1, 2024

    Yes it’s good Idea