Oh another limitation that I encountered with deepfakes was that it only really substitutes the middle of the face (i.e. from the eyebrows and below). The edges of the face and forehead don't always get swapped. I encountered this issue when trying to swap a girl with tiger face paint onto another girl. The result was ridiculous without any stripe patterns on the forehead and edges of the cheeks/chin.
I tried with leopard makeup with even worse results. The AI was mistaking some spot patterns for eyes and nostrils etc and resulted in completely unusable swaps. We're a LONG way away from doing the sort of stuff you guys are wanting, and likely will never get there with the GAN method of neural net learning. I think an AI would need to be trained by a significant amount of human input using reinforcement learning in the training process. You basically have the AI attempt to tag each part of the face and then get a human to correct the tags on every single image, then feed the set back through again until the AI gets it right. Only once the AI can recognise each part of the face accurately, can it start to map important points from image set A to image set B.
Below are my attempts at swapping a girl with basic cat makeup onto Kate Upton using deepfakes and Snapchat AR (you'll notice that Snapchat actually does a way better job)
Deepfake:
https://gfycat.com/recklessmindlessalligator
Snapchat:
https://gfycat.com/honorableactivekinglet