Augmented reality holds unique potential to provide a bridge between real-world activities and digital enhancements. Disney has developed a new technology to live map color from a 2D coloring book onto a animated 3D model. The software takes the image of the character in the coloring book and maps the colors applied by the user to a pre-made 3D animation of the character.
An app in which children color characters in a printed coloring book and inspect their work using a mobile device. The drawing is detected and tracked, and the video stream is augmented with an animated 3-D version of the character that is textured according to the child’s coloring.
A texturing process applies the captured texture from a 2-D colored drawing to both the visible and occluded regions of a 3-D character in real time. Disney developed a deformable surface tracking method designed for colored drawings that uses a new outlier rejection algorithm for real-time tracking and surface deformation recovery.
Source: Disney Research