Blendartrack is an augmented reality camera motion and face tracking app for 3D artists. Blendartrack comes with an Add-On for Blender 2.9+ to import the captured data easily.
The aim with blendartrack is to contribute a toolset for the blender community which provides access to the augmented reality technologies featured by ArCore and ArKit.
understanding the environment
To create a decent motion capturing it is recommend to move the device around slowly. Physically moving around helps the device to understand the environment faster. It’s recommend to point the device to sufficiently textured surfaces while moving around while avoiding reflective and transparent surfaces. Reflections and strong backlight might lead to drifting or similar unexpected behaviours. Slow and calm movements preferably in diffuse lightning conditions will provide the best results. Yellow dots represent the dot cloud. White dots are showing planar surfaces and are an indicator for a good environmental understanding.
tracking the camera motion
Basically, it is what you see is what you get along you have a reference point in frame. It’s possible to experience a short lag or drift when starting the recording and may takes a shot time to readjust. If you experience frame drops during your recording session it’s recommended to reduce the video quality in the app settings. Placing reference objects on surfaces will greatly enhance the tracking quality. It is highly recommend to always have a reference object in frame. It’s also recommended to refresh the tracking session from time to time to clear the cache.
tracking facial expressions
Tracking facial expressions is usually pretty straight forward. Just position you face within the camera frame and hit the record button. Consider to mount your device at a static location and avoid head movements if you plan to clamp the face to a character in blender. While recording it is recommended to have good lightning conditions, avoid backlights and remove reflective surfaces in the background.
The Add-On currently supports Blender 2.9+. After installing it in Blenders Preferences you’ll find in the Viewport Sidepanel (reached by pressing ‘n’) the import functions. Just navigate to your (.zip) tracked footage and press the import button!