3D Face Reconstruction

From a photo of a face, service predicts gender, shape blendshapes, UV face texture, hair color, skin color, the presence of glasses.

1. Getting a token

Getting a token

2. Choice of photo

Choose a photo of a person.

Note: Please send a photo of one person.

Note: The quality of the photo must be a fairly good resolution.

Note: The face should not be a full face and not covered by foreign objects.

Note: Lighting should not have hard drops and shadows.

Note: The photo should have neutral lighting. Otherwise the skin color will be unrealistic.

Note: The correct result is issued for persons over 18 years of age. Children may have problems with gender determination.

Note: The best result will be with neutral facial emotions.

Need a successful image? Download a sample

3. Reconstruct 3D Face

Make a request by sending your chosen photo.

curl

token="ENTER_YOUR_TOKEN"
image_path="ENTER_IMAGE_PATH"

curl -X 'POST' 'https://api.metahumansdk.io/face_recon/run_pipeline' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F "token=$token" \
  -F "image_bytes=@$image_path;type=image/png" > head_config.json

python

import os
import requests
import json

token = "ENTER_YOUR_TOKEN"
image_path = "ENTER_IMAGE_PATH"

if __name__ == "__main__":
    service_url = "https://api.metahumansdk.io/face_recon"
    service_headers = {"accept": "application/json"}
    files = {
        "token": (None, token),
        "image_bytes": ("image", open(image_path, "rb"), "image/png")
    }
    response = requests.post(service_url + "/run_pipeline", headers=service_headers, files=files, timeout=60)
    if response.status_code == 200:
        with open("head_config.json", "w+") as f:
            json.dump(response.json(), f)

Note: Do not forget to replace the details in the request with your own.

Expected output:

After successful execution of the code, you should receive the following type of json in head_config.json file:

{
  "result": {
    "face": {
      "textureUrl": str,
      "blendShapes": list,
      "blinkCorrectionCoeff": double
    },
    "hair": {
      "color": hex color,
      "type": str
    },
    "skin": {
      "color": hex color
    },
    "accessories": list,
    "gender": str
  }
}

At the moment we have received information about the person. Now let's download the face texture.

4. Download UV face texture

The json we received in the last step contains the url where we can download the texture.

curl

texture_url=$(cat head_config.json | python -c 'import sys, json; print(json.load(sys.stdin)["result"]["face"]["textureUrl"])')

curl -X 'GET' "https://api.metahumansdk.io/face_recon/${texture_url}" > uv.png

python

import os
import requests
import json

if __name__ == "__main__":
    service_url = "https://api.metahumansdk.io/face_recon"
    service_headers = {"accept": "application/json"}
    with open("head_config.json", 'r') as f:
        config = json.load(f)
    response = requests.get(service_url + config["result"]["face"]["textureUrl"])
    assert (response.status_code == 200)
    with open("uv.png", "wb") as f:
        f.write(response.content)

After the completed stages, you get a uv texture uv.png and information about the face head_config.json.

5. Visualization

In order to get a 3D model of the head, we will use GLB Constructor:

curl

token="ENTER_YOUR_TOKEN"

curl -X 'POST' 'https://api.metahumansdk.io/glb_const/assemble' \
    -H 'accept: application/json' \
    -H 'Content-Type: multipart/form-data' \
    -F "token=$token" \
    -F 'head_uv=@uv.png;type=image/png' \
    -F 'head_config=@head_config.json;type=application/json' \
    -F 'add_facs=true' > head.glb

python

import json
import requests

token = "ENTER_YOUR_TOKEN"

if __name__ == "__main__":
    service_url = "https://api.metahumansdk.io/glb_const"
    service_headers = {'accept': 'application/json'}
    
    with open('uv.png', 'rb') as out:
        head_uv = out.read()
    with open('head_config.json', 'r') as out:
        head_config = json.loads(out.read())

    files = {
        'token': (None, token),
        'head_uv': ('head_model', head_uv, 'image/png'),
        'head_config': ('head_config', json.dumps(head_config), 'application/json'),
        'add_facs': (None, True),
    }

    response = requests.post(service_url + "/assemble", headers=service_headers, files=files)
    assert (response.status_code == 200)

    with open("head.glb", "wb") as f:
        f.write(response.content)

After the completed stages, you get a glb model head.glb .

image.png

head.glb

To add hair to the head model please follow to 3D Hair Reconstruction.