3D Face Reconstruction

From a photo of a face, service predicts gender, shape blendshapes, UV face texture, hair color, skin color, the presence of glasses.

1. Getting a token

Getting a token

2. Choice of photo

Choose a photo of a person.

Note: Please send a photo of one person.

Note: The quality of the photo must be a fairly good resolution.

Note: The face should not be a full face and not covered by foreign objects.

Note: Lighting should not have hard drops and shadows.

Note: The photo should have neutral lighting. Otherwise the skin color will be unrealistic.

Note: The correct result is issued for persons over 18 years of age. Children may have problems with gender determination.

Note: The best result will be with neutral facial emotions.

Need a successful image? Download a sample

3. Reconstruct 3D Face

Make a request by sending your chosen photo.

token="ENTER_YOUR_TOKEN"
image_path="ENTER_IMAGE_PATH"

curl -X 'POST' 'https://api.metahumansdk.io/face_recon/run_pipeline' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F "token=$token" \
  -F "image_bytes=@$image_path;type=image/png" > head_config.json

import os
import requests
import json

token = "ENTER_YOUR_TOKEN"
image_path = "ENTER_IMAGE_PATH"

if __name__ == "__main__":
    service_url = "https://api.metahumansdk.io/face_recon"
    service_headers = {"accept": "application/json"}
    files = {
        "token": (None, token),
        "image_bytes": ("image", open(image_path, "rb"), "image/png")
    }
    response = requests.post(service_url + "/run_pipeline", headers=service_headers, files=files, timeout=60)
    if response.status_code == 200:
        with open("head_config.json", "w+") as f:
            json.dump(response.json(), f)

Note: Do not forget to replace the details in the request with your own.

Expected output:

After successful execution of the code, you should receive the following type of json in head_config.json file:

{
  "result": {
    "face": {
      "textureUrl": str,
      "blendShapes": list,
      "blinkCorrectionCoeff": double
    },
    "hair": {
      "color": hex color,
      "type": str
    },
    "skin": {
      "color": hex color
    },
    "accessories": list,
    "gender": str
  }
}

At the moment we have received information about the person. Now let's download the face texture.

4. Download UV face texture

The json we received in the last step contains the url where we can download the texture.

texture_url=$(cat head_config.json | python -c 'import sys, json; print(json.load(sys.stdin)["result"]["face"]["textureUrl"])')

curl -X 'GET' "https://api.metahumansdk.io/face_recon/${texture_url}" > uv.png

import os
import requests
import json

if __name__ == "__main__":
    service_url = "https://api.metahumansdk.io/face_recon"
    service_headers = {"accept": "application/json"}
    with open("head_config.json", 'r') as f:
        config = json.load(f)
    response = requests.get(service_url + config["result"]["face"]["textureUrl"])
    assert (response.status_code == 200)
    with open("uv.png", "wb") as f:
        f.write(response.content)

After the completed stages, you get a uv texture uv.png and information about the face head_config.json.

5. Visualization

In order to get a 3D model of the head, we will use GLB Constructor:

token="ENTER_YOUR_TOKEN"

curl -X 'POST' 'https://api.metahumansdk.io/glb_const/assemble' \
    -H 'accept: application/json' \
    -H 'Content-Type: multipart/form-data' \
    -F "token=$token" \
    -F '[email protected];type=image/png' \
    -F 'head_config=@head_config.json;type=application/json' \
    -F 'add_facs=true' > head.glb

import json
import requests

token = "ENTER_YOUR_TOKEN"

if __name__ == "__main__":
    service_url = "https://api.metahumansdk.io/glb_const"
    service_headers = {'accept': 'application/json'}
    
    with open('uv.png', 'rb') as out:
        head_uv = out.read()
    with open('head_config.json', 'r') as out:
        head_config = json.loads(out.read())

    files = {
        'token': (None, token),
        'head_uv': ('head_model', head_uv, 'image/png'),
        'head_config': ('head_config', json.dumps(head_config), 'application/json'),
        'add_facs': (None, True),
    }

    response = requests.post(service_url + "/assemble", headers=service_headers, files=files)
    assert (response.status_code == 200)

    with open("head.glb", "wb") as f:
        f.write(response.content)

After the completed stages, you get a glb model head.glb .

To add hair to the head model please follow to 3D Hair Reconstruction.

PreviousGetting a token Next3D Hair Reconstruction

Last updated 2 years ago

hashtag1. Getting a token

hashtag2. Choice of photo

hashtag3. Reconstruct 3D Face

hashtag4. Download UV face texture

hashtag5. Visualization

1. Getting a token

2. Choice of photo

3. Reconstruct 3D Face

4. Download UV face texture

5. Visualization