Conversion of vertex-colored meshes to UV-mapped textured meshes

synopsis

Vertex shading is an easy way to apply color information directly to mesh vertices. This approach is commonly used in the construction of generative 3D models such asInstantMeshHowever, most applications prefer to use UV-mapped texturized meshes. However, most applications prefer textured meshes with UV mapping.

This tutorial will present a quick solution for converting vertex-colored meshes to UV-mapped and textured meshes. Contents include [Short Version](# Short Version) to help you get results quickly, and [Detailed Version](# Detailed Version) for in-depth how-to instructions.

short version

mountingInstantTexture library for easy conversion. The library implements the followingdetailed edition The specific steps described in the

pip install git+/dylanebert/InstantTexture

usage

The following code colors the vertices of the.obj Mesh conversion to UV-mapped texture.glb grid and save it as Documentation.

from instant_texture import Converter

input_mesh_path = "/dylanebert/InstantTexture/refs/heads/main/examples/"

converter = Converter()
(input_mesh_path)

Visualize the output of the grid.

import trimesh

mesh = ("")
()

That's it!

If you need more detailed steps, you can continue reading below.

detailed edition

First install the following dependencies.

• numpy For numerical operations
• trimesh For loading and saving grid data
• xatlas For generating UV maps
• Pillow For image processing
• opencv-python For image processing
• httpx For downloading input grids

pip install numpy trimesh xatlas opencv-python pillow httpx

Import dependencies.

import cv2
import numpy as np
import trimesh
import xatlas
from PIL import Image, ImageFilter

Load the input mesh with vertex colors. The file should be.obj format, located in theinput_mesh_path 。

If it is a local file, use the() rather thantrimesh.load_remote() 。

mesh = trimesh.load_remote(input_mesh_path)
()

View the vertex colors of the mesh.

If it fails, make sure the grid is valid.obj file and with vertex colors.

vertex_colors = .vertex_colors

Use xatlas to generate UV maps.

This is the most time-consuming part of the whole process.

vmapping, indices, uvs = (, )

Remaps vertices and vertex colors to UV maps.

vertices = [vmapping]
vertex_colors = vertex_colors[vmapping]

 = vertices
 = indices

Defines the desired texture size.

Constructs a texture buffer byupscale_factor to create higher quality textures.

texture_size = 1024

upscale_factor = 2
buffer_size = texture_size * upscale_factor

texture_buffer = ((buffer_size, buffer_size, 4), dtype=np.uint8)

Fills the texture of the UV-mapped mesh using center-of-mass interpolation.

1. center-of-mass interpolation: Calculated on the vertex of the vertex that consists of thev0 、v1 respond in singingv2 Points within a defined trianglep of the interpolated colors, corresponding to the colorsc0 、c1 cap (a poem)c2 。
2. Points in Triangles Test:: Determination pointp Whether or not it is located at the point defined by the vertexv0 、v1 cap (a poem)v2 within the defined triangle.
3. Texture Fill Loop:

• Iterate over each face of the mesh.
• Retrieve the current UV coordinate (uv0 , uv1 , uv2 ) and color (c0 , c1 , c2 )。
• Converts UV coordinates to buffer coordinates.
• Determines the bounding box of the triangles in the texture buffer.
• For each pixel in the bounding box, check to see if the pixel is inside a triangle, using the point-in-triangle test.
• If internal, use center of gravity interpolation to calculate the interpolated color.
• Assigns the color to the corresponding pixel in the texture buffer.

# Barycentric interpolation
def barycentric_interpolate(v0, v1, v2, c0, c1, c2, p):
    v0v1 = v1 - v0
    v0v2 = v2 - v0
    v0p = p - v0
    d00 = (v0v1, v0v1)
    d01 = (v0v1, v0v2)
    d11 = (v0v2, v0v2)
    d20 = (v0p, v0v1)
    d21 = (v0p, v0v2)
    denom = d00 * d11 - d01 * d01
    if abs(denom) < 1e-8:
        return (c0 + c1 + c2) / 3
    v = (d11 * d20 - d01 * d21) / denom
    w = (d00 * d21 - d01 * d20) / denom
    u = 1.0 - v - w
    u = (u, 0, 1)
    v = (v, 0, 1)
    w = (w, 0, 1)
    interpolate_color = u * c0 + v * c1 + w * c2
    return (interpolate_color, 0, 255)

# Point-in-Triangle test
def is_point_in_triangle(p, v0, v1, v2):
    def sign(p1, p2, p3):
        return (p1[0] - p3[0])*(p2[1] - p3[1]) - (p2[0] - p3[0])*(p1[1] - p3[1])

    d1 = sign(p, v0, v1)
    d2 = sign(p, v1, v2)
    d3 = sign(p, v2, v0)

    has_neg = (d1 < 0) or (d2 < 0) or (d3 < 0)
    has_pos = (d1 > 0) or (d2 > 0) or (d3 > 0)

    return not (has_neg and has_pos)

# Texture-filling loop
for face in :
    uv0, uv1, uv2 = uvs[face]
    c0, c1, c2 = vertex_colors[face]

    uv0 = (uv0 *(buffer_size - 1)).astype(int)
    uv1 = (uv1 *(buffer_size - 1)).astype(int)
    uv2 = (uv2 *(buffer_size - 1)).astype(int)

    min_x = max(int((min(uv0[0], uv1[0], uv2[0]))), 0)
    max_x = min(int((max(uv0[0], uv1[0], uv2[0]))), buffer_size - 1)
    min_y = max(int((min(uv0[1], uv1[1], uv2[1]))), 0)
    max_y = min(int((max(uv0[1], uv1[1], uv2[1]))), buffer_size - 1)

    for y in range(min_y, max_y + 1):
        for x in range(min_x, max_x + 1):
            p = ([x + 0.5, y + 0.5])
            if is_point_in_triangle(p, uv0, uv1, uv2):
                color = barycentric_interpolate(uv0, uv1, uv2, c0, c1, c2, p)
                texture_buffer[y, x] = (color, 0, 255).astype(
                    np.uint8
                )

Let's visualize the texture effect so far.

from  import display

image_texture = (texture_buffer)
display(image_texture)

As we can see, the texture has a lot of voids.

To address this issue, we will combine four techniques.

1. Image Restoration: Fills the void with the average color of the surrounding pixels.
2. median filter: Removes noise by replacing each pixel with the median color of the surrounding pixels.
3. Gaussian blur: Smooth the texture to remove any remaining noise.
4. downsampling: Use LANCZOS resampling to reduce totexture_size 。

# Inpainting
image_bgra = texture_buffer.copy()
mask = (image_bgra[:, :, 3] == 0).astype(np.uint8)* 255
image_bgr = (image_bgra, cv2.COLOR_BGRA2BGR)
inpainted_bgr = (
    image_bgr, mask, inpaintRadius=3, flags=cv2.INPAINT_TELEA
)
inpainted_bgra = (inpainted_bgr, cv2.COLOR_BGR2BGRA)
texture_buffer = inpainted_bgra[::-1]
image_texture = (texture_buffer)

# Median filter
image_texture = image_texture.filter((size=3))

# Gaussian blur
image_texture = image_texture.filter((radius=1))

# Downsample
image_texture = image_texture.resize((texture_size, texture_size), )

# Display the final texture
display(image_texture)

As we can see, the textures are now smoother and free of voids.

It can be further improved with more advanced techniques or manual texture editing.

Finally, we can build a new mesh with the generated UV coordinates and texture.

material = (
    baseColorFactor=[1.0, 1.0, 1.0, 1.0],
    baseColorTexture=image_texture,
    metallicFactor=0.0,
    roughnessFactor=1.0,
)

visuals = (uv=uvs, material=material)
 = visuals
()

That's it! The mesh has been UV mapped and textured.

At the local runtime, you can call the("") to export it.

limitations

As you can see, the mesh still has many small artifacts.

The quality of UV maps and textures is still far from the standard for production-grade meshes.

However, if you are looking for a quick solution to map a vertex shading mesh to a UV mapping mesh, this approach may be helpful.

reach a verdict

This tutorial describes how to convert a vertex shading mesh to a UV-mapped texture mesh.

If you have any questions or feedback, please feel free to ask at theGitHub maybeSpace Ask a question on.

Thank you for reading!

---

Link to original article./blog/vertex-colored-to-textured-mesh
Original article by Dylan Ebert

Translator: cheninwang