The related work of 3D human reconstruction algorithm proposed in this paper was three-dimensional scanning technology, model denoising, model simplification, model fusion, and simulation experiment. Firstly, the head model was obtained by using a cheap Kinect depth camera, and then a smooth model was obtained by removing the fragments in the model, the model denoising, and other preprocessing operations. Secondly, we simplified the complex head model by using the simplified method of preserving model features based on edge curvature and area error proposed in this paper. Finally, the improved FCF model fusion method was used to fuse the head model and the body model, and the personalized hierarchical model library was built, which was finally applied to group simulation.
Preprocessing
The denoising of the initial model can be divided into two steps: One is to delete the fragments generated during scanning. In this paper, we used DFS (depth first search) algorithm to delete fragments. The other is to use the weighted Laplace smoothing algorithm [19] to remove the tiny noise on the model and make the model as smooth as possible. The Laplace smoothing algorithm has a low computational complexity and can control the details of the model very well in operation. Therefore, this paper used the Laplace smoothing algorithm to remove the small noise in the model.
The topological structure of the model was mapped to a graph, and the connectivity of the whole graph can be obtained by using the method of DFS. The largest connected graph was the structure that needed to be retained, and the rest of the disconnected graph was deleted. Figure 1a is the graph before deleting the fragment, and Fig. 1b is the graph after the fragment is deleted.
The specific operation of the Laplace algorithm is as follows: The 3D position of the vertex is moved to the center of gravity position of the surrounding vertexes to minimize the difference between the vertex and the surrounding vertexes. For every point in the model, the space position of the point is recalculated according to the position information of the surrounding vertexes. The Laplace smoothing formula is as follows:
$$ \overline{x_i}=\frac{1}{N}\sum \limits_{j=1}^N{x}_j $$
(1)
N is the number of vertexes around the current point and \( \overline{x_i} \)is the new coordinate of the i-th vertex.
In practice, N cannot be too large. If it is too large, the details may be lost. If N is too small, the smooth effect will not be achieved. Therefore, an improved Laplace smoothing algorithm was used in this paper, which was weighted by Gauss weighted method. For the point far away from the center point, its influence on the center point was reduced, and for the point near the center point, its influence on the center point was increased. Figure 2a is a model before denoising, and Fig. 2b is a model after denoising. It is obvious that the preprocessing method proposed in this paper can effectively remove the small noise on the model.
Detail features preserving
According to the different geometric elements, mesh model simplification method is divided into vertex deletion, edge collapse, triangle deletion, and patch deletion. Edge collapse operation is based on half edge structure and an edge with the smallest triangular patch cost is collapsed in order to achieve the purpose of deleting the simplified model of triangular patch.
When the mesh model is simplified by the simplification method based on geometric elements, the higher the simplification rate, the more serious loss of the details of the model, and the detail features of the model cannot be preserved. The purpose of model simplification is to maintain the detail features of the model to the maximum extent on the basis of reducing the scale of the model. Based on the edge collapse method, an interactive method was proposed in this paper, which can effectively retain the detail features of the model and make the model have higher identifiability.
The detail features of the model can be reflected by the edge curvature and the area change of the triangular patch on the model. If the curvature of the edge is large in a region, the features of the model are obvious here. The change of the area of the related triangle patch caused by the collapse edge also reflects the features of the model here. The smaller the change, the smoother. Therefore, the edge collapse cost can be defined as two errors: one is the curvature of the edge, and the other is the change of the area brought by the deletion point.
As shown in Fig. 3, the way to calculate the edge curvature is:
$$ {E}_c=\beta \left(1-\mathit{\cos}\alpha \right)=\left(\frac{l}{h_1}+\frac{l}{h_2}\right)\left(1-{n}_1{n}_2\right) $$
(2)
Among them, l = ‖q − p‖, hi = d{vi, e} 。.
As shown in Fig. 4, t′ is the reproduced triangular patch while deleting the triangular patch t. This process can be achieved by rotating the triangular patch T by an angle. When the angle of triangle rotation is set as θ, the error Qt resulting from the deletion of one surface t is as follows:
$$ {Q}_t={l}_t\times \theta $$
(3)
Among them, lt = (A1 + A2)/2,A1 is the area of the triangle t = ( v0, v1, v2), A2 is the area of the triangle t′ = (v, v1, v2) . The computation angle θ involves trigonometric operation, so the computation speed is slow. In order to improve the computation speed, let \( \theta =1-{n}_t{n}_{t^{\prime }} \), in which ni is a normal vector of the surface i. As shown in Fig. 4, the area error Ea caused by the deletion edgy is:
$$ {E}_s=\sum \limits_{t\in P}{Q}_t $$
(4)
P is the set of all triangle patches connected to v0. Therefore, the error of the edge collapse is as follows:
Although the detail features of the model can be retained by using this method, with the improvement of the rate of simplification, the detail features of the model will also be lost gradually. In order to keep the detail features of the model, an interactive method was used in this paper to retain the detail features of the model. The simplified operation to preserve the detail features is shown in Algorithm 1:
The complexity of the algorithm is the calculation of the folding cost of each edge. The folding cost of each edge includes two parts: the edge curvature and the area difference. For each edge, the value of both of the two parts should be calculated. Generally, the simplification of the whole model can be realized if the order of all edges is sorted by the size of all edges according to the folding cost of all edges and the edge folding operation. However, the specific details on the model cannot be preserved and it cannot guarantee the real sense of the model through the simple model can be obtained according the simplification method mentioned above. Therefore, we introduce an interactive method to mark the area that needs to be preserved by a manual way when choosing the folding edge, so as to retain the local features with special significance. In this paper, a random weight is set up for the folding cost of the edge of the reserved region, that is (rand() + a). The average folding cost of all edges is added to obtain the edges of the reserved area with a larger folding cost, so that the edges of the feature area can be retained effectively. For the head model collected by Kinect, we can adjust the value of a to control the simplification rate of the reserved area. We introduce a function rand() to reduce the effect of noise on the model. When a takes a large value, it can completely preserve the details of the feature area; when a is taken for smaller values, the simplification ratio of the reserved area becomes larger, which cannot achieve the purpose of preserving the details of the model. When a is taken between the two values, the reserved area can be simplified to a suitable level.
The proposed algorithm can effectively retain the detail features of the model. Experiments show that this way of processing can effectively retain the important features of the model, and it can also retain the features of the model well when the simplification ratio is high.
Fusion of models
The simplified head model also needs to be fused with the body model to get a personalized human model. Kanai et al. propose a FCF-based mesh model fusion method. This method needs to map the model to two-dimensional space and then map it to 3D space according to FCF function, so as to achieve the integration between models. However, the high computational complexity does not apply to the fusion of large models, and it needs to choose the fusion area manually, which cannot realize the automatic integration between models. Therefore, in this paper, an improved FCF fusion method based on boundary edge was applied to achieve the automatic fusion between two large-scale models. The first step was to determine the fusion area according to the boundary edge of the two models. As shown in Fig. 5, the fusion areas F1, F2 were determined and then mapped to the two-dimensional space H1, H2to achieve the mapping from three dimensions to two dimensions. By using the method in literature [20], we can approximately calculate the locations of all points on H1 and H2. The second step was to merge H1 and H2 and get Hc. Hc included two kinds of vertexes: one was the original vertex that is directly inherited from F1, F2, and the other was the new point generated by cross computing. The third step was to reconstruct the blending surface Fc according to every point in Hc. Each point in Hccontained two parts of three-dimensional information, one was the three-dimensional information on F1, and the other was the three-dimensional information on F2. The transition curved surface Fc of F1 and F2can be obtained by FCF method. As shown in Fig. 6, we define a non-uniform cubic B-spline curve interpolation algorithm, called f(s) (0 ≤ f(s) ≤ 1), to control the fusion models.
The coordinate of the vertex vc in Fcis as follows:
$$ {v}^c=f(s){v}^1+\left(1-f(s)\right){v}^2 $$
(6)
Among them, v1 represents the coordinate of vc onF1, v2 represents the coordinate of vc on F2, s = 1 − l/L,l. The definition of L is shown in Fig. 6. l is the distance from point to bottom boundary of Hc, and L is the distance between upper and lower boundaries. It can be concluded that the value of s is closer to 1 when the distance between a point and the corresponding lower boundary is smaller. On the other hand, when the distance between a point and the lower boundary is larger, s is closer to 0.