Xi'an Technological University

**Subject:** Computer Science, Software Engineering

**eISSN:** 2470-8038

SEARCH WITHIN CONTENT

Related articles

Yi Guo
** ^{*}**
/
Xiaoyi Ruan

**Keywords : **
Camera Calibration,
Point Cloud Stitching,
Dental Model,
Three-Dimensional Measurement

**Citation Information : **
International Journal of Advanced Network, Monitoring and Controls. Volume 6,
Issue 2,
Pages 56-64,
DOI: https://doi.org/10.21307/ijanmc-2021-017

**License : **
(CC-BY-NC-ND 4.0)

**Published Online: ** 12-July-2021

- ARTICLE
- FIGURES & TABLES
- REFERENCES
- EXTRA FILES
- COMMENTS

On the basis of studying the calibration method of the camera and the rotating platform, we designed an automatic camera calibration scheme based on four azimuth circles, which can realize the automatic sorting of the mark points. In order to improve the efficiency of identifying the center of the calibration plate, this paper uses RANSAC to improve the RED ellipse center detection algorithm. Experimental verification shows that the improved RED algorithm has increased 29.34% in anti-noise interference ability and 30.10% in center detection accuracy, which effectively guarantees the measurement stability and accuracy of the measurement system. Then, with the aid of the designed calibration board, we fit the rotation center of the turntable using the principle of three points in a circle, which provides a basis for the subsequent point cloud splicing. Experiments show that the back-projection error of the calibration method in this paper is less than 1 pixel, and the calibration accuracy is better than Zhang’s algorithm.

Stomatology is one of the first disciplines to introduce digital technology, and it is the dominant discipline in digitalization. In 1987, CAD/CAM technology has been applied in clinical denture restoration. The three-dimensional measurement of the dental jaw model is the basis of the oral CAM/CAD system. Only on the basis of obtaining the digital model of the dental jaw by measuring the three-dimensional point cloud, the doctor can use computer-aided technology to design and process the digital plan model of the dental jaw. In addition, most of the dental models commonly used in hospitals today are plaster models [1−2]. The plaster models are made by inverting the patient with alginate and other materials with the help of doctors. The plaster model can well restore the patient’s jaw geometry. However, due to the characteristics of the plaster itself and the influence of the surrounding environment such as light and humidity, the dental jaw plaster model will generally change its shape after a period of time, which is not convenient for the diagnosis and treatment for patients, and is not conducive to the review of medical records [3]. On the other hand, due to the volume of plaster models is relatively large. As the number of cases increases, the storage and management of plaster models also face greater problems. Therefore, the establishment of a digital dental model by acquiring dental jaw point cloud data is also to establish a personal dental information database, improve personal electronic medical records, and provide convenience for doctors to diagnose the condition and design treatment plans. In addition, digital dental information can be accessed through the Internet Transmission and 3D printing bring great convenience to doctors for remote consultation.

The dental model measurement system needs to include the following functions:

*a)* *Dental model fixing and rotating device;*

*b)* *Project grating fringes to the measured object;*

*c)* *Capable of acquiring images from multiple angles;*

*d)* *Process the fringe image obtained in step c) and generate the corresponding 3D point cloud data.*

Aiming at the dental model, which is a small object with smooth surface and complex texture adjustment, this topic specially designed and processed a rotating table dedicated to the dental jaw model; built a dental jaw model measurement platform, and determined the hardware selection of the measurement system Models and software modules.

The software and hardware design diagram of the system is shown in Figure 1. The hardware part is mainly composed of two CCD cameras, a grating projector and a table rotation control module. The software part includes system calibration, three-dimensional information solving and model reconstruction.

Selection requirements: This project is expected to control the average error of the dental model measurement to 0.05mm, the camera measurement range is 110mm×90mm, the camera to the object distance is 200~350mm, and the measurement depth of field is required to be 50mm.

CCD camera pixel resolution: The resolution of the captured image is determined by the camera resolution and the size of the target scene. In this measurement system, the resolution of the CCD camera in the horizontal and vertical directions should be 110mm/0.05 respectively. Only when mm=2200 and 90mm/0.05mm=1800 can meet the measurement standards.

The three-dimensional point cloud measurement system can ignore the color and texture characteristics of the tooth surface, and the dental model measurement system has certain requirements for noise and transmission speed. Therefore, in order to introduce as little noise as possible to increase the transmission speed, two models are selected in this article It is a CCD series USB3.0 camera of Hikvision MV-CE100-30M. The resolution of the camera is 3840×2748, the maximum frame rate is 7.1 fps, and the target surface size is 1/2.3 inches (6.41mm×4.59mm).

System magnification:

Lens focal length:

Normally, it is required that the size of the target surface of the lens should not be smaller than the size of the camera’s target surface. This topic chooses ZX-SF1620C, the target surface size is 1/3 inches, the focal length is 16mm, the TV distortion of the lens is less than 0.1%, and the aperture is F1.4. ~F16C.

The clarity of the projection of the projector directly affects the accuracy of the measurement, and the image of a general projector is not clear within 110mm×90mm. This measurement system selects the micro projector optical machine with LCoS projection technology. The optical machine projection control chip model is Texas Instruments DLP3010EVM-LC. The projector luminous flux is 50 lumens, the projection size is 5-200 inches, and the projection distance is 100mm~2500mm. The pixel resolution is 1280×720, and the micro-mirror pitch is 5.4um. Experiments show that it can clearly project in the range of 110mm×90mm at a distance of 200~400mm. DLP3010EVM-LC supports the burning of programming graphics. Burn the raster pictures of the specific frequency needed during the measurement, the image capture card will save the images taken by the camera and transmit them to the computer. The turntable control module is connected to the computer and cooperates with the rotating platform to perform high-precision object postures for the rotation angle.

The angle between the two CCD cameras is about 30°, the angle between the single camera and the projector is 15°, and the actual working distance is about 317mm. The measurement software part mainly completes the system calibration and the resolution of the grating structured light. Combining the results of the camera calibration and the rotation center calibration module, the software system can solve the three-dimensional information of the object surface under a certain angle of view, and finally according to the rotation direction and rotation of the turntable The angle unifies the measurement results under the same coordinate system, and obtains the surface profile information of the object.

Figures 1 how the physical objects of the in vitro dental model measurement system. The operation process of the instrument is as follows: adjust the angles of the two cameras until the calibration plate is fully exposed in the field of view of the left and right cameras, project the grating structured light onto the dental model to be tested, and the rotating platform drives the dental model to start rotating at a certain angle Complete the scan in one week. The left and right cameras collect data information, and the obtained information is processed by a computer to obtain a three-dimensional model of the tooth.

Ideally, the camera’s imaging process is small hole imaging (as shown in Figure 2), but subject to the processing conditions of the lens, in fact, the light will shift to a certain extent during the process of passing through the lens, resulting in the mapping of three-dimensional objects in two-dimensional Distortion appears on the image. In order to correctly restore the position information of the measured object, the camera distortion must be corrected. The process of correcting the camera distortion is the camera calibration. Camera calibration technology has been studied for decades at home and abroad. Generally, it is necessary to design corresponding calibration methods and calibration devices for different measured objects and design an ingenious calibration scheme to improve the overall measurement accuracy of the system. The stability of the system is of great significance [4].

• Coordinate system

The optical imaging system of the camera uses a convex lens to map the image of a three-dimensional object in reality onto a two-dimensional plane, as shown in Figure 3 This process requires the conversion of three coordinate systems. The three coordinate systems are: world coordinate system, camera coordinate system, and image pixel coordinate system. The world coordinate system is considered to be the coordinates of the real three-dimensional space where the measured object is located. The camera coordinate system is a coordinate system established with the camera’s optical center as the circle point and the camera’s optical axis as the Z axis. The image coordinate refers to a single pixel.

• Coordinate system

The optical imaging system of the camera uses a convex lens to map the image of a three-dimensional object in reality onto a two-dimensional plane, as shown in Figure 3 This process requires the conversion of three coordinate systems. The three coordinate systems are: world coordinate system, camera coordinate system, and image pixel coordinate system. The world coordinate system is considered to be the coordinates of the real three-dimensional space where the measured object is located. The camera coordinate system is a coordinate system established with the camera’s optical center as the circle point and the camera’s optical axis as the Z axis. The image coordinate refers to a single pixel.

As shown in Figure 6, assuming that the point *W(Xw,Yw,Zw)* is a point in the three-dimensional space, the point of its projection on the imaging plane is *p(u,v)*, *(x,y)*, *(xw,yw,zw)* and *(xc,yc,zc)* respectively represent the three-dimensional coordinates of the point p in the world coordinate system and the camera coordinate system coordinate. Suppose the projection of the origin of the camera coordinate system in the imaging plane coordinate system is *(u0,v0)*, *sx, sy* denote the number of pixels per unit distance, and f denote the effective focal length of the system. There is a perspective relationship:

The relationship between image coordinates and camera coordinates is:

The relationship between camera coordinates and world coordinates is shown in formula (3), R in the formula represents the rotation matrix, and represents the translation matrix:

The pixel coordinates to world coordinates are sorted out as shown in formula (4):

The above formula can be converted as:

Among them, the parameters of the x matrix and the actual focal length matrix and the actual focal length, the distortion coefficient is related to the coordinates of the image center (principal point) and the angle between the camera and the lens. These are the internal parameters of the camera and have nothing to do with the movement of the camera and the pose of the camera. Therefore, the *M*_{1} matrix is also called the internal parameter matrix. The important parameters of *M*_{2} are the rotation matrix and translation matrix of the camera. The solution of these two matrices is related to the movement of the camera and the position and angle of the camera. These are the non-internal parameters of the camera, the matrix is called the external parameter matrix.

• Coordinate system

Ideally, the camera is a pinhole model, but due to component manufacturing and processing technology, the optical imaging components of the camera often have nonlinear geometric distortions. In the 3D reconstruction process, if you do not consider correcting the camera distortion, then you cannot Restore the true coordinate value of the measured object, resulting in measurement failure. Camera distortion models can generally be divided into two types: radial distortion and tangential distortion. Radial distortion shows that all light rays are closer to the center of the image, so it is also called pincushion distortion, while tangential distortion is the opposite. The light will deviate to the surroundings. Also called barrel distortion, the two distortions are shown in Figure 4.

The radial distortion model of the camera can be expressed as:

The tangential distortion model of the camera can be expressed as:

Unlike tangential distortion, radial distortion may need to be considered for higher orders but considering higher-order radial distortion coefficients may cause the calculation results to not converge and obtain high-precision calibration results. The calculation also increases the difficulty of solving. In actual measurement, only second-order or third-order distortion can be considered in camera calibration.

Zhang proposed a non-linear optimization method for camera calibration. The camera calibration can be completed by shooting the checkerboard pattern at multiple angles. The algorithm only needs to print a specific checkerboard pattern to achieve high-precision calibration of the camera., Low cost and simple to implement. However, in the process of using this method, the characteristic points on the calibration board, that is, the corner points between the checkerboards, must be accurately extracted, and the extracted points must be arranged in a certain order.

In order to ensure the accuracy of corner extraction, high-quality images are usually required. However, in practical applications, considering the cost factor, the quality of the camera and the calibration board are often uneven, making it possible to rasterize during the calibration process, leading to deviations in corner recognition, as shown in Figure 5.

In order to solve the rasterization problem of the checkerboard calibration board, Xia Renbo et al. proposed a fully automatic camera calibration method based on circular marking points. The essence of this algorithm is to improve the corner detection to the center detection of the circular mark. The circular mark points have lower requirements for image quality and stronger anti-interference ability, but the mark point sorting process of the calibration algorithm proposed by Xia Renbo is more cumbersome. Liang Li et al. proposed a design plan for a plane calibration board with a progressive circle as the primitive. The checkerboard calibration template was retained, and the detection and sorting algorithm for the characteristic points of the circle was given. However, when the plane of the calibration target is relative to the optical axis of the camera, there is a risk of failure of the automatic sorting algorithm when it exceeds 90°. Hou Junjie and others proposed a calibration scheme based on concentric circles. The advantage of concentric circles is that the concentric circles in the world coordinate system are still concentric circles when they are mapped to the image coordinates, which can better ensure the accuracy of the center of the circle, and finally use the double Obtain the camera parameters based on the constraint relationship of the eye vision [8].

In view of this, this paper proposes an automatic detection and matching algorithm for the feature points of the calibration plate based on 4 concentric circular mark points. With Zhang’s camera parameter calculation method, it can realize the automatic calibration of the camera. The calibration board used in this paper is shown in Figure 6. The diameter of the big circle in the calibration board is 9mm, the diameter of the small circle is 3mm, and the interval between the small circles is 12mm. The parameter is the number of circular marker points along the X direction is 17, the number of circular marker rows along the Y direction is 14, and the four concentric circles in the sign orientation are as follows:

The calibration process based on circular marking points is shown in the Figure 7:

In order to facilitate the extraction of the dots directly through threshold segmentation after the image is collected, the calibration plate uses a black background and white pattern, and the surface is treated with aluminum oxide to prevent reflection. The physical picture of the calibration plate is shown in Figure 8.

Since the dot will become an ellipse after the perspective transformation, the ellipse detection method can be used to obtain the set P of ellipses in the calibration plate.

The four large circles on the calibration board are arranged as shown in Figure 5. According to the semi-major axis of the ellipse, the positions of these large circles can be measured as, and the next step is to identify these mark circles and number them:

*a)* Find the two closest circles in E and mark them as E1 and E2 (they correspond to III, IV or IV, III);

*b)* Find the great circle closest to the line where E1 and E2 are located among the remaining two circles, and record it as E3( II ), then the remaining E4 is circle I;

*c)* Then the great circle closest to II is III.

As shown in Figure 8, the dots at the four corners of the calibration plate are A, B, C, D, and the matching relationship of all dots can be determined according to the following method:

According to the positioned ellipses I and II, the point on the left side of the line and the farthest distance from the line in the calculation P is point A. Similarly, the position of the BCD point can be determined. The direction and position relationship between the calculated point and the straight line is as follows: If *c*(*x,y*), *s*(*x _{s},y_{s}*), and

In formula (8), the observation is from the point to, if the point is on the left, the calculation result F>0, otherwise, it is considered to be on the right of the observer. This method can cleverly judge the point on the calibration board the azimuth relationship with the great circle.

The sorting method of circular marker points in this article is as follows:

*Step1:* Find the point on the straight line AB. This step can be considered as searching for the 14 mark points that are closest to the straight line where AB is located in the set of all points p, denoted as *P*_{i}(*i* = 1,2…14) Arrange and number the points in according to the distance from A, so that the first column of the calibration board is found, and the set is updated *P* = *P*-{*p*_{i}};

*Step2:* The point farthest from the straight line l in the updated point set, Calculate the straight line parallel to the straight line with as the point, repeat Step1, and find the second column of the calibration plate, and so on until you find the ninth column of the calibration plate;

*Step3:* The next two steps are similar to the previous two steps. First, find 14 points on the straight line, sort and number these points according to the distance from point C from far to near, and record as *P _{i}* and update the collection

*Step4:* The point farthest from the straight line in the updated point set, mark as point, use calculate the line parallel to the line for the Repeat Step 3 to find the 13th column of the calibration board, and so on until you find the 8th column of the calibration board.

• Camera calibration experiment

In order to verify the feasibility of the system calibration algorithm in this chapter, this article has photographed the calibration board from multiple angles. Using the method in this chapter, the matching effect of different angles is shown in Figure 9. It can be seen that the camera calibration algorithm used in this article can identify the centers of the calibration plates one by one at different angles.

Reprojection Error, the correctness and accuracy of the algorithm can be matched with the verification mark point, which refers to the error between the point on the projection and the point on the image. Assuming that the mark point A on the calibration board is in the calibration process, the theoretical pixel point a will be obtained after the projection transformation, and the pixel point of the measured point after the distortion correction is set to a’, then at this time, between the two points Euclidean distance ǁ*a*–*a*′ǁ is the reprojection error.

In order to verify the accuracy of the calibration algorithm, this paper collected three sets of calibration data under different lighting environments and different angles, and compared the collected data with Zhang’s calibration method under the same conditions. In order to verify the reliability of the calibration algorithm, random noise was added to the collected calibration images. Let be the average back-projection error and be the maximum back-projection error. The calculation results of the back-projection error of the two calibration algorithms are shown in Tables 1 and 2. It can be seen that for experiments in different environments, the calibration errors are better than Zhang’s calibration method. The calibration method used in this article has a maximum backprojection error of less than 1 pixel, which shows that the detection and matching of marker points have high accuracy.

On the basis of camera calibration, we used structured light three-dimensional measurement equipment to measure the dental jaw model and obtained the point cloud model.

Due to the noise of the system, after the complete 3D point cloud data of the dental jaw is obtained, since the point cloud data is discrete, it is also necessary to use OpenGL to render the point cloud data to obtain a complete 3D model of the dental jaw, and then the dental jaw model can be performed. In the measurement comparison, the standard dental jaw model was provided by the School of Stomatology of the Fourth Military Medical University (Air Force Military Medical University), and the standard dental jaw was used as the object to be measured. Then use the LJV7000 (with a measurement accuracy of 0.005mm), and use the measurement result as the true three-dimensional value of the standard jaw. Geomagic measures the distance between the feature points of the digital model of the tooth in this article, and compares it with the measurement result of the LJ-V7000 Compare [10]. The measurement results are shown in Table 3:

Based on the calibration methods of Zhang Zhengyou and Xia Rinpo, a calibration plate based on four azimuth circles is designed. Based on this calibration plate, a corresponding camera calibration is proposed. The calibration method of the rotating platform and the ellipse detection part of the traditional RED ellipse detection algorithm uses RANSAC instead of the least square method to improve calibration accuracy. Finally, the calculation experiment proved the accuracy of the calibration method, guaranteed the accuracy and stability of the measurement system, which provides a theoretical basis for the subsequent point cloud splicing.