- Open Access
Mobile robot location algorithm based on image processing technology
© The Author(s). 2018
- Received: 4 June 2018
- Accepted: 2 October 2018
- Published: 19 October 2018
To improve the reconfigurable micro mobile robot cluster system based on precision detection, a positioning and tracking system based on computer digital image processing technology was developed. The system consisted of three subsystems: image acquisition and preprocessing subsystem, rapid positioning subsystem based on robot marker recognition, and tracking subsystem based on position estimation. First, after studying the related algorithms in the subsystem, the threshold selection method of adaptive gray weight conversion was proposed for image preprocessing. Then, a fast positioning method based on marker recognition for miniature mobile robots was proposed. The selection of micro-robot markers and the basis for judging the selection of moments were given. A triangular projection positioning method was implemented, and related experimental results were given. Finally, the windowing scanning algorithm was optimized. According to the speed and direction of the robot, a tracking algorithm for position estimation was proposed. Through the simulation experiment, the effect of system positioning tracking and the system reference time in 0.270 s were given. The results showed that the system had high real-time performance.
- Mobile robot
- Positioning and tracking
- Image processing technology
MEMS are microelectronic mechanical systems. It is a very important technology for micro/nanotechnology . After more than 10 years of development, the demand for micro-electromechanical systems that integrates micro-sensors, microprocessors, and micro-actuators has been increasing . The development of multi-functional MEMS products has become a hot topic. Miniature mobile robots are new robotics technologies based on MEMS technology. The miniature mobile robot is small in size. Its adaptability is strong. It has unique advantages in the detection and precise detection of complex and unknown environments .
The micro mobile robot is a new robot technology developed based on MEMS technology. Compared with conventional robots, micro mobile robots can be used effectively in unknown and complex environments . For example, rescue operations in complex and dangerous environments are difficult for ordinary robots to perform operations in an unknown environment. However, a miniature mobile robot can adapt to various terrains with its strong adaptability and it can detect unknown environments and carry out rescue work through small caliber pipes. Therefore, the research of micro-robots has received extensive attention, and its superiority has also been recognized .
To realize the micro mobile robot reconstruction technology, the miniature mobile robot must be precisely controlled. However, for precise control, accurate positioning of the mobile micro-robot and its path tracking are particularly important. If the location tracking system is not equipped, the controller does not know the actual situation of the miniature mobile robot in a complex environment. It not only greatly restricts the development of micro-robot technology but also sets obstacles for realizing the reconstruction of micro-mobile robots. Based on the above reasons, a set of precise positioning of multiple micro mobile robots is proposed. It provides precise control of micro-robots through a real-time tracking system. Therefore, accurate positioning and tracking of the micro mobile robot is achieved. This lays a solid foundation for the reconstruction technology of micro mobile robots .
The difference between a modular robot and a conventional robot is that it can be used very effectively in unknown environments and complex environments. Proteo is an unconventional reconfigurable module robot. It was proposed by Japanese scholars in 2000. Each module of the robot is a rhombic dodecahedron. Each surface has a certain connection mechanism, so that modules and modules can be interconnected. The bindiny mechanism is mainly composed of electromagnets. The electromagnet bindiny mechanism is mainly distributed on the edge of each surface. The robot system is made up of a series of diamond homogeneous units.
The Fractal reconfigurable robot was proposed by the German scholar Michale in 1994. Fractal is a robot consisting of a series of identical cubes and bolts and a series of slots. Each surface of the cube has bolts and grooves. Through these mechanisms of bolts and grooves, the robot completes geometric transformations to accomplish specific tasks. The structure of the robot is achieved by sliding a cube or series of cubes on the contact surface. This mechanism seems to be difficult to achieve in reality. Research results show that the vast majority of it only appears in simulations.
Robot positioning is an indispensable part of the reconfigurability of micro-mobile robots. Meng et al. used eight Polaroid sonars to extract features such as planes, corners, and cylinders in an indoor environment and used AEKF fusion encoders and sonar information to locate robots. Drumheller extracts line segment features from the indoor environment information acquired by the sonar and matches the map to locate the robot. However, the accuracy of sonar measurement is not high. It has a “blind” defect. There is a great deal of uncertainty in the data returned by sonar.
According to the characteristics of the mobile robot platform, the global positioning system of the micro mobile robot consists of three major subsystems. The three major subsystems are the image acquisition and image preprocessing module, the micro mobile robot label identification and positioning module, and the micro mobile robot position estimation and tracking module.
The image processing method mainly performs preprocessing, filtering, and segmentation on the image. This lays the foundation for subsequent location tracking. The complexity of subsequent operations is reduced, the accuracy is improved, and the real-time effects of the overall system are improved.
The choice of the best filter is to optimize this relationship. The Fourier transform of the Gaussian function is still a Gaussian function. Therefore, the Gaussian function can form a low-pass filter with smooth performance in both the time domain and the frequency domain. Gaussian filters can be optimized in both the time and frequency domains .
Image segmentation is based on the criteria of image features or feature sets to divide the image plane into a series of “meaningful” regions, which is the first thing that needs to be done to achieve automatic image analysis. In the Figs. 1, 2, 3, 4, and 5 represents the part that needs to be smoothed.Taking the system processing reference time as the standard, the grayscale threshold segmentation is used to binarize the image. The theory of classical grayscale adaptive threshold selection was introduced. Combined with the particularity of the experimental scene, a method of scene binarization based on adaptive gray threshold selection is proposed. This method has a fast processing speed, good real-time performance, and good segmentation effect.
The 50*50-pixel window is used for the alignment scan. In this way, the adaptive threshold selection can be completed with the highest efficiency and the image segmentation binarization processing can be completed.
Grayscale histogram extraction is performed on the image in the scanning window, and different grayscale intensities are weighted. The grayscale threshold of the image area is between 250 and 255. Therefore, the weight of the grayscale intensity in the threshold of the region is increased, and the weight of the grayscale intensity of the image background is correspondingly reduced.
In Eq. (4), Tbest represents the optimal gray threshold. Imax represents the strongest grayscale intensity in the window. TImax represents the grayscale threshold of the grayscale corresponding to the strongest grayscale intensity. β is a fixed constant. The optimal gray threshold is to find the first gray point with the attenuation degree β from the strongest point of gray to the minimum gray direction and use this point as the optimal gray threshold. In the system, the value of β is 2.3.
5.1 Feature extraction and marker selection of robot markers
The feature description of the robot tag refers to the use of some special algorithms to extract the imaged robot tag information. Its unique features are preserved. It can be described in computer-cognizable form, such as numerical value and display, so as to realize the description of the mark. This form of computer-recognized image information is called the feature value of the image . The characteristic value of the image may be a single value, but it may also be a set of single values. Regardless of the image feature value has several numeric elements, it must have a one-to-one correspondence with the image tag itself.
Verification of geometric moments
Rotation angle (°)
Geometric moment value
Geometric moment value
Rotation angle (°)
Geometric moment value
It can be seen from the experimental results that the geometric moment value is outstanding in the rotation and translation, which meets the system requirements. There is a slight deviation in the dimensional invariance, but the system does not require high-dimensional invariance. Therefore, the geometric moment image feature operator satisfies the needs of the system, and the algorithm is used to match the robot markers on the scene image.
When the tag recognition system completes the identification of the image scene robot, and locks the area, it is necessary to calculate the center point of the marker, so as to determine the location of the robot. The common practice is to scan the entire area of the black pixel in the area and calculate the center position of the black pixel. Then, the white pixel center position is calculated in the same way. The weighted average of two center positions and the marked center position is considered to be the position of the robot’s scene.
The circular mark is reduced to the base of the triangle, and the diameter D of the circular mark is the base of the triangle. For a given base, an isosceles triangle with a height of H can always be made. Obviously, the apex O of the isosceles triangle at this time is the projection of the circular mark center C in the isosceles triangle region. The remaining points can also be converged to the apex through an isosceles triangle projection. According to the triangular projection mapping method, only the black pixel point information of the edge of the mark is needed to calculate the position of the center point, so that the center position of the mark is accurately calculated. The positioning of the miniature mobile robot in the global scene is realized.
5.2 Implementation of robot marking recognition and positioning subsystem
Robot tracking is the same as robot identification and positioning. It is also a hot issue in recent years. The entire micro mobile robot global positioning tracking system is based on image processing technology. The trajectory of the robot movement also changes in real time with the interference factors of the control system, such as noise and mechanical factors, and the given path is not determined. The robot itself has very low random mobility. Therefore, the image window scanning method tracks the positioned micro mobile robot.
6.1 Traditional scans and tracking
The traditional fenestration scan tracking algorithm is relatively simple, but its limitations are very large. Traditional window scanning and tracking algorithms have high requirements on the real-time performance of the system. If the system’s real-time performance is not enough or the robot moves too fast, it will cause the robot to run out of the established window scanning range, resulting in the failure of the tracking algorithm. To solve this problem, the window scanning window can only be expanded. However, the risk of expanding the window scanning window is to further reduce the real-time performance of the system. Since the number of scanned pixels is squared-increase, the time complexity of the algorithm increases with the increase of the windowing window into a geometric progression, which will greatly reduce the real-time performance of the system. It can be seen from the above that the traditional method of window selection for window scanning and tracking has certain blindness. Therefore, traditional window tracking algorithms cannot be applied to global positioning and tracking systems.
6.2 Velocity-based location prediction and tracking algorithm for mobile robots
Experimental comparison of position prediction tracking algorithm
Location estimation tracking algorithm
Traditional window scanning
Reference time (s)
Obviously, the position estimation tracking algorithm greatly reduces the number of points for window scanning, increases the real-time performance of the system, and achieves the purpose of rapid capture and tracking.
6.3 Implementation of tracking algorithm based on position prediction
A global positioning and tracking system based on miniature mobile robots was discussed. The system mainly includes three major subsystems: image acquisition and preprocessing subsystems, robot tag identification and positioning subsystems, and position estimation-based tracking subsystems.
In the image acquisition and preprocessing subsystem, the method of gray adaptive selection suitable for the system is proposed, and the experimental results are given. Based on the marker recognition, a robot-based marker image recognition method is proposed to locate the robot. The shape of the micro-robot mark is selected, and the image moment that describes the mark feature is most suitable. The triangular projection mapping positioning method for fast positioning is described. Finally, the specific implementation of the subsystem is given, as well as the reference time and scene effects. In the location estimation and tracking research, a window estimation scanning algorithm based on position estimation is proposed. Based on the comparison of experimental results, the correction of the traditional window-opening scanning tracking algorithm based on the position estimation-based window scanning tracking algorithm is given. The specific process and simulation results of the subsystem implementation are given.
A global positioning tracking system for miniature mobile robots was studied. At present, the system reference period is 0.277 s. Real-time performance basically meets the requirements. However, if the system reference time can be further shortened and the real-time performance is further improved, the positioning accuracy of the system will also be improved. This system is based on the reconstruction of micro-robots. The positioning and tracking of the micro mobile robot is realized, but it does not have the function of detecting the direction of motion of the robot. Since the reconstruction of the micro mobile robot requires the docking of the connector, there is also a requirement for direction detection with the robot. To serve the micro-robot reconstruction, the global positioning and tracking system should also add the function of detecting the direction of the micro-mobile robot, which is convenient for the reconstruction of the micro-mobile cluster. In addition, the global positioning and tracking system should also have the monitoring function of the overall coordinated control of the robot cluster. The system should be added to the overall coordinated control and monitoring of the robot cluster to realize the role of the robot barrier and the overall motion path planning, so as to prepare for the real reconstruction of the micro mobile robot.
The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.
About the Authors
Guifeng Wu, Master of Engineering, Associate Professor, graduated from the Southeast University in 2004 and worked in Yangzhou University. His research interests include modern measurement and control technology, human-computer system, and blended teaching.
Jie Zheng, Master of Engineering, Associate Professor, graduated from the Southeast University in 1999 and worked in Yangzhou University. Her research interests include communication and electronic technology.
Jiatong Bao, Doctor of Technical Science, Lecturer, graduated from the Southeast University in 2013 and worked in Yangzhou University. His research interests include human-computer interaction technology, robot sensing, and control.
Shengquan Li, Doctor of Technical Science, Associate Professor, Graduated from the School of Aerospace Engineering, Nanjing University of Aeronautics and Astronautics in 2012, worked in Yangzhou University. His research interests include disturbance estimation and compensation and its application to mechatronics systems.
1. National Natural Science Foundation of China (61773335)
2. Natural Science Foundation of Jiangsu Province (BK20150454)
3. Excellent teaching team of Yangzhou University (ETT20171066)
Availability of data and materials
Authors can provide the data.
All authors take part in the discussion of the work described in this paper. The author GW wrote the first version of the paper. The author JZ and JB did part experiments of the paper. SL revised the paper in different version of the paper, respectively. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- S. Park, K.S. Roh, Coarse-to-fine localization for a mobile robot based on place learning with a 2-D range scan. IEEE Trans. Robot. 32(3), 528–544 (2016)View ArticleGoogle Scholar
- J. Simanek, V. Kubelka, M. Reinstein, Improving multi-modal data fusion by anomaly detection. Auton. Robot. 39(2), 139–154 (2015)View ArticleGoogle Scholar
- P. Yang, W. Wu, Efficient particle filter localization algorithm in dense passive RFID tag environment. IEEE Trans. Ind. Electron. 61(10), 5641–5651 (2014)View ArticleGoogle Scholar
- B. Benjamin, G. Erinc, S. Carpin, Real-time WiFi localization of heterogeneous robot teams using an online random forest. Auton. Robot. 39(2), 155–167 (2015)View ArticleGoogle Scholar
- J. Kim, W. Chung, Localization of a mobile robot using a laser range finder in a glass-walled environment. IEEE Trans. Ind. Electron. 63(6), 3616–3627 (2016)View ArticleGoogle Scholar
- S. Safavi, U.A. Khan, An opportunistic linear–convex algorithm for localization in mobile robot networks. IEEE Transactions on Robotics PP 99, 1–14 (2017)Google Scholar
- S. Halder, A. Ghosal, A survey on mobile anchor assisted localization techniques in wireless sensor networks. Wirel. Netw 22(7), 1–20 (2016)View ArticleGoogle Scholar
- G. Feng, Y. He, J. Han, Active persistent localization of a three-dimensional moving target under set-membership uncertainty description through cooperation of multiple mobile robots. IEEE Trans. Ind. Electron. 62(8), 4958–4971 (2015)View ArticleGoogle Scholar
- H.Y. Chung, C.C. Hou, Y.S. Chen, Indoor intelligent mobile robot localization using fuzzy compensation and Kalman filter to fuse the data of gyroscope and magnetometer. IEEE Trans. Ind. Electron. 62(10), 6436–6447 (2015)View ArticleGoogle Scholar
- I. Becerra et al., Reliable confirmation of an object identity by a mobile robot: a mixed appearance/localization-driven motion approach. Int. J. Robot. Res. 10, 35 (2016)Google Scholar