|SYSTEMATIC REVIEW AND META-ANALYSIS
|Year : 2022 | Volume
| Issue : 3 | Page : 354-358
Accuracy of machine learning in identification of dental implant systems in radiographs – A systematic review and meta-analysis
Veena Benakatti1, Ramesh P Nayakar1, Mallikarjun Anandhalli2, Vasanti Lagali-Jirge3
1 Departments of Prosthodontics and Crown and Bridge, KAHER'S KLE VK Institute of Dental Sciences, Belagavi, Karnataka, Inaia
2 Department of Electronics and Communication Engineering, KLS Gogte Institute of Technology, Belagavi, Karnataka, India
3 Departments of Oral Medicine and Radiology, KAHER'S KLE VK Institute of Dental Sciences, Belagavi, Karnataka, Inaia
|Date of Submission||12-Mar-2022|
|Date of Decision||01-Sep-2022|
|Date of Acceptance||04-Sep-2022|
|Date of Web Publication||26-Sep-2022|
Department of Prosthodontics and Crown and Bridge, KAHER'S KLE VK Institute of Dental Sciences, Neharu Nagar, Belagavi - 590010, Karnataka
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Machine learning has played a promising role in medical diagnosis. The aim of this systematic review was to evaluate the accuracy of machine learning in identification of dental implant systems from radiographs. This systematic review was conducted by searching four electronic databases, PubMed, SCOPUS, Cochrane Library, and Google Scholar. Inclusion criteria were studies that used machine learning for implant identification. Our search yielded 87,189 studies, of which a total of eight studies were found which used machine learning for implant identification. Of the included studies, three studies provided the required data to conduct meta-analysis. The overall pooled estimate of accuracy of the three included studies was 95.43%. Machine learning appears to be practically efficient in implant recognition. The findings of this review suggested an inadequate reporting of studies due to a lack of standardized guidelines for reporting and conducting the studies that investigate machine learning in implant identification. This could limit the reliable interpretation of the reported accuracy.
Keywords: Artificial intelligence, classification, dental implants, dental radiography, machine learning
|How to cite this article:|
Benakatti V, Nayakar RP, Anandhalli M, Lagali-Jirge V. Accuracy of machine learning in identification of dental implant systems in radiographs – A systematic review and meta-analysis. J Indian Acad Oral Med Radiol 2022;34:354-8
|How to cite this URL:|
Benakatti V, Nayakar RP, Anandhalli M, Lagali-Jirge V. Accuracy of machine learning in identification of dental implant systems in radiographs – A systematic review and meta-analysis. J Indian Acad Oral Med Radiol [serial online] 2022 [cited 2022 Dec 10];34:354-8. Available from: http://www.jiaomr.in/text.asp?2022/34/3/354/356973
| Introduction|| |
In today's world, dental implants form the most accepted and promising treatment modality for missing teeth. Owing to their popularity and versatility, many manufacturers have entered the industry and produce a variety of implants that vary in make and shape. This makes implant identification difficult when necessity arises.
Implant maintenance and repair require information about the implant system as the implant components are unique to the manufacturer and need specific tools or components. When patients do not have implant records or have traveled from other region or country, clinicians face difficulty in identifying the system. Clinicians either rely on radiographic interpretation or contact the company or colleagues to seek assistance in identifying the system, making it an assumptive process and not a definite approach.
Several attempts have been made to document basic design and features of implant systems on radiographs.,,, However, none of them are easy or quick, and they require significant amount of human effort, knowledge, experience, and time.
Considering these challenges, recently, artificial intelligence (AI) has been experimented in implant identification and found to be a promising technique.,,,,,, Machine learning, a subset of AI, offers an array of algorithms that classify the implant systems based on similarity in the pattern they exhibit; thus, the trained model predicts the type of implant system, making the process quick and effortless., This review was undertaken to assess the accuracy of machine learning in dental implant identification from radiographs and to gauge if it can be advocated as a technique of implant identification.
| Materials and Methods|| |
Population, intervention, comparison, and outcome (PICO) framework was formulated. The population included dental implants, the intervention included machine learning as a method to identify dental implants, the comparison was for human participants (dental professionals), and the outcome was accuracy of machine learning in identification of dental implant systems.
Searches were conducted in four electronic databases: PubMed, SCOPUS, Cochrane library, and Google Scholar. Additional searches were made in gray literature, trial registries, and thesis repository, and manual searches were performed in the references of the included studies. For PubMed, a literature search was made with MeSH terms and for other databases same search terms were used. The search terms used were machine learning, artificial intelligence, dental implants, radiography, and classification (MeSH terms). These search terms were used in combination to search the database. Filters included the articles published during the period June 2015–June 2021, accounting for 6 years, and in English language. Filters also included the exclusion of patents and books.
The systematic review was conducted adhering to Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. [Figure 1] depicts the flowchart showing the process of study selection in accordance with PRISMA guidelines. The titles and abstracts were screened for the inclusion criteria, that is, machine learning models for implant system identification. The full text of articles was screened adhering to the inclusion criteria. Studies that used methods other than machine learning to identify dental implant systems, review articles, studies that used means other than intraoral periapical radiographs (IOPA) and orthopantomogram (OPG) images, and studies that used machine learning for purposes other than identification of dental implant systems were excluded. Two reviewers screened each record (title/abstract) and retrieved the report independently, and disagreements were resolved by discussion with the third examiner. This study was registered in PROSPERO (CRD42021252085).
|Figure 1: Flowchart showing the process of study selection by PRISMA guidelines. PRISMA = Preferred Reporting Items for Systematic reviews and Meta-Analysis|
Click here to view
| Results|| |
The search strategy yielded eight articles that applied machine learning for implant identification. After screening the full text and eliminating duplicates, articles were exported in Zotero software. The data from selected articles were extracted in Microsoft Excel sheet, which included author, title, journal, year, country, and language of publication, population, keywords, type of manuscript, study design, location of study, statistical analysis, intervention, sample size, publication and other biases, and type and accuracy of the machine learning model used. The included studies had been published between June 2015 and June 2021 in the USA, Switzerland, and Germany.
Of the eight included studies, three studies used OPG, two studies used IOPAs, and three studies used both OPG and IOPAs to train AI models. Lee et al., compared the accuracy of AI model based on OPG and periapical radiographs; both studies concluded that accuracy was marginally high for periapical radiographs (0.956, 0.979 and 0.929, 0.961, respectively). The reason for this could be that IOPAs generally have better image quality in comparison to OPGs.
Dental implant systems used and their characteristics
Studies included in the review used different dental implant systems for training AI model: Zimmer Biomet, Dentsply, Nobel Biocare, Implantium, Straumann, Brånemark System, Swiss plus, Osstem, Dentium, and Kyocera. Straumann and Nobel Biocare were the most commonly used dental implant systems. Lee et al. used Osstem TSIII SA, Superline Dentium, and Straumann BLT and found the highest accuracy for Straumann BLT systems. The authors proposed that this could be due to the largest taper of the Straumann BLT implant system. Lee et al. used Dentsply Astra Osseospeed, Dentium Implantium, Dentium Superline, Osstem TSIII, Straumann SLA active BL, and Straumann SLA active BLT and found the highest accuracy for Straumann SLA active BLT. However, Osstem TSIII and Dentium Superline implant systems that do not have prominent characteristic elements showed lesser accuracy. In a study conducted by Sukegawa et al., the accuracy was similar for all implant systems used in the study, but it was marginally higher for the Straumann tissue 4.1 implant system. In a study by Said et al., the AI model achieved high accuracy for Zimmer Biomet Dental Tapered Screw-Vent, which had an apex hole, and for Nobel Biocare and Branemark system with external hexagon platform. Kim et al. found that neural networks searched for distinctive parts of each implant for recognition of the system, and this proves that deep learning can identify the discriminative features of the implant type well. Outcomes of these studies suggest that the AI model could easily identify implants that have sharp and discriminative features. However, parts, features, and ranges of implants that are focused on for implant identification differ from one AI model to another.
Machine learning models
In a study conducted by Sukegawa et al., finely tuned VGG16 achieved the best performance, followed by finely tuned VGG19, VGG16 with transfer learning, VGG19 with transfer learning, and basic convolutional neural networks (CNNs). The authors found that basic CNN with lesser convolutional layers has limited capacity for image classification. With appropriate transfer learning and fine-tuning, pretrained CNN networks were capable of performing better in image classification, even with smaller dataset. In a study conducted by Kim et al., SqueezeNet, ResNet 18, and ResNet 50 showed the highest accuracy among the models tested. Sukegawa et al. found that the larger the number of parameters and deeper the network, the better was the accuracy. Algorithm performances cannot be compared as studies exhibit heterogenicity in terms of sample size, method and type of data acquisition, training, and the algorithm itself.
Performance comparison between machine learning and dental professionals
Two studies compared the accuracy of AI models with that of dental professionals. A study by Lee et al. demonstrated the superiority of deep CNN architecture with an area under curve (AUC) of 0.971 over board-certified periodontists with an AUC of 0.925. In a study by Lee et al., the performance accuracy of 25 dental professionals was compared to trained automated deep CNN (DCNN), and automated DCNN surpassed most of the dental professionals.
Risk of bias
Systematic reviews of AI in diagnosis are sparse in dentistry. We could not find a suitable tool to perform a risk of bias assessment, although the possible risk of bias has been discussed in the discussion section.
[Table 1] describes the study characteristics of the included studies. Excluding one study that used supervised machine learning (k nearest neighbor [KNN]), the rest of the studies used different CNN architectures that varied from basic CNN to visual geometry group (VGG), ResNet, and GoogLeNet Inception. Six studies applied transfer learning to CNNs; transfer learning improves efficiency, saves training time, and reduces data needs. Data augmentation was considered in three studies. Data augmentation either creates slightly modified copies of existing data or artificially creates new training data from the existing data to overcome overfitting. The limitation of data augmentation is data bias; augmented data is quite different from the original that can lead to suboptimal performance.
Meta-analysis was conducted on three eligible studies. Other studies were not eligible as the outcomes of interest were not measured or were not reported. The random-effects model was used. [Figure 2] shows the forest plot for accuracy of machine learning in identification of dental implant systems based on three studies. The forest plot denotes a highest accuracy of 97.10% (95% confidence interval [CI]: 96.32, 97.88) and a lowest accuracy of 93.80% (95% CI: 87.20, 99.40). The overall pooled estimate of accuracy when all individual studies were combined was 95.43% (93.37, 97.48), with heterogenicity among three studies being very high (92.44% and P < 0.001). We observed a clear heterogenicity among studies in terms of sample size, method of data acquisition, training AI model, presentation of data, data analysis, and validation of the model. [Figure 3] shows the evidence of publication bias as it is apparent that the funnel plot does not exhibit expected normality in the estimates of accuracy.
|Figure 2: Forest plot for accuracy of machine learning in dental implant identification|
Click here to view
|Figure 3: Funnel plot for accuracy of machine learning in dental implant identification|
Click here to view
| Discussion|| |
Science and technology have made life easier by providing solutions to many challenges. Implant identification, being an unsolved problem in implant dentistry, needs an immediate solution and AI can be the answer. A total of eight studies were found that used machine learning in implant identification and found machine learning algorithms to be proficient in performing the task.
In comparison to the existing literature, this review addresses the key elements of the research, that is, elaboration on different AI algorithms used, their outcomes based on different implant systems used, performance comparison with dental professionals, and meta-analysis of the eligible studies.
Morais et al. tested the supervised machine learning model KNN in implant identification in 2015. Thereafter, there was a substantial increase in studies using deep learning for implant identification during 2020. A KNN classifier compares the unknown input data with all the labeled data present in the reference database and puts it into a category that is most similar to input data. The aim of this study was to introduce the concept of identifying an unknown dental implant using a framework based on image processing concepts. The remaining studies tested deep learning techniques that use multiple layers known as CNNs to extract higher-level features from the input data. The information of the image passes from the first layer (input) to the last layer (output) by being filtered through each layer according to its features. The first layer identifies the general shape of the image, the next layer its edges, and then angles, points, and so on, until a prediction is made at the exit of the last layer on the identity of the image. Six out of eight studies reported an accuracy of over 90%, which supports the use of machine learning in implant recognition; the other two studies were pilot in nature.
A clear heterogenicity was noticed among studies in terms of data acquisition, sample size, data presentation, and analysis. A formal sample size calculation was not mentioned in the studies included. Many times, the historical data is collected from clinical care repositories or open-source databases to meet the data demands. These repositories are not always quality controlled and may lead to mistakes and unrecognized biases. Thus, insufficient sample size and poor data quality subject the studies to high risk of bias.
Considering the reporting of relevant model performance measures, different range of metrics were used to report the performance of the machine learning algorithms, such as accuracy, precision, recall, F1 score, sensitivity, specificity, positive predictive value, and negative predictive value. As these were the only measures of performance reported, it leads to difficulty in comparison and summarizing the data during meta-analysis. Studies conducted using AI in implant identification are few and the research is in the initial stage. In the future, there is a need for comprehensive and structured reporting and standardization of the methodologies. Problems like overfitting and model validation need to be addressed and a standardized guideline for reporting these has to be followed. External validation of the models will help to set standards to ensure the applicability of machine learning-based prediction model studies.
The final estimate of the accuracy performance of machine learning is based on relatively few studies. Studies were excluded from meta-analysis due to lack of standardized reporting and inadequate data. Consequently, there could be considerable uncertainty around the estimates of meta-analysis.
The outcome of this systematic review and meta-analysis suggests that machine learning is practically efficient in implant recognition. Additional studies are required to develop algorithms for other implant systems across the world.
AI has revolutionized the health-care industry. The development of machine learning has created an unprecedented transformation in medical diagnosis. The use of AI to identify the implant system in a patient will increase efficiency and reduce the effort and time spent in identification using conventional methods or by human intelligence.
Recommendations for future research
- Training of algorithms needs to be done with more implant systems across the world and more sample sizes to achieve maximum accuracy to be useful for the dental community.
- A wide array of algorithms is available to solve classification problems. More algorithms can be tested to achieve the highest accuracy for implant recognition.
- Standardized guidelines need to be formulated for conducting and reporting of studies that investigate AI in implant identification. Standardization of radiographic images plays an important role in the accuracy of the algorithm. AI has demonstrated better performance with 3D imaging. Studies using cone-beam computed tomography images would yield higher accuracy and better performance of the models.
We would like to extend our sincere thanks to Mr. Jang Bahadur Prasad, Department of Epidemiology and Biostatistics, K. L. E Academy of Higher Education and Research, Belagavi, Karnataka, India, for his contribution in carrying out statistical analysis.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Takahashi T, Nozaki K, Gonda T, Mameno T, Wada M, Ikebe K. Identification of dental implants using deep learning-pilot study. Int J Implant Dent 2020;6:53.
Saghiri MA, Freag P, Fakhrzadeh A, Saghiri AM, Eid J. Current technology for identifying dental implants: A narrative review. Bull Natl Res Cent 2021;45:1-11.
Kim JE, Nam NE, Shim JS, Jung YH, Cho BH, Hwang JJ. Transfer learning via deep neural networks for implant fixture system classification using periapical radiographs. J Clin Med 2020;9:1117-27.
Sahiwal IG, Woody RD, Benson BW, Guillen GE. Macro design morphology of endosseous dental implants. J Prosthet Dent 2002;87:543-51.
Sahiwal IG, Woody RD, Benson BW, Guillen GE. Radiographic identification of nonthreaded endosseous dental implants. J Prosthet Dent 2002;87:552-62.
Michelinakis G, Sharrock A, Barclay CW. Identification of dental implants through the use of Implant recognition software (IRS). Int Dent J 2006;56:203-8.
Jandali R. Global Implant Solutions LLC. Dental Implant Identification System. United States Patent US20090155744A1; 18 June, 2009.
Sukegawa S, Yoshii K, Hara T, Yamashita K, Nakano K, Yamamoto N, et al
. Deep neural networks for dental implant system classification. Biomolecules 2020;10:984-96.
Lee J-H, Kim YT, Lee J-B, Jeong S-N. A performance comparison between automated deep learning and dental professionals in classification of dental implant systems from dental imaging: A multi-center study. Diagnostics (Basel) 2020;10:910.
Lee J-H, Jeong S-N. Efficacy of deep convolutional neural network algorithm for the identification and classification of dental implant systems, using panoramic and periapical radiographs: A pilot study. Medicine (Baltimore) 2020;99:e20787.
Saïd MH, Le Roux M-K, Catherine J-H, Lan R. Development of an artificial intelligence model to identify a dental implant from a radiograph. Int J Oral Maxillofac Implants 2020;36:1077-82.
Sukegawa S, Yoshii K, Hara T, Matsuyama T, Yamashita K, Nakano K, et al
. Multi-task deep learning model for classification of dental implant brand and treatment stage using dental panoramic radiograph images. Biomolecules 2021;11:815.
Revilla-León M, Gómez-Polo M, Vyas S, Barmak BA, Galluci GO, Att W, et al
. Artificial intelligence applications in implant dentistry: A systematic review. J Prosthet Dent 2021;S0022-3913(21)00272-9. doi: 10.1016/j.prosdent. 2021.05.008. Online ahead of print.
Grischke J, Johannsmeier L, Eich L, Griga L, Haddadin S. Dentronics: Towards robotics and artificial intelligence in dentistry. Dent Mater 2020;36:765-78.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al
. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
Adarsh K, Sharma P, Juneja A. Accuracy and reliability of tooth length measurements on conventional and CBCT images: An in vitro
comparative study. J Orthod Sci 2018;7:17.
Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al
. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis. Lancet Digit Health 2019;1:e271-97.
Mirbabaie M, Stieglitz S, Frick NRJ. Artificial intelligence in disease diagnostics: A critical review and classification on the current state of research guiding future direction. Health Technol 2021;11:693-731.
Hung K, Yeung AWK, Tanaka R, Bornstein MM. Current applications, opportunities, and limitations of AI for 3D imaging in dental research and practice. Int J Environ Res Public Health 2020;17:4424-41.
[Figure 1], [Figure 2], [Figure 3]