Optical Prosthesis Image Processing Using Computer Vision and Convolutional Neural Network
Keywords:
Computer vision, Convolutional Neural Network, Retinal Prosthesis, Mask-RCNNAbstract
Optical prosthesis is a way to restore vision to millions of people who lost their eye-sight due to diseases or accident causing degradation of vision. The optical prosthesis device transforms the recorded images into corresponding electrical stimulation patterns, which are then used to create phosphenes. However due to some uncertainty in the internal electrodes the induced perception is far from ideal. Therefore, in this study a novel approach is proposed that can convert the object from video feed into phosphene image. The proposed approach comprises of four phases. The proposed approach extracts frame by frame of the video feed and recognizes the object images with the help of a pre-trained mask-RCNN model. The objects identified in the images are separated from the background by semantic segmentation. Then the object images are converted into phosphene images which are then superimposed to recreate the scene. The proposed approach is repeated for each frame of the video. The strength of a proposed model lies in its practical applicability. Therefore the approach is experimentally run on a video and tested. The result obtained from the experimentation can confirm that the proposed model is effective as well as efficient.
References
Zrenner, E. (2002). Will retinal implants restore vision?. Science, 295(5557), 1022-1025.
Sharmili, N., Swapna, N., & Ramakrishna, G. (2017, April). Comparative analysis of image processing algorithms for visual prosthesis. In 2017 International Conference on Communication and Signal Processing (ICCSP) (pp. 1120-1124). IEEE.
Beyeler, M., Boynton, G. M., Fine, I., & Rokem, A. (2017). pulse2percept: A Python-based simulation framework for bionic vision. BioRxiv, 148015.
Ayton, L. N., Barnes, N., Dagnelie, G., Fujikado, T., Goetz, G., Hornig, R., ... & Petoe, M. A. (2020). An update on retinal prostheses. Clinical Neurophysiology, 131(6), 1383-1398.
Han, N., Srivastava, S., Xu, A., Klein, D., & Beyeler, M. (2021, February). Deep learning–based scene simplification for bionic vision. In Augmented Humans Conference 2021 (pp. 45-54).
Yuan, J. C. C., Kaste, L. M., Lee, D. J., Harlow, R. F., Knoernschild, K. L., Campbell, S. D., & Sukotjo, C. (2011). Dental student perceptions of predoctoral implant education and plans for providing implant treatment. Journal of dental education, 75(6), 750-760.
Walny, J., Carpendale, S., Riche, N. H., Venolia, G., & Fawcett, P. (2011). Visual thinking in action: Visualizations as used on whiteboards. IEEE Transactions on Visualization and Computer Graphics, 17(12), 2508-2517.
Pérez L, Rodríguez Í, Rodríguez N, Usamentiaga R, García D, et. al. Robot guidance using machine vision techniques in industrial environments: a comparative review. Sensors. 2016;16:335.
Wang, J., Zhu, H., Liu, J., Li, H., Han, Y., Zhou, R., & Zhang, Y. (2021). The application of computer vision to visual prosthesis. Artificial Organs, 45(10), 1141-1154.
Ilea, D. E., & Whelan, P. F. (2011). Image segmentation based on the integration of colour–texture descriptors—A review. Pattern Recognition, 44(10-11), 2479-2501.
Sanin, A., Sanderson, C., & Lovell, B. C. (2012). Shadow detection: A survey and comparative evaluation of recent methods. Pattern recognition, 45(4), 1684-1695.
Chakraborty, A., Staib, L. H., & Duncan, J. S. (1996). Deformable boundary finding in medical images by integrating gradient and region information. IEEE Transactions on Medical Imaging, 15(6), 859-870.
Delahoz, Y. S., & Labrador, M. A. (2014). Survey on fall detection and fall prevention using wearable and external sensors. Sensors, 14(10), 19806-19842.
Wang, J., Wu, X., Lu, Y., Wu, H., Kan, H., and Xinyu, C. Face recognition in simulated prosthetic vision: Face detection-based image processing strategies. Journal of neural engineering 11 (06 2014), 046009.
Guo, F., Yang, Y., Xiao, Y., Gao, Y., and Yu, N. Recognition of moving object in high dynamic scene for visual prosthesis. IEICE TRANSACTIONS on Information and Systems E102-D (2019), 1321{1331.
Han, N., Srivastava, S., Xu, A., Klein, D., and Beyeler, M. Deep learning-based scene simplification for bionic vision. In Augmented Humans Conference 2021 (New York, NY, USA, 2021), AHs'21, Association for Computing Machinery, pp. 45-54.
Sanchez-Garcia, M., Martinez-Cantin, R., and Guerrero, J. J. Semantic and structural image segmentation for prosthetic vision. PLoS ONE 15 (2020).
Waldrop, M. M. (2019). What are the limits of deep learning?. Proceedings of the National Academy of Sciences, 116(4), 1074-1077.
Qin, Y., He, S., Zhao, Y., & Gong, Y. (2016, November). RoI pooling based fast multi-domain convolutional neural networks for visual tracking. In 2016 2nd International Conference on Artificial Intelligence and Industrial Engineering (AIIE 2016) (pp. 198-202). Atlantis Press.
