Autonomous drone hunter operating by deep learning and all-onboard computations in GPS-denied environments
Autoři:
Philippe Martin Wyder aff001; Yan-Song Chen aff002; Adrian J. Lasrado aff001; Rafael J. Pelles aff001; Robert Kwiatkowski aff002; Edith O. A. Comas aff002; Richard Kennedy aff002; Arjun Mangla aff002; Zixi Huang aff003; Xiaotian Hu aff003; Zhiyao Xiong aff001; Tomer Aharoni aff002; Tzu-Chan Chuang aff002; Hod Lipson aff001
Působiště autorů:
Department of Mechanical Engineering, Columbia University, New York, New York, United States of America
aff001; Department of Computer Science, Columbia University, New York, New York, United States of America
aff002; Department of Electrical Engineering, Columbia University, New York, New York, United States of America
aff003
Vyšlo v časopise:
PLoS ONE 14(11)
Kategorie:
Research Article
prolekare.web.journal.doi_sk:
https://doi.org/10.1371/journal.pone.0225092
Souhrn
This paper proposes a UAV platform that autonomously detects, hunts, and takes down other small UAVs in GPS-denied environments. The platform detects, tracks, and follows another drone within its sensor range using a pre-trained machine learning model. We collect and generate a 58,647-image dataset and use it to train a Tiny YOLO detection algorithm. This algorithm combined with a simple visual-servoing approach was validated on a physical platform. Our platform was able to successfully track and follow a target drone at an estimated speed of 1.5 m/s. Performance was limited by the detection algorithm’s 77% accuracy in cluttered environments and the frame rate of eight frames per second along with the field of view of the camera.
Klíčová slova:
Algorithms – Cameras – Neural networks – Machine learning algorithms – Flight testing – Target detection – Computer imaging – Computers
Zdroje
1. Federal Aviation Administration. FAA Aerospace Forecast, FY2018-38. 2018. doi: 10.1017/CBO9781107415324.004
2. Wall R. U.K. Airport Remains Closed After Drones Disrupt Travel—WSJ. In: The Wall Street Journal [Internet]. 2018 [cited 11 Jan 2019]. Available: https://www.wsj.com/articles/u-s-bound-flights-from-major-u-k-airport-grounded-over-drone-fears-11545295342
3. Rizzo C. Drones Force London Airport to Shut Down, Leaving Thousands Stranded Throughout Europe | Travel + Leisure. In: Travel+Leisure [Internet]. 2018 [cited 11 Jan 2019]. Available: https://www.travelandleisure.com/travel-news/london-gatwick-airport-drone-shut-down
4. FOX 5. Delta jet spots drone near JFK Airport—Story | WNYW. 2015 [cited 11 Jan 2019]. Available: http://www.fox5ny.com/news/delta-jet-spots-drone-near-jfk-airport
5. Wong D. Simmering debate over drone use in condos, as more move to ban the flying objects, Housing News & Top Stories—The Straits Times. In: The Straits Times [Internet]. 2018 [cited 11 Jan 2019]. Available: https://www.straitstimes.com/singapore/housing/condos-move-to-ban-drones-to-protect-residents-privacy
6. Richey E. Peeping drones spying on people in St. Louis | ksdk.com. In: KSDK [Internet]. 2018 [cited 11 Jan 2019]. Available: https://www.ksdk.com/article/news/investigations/peeping-drones-spying-on-people-in-st-louis/63-548590075
7. Gibbons-Neff T. ISIS drones are attacking U.S. troops and disrupting airstrikes in Raqqa, officials say—The Washington Post. In: The Washington Post [Internet]. 2017 [cited 10 Jan 2019]. Available: https://www.washingtonpost.com/news/checkpoint/wp/2017/06/14/isis-drones-are-attacking-u-s-troops-and-disrupting-airstrikes-in-raqqa-officials-say/?utm_term=.06a397b386e9
8. Lohr D. Gang Used Drone Swarm To Thwart FBI Hostage Raid | HuffPost. In: Huffington Post [Internet]. 2018 [cited 10 Jan 2019]. Available: https://www.huffingtonpost.com/entry/drone-swarm-thwarts-fbi-gang-raid_us_5aec995de4b0ab5c3d655cfe
9. Dinan S. Mexican drug cartels using drones to smuggle heroin, meth, cocaine into U.S.—Washington Times. In: The Washington Times [Internet]. 2017 [cited 11 Jan 2019]. Available: https://www.washingtontimes.com/news/2017/aug/20/mexican-drug-cartels-using-drones-to-smuggle-heroi/
10. BBC. Gang who flew drones carrying drugs into prisons jailed—BBC News. 2018 [cited 11 Jan 2019]. Available: https://www.bbc.com/news/uk-england-45980560
11. Lu Y, Xue Z, Xia GS, Zhang L. A survey on vision-based UAV navigation. Geo-Spatial Inf Sci. 2018. doi: 10.1080/10095020.2017.1420509
12. Balamurugan G, Valarmathi J, Naidu VPS. Survey on UAV navigation in GPS denied environments. International Conference on Signal Processing, Communication, Power and Embedded System, SCOPES 2016—Proceedings. 2017. doi: 10.1109/SCOPES.2016.7955787
13. Rady S, Kandil AA, Badreddin E. A hybrid localization approach for UAV in GPS denied areas. 2011 IEEE/SICE International Symposium on System Integration, SII 2011. 2011. doi: 10.1109/SII.2011.6147631
14. Battelle Memorial Institute. Battelle DroneDefender® Counter-UAS Device. [cited 10 Jan 2019]. Available: https://www.battelle.org/government-offerings/national-security/aerospace-systems/counter-UAS-technologies/dronedefender
15. Wyant RT. Skynet is real! An anti-drone round. Small Arms Def J. 2016;8. Available: http://www.sadefensejournal.com/wp/?p=3752
16. OPENWORKS ENGINEERING LTD. SkyWall. [cited 10 Jan 2019]. Available: https://openworksengineering.com/skywall
17. AIRSPACE SYSTEMS I. Airspace. [cited 10 Jan 2019]. Available: https://airspace.co/index.html
18. Mannes J. Airspace Systems’ ‘Interceptor’ can catch high-speed drones all by itself | TechCrunch. In: Tech Crunch [Internet]. 2017 [cited 9 Jan 2019]. Available: https://techcrunch.com/2016/11/18/airspace-systems-interceptor-can-catch-high-speed-drones-all-by-itself/
19. Honegger D, Meier L, Tanskanen P, Pollefeys M. An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications. Proceedings—IEEE International Conference on Robotics and Automation. 2013. doi: 10.1109/ICRA.2013.6630805
20. ARDUPILOT Dev Team. Companion Computers—Dev documentation. [cited 11 Jan 2019]. Available: http://ardupilot.org/dev/docs/companion-computers.html
21. Dronecode Project Inc. Companion Computers · PX4 Developer Guide. [cited 11 Jan 2019]. Available: https://dev.px4.io/en/companion_computer/pixhawk_companion.html
22. Ong T. Dutch police will stop using drone-hunting eagles since they weren’t doing what they’re told. In: The Verge [Internet]. 2017 [cited 10 Jan 2019] p. 1. Available: https://www.theverge.com/2017/12/12/16767000/police-netherlands-eagles-rogue-drones
23. Selk A. Terrorists are building drones. France is destroying them with eagles.—The Washington Post. In: The Washington Post [Internet]. 2017 [cited 10 Jan 2019]. Available: https://www.washingtonpost.com/news/worldviews/wp/2017/02/21/terrorists-are-building-drones-france-is-destroying-them-with-eagles/?noredirect=on&utm_term=.227c762f4ffb
24. Opromolla R, Fasano G, Accardo D. A vision-based approach to uav detection and tracking in cooperative applications. Sensors (Switzerland). 2018;18. doi: 10.3390/s18103391 30309035
25. Xiang T, Jiang F, Lan G, Sun J, Liu G, Hao Q, et al. UAV based target tracking and recognition. IEEE Int Conf Multisens Fusion Integr Intell Syst. 2017; 400–405. doi: 10.1109/MFI.2016.7849521
26. Nvidia. Jetson TX2 Module | NVIDIA Developer. In: developer.nvidia.com [Internet]. [cited 11 Jan 2019]. Available: https://developer.nvidia.com/embedded/buy/jetson-tx2
27. Dronecode Project Inc. Open Source for Drones—PX4 Open Source Autopilot. [cited 11 Jan 2019]. Available: https://px4.io/
28. Redmon JSDRGAF. (YOLO) You Only Look Once. Cvpr. 2016. doi: 10.1109/CVPR.2016.91
29. Shah S, Dey D, Lovett C, Kapoor A. AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. 2017 [cited 10 Jan 2019]. doi: 10.1007/978-3-319-67361-5_40
30. Arreola L, Gudiño G, Flores G. Object recognition and tracking using Haar-like Features Cascade Classifiers: Application to a quad-rotor UAV. 2019 [cited 27 Aug 2019]. Available: http://arxiv.org/abs/1903.03947
31. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2014. doi: 10.1109/CVPR.2014.81
32. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell. 2017. doi: 10.1109/TPAMI.2016.2577031 27295650
33. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. 2017. doi: 10.1109/CVPR.2017.106
34. Redmon J, Farhadi A. YOLO. Plant Soil. 2001. doi: 10.1023/A:1011909414400
35. Redmon J, Farhadi A. YOLO9000: Better, faster, stronger. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. 2017. doi: 10.1109/CVPR.2017.690
36. Redmon J, Farhadi A. yolov3. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017. 2017. doi: 10.1109/CVPR.2017.690
37. Fu C-Y, Liu W, Ranga A, Tyagi A, Berg AC. DSSD: Deconvolutional Single Shot Detector. CoRR. 2017;abs/1701.0. Available: http://arxiv.org/abs/1701.06659
38. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, et al. SSD: Single shot multibox detector. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2016. doi: 10.1007/978-3-319-46448-0_2
39. Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision. 2017. doi: 10.1109/ICCV.2017.324
40. Redmon J. Darknet: Open Source Neural Networks in C. [cited 1 Sep 2018]. Available: http://pjreddie.com/darknet/
41. Stereolabs Inc. SDK Introduction | ZED API. [cited 10 Jan 2019]. Available: https://www.stereolabs.com/docs/api/
42. Marvelmind Robotics. Starter Set—HW v4.9 + IMU—Marvelmind Robotics. [cited 11 Jan 2019]. Available: https://marvelmind.com/product/starter-set-hw-v4-9-imu-plastic-housing/
43. Open Source Robotics Foundation. ROS.org | Powering the world’s robots. [cited 11 Jan 2019]. Available: http://www.ros.org/
44. Nvidia Corp. Jetson AGX Xavier Developer Kit | NVIDIA Developer. In: developer.nvidia.com [Internet]. 2018 [cited 12 Jan 2019]. Available: https://developer.nvidia.com/embedded/buy/jetson-agx-xavier-devkit
45. Wu Y, Sui Y, Wang G. Vision-Based Real-Time Aerial Object Localization and Tracking for UAV Sensing System. IEEE Access. 2017;5: 23969–23978. doi: 10.1109/ACCESS.2017.2764419
46. Nikolic J, Rehder J, Burri M, Gohl P, Leutenegger S, Furgale P, et al. A Synchronized Visual-Inertial Sensor System with FPGA Pre-Processing for Accurate Real-Time SLAM. IEEE Int Conf Robot Autom (ICRA), 2014 May 31, 2014—June 7, 2014, Hong Kong, China. 2014; 431–437. doi: 10.3929/ETHZ-A-010061790
47. Zhang Z, Suleiman AA, Carlone L, Sze V, Karaman S. Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co-design Approach. Sze. 2017 [cited 30 Aug 2019]. Available: https://dspace.mit.edu/handle/1721.1/109522
Článok vyšiel v časopise
PLOS One
2019 Číslo 11
- Metamizol jako analgetikum první volby: kdy, pro koho, jak a proč?
- Nejasný stín na plicích – kazuistika
- Ne každé mimoděložní těhotenství musí končit salpingektomií
- Somatizace stresu – typické projevy a možnosti řešení
- Těžké menstruační krvácení může značit poruchu krevní srážlivosti. Jaký management vyšetření a léčby je v takovém případě vhodný?
Najčítanejšie v tomto čísle
- A daily diary study on maladaptive daydreaming, mind wandering, and sleep disturbances: Examining within-person and between-persons relations
- Molecular validation of clinical Pantoea isolates identified by MALDI-TOF
- Pathways to conspiracy: The social and linguistic precursors of involvement in Reddit’s conspiracy theory forum
- Urticating setae of tarantulas (Araneae: Theraphosidae): Morphology, revision of typology and terminology and implications for taxonomy
Zvýšte si kvalifikáciu online z pohodlia domova
Všetky kurzy