A Characterization of the Utility of Using Artificial Intelligence to Test Two Artificial Intelligence Systems

Other literature type, Article English OPEN
Straub, Jeremy ; Huber, Justin (2013)
  • Publisher: Multidisciplinary Digital Publishing Institute
  • Journal: Computers (issn: 2073-431X)
  • Related identifiers: doi: 10.3390/computers2020067
  • Subject: testing artificial intelligence systems | using artificial intelligence for testing | Electronic computers. Computer science | artificial intelligence test case generation | QA75.5-76.95 | artificial intelligence

An artificial intelligence system, designed for operations in a real-world environment faces a nearly infinite set of possible performance scenarios. Designers and developers, thus, face the challenge of validating proper performance across both foreseen and unforeseen conditions, particularly when the artificial intelligence is controlling a robot that will be operating in close proximity, or may represent a danger, to humans. While the manual creation of test cases allows limited testing (perhaps ensuring that a set of foreseeable conditions trigger an appropriate response), this may be insufficient to fully characterize and validate safe system performance. An approach to validating the performance of an artificial intelligence system using a simple artificial intelligence test case producer (AITCP) is presented. The AITCP allows the creation and simulation of prospective operating scenarios at a rate far exceeding that possible by human testers. Four scenarios for testing an autonomous navigation control system are presented: single actor in two-dimensional space, multiple actors in two-dimensional space, single actor in three-dimensional space, and multiple actors in three-dimensional space. The utility of using the AITCP is compared to that of human testers in each of these scenarios.
  • References (32)
    32 references, page 1 of 4

    1. Halawani, S. Safety Issues of computer Failure. Technical Report. Available online: http://amubaraki.kau.edu.sa/Files/830/Researches/55979_26288.doc (accessed on December 7, 2012).

    2. AdiSrikanth; Kulkarni, N.J.; Naveen, K.V.; Singh, P.; Srivastava, P.R. Test Case Optimization Using Artificial Bee Colony Algorithm. In Proceedings of ACC 2011, Part III, CCIS 192, Springer Verlag: New York, NY, 2011; pp. 570-579.

    3. Mondada, F.; Floreano, D. Evolution of neural control structures: some experiments on mobile robots. Robotics Auton. Syst. 1995, 16, 183-195.

    4. Felgenbaum, E.A. The Art of Artificial Intelligence, Stanford University Technical Report, STANCS-77-621, Stanford University: Cambridge, MA, 1977.

    5. Chandrasekaran, B. On Evaluating AI Systems for Medical Diagnosis. The AI Magazine 1983, 48, 34-37.

    6. Cholewinski, P.; Marek, V.W.; Mikitiuk, A.; Truszczynski, M. Computing with Default Logic. Artif. Intell. 1999, 112, 105-146.

    7. Brooks, R.A. Artificial Life and Real Robots. MIT Artificial Intelligence Laboratory: Cambridge, MA, USA (in press).

    8. Boooks, R.A. Elephants Don't Play Chess. In Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back; Maes, P., Ed.; MIT Press: Cambridge, MA, USA,1990.

    9. Brooks, R.A. Intelligence Without Reason. In Proceedings of the International Joint Conferences on Artificial Intelligence; Morgan Kaufmann Publishers: San Francisco, CA, USA, 1991; pp. 569- 595.

    10. Brooks, R.A. New Approaches to Robotics. Science 1991, 253, 1227-1232.

  • Metrics
    No metrics available
Share - Bookmark