Abstracts

Artificial Intelligence (AI) has taken the world by storm and AI Models are increasingly being used to make critical decisions. However, these models do suffer from problems like Hallucination or generating confident but incorrect or entirely made-up answers, issues with Bias & Fairness, Lack of Understanding or Reasoning, Lack of Transparency as models are black boxes, etc.
 
While testing AI models are important, it is hard to do so due to several factors like Non-Deterministic Behavior, Lack of Clear Ground Truth which prevents objective evaluation, lack of good evaluation metrics, Generalization across domains, dialects, or formats that are especially tough to evaluate, Adversarial and Unexpected Inputs, etc.

This talk is about how to test AI models and overcome some of the challenges associated with it. Specifically, we cover topics like Data Validation testing, Bias & Fairness testing, Adversarial testing, and Model Explainability testing. We also cover some standard industry practices like Unforeseen Attack Robustness (UAR) metric, to evaluate a model's resilience to new, unanticipated types of adversarial inputs, red-teaming exercises to identify potential risks, including the misuse of AI and using external auditors to identify and mitigate emerging risks etc.

Finally, we touch upon some open-source tools like DeepXplore for Automated Whitebox Testing, Foolbox for benchmarking fast adversarial attacks, CleverHans to measure vulnerability to adversarial examples and SHAP for explaining the output of models.

The objective of the talk is to show how to test AI models to ensure they are safe, reliable, and effective.