This post is the second entry in our series on the Ethical AI concept.
In the conclusion of the first post, we discussed the Ethical AI concept and its 10 principles. We briefly mentioned that there was a concept that was closely related to Ethical AI. It was the Explainable AI concept, a concept evident in the first principle for Ethical AI which demands transparent AI systems.
Therefore, we will discuss in detail in this article the first principle for Ethical AI, with a special emphasis on the Explainable AI concept. Let’s begin the discussion.
A Quick Overview of the 10 Principles for Ethical AI
To recap what we learned in the first article in this series on Ethical AI, there are 10 principles for Ethical AI provided by UNI Global Union. The ten principles are:
- Demanding Transparent AI Systems
- Equipping AI Systems with an ‘Ethical Black Box’
- Ensuring AI Serves people and planet
- Adopting a Human-In-Command Approach
- Ensuring an Unbiased AI
- Sharing the AI System’s Benefits
- Securing a Just Transition and Ensuring Support for Fundamental Freedoms and Rights
- Establishing Global Governance Mechanisms
- Banning the Attribution of Responsibility to Robots
- Banning AI Arms Race
We will cover all the 10 principles for Ethical AI in this series. The current article will focus on the first principle for Ethical AI. Following is a look at the first principle for Ethical AI and its importance to artificial intelligence (AI) field.
1. Demanding Transparent AI Systems
The idea behind the first principle is to make sure that AI systems are transparent enough for anyone to find out how an AI system reached a decision. The principle is related to Explainable AI because the basic premise of the Explainable AI concept is that AI algorithms should explain the decision-making process in a way that can be easily understood by humans.
Understanding the Explainable AI Concept
Explainable AI is a concept in which an AI algorithm must be able to explain how it reached a conclusion in a way that is easily understandable to humans. It’s been a topic of frequent debate in recent times.
A term that is used brought up during discussions on Explainable AI concept is discussed is the ‘black box problem’. This ‘problem’ refers to a situation in which an AI algorithm reaches a conclusion but does not reveal the factors that helped it get there. This puts the reliability of an AI algorithm at stake. After all, if AI is going to make decisions for us, then we should be able to know how it made the decision.
Scope of Explainable AI
Since deep learning models comprise several layers of artificial neural networks, they are not unfailing or error-free. They can quickly lose their credibility when they are misled. According to the Department of Defense (DoD), AI-enabled autonomous and symbiotic systems face several challenges.
The DoD believes that Explainable AI, in particular, explainable machine learning, will be the key to understanding, trusting, and effectively managing the next-gen of artificially intelligent partners. As AI systems develop further, the complexity of the AI applications will increase. This will make it more difficult to achieve the goal of Explainable AI.
The Aim of Explainable AI
Explainable AI aims to explain the decision-making process involved in AI/ML systems; this will help in working out the strengths and weaknesses of these systems and predicting their future behavior. To achieve this goal, new, modified artificial learning techniques have been developed to produce more definable models for machine and deep learning systems.
The intention is to integrate these models with advanced human-computer interactive interface techniques so that the ML/deep learning models can be converted into easy-to-understand and useful explanations for the end-user.
XAI, short for Explainable AI, is one of the few current programs initiated by Defense Advanced Research Projects Agency (DARPA) that is geared towards enabling third-wave AI systems. These systems include intelligent machines that can understand the environment and circumstances in which they operate. Additionally, they can build explanatory models over time for delineating real-world phenomena.
From an ethical point of view, this is important because someone’s well-being or life might be at stake. For example, if a physician uses an AI system to make decisions about patient care, the physician should be able to get an explanation behind the AI’s decisions. The physician can then explain the details of the treatment to the patient.
Also, it is the ethical responsibility of the physician to inform the patient that the treatment being proposed by them is based on the recommendation of an AI-enabled decision support system. You simply can’t tell someone that they have a high risk of a heart attack and not explain to them how you reached this conclusion. This would be criminal, both figuratively and literally.
In the case mentioned above, the AI system should be able to explain which data was used to perform the evaluation. Additionally, it should identify what additional data is needed as well as what more needs to be done to perform a proper evaluation free from any bias or errors.
The Thinking Behind Explainable AI
The argument made by Explainable AI is that current AI/ML technology is not good enough because it cannot explain how it reached a conclusion. Weight is added to this argument when you consider how humans make decisions and that the scientific explanation for these decisions is not accepted and often ridiculed.
Say, it’s the winter season and you head to a local coffee shop and order yourself a cold coffee. A friend is with you and asks why you ordered a cold coffee when it is chilly outside. Luckily, you’ve just read an incredible article about how cold brew coffee helps improve brain functions. You tell your friend that you are drinking the coffee to sharpen your mind. Your friend can’t make head or tails of this explanation. After all, they have not read the article and have no context for what you have just said.
This is the current situation of machine learning and deep learning models. The current explanation behind the decisions made by AI/ML systems is beyond the comprehension of most humans. Another important consideration for Explainable AI is that AI approaches will vary based on where they are applied. This means that healthcare providers’ approach to the implementation and operation of AI systems will significantly differ from insurance companies and banks. Different sectors have different ethical requirements and legal regulations which they need to adhere to for maintaining their credibility and avoid penalization.
Therefore, before you equip your AI system with the ability to explain its decisions in simple language, you must make sure that the ethical requirements and legal regulations of your industry have been considered in the development and implementation of the system.
Current research in this area is being done by Google, DARPA, and DeepMind. Regardless of the sector and the way it employs AI systems in its operations, the decision-making function of the system needs to be perfected. AI should not be treated as an oracle that can’t be questioned; AI systems should be able to establish a cause-effect relationship and explain decisions.
In other words, AI and deep learning models must be interpretable and flexible enough to work in harmony with the opinions of various disciplines/sectors and the experts who have the technical and academic knowledge of these systems.
Methods of Developing an Explainable AI
Generally, it is better and more useful to think about AI and machine learning in terms of models. However, in the case of Explainable AI or explainability applications, it would be more fruitful to deliberate over general techniques. The techniques for explainable AI applications can be categorized into two types: ante-hoc and post-hoc. Following is a brief description of each.
1. Ante-Hoc Techniques
In ante-hoc techniques, explainability is incorporated into a model from the start. The Bayesian deep learning (BDL) is one example of this. This is because BDL allows one to determine the uncertainty level of a neural network about its predictions. Further resources, like more data, may be needed if the confidence of a neural network about its predictions is below a reasonable level, such as a 5% confidence level.
2. Post-Hoc Techniques
Unlike ante-hoc techniques, post-hoc techniques do not incorporate explainability into a model from the start. Instead, they allow modes to train in a normal way and incorporate explainability into time at the time of testing. A popular post-hoc technique is Local Interpretable Model-Agnostic Explanations (LIME).
A technique that can be applied to any ML model, the LIME technique perturbs data sample inputs and understands how the predictions change to get an understanding of the model. An example would be applying the LIME technique to pixels of a cat’s image to find out which pixel segments contributed the most to how the model classified that image.
While they are generally more flexible than ante-hoc techniques, post-hoc techniques typically provide less explanation about the model. Currently, several post-hoc techniques are being explored. This includes a technique that utilizes influence functions.
In simple words, an influence function is a method for measuring the change in the parameters of a model with a change in the factor of training data points. This allows one to answer a counterfactual question such as the outcome of a model’s prediction if a data point was altered or not there. Although it shares some qualities with LIME, the ability of the influence function to focus on counterfactuals allows it to effectively debug and elucidate possible errors in a data set.
Understanding the Complexity in Achieving the Goal of Explainable AI
While it is a necessary and admirable feat, achieving the goal of Explainable AI is not at all easy. Therefore, before you set out on a mission to achieve the goal of Explainable AI, you may want to consider the following four things to make an informed choice.
1. Explanations May Not Be the Best Way to Build Trust
Contrary to popular belief, the ability of AI to explain itself to the businesses that use it as well as to the public-at-large does not make it more trustworthy. Instead, AI would be more trustworthy if testing and analyzing an AI system provided us with the answers we seek.
The simple reason that makes working towards the goal of Explainable AI not a great idea is that the conclusions drawn by AI may not be comprehensible to humans, even when explanations are provided. Therefore, it is best to rely on testing and analysis, rather than conclusions drawn by AI, to establish the reliability of an AI system.
2. Explainable AI Relates Directly to Application Design
Making progress with Explainable AI directly correlates to starting at the application level. A mistake that many people make is seeing the Explainable AI concept as the starting point for getting the answers. While starting at the application level does not guarantee the full ‘AI explainability’ for a given application, it is can be an extremely useful strategy for those involved in the development of AI-based applications.
3. Reinforcement Learning Is More Complicated Than First Thought
Reinforcement learning, also known as self-learning, involves an algorithm that categorizes the actions performed by it as undesirable or preferable based on the feedback it gets. To conclude, the AI algorithm first studies its actions and then uses a trial-and-error approach to learn more. This type of AI learning style is widely misunderstood.
Unfortunately, reinforcement learning is more complex than what people think. Since the sum of all actions determines the success of any complicated task, people must account for some established conditions to make AI-based reinforcement learning effectively. Additionally, they must account for every variable in the AI environment. This makes reinforcement learning, which is an important element of the Explainable AI concept, more complicated than previously believed.
4. An Ethics Component Should be Included in All-Encompassing Conversations About Explainable AI
Ethical dilemmas are a part and parcel of AI discussions today. There are some suggestions for holding AI systems and machine to a higher standard than humans to meet the goal of Explainable AI. Gut instincts and transitory ideas are the reasons people act without forethought. On the other hand, AI systems and machine behave according to how they have been programmed. However, this does not reveal the whole story.
While an AI system or machine can be programmed to act ethically, the conversations around Explainable AI should always consider the ethics surrounding AI to meet the goals of both Explainable AI and Ethical AI.
Final Word
As mentioned earlier, Explainable AI is an important concept to consider while developing, implementing, or using an AI system. In this article, we looked at the first principle of Ethical AI and understood the Explainable AI concept including its aim, scope, the thinking behind it, the methods of developing/applying it, and the complexity in achieving its goal.
In the next article in this series, we will try to gain an understanding of the principles 2, 3, and 4 of Ethical AI.
