Why AI Needs to Explain Its Decisions
AI systems can process large amounts of data, identify complex patterns, and generate recommendations much faster than conventional approaches. However, many machine-learning systems operate like a “black box,” making it difficult for users to understand how a particular recommendation was produced.
This lack of transparency can create concerns about accountability, fairness, traceability, and the appropriate use of AI-generated decisions. The problem becomes more serious when an AI recommendation affects a person's health, financial situation, rights, safety, or professional responsibilities.
The concept of Explainable Artificial Intelligence, or XAI, addresses this challenge by making the reasoning behind AI outputs more understandable to humans.
However, Ahmad's findings show that simply adding an explanation to an AI system does not automatically make the system trustworthy. Users need explanations that are understandable, consistent, verifiable, and relevant to their actual work.
This distinction is important because the goal of XAI should not be to make users blindly trust AI. Instead, it should help users develop calibrated trust confidence that matches the system's actual capabilities, limitations, and uncertainties.
How the Research Was Conducted
Ali Ahmad from Universitas Darunnajah used an exploratory qualitative approach to understand what different stakeholders expect from explainable AI.
The research involved 20 informants, consisting of:
- 7 AI developers, or 35 percent;
- 5 information systems experts, or 25 percent; and
- 8 professional users, or 40 percent.
The interview transcripts were then coded and examined using thematic analysis. Rather than statistically testing variables, the analysis looked for recurring patterns in how different stakeholder groups understand transparency, explanations, and trust in AI.
Four Elements Build Trust in Explainable AI
The analysis identified four interconnected components that form the proposed XAI framework.
1. Clear Reasons for AI Decisions
Users want more than a final recommendation. They want to know why the system selected a particular option.
The interviews showed that unclear or overly technical explanations can make it difficult for nontechnical users to evaluate an AI recommendation. Clear explanations allow users to assess whether the factors behind a decision are reasonable and relevant.
As one professional user explained during the interviews, understanding why the system selected an option is essential rather than simply receiving the final result.
For Ahmad of Universitas Darunnajah, this means technical interpretability must be translated into explanations that ordinary professional users can actually understand and use.
2. Consistent Explanations
The second requirement is consistency.
Users can lose confidence when similar inputs produce different explanations without a clear reason. Changes in recommendations may be acceptable when data or system conditions change, but the AI should explain what caused the change.
Consistency helps users build a stable understanding of how the system behaves. The explanation should remain aligned with the input data, recommendation, and conditions influencing the decision.
3. Traceable Decision Processes
The third component is process traceability.
Users should be able to trace the data and processing stages that contributed to an AI recommendation. This includes information such as the data source, processing history, model version, and reasons behind changes in recommendations.
Traceability is particularly important when an AI system makes an inappropriate decision. A clear decision trail can help organizations identify errors, evaluate the system, and determine how the problem should be corrected.
4. Explanations Must Fit the User's Context
The fourth element is contextual relevance.
An AI developer, information systems expert, and professional user may require different levels of information. A technical explanation that is useful to a developer may be confusing or unnecessary for a professional user.
Ahmad's research therefore emphasizes that explanations should reflect the user's knowledge, responsibilities, tasks, and decision-making environment. More information is not necessarily better if it overwhelms users with irrelevant technical details.
What the Framework Means for AI Development
The four elements work together rather than independently.
Clarity helps users understand the reason for a recommendation. Consistency helps them understand the system's behavior over time. Traceability enables verification and auditing. Contextual relevance ensures that the explanation is useful for the person making the decision.
Together, these components form the conceptual XAI framework proposed by Ali Ahmad of Universitas Darunnajah. The framework connects technical interpretability with user understanding, accountability, and responsible use of AI recommendations.
For businesses and organizations, the findings suggest that trustworthy AI requires more than installing a technically capable model. Developers should consider how explanations will be presented, how users will verify recommendations, and how the system will document the information behind its decisions.
The approach may be particularly relevant in healthcare, education, finance, public administration, and industry, where AI recommendations can have significant consequences.
Trust Should Not Mean Blind Acceptance
A central message from Ahmad's research is that trustworthy AI should not encourage people to accept every recommendation automatically.
Instead, an effective XAI system should give users enough information to determine when an AI recommendation is appropriate and when it needs further review.
The study concludes that a trustworthy decision support system needs explanations that are understandable, stable, traceable, and appropriate to the user's context. Organizations should also document data sources, processing histories, model versions, uncertainty levels, and reasons for changes in recommendations.
The proposed framework remains conceptual, however. The research involved only 20 purposively selected informants and did not statistically generalize their views. The framework has also not yet been tested through a working prototype. Future research should therefore develop prototypes and evaluate whether the four components improve user understanding, decision accuracy, error detection, calibrated trust, and decisions to accept or reject AI recommendations.
Author Profile
Ali Ahmad is the author of the article and is affiliated with Universitas Darunnajah. His research in this article focuses on Explainable Artificial Intelligence, AI-based decision support systems, user trust, transparency, accountability, and thematic analysis. The article does not specify Ahmad's academic degree, so no degree is attributed to the author here.
Research Source
Key takeaway: Explainable AI becomes more trustworthy not simply by revealing how a model works, but by giving users clear reasons, consistent explanations, traceable processes, and information that fits their context.
0 Komentar