Explainable AI in Finance: Addressing the Needs of Diverse Stakeholders

To bridge the gap between AI theory and real-world portfolio implementation, access our fully worked quantitative case study. Utilizing real Bloomberg LP factor data, this hands-on Python configuration demonstrates how to train an XGBRegressor model over a 100-month testing horizon. It explicitly details how to apply and code SHAP plots, LIME summaries, individual conditional expectation (ICE) configurations, and partial dependence plots (PDPs) to dynamically break down style factor returns.

[Access the Open-Source Python Repository on GitHub]

This report is an essential strategic playbook designed for:

• Risk managers and credit analysts auditing complex machine learning frameworks.
• Portfolio managers and quantitative traders deploying algorithmic trading strategies.
• Compliance officers and internal auditors validating corporate model governance.
• Regulators and policymakers shaping systemic AI transparency guidelines.

XAI refers to a suite of artificial intelligence and machine learning techniques that provide human-understandable explanations for algorithmically generated outputs. It creates a critical cognitive bridge between human intuition and machine intelligence, transforming opaque "black-box" configurations into transparent, auditable, and legally compliant systems.

Decision-making systems orchestrate our world, powered by machine learning (ML) systems based on AI. These AI-based systems help underwriters and credit analysts assess risk, portfolio managers optimize security allocation, and individuals select investment and insurance products. As the digital economy grows, so does the need for immense computing power. This power comes at a cost, however: Systems based on deep learning algorithms in particular can become so complex that even their developers cannot fully explain how these systems generate decisions. This, in essence, is the “black-box problem,” which makes it difficult to trust an AI system’s decisions, assess model fairness, and meet regulatory demands. Consequences include actual or perceived discrimination against protected consumer groups and violation of fair lending rules.

This problem has led to the consideration of various proposed solutions — the most well-known being XAI technologies — to create a cognitive bridge between humans and machines. XAI refers to AI and ML techniques, or capabilities, that seek to provide human-understandable justifications for the AI-generated output. Implicit in XAI is the question, “explainable to whom?” In fact, defining “whom” (or the user group) is essential to determining how the data are collected, what data can be collected, and the most effective way of describing the reason behind an action.

This report focuses on the human behind human–machine collaboration. The objective is to generate discussion on the best way to support the diverse needs of AI users. As such, this report explores the role of XAI in modern finance, highlighting its applications, benefits, and challenges, with insights from recent studies and industry practices. It presents a detailed analysis of the explainability needs of six stakeholder groups, most of whom are nontechnical users. The analysis includes matching their needs with their job responsibilities and assessing the most relevant XAI methods. Finally, the report reviews some alternative approaches to XAI — evaluative AI and neurosymbolic AI.

With its focus on AI explainability, this study represents a deeper analysis of transparency and explainability issues raised in earlier CFA Institute works. These publications include “Ethics and Artificial Intelligence in Investment Management" (Preece 2022) and "Creating Value from Big Data in the Investment Management Process" (Wilson 2025).

To ensure complete corporate compliance, financial institutions must align their machine learning models with rapidly emerging international frameworks:

• The EU AI Act: Mandates strict AI transparency, data lineage tracing, and human oversight protocols for high-risk financial applications, requiring granular technical documentation for training datasets.
• GDPR (Right to Explanation): Grants European consumers the explicit legal right to receive meaningful, human-understandable explanations regarding automated credit or profiling decisions.
• US Equal Credit Opportunity Act (ECOA): Requires financial firms to provide "specific and accurate reasons" for adverse credit actions, rendering uninterpretable deep learning scoring systems a direct compliance risk.

Why Finance Needs XAI

• Credit scoring and lending: Deep learning models can provide more detailed assessments by using alternative data (e.g., credit card transactions, social media), but they require explainability to ensure fairness, transparency, and regulatory compliance.
• Investment and portfolio management: AI can enhance financial analysis, asset allocation, and risk management by detecting patterns in large datasets to improve modeling and decision making, but a lack of explainability and model “hallucinations” can lead to misinformed decisions and financial losses.
• Insurance: AI can speed up underwriting, boost fraud detection, and enhance customer service, but its use raises concerns about unintended bias and discrimination created through correlations with sensitive personal attributes. Examples of nonpersonal characteristics that may indirectly correlate with protected attributes include zip codes as proxies for socioeconomic status or ethnicity, as well as purchasing history for gender or ethnicity.
• Regulatory challenges: AI-driven systems present oversight difficulties caused by limited transparency in data sources and decision-making logic.

This report categorizes XAI methods into two main types:

• Feature attribution (SHAP & LIME): Mathematically maps out and isolates exactly which input variables influenced a specific, local automated prediction.
• Counterfactual explanations: Details the minimal parameters required to change an output (e.g., "If your debt-to-income ratio were 4% lower, the loan would be approved").
• Visual explanations: Utilizes interactive toolsets like heatmaps, attention layers, and partial dependence plots (PDPs) to chart model logic.
• Post-hoc (after-the-fact explainability) models:
  • Applied to black-box models (e.g., deep learning, ensemble methods) to generate explanations after predictions are made
  • Examples:
    • Feature attribution methods (SHAP, LIME): Determine which input factors influenced an AI decision
    • Visual explanations: Heatmaps, partial dependence plots, and attention maps to illustrate AI reasoning
    • Counterfactual explanations: Explain how a decision could have changed under different circumstances (e.g., “If income were $5,000 higher, the loan would be approved”)
• Rule-based and simplification approaches: Approximate black-box models with more interpretable versions

This report addresses the following key examples. This list should not be construed as exhaustive, however.

• Credit scoring and lending: XAI methods, such as SHAP and LIME, can help financial institutions justify loan approvals or denials.
• Algorithmic trading and investment strategies: Visual techniques, such as heatmaps, can help traders understand how models generate buy/sell signals.
• Fraud detection and anti–money laundering (AML): Feature attribution techniques are used to improve the interpretability of fraud detection models.
• Regulatory compliance and risk management: Regulators require clear explanations for AI-driven financial decisions, ensuring accountability and fairness.

• Technical challenges:
  • Lack of standardized evaluation metrics: No universal benchmarks exist to assess the quality of AI explanations, leading to inconsistent evaluations.
  • Real-time decision-making constraints: Delivering instant, understandable explanations during fast-paced transactions remains difficult.
• Regulatory challenges:
  • Privacy risks: Detailed explanations can unintentionally reveal sensitive personal or financial data.
  • Absence of universal explainability standards: Differing regional regulations (e.g., EU versus US regulations) create compliance challenges for firms that operate internationally.
• User experience challenges:
  • Overreliance on AI explanations (algorithmic appreciation): Users often trust AI outputs without critical evaluation, leading to confirmation bias.
  • Limited user-friendly tools: Most XAI tools are built for technical users, with a lack of accessible interfaces for business users, regulators, and customers.

Beyond standard XAI frameworks, the report explores the following:

• Evaluative AI: Focuses on hypothesis-driven decision making rather than direct AI recommendations, promoting human engagement
• Neurosymbolic AI: Integrates rule-based reasoning with deep learning to improve interpretability while retaining predictive power

To successfully integrate XAI, firms must tailor model outputs to match the job responsibilities and technical baseline of the end-user:

• Regulators and compliance officers: Require traceable documentation, institutional accountability logs, and fairness metrics to verify protection against bias.
• Internal auditors: Focus on benchmarking automated systems against traditional human logic models to optimize fraud tracking pipelines.
• Risk management teams: Depend on scenario analysis and deep stress-testing metrics to confirm model structural stability under volatile market constraints.
• Data scientists and ML developers: Prioritize feature debugging, hyperparameter tuning, and balancing the classic accuracy vs. interpretability trade-off curve.
• Traders and portfolio managers: Demand fast, visually accessible execution attributions to explain macroeconomic exposure shifts instantly.
• Retail consumers and investors: Require clear, simple, and actionable text-based justifications to evaluate personal credit worthiness and product suitability.

XAI presents a transformative opportunity for financial institutions to enhance transparency, regulatory compliance, and trust in AI-driven decision-making. Although challenges such as overreliance on explanations, privacy risks, and model complexity persist, strategic adoption of XAI can help financial firms navigate these obstacles effectively.

By developing standardized frameworks, tailoring explanations to stakeholders, balancing interpretability with performance, and ensuring privacy protection, financial institutions can use XAI to the extent of its potential while maintaining ethical and responsible AI practices.

Future research should focus on developing hybrid models that balance accuracy with interpretability, creating standardized benchmarks for evaluating XAI methods, and improving computational efficiency in real-time financial applications.