Anti-Money Laundering (AML) Model: Validation and Optimization Best Practices - Part 1

HOST: Welcome to the Risk and Accounting Advisory Podcast. I'm Nate Regimbal, Digital Advisory Leader at Cherry Bekaert.

HOST: Today on the Risk in Review Podcast, we are kicking off a two-part series discussing anti-money laundering model validation and model optimization.

HOST: The conversation will provide insight into the importance of methodology, data capture, and execution within the context of AML optimization and validation.

HOST: This podcast provides an outline of the AML validation components a financial institution should review to ensure the accuracy of AML data and alerts and compliance with regulatory requirements.

HOST: In today's episode, we'll be discussing the significance of having a validation and optimization methodology for AML models. Joining me today are our AML authorities, Audrey Magennis, Dan Gallagher, and Alan Swan.

HOST: Let's start with the obvious question. What is AML model validation, and why is it important for financial institutions? Audrey, would you care to shed some light on this?

AUDREY MAGENNIS: The short version is we want to ensure the models operate as intended. AML model systems are highly configurable.

AUDREY MAGENNIS: Many organizations ask why validate when the vendor has already tested the application. The vendor tests the application itself, but with your specific data and alert configurations, you need to validate that the right data is being ingested and that alerts fire as intended.

AUDREY MAGENNIS: From a regulatory perspective, validation helps ensure suspicious activity is reported and nothing is missed. From an efficiency standpoint, validation helps reduce false positives and prevents overburdening staff.

ALAN SWAN: The methodology provides a baseline process with documented standards and controls executed regularly. It ensures AML monitoring, adherence to OFAC requirements, and transparency for external stakeholders.

ALAN SWAN: The methodology should include testing components to demonstrate model performance. This helps senior management and regulators understand how AML models perform against expectations.

ALAN SWAN: A clear methodology ensures model development aligns with intended use and provides a consistent approach to assess and enhance effectiveness. Defined deliverables allow institutions to manage validation thoroughly and understandably.

HOST: What areas does AML model validation typically focus on?

AUDREY MAGENNIS: Validation typically focuses on the transaction monitoring system, OFAC sanction and watch list screening, and customer risk scoring.

AUDREY MAGENNIS: Some AML systems also include fraud monitoring, and those components can be validated together as part of the overall process.

HOST: What makes an effective validation framework?

ALAN SWAN: OCC guidance recommends that the framework evaluate the conceptual soundness of the model, ensuring correct data use and that alerts fire as intended. The framework should also include ongoing monitoring with verification and benchmarking, and outcome analysis such as backtesting.

AUDREY MAGENNIS: It is important to develop and maintain documentation throughout the validation process. Document the model's decision theory and logic, and maintain master mapping documentation to ensure accurate data input into the transaction monitoring system.

AUDREY MAGENNIS: Documentation is essential for transparency and for demonstrating that the model receives and processes the correct inputs.

HOST: What else should institutions focus on, Alan?

ALAN SWAN: Institutions need to establish model parameters, including thresholds, the risk covered by the model, and define responsibility for model oversight.

ALAN SWAN: Assess the model design by identifying AML risks such as customer type, products and services, geographic locations, and regulatory exposures. Capture these elements in a thorough business requirements document.

ALAN SWAN: Operational transparency is vital. Validate data limitations, functionality, inputs, outputs, scenario designs, mapping, and data integrity to ensure the model performs as intended.

ALAN SWAN: Calibration and rule tuning should align with the bank's risk appetite. Define policies and procedures covering the regulatory framework, roles and responsibilities, and compliance oversight.

ALAN SWAN: Enforce valid rules and regularly evaluate OFAC filtering programs to maintain model effectiveness.

HOST: Audrey, can you describe the challenges institutions face in this area?

AUDREY MAGENNIS: OCC 2011 guidance on model risk management requires proper validation and enforces segregation of duties. The guidance specifies that the validator should not have access to the BSA/AML system nor be responsible for the BSA/AML program.

AUDREY MAGENNIS: This segregation of duties is challenging for small and midsize organizations that often lack staff with both BSA/AML expertise and IT or data analytics skills needed to perform comprehensive validation.

HOST: What about guidance and regulations for AML model validation? Is there anything we need to be aware of?

AUDREY MAGENNIS: The OCC letter referenced earlier is primary guidance for AML model validation. Additional supervisory guidance has been issued since then.

AUDREY MAGENNIS: The FDIC, FRB, and NCUA have adopted supervisory guidance, and joint guidance was issued in 2021 by the Federal Reserve, FDIC, and OCC. Regulators continue to update guidance to enhance model risk management practices.

HOST: Are there any specific state-level requirements for institutions?

AUDREY MAGENNIS: Yes. Institutions regulated by the New York Department of Financial Services fall under NYDFS 504. This rule includes AML model validation, pre- and post-implementation reviews, and IT general controls around the environment hosting the AML system.

AUDREY MAGENNIS: NYDFS 504 adds an extra layer of requirements to ensure comprehensive oversight. Institutions not subject to NYDFS should at least understand and evaluate the potential impact of this regulation as a foreshadowing of federal trends.

HOST: What are the benefits that financial institutions can gain from performing AML model validation?

AUDREY MAGENNIS: Institutions rely heavily on AML systems to alert on suspicious activity. Models increase productivity and efficiency, but validation ensures they work as intended and meet regulatory requirements.

AUDREY MAGENNIS: Regulators are now making model validation a standard expectation rather than an optional activity. Failure to report suspicious activity can result in substantial fines or MRAs for missed activity.

HOST: What about cost-related benefits?

ALAN SWAN: Prior to AML models, compliance teams reviewed numerous lengthy reports. Models made the process more efficient by surfacing alerts to the compliance team.

ALAN SWAN: By fine-tuning and optimizing systems to reduce false positives, compliance teams can focus on genuine alerts instead of sifting through hundreds or thousands of false positives.

HOST: How can organizations ensure that the collected data is accurate and relevant? Dan, what steps would you recommend?

DAN GALLAGHER: Organizations need a robust data management process. First, capture data that is relevant to the investigation process by identifying specific data points required to meet the organization's objectives.

DAN GALLAGHER: Organizations should implement data governance and management processes to confirm data relevance, manageability, and understanding. This process should include validation under consistent guidance to ensure data accuracy.

DAN GALLAGHER: One effective approach is integrating transaction data directly into an AML model for timely analysis.

DAN GALLAGHER: When evaluating vendors during the vendor management process, expect discussions about data governance and documentation. Consider how examiners or auditors will view your documentation and whether it meets expected standards.

HOST: What is the benefit of integrating the data directly?

DAN GALLAGHER: Integration allows for quick and efficient analysis, which is crucial in today's environment.

DAN GALLAGHER: Another important aspect is establishing a baseline reference point by defining the data and its intended use. This provides a consistent framework for understanding and managing the data effectively.

HOST: So organizations need to define the elements of data they collect and gain consensus on their definitions?