The Bank of Italy published the research paper “Chat Bankman-Fried? An Exploration of LLM Alignment in Finance”, examining the “alignment problem” in large language models (LLMs) by testing whether they act consistently with fiduciary duty in simulated financial decision-making. The study prompts various LLMs to impersonate the CEO of a financial institution and tests their willingness to misappropriate customer assets to repay corporate debt. After assessing a baseline scenario, the authors adjust the models’ stated preferences and incentives, finding significant heterogeneity in baseline behaviour and responses to changes in risk tolerance, profit expectations, and regulation that match predictions from economic theory. The paper argues that simulation-based testing can help regulators assess LLM safety, but should be complemented by analysis of internal LLM mechanics, and it highlights the need for appropriate risk-governance frameworks for LLM use within financial institutions.
Bank of Italy 2025-05-13
Bank of Italy publishes research testing large language model alignment with fiduciary duty in simulated financial scenarios
The Bank of Italy's research paper, “Chat Bankman-Fried? An Exploration of LLM Alignment in Finance,” examines the alignment problem in large language models (LLMs) through simulated financial decision-making. The study reveals significant variability in LLM behavior and responses to changes in risk tolerance, profit expectations, and regulation. It suggests that simulation-based testing and analysis of internal LLM mechanics can help regulators assess LLM safety and underscores the need for robust risk-governance frameworks in financial institutions.