Bank of England publishes research finding GPT-3.5 tracks short-horizon inflation attitudes but lacks a consistent inflation model

The Bank of England has published a staff working paper examining whether large language models can form inflation perceptions and expectations from macroeconomic price signals. Using GPT-3.5 Turbo and a synthetic version of the Bank’s Inflation Attitudes Survey, the paper finds that the model can reproduce aggregate survey results and official statistics at short horizons, and can mirror some household patterns across income, housing tenure and social class. But it also finds important limits: the model shows weak and unstable alignment with individual responses, performs poorly on longer-horizon inflation expectations and does not display a coherent underlying model of consumer price inflation. The study relies on GPT’s September 2021 training cut-off, which meant the model had no knowledge of the subsequent UK inflation surge and allowed the authors to test how it responded to post-cut-off price information. The paper finds GPT is especially sensitive to food inflation, similar to human respondents, but also shows unexplained inconsistencies in how it reacts to inflation components. Later GPT releases displayed an upward drift in unconditioned inflation perceptions, which the authors say makes the fixed pre-surge cut-off important for valid testing. A Shapley value decomposition adapted to the synthetic survey setting is used to identify which prompt inputs drive responses, and the authors suggest the framework could be applied to assess other models, support social science research and improve survey design.