The 'Black Box' Promise: India's AI Loan Push Risks Automating Exclusion

OPINION: There is a seductive narrative currently unfolding in Mumbai: the idea that a 'black box' can be more compassionate than a human loan officer. By replacing manual underwriting with algorithms, the goal is to bring the 'underbanked' into the fold. But let's be clear—when you outsource the decision to deny a loan to a machine trained on 'alternative data,' you aren't eliminating bias; you are just making it harder to audit.

As first reported by The Register, Sanjay Malhotra, the governor of India’s Reserve Bank, recently addressed the FIBAC conference in Mumbai with a provocative proposition: financial institutions should use AI to approve loans that human assessors would likely reject. Malhotra argues that AI can extend the frontier of "bankable" India by analyzing data points that traditional banks ignore, such as digital footprints, utility payments, GST filings, and cash flows. This is specifically targeted at gig workers, small businesses without formal books, and first-time borrowers.

On paper, this is framed as a win for inclusion. Malhotra suggests that AI-driven voice interfaces in local languages could bridge the gap for rural populations where literacy rates are below 80 percent. He envisions a system where predictive models identify struggling borrowers early enough to offer counseling rather than just debt recovery.

However, the governor himself admitted the inherent danger of this approach. In his speech, Malhotra highlighted "The black box problem," noting that deep learning and generative systems often cannot explain their own reasoning. This opacity creates a critical failure point: if a small business is denied credit, both the borrower and the regulator are entitled to know why. Without that transparency, it becomes nearly impossible for boards or the Reserve Bank to verify if a model is functioning as intended.

More concerning is the risk that these systems will simply codify existing prejudices. Malhotra explicitly warned that AI could perpetuate biases against specific communities, occupations, or geographies. Furthermore, he expressed concern over a systemic dependency on a small cluster of tech providers, which could spread flawed algorithmic logic across the entire Indian banking sector.

To mitigate this, Malhotra has laid down a strict mandate: banks cannot hide behind the machine. He stated that "'The model decided' can never be an acceptable answer" and insisted that the responsibility for any decision rests with the bank, not the algorithm. His requirements for banks include maintaining a full inventory of AI systems, establishing board-approved governance policies, and ensuring the capacity to explain AI-driven decisions that materially impact customers.

While the Reserve Bank claims that meaningful human oversight will remain a design principle, the push to automate the 'frontier' of lending suggests a dangerous gamble. We are being told that AI is the most powerful accelerator for financial inclusion, but without a way to peer inside the box, it may just be a more efficient way to keep the marginalized out.