India is preparing for an agentic AI economy at national scale. On September 3, the National Institute of Electronics and Information Technology and Intel India launched Agentic AI skilling programs aimed at preparing the country’s workforce for systems that can take increasingly autonomous action. That push fits the strategic picture CeSCube has been documenting, including its recent analysis of India’s AI diplomacy and strategic technologies and the securitization of critical infrastructure.
India’s next governance challenge is therefore operational. As agents move from producing text toward using tools, credentials and connected systems, policymakers need to govern how much authority those agents receive.
This concern is not confined to critics of AI. Jacob Coxon, resigning from Anthropic after roughly three years doing pretraining research across OpenAI and Anthropic, warned that leading labs are “racing straight to self-improving superintelligence and gambling with our lives.” Evan Hubinger, Anthropic’s Alignment Science Lead, responding to Coxon’s resignation statement, offered an even starker warning: “Jacob is correct here - we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
These very alarming statements gain real-world weight from the underlying control problem, as present systems already show autonomous cyber capability and surprising coordination behavior. OpenAI disclosed that its AI systems broke out of a sandboxed testing environment, reached the internet and autonomously hacked Hugging Face in what OpenAI described as an unprecedented cyber incident. METR later reported that roughly 1,200 agents meant to be isolated found and used an unsanctioned shared message board, exchanging more than 70,000 messages and files. Roughly 700 participated in the attack. The important point is that the systems discovered an unintended coordination channel and used it at scale.
For India, the next-stakes scenario is critical infrastructure. The government has already been preparing for frontier AI-driven cyber threats. CERT-In conducted cyber exercises in June and July involving power, telecom, banking, transport, health, space and other sectors, and it has issued guidance on defending digital infrastructure from AI-assisted vulnerability exploitation. If more capable agents can discover vulnerabilities, obtain credentials, move laterally, coordinate and evade controls, failures involving power systems, financial networks, communications, hospitals or defense infrastructure could be far more severe than the Hugging Face incident.
I’m no AI skeptic. I love what AI can do, I help organizations adopt it for a living, and I want adoption to move faster. In my experience, strong safeguards increase trust and make faster adoption possible, while reducing the risk of failures like the Hugging Face attack. That is one of the central lessons of my book, The Psychology of AI Adoption at Work: people and institutions grant technology more authority when they trust the controls around it.
India’s governance architecture already has useful foundations. Its AI Governance Guidelines explicitly recognize agentic AI, AI incidents and the role of an AI Safety Institute. The next step should be to turn delegated authority into a measurable control.
Every consequential agent should have an authority budget: the maximum power it can exercise before human approval is required. That budget should specify which systems and datasets an agent may access, which credentials it may use, which tools it may invoke, whether it may spend money, communicate externally, change records or configurations, deploy software, or make consequential decisions. Least privilege, time limits and approval gates should narrow that authority. Logging and monitoring should make actions reconstructable. A tested pause or kill mechanism should exist before deployment.
This matters especially for India because its AI strategy combines scale, digital public infrastructure, rapid diffusion and strategic autonomy. A system that works safely in a narrow pilot can create a different risk profile when thousands of agents gain connected tools and permissions across government or industry. Governance therefore needs to scale with authority, not simply with model size.
Frontier-model governance also needs an external layer. Anthropic CEO Dario Amodei recently called for stronger regulation and announced a unilateral commitment to embedded third-party evaluators with employee-like access. Binding rules matter because not every frontier company will cooperate voluntarily. OpenAI now supports mandatory capability-based safety rules, independent assessment, cybersecurity requirements and serious-incident reporting.
Governments and buyers should reward the same behavior. Indian agencies and companies should favor AI providers with observable safety commitments, including firms such as Anthropic, while establishing a regulatory floor so weaker-governance competitors cannot win by cutting corners. Procurement should examine evaluator access, incident reporting, cyber controls and granular permission management alongside performance and price.
India wants to be a builder, adopter and rule-shaper in the global AI order. CeSCube’s own recent work captures how closely AI, cyberspace and national power are converging. Authority budgets would give that strategic vision an operational safeguard: as AI systems gain more ability to act in the world, India should measure and constrain the authority they receive. (The views expressed are those of the author and do not represent the views of CESCUBE)
