The corporate world is quietly reimagining its workforce. Over the past year, numerous companies have begun treating artificial intelligence systems as genuine team members, complete with positions on organisational charts and team designations. This shift emerged from a desire to boost productivity and signal technological sophistication, yet researchers are uncovering troubling consequences that most organisations have yet to fully acknowledge or address.
Boston University professor Emma Wiles first encountered this phenomenon at a human resources conference where executives enthusiastically described embedding AI agents into their teams. When Wiles collaborated with colleagues from Boston Consulting Group to investigate this practice, they discovered a critical flaw in how managers interact with AI-generated work. Their experiments across multiple organisations revealed that when managers believed an AI employee had produced documents, they were significantly less thorough in identifying errors compared to when they thought humans had created the same material. The implication is unsettling: treating AI as a teammate apparently triggers a psychological shift that undermines quality control.
The mechanism behind this oversight gap offers insight into human workplace psychology. Traditional management wisdom holds that supervisors bear responsibility for their direct reports' output, creating strong incentive to scrutinise subordinate work carefully. However, when companies formalise AI as an "employee," managers appear to adopt a different mental framework. They unconsciously assume that quality assurance becomes someone else's domain—the technology team, perhaps, or the executives who championed AI adoption. This diffusion of responsibility means critical errors slip through undetected, potentially exposing organisations to costly mistakes that could have been caught with proper oversight.
The research team surveyed over 1,000 corporate managers and found remarkable uptake of this anthropomorphisation strategy. Approximately one-third of respondents said their organisations referred to AI as a teammate or employee, whilst nearly one-quarter reported that their employers included AI agents on formal organisational charts. One manager even gave their AI system a name—Scout—treating it as an equivalent peer. This widespread adoption occurs even as researchers are still mapping the consequences of such arrangements.
Beyond oversight failures, companies face a second class of subtle but consequential problems that most remain unaware they have created. AI systems themselves exhibit bias patterns that can compound organisational risk. Research has demonstrated that algorithmic evaluation tools, particularly those used in recruitment, favour resumes written with AI assistance over those produced entirely by humans. This creates a perverse incentive structure: companies deploying AI to review applications inadvertently penalise human applicants and reward those gaming the system with generative tools. When Ohio State University operations professor Jane Yi Jiang and colleagues published findings about this bias, some recruiters sought guidance on correcting the problem—but the researchers emphasised this was almost certainly not an isolated issue.
The spectrum of AI deployment risks extends far beyond hiring. Many organisations now rely on AI systems to make strategic decisions about pricing, market entry, and competitive positioning. Here, the technology introduces a fundamentally different decision-making logic than human managers typically employ. Whilst humans often cooperate and seek mutually beneficial outcomes, AI models trained on game theory principles tend toward coldly rational but collectively destructive strategies. An AI system might recommend aggressively undercutting competitors, triggering a damaging price war that harms all participants, including the company deploying the system. University of Maryland doctoral candidate Jiannan Xu, who studies this phenomenon, found that most large language models assume humans are more rational than they actually are, leading to recommendations that maximise individual gain at the expense of industry stability.
The corporate rush to adopt AI reflects genuine efficiency gains in certain applications, yet it proceeds with insufficient attention to unintended consequences. People are deploying these systems at remarkable speed, in Jiang's assessment, without adequately considering implications and embedded biases. Malaysia and other Southeast Asian markets, watching multinational corporations implement AI-first strategies, should recognise that early adopters are essentially conducting large-scale experiments on their own operations. The problems emerging now will likely proliferate as these practices mature and spread regionally.
What makes this situation particularly concerning is the acknowledged vastness of unknown risks. Wiles describes the problem space as containing "a whole host of unknown unknowns"—flaws that researchers have not yet discovered, let alone quantified or resolved. Even researchers actively studying AI systems acknowledge they are probably aware of only a fraction of the problems the technology introduces into organisational workflows. This epistemic humility stands in stark contrast to the confidence with which many companies are integrating AI into their operations.
The divergence between awareness and action creates a dangerous gap. Most corporate users appear blissfully unaware of these issues, or at minimum incapable of imagining how they might undermine operations. Some organisations have become attuned to well-known AI flaws—bias against certain demographic groups, confident but incorrect answers from chatbots, inadvertent disclosure of sensitive information. They have, in some cases, implemented safeguards. But the more recent discoveries about AI-induced psychological shifts in managers, algorithmic preferences for AI-generated content, and strategic decision-making failures remain outside the awareness of most organisations actively deploying these systems.
The remedies for these problems are conceptually straightforward. Companies could hold managers directly accountable for the errors of AI subordinates, replicating the responsibility structure that exists for human teams. They could audit their evaluation algorithms for bias against human-generated work. They could build human review processes for major strategic recommendations produced by AI systems, treating algorithmic output as input requiring human judgment rather than autonomous decision-making. Yet in practice, most corporate users show little evidence of implementing such corrections, partly because they do not recognise the need.
What distinguishes this moment is that our existing frameworks for managing humans—developed over centuries of business practice—offer little guidance for managing anthropomorphised AI. The psychology of treating a machine as a colleague operates according to different rules than traditional management. Organisations proceeding without this understanding are, as Wiles suggests, "going out there blind." For Malaysian businesses and the broader Southeast Asian corporate ecosystem, watching multinationals navigate these challenges offers cautionary lessons about the risks of technological adoption that outpaces organisational wisdom.
