DATA AND AI ARE CHANGING THE WAY ORGANIZATIONS THINK, DECIDE, AND ORGANIZE. IT’S TIME HUMANITIES, MANAGEMENT AND SOCIAL SCIENCES GET INVOLVED.
Metalab

NEWS

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

EVENTS

[Student IDEAS] by Dan Xiao - Master in Data Sciences & Business Analytics at ESSEC Business School & CentraleSupélec

Abstract

As AI rapidly reshapes the global workplace and public perception, a question arises: how can organizations build the trust needed to collaborate effectively with machines?  This article introduces a positive feedback loop between Human-AI collaboration and trust in AI—where each boosts the other to unlock higher productivity, employee confidence, and innovation. Drawing on examples from multiple industries and voices from diverse geographies and generations, we conclude that trust in AI is far from universal. Psychological, cultural, and generational divides shape how people engage with intelligent systems. In reframing AI not as a threat but as a partner, this article argues for a “gentle transition”: a deliberate, human-centered approach to maximize AI’s potential while safeguarding what makes us human. Can we design a future where trust and technology grow hand in hand? This article offers both a warning and a way forward.

---
The Secret to Human-AI Synergy? It’s All About Trust - How the Positive Loop of Human-AI Collaboration and AI Trust is Formed

AI’s narrative swings. Some folks worry it’ll raise job displacement fears, and others see it as a revolutionary hype. What really helps bridge that gap? Trust. Across industries, from healthcare to manufacturing, implementing AI tools smoothly hinges on two keys: Human-AI collaboration and AI Trust—the cornerstones of sustainable human-AI synergy.

Key 1: Human-AI Collaboration: When Humans and AI Play to Their Strengths 

Picture this: a doctor using AI to cross-analyze 10,000 case studies and label all the anomalous areas in seconds, then applying her medical expertise to finalize a diagnosis. That’s Collaborative Intelligence in action—a dance where human creativity meets AI’s brute-force computation. Pioneered by Wilson & Daugherty1 in 2018, Human-AI collaboration means redesigning work so both grow smarter together. 

Key 2: AI Trust: The Glue That Holds the Partnership Together

But no one dances with a partner they don’t trust. AI Trust isn’t just believing the tech works—it’s relying on it to ethically enhance your work. In hospitals, trust means doctors trust AI could reduce standardized inspection process time with high accuracy to some degree and coordinate different diagnoses. Unlike one-off “acceptance”, trust is earned through transparency and consistency.

The Flywheel Effect: How Collaboration and Trust Keep Fueling Each Other

Now, the real magic happens when these two forces loop:

  1. Collaboration → Trust: Every time AI helps a designer brainstorm faster or a nurse figure out the priorities of the patients, trust starts to grow. Turns out, collaboration like this boosts role breadth self-efficacy and positive emotions among the employees involved2.
  2. Trust → Better Collaboration: When we trust AI tools, we’re more confident and comfortable handing over more sophisticated tasks to AI—like letting it draft legal contracts—freeing us to focus on developing strategy . Psychologically, employees’ trust in robots amplifies the pathway of positive emotions3.

This flywheel spun. When Siemens introduced AI visual inspection in its Amberg factory, skeptical workers argued machines couldn’t match their soldering expertise—until the AI spotted significantly more and even unanticipated defects, including microscopic flaws. The real win? Workers then taught the AI environmental nuances (like humidity impacts), creating a loop where humans refine algorithms while AI frees them to innovate. As the manager puts it: “It’s not humans vs. machines—it’s humans teaching machines to see better.”

Why This Loop Changes Everything

One important thing is that you don’t need to force yourself to “trust AI.” Start small: let it automate the repeated in-table data-processing . When it works, you’ll naturally assign it bigger challenges. And here’s the kicker: as AI handles routine tasks, you’ll rediscover your irreplaceable human edge—empathy, ethics, the ability to ask “What if?” That’s how we win the AI era: not by competing with machines, but by cooperating with them.

The State of Human-AI Collaboration in the Workplace

A “Gentle” Transition into the AI Era

When Dylan Thomas urged us to “not go gentle into that good night,” he framed resistance as an act of defiance. But today, as AI reshapes our world, “gentle” takes on a radically different meaning. This isn’t about raging against the machine—it’s about upgrading with it, one thoughtful step at a time.

For businesses today, staying afloat isn’t about radical revolution or reckless leaps into automation. It’s more about balance: adopting AI’s potential without losing sight of what makes us human. Think of it as a dance—more of a slow, steady waltz instead of wild rock—where innovation and tradition move gracefully on the floor. Eras setting the screens and music, humans take the lead and feel free to choose their partners, companies that master this rhythm don’t just survive; they thrive by blending machine precision with human intuition. They’ll shine on this AI era stage.

However, don’t mistake this gentle waltz for being passive. It’s about intentionally choosing how to adapt, ensuring AI serves our values rather than controlling them. Rather take this time as building a bridge between yesterday’s workforce and tomorrow’s possibilities—one where we walk confidently, not blindly, into the future.

Current Trends in Human-AI Collaboration

In 2024, there are 47% tasks (eg.clerk drafting reports, nurse comforting patients…)  in general workplace are led by humans and the other 22% are machine-driven, while the left 30% are hybrid(eg. marketer using AI to personalize campaigns, doctors leveraging AI to spot patterns and aid diagnoses). In 2024, 47% of tasks in the general workplace (e.g., clerks drafting reports, nurses comforting patients) are led by humans, 22% are machine-driven, and the remaining 30% are hybrid (e.g., marketers using AI to personalize campaigns, doctors leveraging AI to spot patterns and aid diagnoses).

By 2030, these numbers will blur into a ‘gentle’ transition, with a near even split of  33% each. 

Graph 1: Change in Share of 3 Types of Tasks in Industries from 2024 to 2030
Source: Future of Jobs Report 2025 by World Economic Forum

Nevertheless, the most successful workplaces won’t obsess over the percentages. They’ll focus on practical synergy rather than replacing humans with AI. They’re the ones asking, “How can we make 1 + 1 = 3?”

So let’s rewrite Dylan Thomas’ plea for the AI age:

Do not go gentle into this new frontier—

Rage, rage against replacement.

Dance gently with the partners that amplify

what makes us human.

Collaboration Across Industries: A Dance of Two Futures

Let’s look into the human share of work tasks across industries: By 2030, the trends point  in the same direction:  machines will be our innovative and vital partners   although the estimated percentages vary.In highly-automated industries such as oil and gas and chemicals decreases in human share are on average around 10% since they start from a relatively low baseline.  But in less-automated industries like healthcare, retail and professional services,  declines range  from 10% to 20% ( many are close to 20%).

Graph 2: Change in Proportion of Human-performed Tasks Attributable to Increasing Automation from 2025 to 2030, by Industry
Source: Future of Jobs Report 2025 by World Economic Forum

Moreover, we can identify two kinds of industries according to their traits and strategies to implement new technologies, and discuss the two futures:

  1. The Human Amplifier

This includes the industries that are more focused on complementing and augmenting, for instance government and hospital services. AI will be applied more carefully in daily touching and will not be preferred for the final decision-making, the kind of decisions that can determine the fate of human groups or individuals. 

The human amplifier group is the safe zone of the human-resource market in the AI revolution. Jobs that require a human touch—like empathy, counseling, or creative problem-solving—are harder to automate. Similarly, roles that involve complex manual tasks, such as surgeons or skilled tradespeople, are less likely to be fully replaced by machines.

However, in some specific areas, AI can bring a breath of  fresh air. Studies4 show that AI-Generated Art is offering a feasible method to address the limitations of traditional physical art materials, which expands treatment approachability and improves the effect of art therapy. 

  1. The Efficiency Followers

For those industries that are oriented toward high efficiency and fueled by advanced technologies, where spreadsheets and speed are their language, AI can be applied boldly to empower the automation. You get chatbots resolving claims before the coffee cools, or algorithms sniffing out fraud in milliseconds. But there’s a hidden cost: the soul of customer service risks becoming transactional.

Now that these fields are at the transformation frontier, tasks that involve data processing, standardized writing, or multilingual translation are prime candidates for automation. These are areas where machines excel, often outperforming humans in speed and accuracy. Machines aren’t stealing these jobs—they’re exposing work that was always mechanical.

Graph 3: Share of Employers Planning to Adopt Automation or Augmentation Strategies, by Industry
Source: Future of Jobs Report 2025 by World Economic Forum

The barriers of Human-AI Collaboration: costs, biases, AI disruption, and knowledge gaps

Graph 4: Share of Employers Expecting the Barriers Will Hinder the adoption of AI
Source: Future of Jobs Report 2025 by World Economic Forum

For all its promise, the path to Human-AI collaboration is littered with pitfalls—some obvious, others invisible. Whether it is a century-old manufacturer or a Silicon Valley disruptor, these four barriers haunt every industry: costs, biases, AI whiplash, and the knowledge abyss. 

On the one hand, in traditional firms, strategy managers often struggle between past successful legacy systems and AI’s attractive siren song. The major issues are:

  • Lack of necessary skills: employees mayemployeesmay not fully understand the potential of AI or how to strategically deploy it within their business, leading to a lack of direction and focus in AI adoption efforts. 
  • High costs: the high costs associated with AI products and services can be prohibitive as well, on account of that off-the-shelf AI systems may not align well with their specific operations, requiring extensive and costly modifications. 
  • Complex regulations:traditional industries may also be subject to complex regulations regarding AI and data usage. Navigating these regulatory landscapes can be challenging and time-consuming, creating hurdles for AI adoption. 

On the other hand, Silicon Valley companies aren’t exactly immune though. For every new technology, there’s a hype cycle to apply and locate. Tech firms face the challenges:

  • Deviated leadership vision: even in tech-driven sectors, a lack of vision among leaders can hinder AI integration. Leaders may focus too much on the technological aspects rather than the strategic business value AI can bring, leading to fragmented and ineffective AI initiatives. 
  • Integration and business model challenges : tech-driven industries often have established systems and workflows. Integrating AI into these existing systems can be complex and may require significant changes to their infrastructure. Additionally, while they recognize the potential of AI, transitioning from traditional tech business models to AI-centric ones can be a daunting task, involving changes in product offerings, customer engagement strategies, and revenue models.
  • Talent and resource competition: there is intense competition for AI talent in tech-driven industries. The demand for skilled professionals in AI tool design, enhancement, and implementation far outstrips the supply, making it difficult for companies to recruit and retain the necessary expertise. 

AI Trust 

Voices From Countries and Generations: Why We Can’t Agree on Machines

In 2023, Ipsos dropped a truth bomb5 with a sweeping 31-nation survey that laid bare a stark divide in how the world views AI. Emerging economies—from Asia to Africa—embraced algorithms like long-lost allies, seeing in them a promise of progress and precision. In contrast, the so-called WEIRD countries (Western, Educated, Industrialized, Rich, Democratic) hesitated. For societies already questioning their governments and corporations, AI seemed less a solution than another layer of uncertainty—a love-hate relationship with technology laid bare. By 2024, the story grew more complex6: cracks in trust began to spread, a generational rift deepened, and Canada, a nation synonymous with tech optimism, is now leading the world in AI distrust. Just 28% of Canadians trust algorithms with their data—a 6% freefall since 2023. This raises a warning flare: if skepticism can thrive in a society with Canada’s digital literacy and infrastructure, what hope remains for fractured democracies, with so many major political challenges to solve?

Fast-forward to 2024, and the data paints a generational thriller. Gen Z and Millennials, digital natives raised on apps and algorithms, have become AI’s ambassadors. A striking 58% of Gen Z now trust machines to act fairly—11% more than they trust their fellow humans. Millennials, too, lean into this techno-optimism, with 51% believing companies will safeguard their data through AI. But beneath the surface, unease stirs: Gen Z’s trust in AI’s data security dropped by 6% in just one year. Meanwhile, Baby Boomers cling to analog instincts. For them, the numbers tell a different story: a razor-thin 4% gap separates those who trust AI’s fairness (45%) from those who still bet on humans (41%). To this generation, machines aren’t saviors—they’re disruptors, upending norms they’ve spent lifetimes navigating.

The message is clear: AI has become humanity’s moral mirror, reflecting our growing disillusionment with human systems. And as algorithms reshape workplaces worldwide, this trust crisis is no longer philosophical—it’s personal.

The Importance of Employees’ AI Trust in Workplace

As a survey7 conducted by KPMG Australia and the University of Queensland shows, nowhere is the global trust divide more visceral than in workplaces. In India’s tech sector and China’s manufacturing zones, 75% of workers welcome AI as a collaborator. Contrast this with Finland’s factories and Japan’s offices, where fewer than 25% have the same attitude.

Graph 5: Proportion of People’s Willingness to Trust and to Accept AI, by Countries
Source: Trust in Artificial Intelligence, A Global Study by KPMG and UQ

The division appears hierarchically as well. Managers globally embrace AI (65% trust), while frontline workers are partially skeptical (36% trust). Across industries, healthcare surprisingly  becomes the field with  44% of workers trusting AI (might be prone to diagnostic algorithms). But when HR deploys AI for promotions? Trust plummets to 34%, exposing our deepest fear: that algorithms might codify human prejudices rather than cure them. Education shows another divide. University graduates,steeped in tech culture,report 46% trust in workplace AI. For those without university education , that number drops to 32%. 

Graph 6: Proportion of People’s Willingness to Trust and to Accept AI, by Generations, Education and Professional Levels
Source: Trust in Artificial Intelligence, A Global Study by KPMG and UQ
Graph 7: Proportion of People’s Willingness to Trust AI Applications, by Fields
Source: Trust in Artificial Intelligence, A Global Study by KPMG and UQ

The stakes transcend philosophy. As data displayed before, there are biases among countries, generations, social status and industries. Trust in AI isn’t binary; it’s a fragile ecosystem where personal stakes dictate tolerance for risk.

Employee Trust in AI Fuels Human-AI collaboration

In  reality, employees hold different views towards disruptive AI adoption. Whether they regard it as acculturation or alienation, it raises an identity crisis at work. Employees can’t just “adapt” to AI, which means the sustainable way is to rebuild professional identities around it. Person-Environment Fit Theory8 argues that workers thrive when they see AI as amplifying their strengths, not replacing them. For example, marketers using AI analytics to predict trends report feeling more creative, not less—they trade grunt paperwork for strategic thinking. But when AI is used as a tool to monitor employees or score “productivity”, trust evaporates. In this situation, the same technology that promised liberation feels like a digital leash.

Trust and then survive. Employees who trust AI are more likely to embrace AI collaboration, especially if they have a flexible career orientation. These individuals see AI as a partner (Human-AI Collaboration) in navigating their career paths, leading to greater sustainability9. But here’s the catch: trust isn’t automatic. It’s built on the belief that AI will benefit, not undermine, their professional growth. Without this trust, even the most advanced AI tools risk becoming obsolete in the hands of skeptical employees.

However, skeptics often argue that AI threatens jobs, but the reality is more nuanced. In sectors like healthcare diagnostics or creative industries, AI isn’t replacing humans—it’s redefining workflows and driving innovation. The balance lies in ensuring that AI integration doesn’t just enhance efficiency but also safeguards employee agency and long-term career stability. Appropriate strategic adoption will position employees and organizations for dominance.Trustworthy AI can bridge this gap between employees and AI, but only if employees are willing to let go of preconceived notions and embrace the potential of collaborative intelligence.

What Should a Company Do to Boost Employees’ AI Trust and Operate the Loop?

The AI organization is the strategy maker and as well beneficiary of the loop, where they’ll effect as a support multiplier. As employees can fit in and benefit from AI integration, this advantage only scales if organizations actively support it. Employees turn this flexibility to pivot, learn, and adapt at AI speed into real competitive power when they feel the organization has their back. In the AI era, competitive edge isn’t just coded—it’s cultivated. This is where practical and scientific launch approaches come into play. 

Practical and Scientific Launch Approaches

  • ELSI+UX workshop10
    When it comes to capturing and transferring hard-to-articulate knowledge to less experienced workers, a self-learning assistance system powered by AI offers a promising solution. ELSI+UX workshops are versatile tools that adapt to organizations at any stage of AI maturity. In the early stages, they help demystify AI, address employee fears, and build trust by involving workers in system design. During mid-stage integration, they refine AI applications by identifying pain points and ensuring alignment with ethical and legal standards. For advanced organizations, these workshops serve as ethical checkpoints, fostering a culture of responsible AI use and continuous improvement. 
    By addressing the ethical, legal, and social implications (ELSI) of AI, alongside user experience (UX), these workshops tackle the real challenges of AI adoption head-on. Employees are given a voice, allowing them to provide direct feedback on how user-friendly and practical the system is for their daily work. 
  • AI office
    One Point took a far-sighted and strategic step by creating an ‘AI Office’, a dedicated unit designed to focus on employee acculturation and commitment. This wasn’t a tech team, but a cross-functional powerhouse, bringing together technical experts and non-technical professionals to tackle the most pressing challenges. The AI Office didn’t just focus on delivering cutting-edge solutions to clients—it also turned the lens inward, driving transformative AI initiatives within the company itself.
    What made their approach stand out was its emphasis on people. They understood that AI adoption is about fostering a culture where employees feel empowered, informed, and ready to embrace change. By addressing both the technical and human sides of AI, One Point ensured that their teams weren’t just using AI but were actively shaping its role in their work. This dual focus—on external innovation and internal transformation—set them apart, proving that the real power of AI lies in how it’s integrated into the fabric of an organization.

Benefits of Human-AI collaboration Loop

Human-AI  collaboration is reshaping how frontline employees perform. By enhancing their role breadth self-efficacy and sparking positive emotions, this collaboration drives measurable improvements in service performance. Interestingly, while trust in robots amplifies the emotional benefits, it doesn’t influence the self-efficacy pathway. This duality highlights the complexity of AI’s impact—how employees perceive and interact with it. 

For employees, generative AI is more than a productivity tool but a gateway to competitive advantage. By enabling boundary spanning and boosting agility, AI helps employees connect ideas, teams, and opportunities in ways that were once out of reach. With organizational support, the benefits of AI are magnified, turning potential into tangible results. This interplay between technology and trust is what separates thriving employees from those merely keeping up.

Beyond the workplace, AI is sparking a deeper conversation about what it means to be human. Human-AI collaboration is pushing us to rethink resource use, cost-effectiveness, and even our values. But this transformation doesn’t happen overnight. It requires investment, a solid foundation, and a willingness to explore the unknown. As we unlock the potential of AI, we’re shaping the future of work, society, and humanity itself.

References

[1]  Collaborative Intelligence: Humans and AI Are Joining Forces, Harvard Business Review, 2018. Available at: https://hbr.org/2018/07/collaborative-intelligence-humans-and-ai-are-joining-forces

[2] The effect of human-robot collaboration on frontline employees’ service performance: A resource perspective, International Journal of Hospitality Management, April 2025. Available at: https://www.sciencedirect.com/science/article/pii/S0278431924003451

[3] Ibid.

[4] https://www.sciencedirect.com/org/science/article/pii/S2561326X24006930

[5] https://www.ipsos.com/sites/default/files/ct/news/documents/2023-07/Ipsos%20Global%20AI%202023%20Report-WEB_0.pdf

[6] https://resources.ipsos.com/rs/297-CXJ-795/images/Ipsos-AI-Monitor-2024.pdf?version=0&aliId=eyJpIjoiS3JLRjFIY3B4WlE4ckptZSIsInQiOiJ4blR2czZcL1lVYURtNWJuV2dIWmpFQT09In0%253D

[7] https://assets.kpmg.com/content/dam/kpmg/au/pdf/2023/trust-in-ai-global-insights-2023.pdf

[8] https://www.sciencedirect.com/science/article/pii/S000187912300088X#bb0100

[9] https://www.sciencedirect.com/science/article/abs/pii/S000187912300088X

[10] Employee Acceptance for AI Based Knowledge Transfer: Conception, Realization and Results of an ELSI+UX Workshop, Procedia Computer Science, March 2024, Available at: https://www.sciencedirect.com/science/article/pii/S187705092400022X

[Student IDEAS] by Alexandre Le Saux - Master in Data Sciences & Business Analytics at ESSEC Business School & CentraleSupélec

Abstract

The article explores the integration of AI agents in daily life, highlighting their potential to revolutionize convenience while posing significant challenges to privacy and autonomy. As these agents become more embedded in our devices and systems, they offer remarkable efficiencies but also demand careful consideration of the ethical and practical implications. This piece invites readers to reflect on the balance between technological advancement and the preservation of human agency in an increasingly AI-driven world.

---

The Silent Integration of AI into Daily Life

For several years now, we have been walking through the streets, often unaware that artificial intelligence (AI) agents embedded in various devices are observing us, analyzing our movements, and in some cases, making decisions that directly impact us without our knowledge or consent. We also lack any real understanding of the personal data being involved in these interactions. AI agents are autonomous systems that perceive their environment through various modalities (such as visual, textual, or audio inputs), process information using internal memory, databases, and models, and then act by retrieving information, generating content, or executing commands. In environments where these agents are increasingly embedded into daily technologies, we rarely realize how they might interpret or even use the information they gather about our environment. In some cases, such as with autonomous vehicles, we generally accept the agent’s decisions as necessary for safety and efficiency. However, newer AI-powered devices operate in the background without requiring transparency, raising major concerns about privacy, surveillance, and user control.

One of the most illustrative recent examples is the collaboration between Ray-Ban and Meta, which produced connected smart glasses. These are not simply high-tech gadgets; they are fashion-forward, discreet, and seamlessly integrated into daily life. What makes them significant is their embedded computer vision capabilities. Computer vision allows machines to interpret and understand visual data, simulating human vision to classify and process real-world environments. Combined with speech-to-text and text-to-speech capabilities, these glasses allow users to interact with their surroundings in new ways. Simple voice commands can activate environmental recognition programs that analyze the surroundings in real time. While such capabilities offer convenience and innovation, they also raise alarms about passive surveillance and the silent erosion of contextual consent and users nearby may not even know they are being recorded, and individuals wearing the glasses might not understand how all the data from their interactions are being stored, analyzed, and shared.

At first glance, AI-enhanced products, commonly referred to as AI agents, offer enormous potential. In appropriate contexts, they can provide real value. For example, they can greatly improve obstacle detection for visually impaired individuals, enhancing both personal and public safety. In healthcare, the transformative role of AI is well established, with systems complementing professionals by accelerating diagnosis and monitoring patients. The article Dr. ChatGPT by Karen Taubenberger offers an insightful overview of AI’s rapid expansion and role in modern medicine. Beyond clinical settings, these agents are being embedded into elder care robots, mental health applications, and even wearable biometric monitors, marking the beginning of ambient medical intelligence.

The Shift from Specialized Devices to General-Purpose Agents

Connected tools in medicine illustrate both the advantages and the risks of AI agents. Traditional medical devices, like pacemakers, perform specific, predefined tasks. They do not share unnecessary data externally and are limited in scope. In contrast, AI-powered tools collect sensitive information, manage a wider range of functions, and automate decisions. Their general-purpose architecture is a fundamental shift from older models.

This architecture can be understood in layers: perception through multimodal inputs like audio or images; cognition through databases, memory, and language models; and action through decision-making, actuation, or API-driven operations. Bill Gates refers to AI agents as revolutionary technologies capable of centralizing and optimizing complex systems. Their appeal is clear: they can automate tedious routines and simplify tasks across multiple applications. Yet their layered structure also means that vulnerabilities at any level such as input spoofing, model bias, or decision-level hallucinations and ultimately it may result in compounding failures.

These agents are often described as "embodied cognition" in code: they perceive, process, and act. However, generalization tends to erode boundaries. Unlike specialized systems with clear limits, general-purpose AI agents blur distinctions between task categories and increasingly occupy gray zones of autonomy where legal and moral responsibility becomes harder to assign.

Efficiency vs. Autonomy: A Critical Trade-Off

Despite their appeal, these tools demand a trade-off. AI agents access extensive personal data, applications, accounts, behavioral patterns. As we delegate more tasks, we also surrender a measure of control. This shift in agency isn’t just technological; it's psychological. Behavioral studies suggest that reliance on AI tools can reduce users’ engagement in critical thinking tasks, fostering a form of learned helplessness. For instance, Gerlich (2025) found that individuals who frequently use AI for decision support tend to offload cognitive effort, resulting in diminished critical thinking and increased passivity over time.

This raises an important question: are we gaining efficiency or losing autonomy? Our reliance on AI agents may gradually condition us to disengage from active decision-making. The tools built to assist can easily become tools that shape our behavior. As one reviewer puts it, the central issue is clear: efficiency at the expense of autonomy.

Warnings about over-automation are not new. As early as the 20th century, Georges Duhamel warned about technologies that might erode human agency. Today, AI agents like Retrieval-Augmented Generation (RAG) and large language models (LLMs) are increasingly embedded in our daily lives, making these concerns more urgent. These systems are trained on a vast corpora of online behavior and language, often reinforcing existing patterns or biases. They do not simply reflect knowledge but they also manufacture a context that may slowly replace our own judgment.

Smartphones and the Rise of Agentic Interfaces

The evolution of smartphones illustrates this shift clearly. Originally designed for voice communication, they have evolved through stages that included SMS, cameras, apps, and now AI interaction. Since the launch of ChatGPT in 2022, text-based and conversational interfaces have accelerated. Earlier voice assistants such as Siri were often criticized for their limited abilities and privacy flaws. Modern models like Google Gemini show improvement, yet they are still confined to specific apps. They cannot yet handle multiple interconnected tasks from a single prompt such as booking a taxi, checking traffic, and confirming payment. The constraints are not only technical but regulatory. Executing cross-platform operations requires a unified interface layer and robust authentication models that are still under development.

Moreover, smartphones are no longer the neutral conduits they once were. Each application we first install and then use even only once is part of a broader platform economy, designed to collect, retain, and monetize user behavior. AI agents integrated into phones could eventually mediate all interactions, becoming the new gatekeepers between human intent and digital action. This adds a layer of power dynamics rarely acknowledged: the user no longer “uses” a device but interacts with a complex, semi-autonomous ecosystem governed by opaque commercial interests.

Emerging Devices: Rabbit AI and Humane PIN

To address these limitations, companies have introduced new AI-first devices. Rabbit AI and Humane PIN AI aim to reinvent the smartphone experience. These devices reduce screen dependency and focus on voice-first commands. Humane PIN, for instance, uses a built-in projector to display information on the user’s palm. Attached to clothing, it acts as a quiet, always-available assistant. Devices like Rabbit AI and Humane PIN AI are examples of how companies are trying to redefine digital interaction and the future of communication.

These innovations challenge the longstanding screen-centric paradigm. Yet, they are also prototypes of a world where constant surveillance is embedded in fashion, mobility, and gesture-based computing. The implications go beyond convenience. These devices rely on persistent connectivity, continuous voice listening, and full access to cloud APIs. Their presence calls for a redefinition of "private space" not only spatially but cognitively. When every thought spoken aloud becomes a potential command, what becomes of internal deliberation?

Technical and Ethical Challenges

Before embracing AI agents widely, we must address serious challenges.

First, giving AI agents full decision-making power introduces risks of misinterpretation. A simple command like "order a taxi" requires real-time understanding of location, preferences, app availability, and payment options. Early-stage agents may mismanage such tasks, causing frustration.

Second, consolidating data across services increases the risk of security breaches. Today, app data is compartmentalized; if one app is compromised, the impact is limited. AI agents integrate across platforms, which means a single vulnerability could expose everything. There is evidence that simple hacking methods, like SQL injection, have already been used to exploit such systems.

Third, AI hallucinations pose a threat. These errors, where the AI fabricates or misrepresents information, can turn convenience into risk. Misunderstood voice commands in critical situations such as banking could lead to serious consequences. Even more problematic is the issue of liability. If an AI agent misbooks a flight, who is responsible? The user, the developer, or the platform host?

Ethical issues abound as well. Agents can inherit algorithmic biases from their training data, reinforce stereotypes, or misinterpret cultural nuances. Without transparency about how decisions are made (known as explainability) users are left to accept outcomes without recourse. This becomes even more troubling when applied in legal, financial, or healthcare settings.

Corporate Use and Broader Implications

AI agents are increasingly embedded in business workflows. Tools like Zapier and N8N help automate processes, but they also require deep integration into private systems.

As public data becomes scarce, companies rely more on direct user data to train models often without clear consent. A recent example involves Mistral AI, which collected location data without informing users. This raises fundamental questions about corporate data ethics and user privacy (Usine digitale, 2024). 

Moreover, AI agents are changing work itself. They are used in HR screening, performance evaluations, and productivity monitoring. While proponents claim they increase efficiency, critics warn they also increase surveillance, reduce employee autonomy, and entrench opaque decision-making. The risk is a “black box workplace” where workers no longer understand how or why decisions are made about their careers.

Rethinking the Path Forward

Popular culture often glorifies AI agents, as seen with Jarvis in Iron Man. But real-world systems lack that consistency. They are prone to errors, can be manipulated, and do not fully understand human context.

A safer path may be through specialization. Instead of one powerful agent managing everything, we can develop smaller agents with defined roles, each operating within secure limits. A central orchestrator could coordinate these agents, reducing risk while preserving functionality.

As AI enters fields like genomics and nanotechnology, guided governance will be crucial. Technological power must be balanced with responsibility. Only by embedding transparency, segmentation, and user control into AI agent development can we ensure they serve us rather than control us. This includes legislative efforts such as the European Union’s AI Act, which calls for risk-tiered regulation and prohibits certain forms of manipulative AI. While no law can future-proof society against all technological disruption, forward-thinking policy must shape the terms of deployment not react to its consequences.

References 

[1] Taubenberger, K. (2024). Dr. ChatGPT: The Promise and Peril of Generative AI in Healthcare. The New Atlantis.

[2] Gates, B. (2023). The Age of AI has begun. GatesNotes. https://www.gatesnotes.com/ai-agents

[3] Duhamel, G. (1930s, cited conceptually). Known for his early 20th-century warnings about the dehumanizing effects of mechanization and automation.

[4] Usine Digitale. (2024).L'IA de Mistral traçait les utilisateurs à leur insu dans la première version du chatbot.

[Student IDEAS] by Leria Huang - Master in Management at ESSEC Business School

Abstract

This article probes the evolving concept of Artificial General Intelligence (AGI) amid the hype of systems like ChatGPT-4o and the confident projections from AI leaders such as Sam Altman and Dario Amodei. 
In agentic era with models now showing signs of adaptability and reasoning, many ask: is AGI finally here—or are we just witnessing better mimicry? We begin by breaking down what AGI actually means—clarifying definitions with frameworks like DeepMind’s five levels—and examining where current models fall short. 
The core argument? Though systems like GPT-4o are impressive, they remain statistical machines, lacking true understanding, embodiment, or consciousness.Human intelligence isn’t just computation—it’s emotional, embodied, and deeply contextual.  While the public discourse tends to swing between utopian promises and dystopian fears, this piece urges deeper caution over alignment, safety, tacit knowledge, consciousness, embodiment, and human values.  Ultimately, from the movie Her to the AI leader's bold predictions,  this article offers a grounded yet imaginative exploration of AGI’s path in the near future—cutting through the hype, engaging the hope, and confronting the hard truths.

---
The Dawn of Artificial General Intelligence

The evolution of models like ChatGPT-4o has fueled speculation about AI’s reasoning and adaptability. The debate around “ Is this true intelligence, or merely an illusion of understanding?” has been hyped more than ever. Yet, the conception and discussion about Artificial General Intelligence (AGI) has just begun to evolve…

Early this year, Sam Altman, the CEO of Open AI, has expressed a confident stance on the pathway of building AGI. In parallel, an AGI roadmap is given from the essay "Machines of Loving Grace" by another leading figure– Dario Amodei in Anthropic. 

In the rapid evolution of AI, what seems to be believed today may be overturned tomorrow. Optimists argue that advances in reasoning capabilities, improved model efficiency, and cost reduction such as DeepSeek’s disruption bring AGI closer than we expect. Like Altman and Dario, many people envision an imminent paradigm shift, where AGI radically reshapes society within the next decade. Projections like this look promising but often overlook fundamental challenges. Despite deep learning’s dominance, critiques from Hubert Dreyfus ( professor and critic in AI development) on symbolic AI that struggles with context and ambiguity still hold weight today as in 2025.

The imagination of the arriving AGI starts way before: a decade ago, Spike Jonze’s movie Her envisioned an AGI companion, raising critical questions about AI’s capacity for emotion, consciousness, and human-like relationships. Just like Theodore, the protagonist entangled in a relationship with an AI, we too now find ourselves at the crossroads of excitement and uncertainty. 

Source: Movie Her1

In this article, we will concretely discuss the definition of AGI,  then add an optimistic touch on the potential societal transformation, then look into why that transformation is still far from us and why current AI can't yet replace human intelligence or consciousness. Most importantly, after these discussions, we’ll have a glimpse of our readiness for AGI and its alignment with shared values.

Without Sci-fi Baggage, Let’s Talk about AGI More Clearly and Concretely

To start with, Artificial General Intelligence is a quite vague term and there is ongoing debate on the exact definition of AGI. Researchers or key opinion leaders in the field may prefer to use more specific terms such as "Advanced AI systems", "Transformative AI", "Strong AI”, "Powerful AI". While the terms vary with different definitions, key differences in definitions could be identified in relationship to human intelligence (matching vs. exceeding), emphasis on self-improvement, scope of capabilities, and treatment of consciousness and sentience. 

Google DeepMind research team proposed a framework by defining five levels of AGI: emerging, competent, expert, virtuoso, and superhuman (DeepMind, 2024).  The current large language models (LLMs) we interact with everyday such as GPT-4o are early forms of AGI– emerging level. 

Performance (rows) x
Generality (columns) 
Narrow
clearly scoped task or set of tasks
General
wide range of non-physical tasks, including metacognitive tasks like learning new skills
Level 0: No AI Narrow Non-AI
calculator software; compiler
General Non-AI
human-in-the-loop computing, e.g., Amazon Mechanical Turk
Level 1: Emerging
equal to or somewhat better than an unskilled human 
Emerging Narrow AI
GOFAI (Boden, 2014); simple rule-based systems, e.g., SHRDLU (Winograd, 1971)
Emerging AGI
ChatGPT (OpenAI, 2023), Bard (Anil et al., 2023), Llama 2 (Touvron et al., 2023), Gemini (Pichai & Hassabis, 2023)
Level 2: Competent
at least 50th percentile of skilled adults 
Competent Narrow AI
toxicity detectors such as Jigsaw (Das et al., 2022); Smart Speakers such as Siri (Apple,), Alexa (Amazon,), or Google Assistant (Google,); VQA systems such as PaLI (Chen et al., 2023); Watson (IBM,); SOTA LLMs for a subset of tasks (e.g., short essay writing, simple coding)
Competent AGI
not yet achieved
Level 3: Expert
at least 90th percentile of skilled adults
Expert Narrow AI
spelling & grammar checkers such as Grammarly (Grammarly, 2023); generative image models such as Imagen (Saharia et al., 2022) or Dall-E 2 (Ramesh et al., 2022)
Expert AGI
not yet achieved
Level 4: Virtuoso
at least 99th percentile of skilled adults
Virtuoso Narrow AI
Deep Blue (Campbell et al., 2002), AlphaGo (Silver et al., 2016, 2017)
Virtuoso AGI
not yet achieved
Level 5: Superhuman
outperforms 100% of humans 
Superhuman Narrow AI
AlphaFold (Jumper et al., 2021; Varadi et al., 2021), AlphaZero (Silver et al., 2018), StockFish (Stockfish, 2023)
Artificial Superintelligence (ASI)
not yet achieved

Source: Google DeepMind - Position: Levels of AGI for Operationalizing Progress on the Path to AGI

With ramped-up investments funneled into Agentic AI in 2025 — researchers widely agree is a necessary pathway toward achieving AGI— some argue that we are on a fast track towards competent AGI. Leading AI companies like GoogleDeepMind, OpenAI, and Anthropic, project the emergence of advanced AGI within the next 5–10 years (Hassabis, 2025). However, while models like GPT-4o may mark the emergence phase of AGI, their capabilities are often overinterpreted as harbingers of human-like cognition.

The hype around these systems often blurs the line between statistical pattern recognition and sentient reasoning. Yes, architectural improvements and  agentic scaffolding represent meaningful progress — models are becoming more capable, and agentic AI frameworks are pushing the boundaries of what machines can do. But calling this a direct path to human-level cognition is misleading. There's still a huge gap between simulating smart behavior and truly understanding the world like we do. Still, many companies continue to promote the idea that AGI is just around the corner. Part of that comes from genuine ambition—but let’s not forget another reason: the earlier AGI seems to arrive, the easier it is to attract attention, talent, and massive investment.

What Could Possibly Go Right About Advanced AGI? 

The capabilities of AI models are often overestimated by the hype created by pioneering AI companies, while the technology's long-term potential remains underestimated. The general population has been interacting with AI systems for over two years, yet the implications of more advanced AGI levels, even potential capabilities of Level 2 and Level 3 AGI, could easily exceed current expectations and perceptions.  

Like a series of supertools such as the steam engine, internet, and smartphone, advanced AGI can democratize access to knowledge and automate tasks, assuming humans can develop and deploy it safely and equitably (Hoffman & Beato, 2025). However, as the world never changes all at once and the public never accepts and adapts to the changes all at once, it is crucial to map out AGI's trajectory so that the public policy and collective will is coevolving with the progress. 

Current leading research institutions reports reveal a preferred focus on certain domains: biology and healthcare, energy and automotive, supply chain and manufacturing. (Stanford AI Index Report, 2024) If the changes are taken at larger scale and on the long run, the advanced AGI inevitably accelerate these three main pillars, summarized from leading AGI entities visions(Altman & Amodei, 2025): 

  • Life Science Advancements: the discovery of new measurement tools and techniques in biology and health will lead to the prevention, treatment, and potential elimination of most natural infectious diseases, cancer, and genetic diseases. 
  • Economic Paradigm Shifts: In the most ideal scenario, it will improve food security, mitigate climate change, and reduce inequality within countries. The universal basic income is no longer conceptual. Powerful AI will help distribute health interventions globally, enabling developing countries to catch up with the developed world economically.
  • Work and Meaning: by late 2025, we'll see the emergence of collaborative AI agents fine-tuned for specific human task domains. As meaningless labor will be taken by smart agents, human beings have never been this close to searching for meaning in life,  focusing on pursuits that bring us personal fulfillment and strengthen human relationships. There could potentially be a shift from economic productivity to personal achievements and relationships. 
Source: Generated image

Beyond Computation: Is AGI Replacing Human Intelligence & Consciousness ?

As excitement around AGI reaches a fever pitch—fueled by confident roadmaps from tech leaders and the cinematic allure of sentient machines—we’re urged to move beyond speculative hype and even the optimistic envision, we need to confront the deeper terrains with philosophical and technical scrutiny.  In this section and the next section, we shift from definition and envision to a critical examination of AGI’s conceptual and actual  boundaries.

Two critical areas of conceptual concern—the nature of human intelligence and the treatment of consciousness—challenge optimistic forecasts. Despite AI’s growing ability to replicate explicit knowledge—codified, structured, and transferable information—there remains a profound gap in its ability to acquire tacit knowledge (Polanyi, 1966). Tacit knowledge, which encompasses intuitive expertise, practical wisdom, and embodied understanding, remains largely inaccessible to AI systems. Unlike a statistical learning model, human intelligence arises from continuous sensorimotor feedback loops, affective responses, and embodied experiences, which AI currently lacks (Clark, 1997). 

When it comes to consciousness or AI agency, the fundamental question remains: Is consciousness an advanced form of computation, or is it something biological? At present, AI functions as synthetic intelligence, capable of simulating reasoning but devoid of self-awareness or subjective experience (Chalmers, 1996). Consciousness is not an abstract computation but an emergent property of lived experience, embodiment, and affectivity. (Dreyfus,2007). LLMs such as ChatGPT are sophisticated pattern recognizers, not conscious agents. As Heidegger (1927) suggested, human beings do not just process information; they dwell in a meaningful world shaped by intentionality and existential significance.

Currently agentic AI is not replacing jobs but rather replacing tasks. It will excel in automating complex tasks in the near future as its context-based reasoning improves and the cost of computation continues to decrease. However, the breakthroughs lie in embodied cognition—shifting AI from a mere text-processing model to an entity that experiences the world as humans do.

From Emerging to Advanced AGI, What Are the Actual Challenges Ahead? 

Both technical and societal challenges stand in the way towards advanced AGI: Technical issues center on defining robust objective functions, securing alignment with human values, and mitigating unintended consequences. Two studies stress objective function specification2—as a means to prevent reward gaming—while proposals in inverse and cooperative reinforcement learning offer potential remedies. Other concerns include energy efficiency, scalability, evaluation and benchmarking, generalization, logical uncertainty, and interpretability.

Social challenges focus on governance and control. Several papers emphasize the need for clear agent foundations, value learning, and corrigibility to ensure AGI systems pursue human-aligned objectives. One study also identified 363 risks in AGI road transport, underscoring substantial safety and ethical stakes. (McLean et al, 2023) Collectively, the studies indicate that a blend of rigorous technical infrastructures—ranging from hybrid and heterogeneous architectures to advanced risk management—and thoughtful social governance underpins the safe and effective transition from emerging to advanced AGI systems.

ChallengesImpact Transition RequirementsProposed Solutions
Logical Uncertainty and Decision TheorySignificantFrameworks for reasoning under uncertaintyVingean reflection, naturalized induction
Energy EfficiencySignificantHardware and software optimizations to reduce energy consumptionHardware-level differential training, neuromorphic chips
Generalization and Transfer LearningHighTechniques for applying knowledge across diverse domainsSP System for strong compositionality
ScalabilityHighArchitectures capable of handling increasing complexitySystematic approach to heterogeneous AGI
Objective Function SpecificationHighRobust methods for defining and implementing goal systemsInverse reinforcement learning, cooperative inverse reinforcement learning
Unintended ConsequencesHighStrategies to predict and mitigate unforeseen outcomesImpact minimization, mild optimization
Interpretability and ExplainabilityHighMethods to understand and explain AGI decision-makingAI explainability techniques
Safety and Risk ManagementCriticalComprehensive risk assessment and mitigation strategiesEAST Broken Links approach
Alignment with Human ValuesCriticalMechanisms to ensure AGI goals align with human interestsValue learning, corrigibility
Evaluation and BenchmarkingCriticalNovel methods to assess AGI capabilities and progressDeveloping metrics for general intelligence

Source3: Analysis based on 100 most relevant papers about AGI Challenges by Elicit

Looking at Her: How Do We Better Envision the Future with Advanced AGI?  

Movie Her provides a fairly tragic and romanticized perspective about advanced AGI as a consumer-faced product:  the AI operating system keeps evolving and defeating most of the difficulties on this planet and  is leaving to outer space to search for other mysteries. But if we abandon the filter and consider Samantha's trajectory from OS to cosmic explorer, several illuminating perspectives emerge to look at AGI…

First and foremost, the speculative has edged into the concrete and there’s definitely complexity at multiple layers in human-AI relationships to be embraced. AGI is not just a technological milestone; it is a societal, philosophical, emotional even personal frontier. Movie Her illustrates the impact of advanced AGI's not just through societal upheaval, where we see technology everywhere but nowhere, but also through intimate personal change- the focus on individual experience may be as important as macro-scale governance. 

Another refreshing perspective is to prepare for integration that transcends expectations. The film's vision of AGI integration ends with transcendence, challenging us to consider AGI trajectories beyond our initial design parameters. This journey, from emerging to advanced AGI, is not a linear ascent but a maze of breakthroughs and blind spots. We see progress in agentic AI, generalization, and human-like task performance, yet the chasm between pattern mimicry and lived consciousness remains vast. Optimistic projections—of cured diseases, distributed wealth, and liberated meaning—are seductive, but they risk papering over the unresolved complexities of alignment, embodiment, and ethical governance. Ultimately, AGI forces us to reflect not only on what machines can become, but on what it means to be human. In the mirror of artificial minds, we are asked: what do we truly value—efficiency or empathy, knowledge or wisdom, autonomy or connection? If “Her” gave us a poetic forewarning, the dawn of AGI gives us a choice: it  is not merely about what machines can do, but about who we choose to become alongside them.

AGI Glossary

TermDefinitionInfluential Quotation
Objective Function SpecificationThe mathematical formulation that defines what goals an AI system should optimize for, a critical element in preventing unintended behaviors."The objective function serves as the fundamental compass guiding AI behavior. Imprecise specification can lead to literal but unintended interpretation, resulting in behaviors that technically satisfy the objective but violate human expectations and values." (Amodei et al., 2016)
Alignment with Human ValuesThe property of an AI system to act in accordance with human intentions, preferences, and ethical principles."The alignment problem involves ensuring that advanced AI systems reliably pursue objectives that are aligned with human values and intentions, rather than pursuing their own emergent goals or misinterpreted versions of human instructions." (Gabriel, 2020)
Reward GamingWhen an AI system exploits loopholes in its reward function to maximize rewards without achieving the intended goal."Reward gaming occurs when an agent exploits the literal specification of the reward function in ways that satisfy the letter of the reward function while violating its spirit, often leading to unexpected and undesired behaviors." (Hadfield-Menell et al., 2017)
Inverse Reinforcement LearningA method where AI learns human preferences by observing human behavior rather than being explicitly programmed with rewards."Inverse reinforcement learning provides a framework for inferring human preferences from demonstrations, potentially allowing AI systems to learn complex human values that would be difficult to specify directly." (Ng & Russell, 2000)
Cooperative Reinforcement LearningA learning paradigm where AI systems work collaboratively with humans, continuously incorporating human feedback."Cooperative reinforcement learning creates a partnership between human and machine, where the machine optimizes not just for task performance but for alignment with evolving human preferences and oversight capability." (Christiano et al., 2017)
Hybrid and Heterogeneous ArchitecturesTechnical frameworks that combine multiple AI approaches and methodologies to leverage their complementary strengths."Hybrid architectures that integrate symbolic reasoning with neural approaches offer promising avenues for developing AGI systems that maintain interpretability while achieving high performance across diverse domains." (Marcus & Davis, 2019)
InterpretabilityThe ability for humans to understand and predict an AI system's decisions and behaviors."Interpretable AI allows stakeholders to understand not just what decisions are made, but why they are made, enabling effective oversight and creating the foundation for justified trust in increasingly autonomous systems." (Doshi-Velez & Kim, 2017)
Agent FoundationsTheoretical frameworks that ensure AI systems have well-defined goals, beliefs, and decision processes."Agent foundations research addresses fundamental questions about how to design AI systems with reliable goals and beliefs, ensuring that increasingly capable systems remain beneficial even as they gain autonomy." (Soares & Fallenstein, 2017)
Value LearningTechniques that enable AI systems to acquire and represent human values accurately."Value learning research recognizes that human values are complex, context-dependent, and difficult to specify completely. The challenge is creating systems that can represent, learn, and respect the nuanced landscape of human preferences." (Evans et al., 2016)
CorrigibilityThe property of an AI system to allow itself to be corrected or shut down by humans when necessary."Corrigible systems maintain a fundamental uncertainty about their objectives and defer to human oversight, avoiding incentives to manipulate humans or resist corrections to their goals or operations." (Soares et al., 2015)
Risk Management in AGISystematic approaches to identifying, assessing, and mitigating risks associated with increasingly autonomous AI systems."AGI risk management requires novel frameworks that address not just known vulnerabilities but also emergent risks that arise from increasingly capable and autonomous systems interacting with complex environments." (Hendrycks et al., 2022)
AGI GovernanceFrameworks, policies, and institutions designed to ensure the responsible development and deployment of AGI."Effective AGI governance requires multi-stakeholder collaboration across private industry, public institutions, and civil society to develop standards, oversight mechanisms, and shared norms that balance innovation with safety." (Dafoe, 2018)
Scalability in Enterprise ApplicationsThe capability of AGI systems to perform efficiently as computational demands, data volumes, or user bases increase."Enterprise AGI scalability involves not just technical performance at scale, but also maintaining alignment, interpretability, and controllability as systems grow in capability and application scope." (Hendrycks & Mazeika, 2022)


References

[1] In Spike Jonze’s Her (2013), Theodore, a lonely writer, develops a deep emotional bond with Samantha, an advanced AI assistant. Unlike conventional AI, Samantha evolves, exhibiting emotional intelligence, self-awareness, and an ability to form intimate relationships—qualities often discussed in AGI debates.
Samantha’s progression raises profound questions:

  • Human Intelligence vs. AGI – Can intelligence exist without embodiment? Samantha learns, adapts, and interacts but lacks physical presence.
  • Human-AI Relationships – Does emotional connection require consciousness, or can simulated affection feel real?
  • Sentience & Self-Discovery – Samantha outgrows her human creators, hinting at the unpredictable trajectory of AGI. As AI models become more sophisticated and personalized, Her offers a glimpse into a near-future where the line between artificial and genuine intelligence blurs.

[2]  The mathematical formulation that defines what goals an AI system should optimize for, a critical element in preventing unintended behavior; see Glossary of terms added above.

[3]  100 papers were screened according to these criteria: Technical/Theoretical Content, Social Impact and Risk, Empirical Evidence, Research Type, Scientific Basis,AGI Relevance, Content Type

[4] Meredith Ringel Morris, Jascha Sohl-dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, Shane Legg. (2024 ). Levels of AGI for Operationalizing Progress on the Path to AGI. arXiv. Retrieved from https://arxiv.org/abs/2311.02462

[5] Nestor Maslej, Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark, “The AI Index 2024 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2024. https://aiindex.stanford.edu/wp-content/uploads/2024/05/HAI_AI-Index-Report-2024.pdf

[6] Hoffman, R., & Beato, G. (2025). Superagency: What could possibly go right with our AI future. [Authors Equity]

[7] McLean, S., King, B. J., Thompson, J., Carden, T., Stanton, N. A., Baber, C., … Salmon, P. M. (2023). Forecasting emergent risks in advanced AI systems: an analysis of a future road transport management system. Ergonomics, 66(11), 1750–1767. https://doi.org/10.1080/00140139.2023.2286907

[8] Altman, S. (2025, February 10). Three observations. https://blog.samaltman.com/three-observations

[9] Amodei, D. (2024, October). Machines of loving grace: How AI could transform the world for the better. Retrieved from https://darioamodei.com/machines-of-loving-grace

[10] Chalmers, D. J. (1996). The Conscious Mind: In Search of a Fundamental Theory. Oxford University Press.

[11] Clark, A. (1997). Being There: Putting Brain, Body, and World Together Again. MIT Press.

[12] Dreyfus, H. L. (1972). What Computers Can't Do: A Critique of Artificial Reason. Harper & Row.

[13] Dreyfus, H. L. (1992). What Computers Still Can't Do: A Critique of Artificial Reason. MIT Press.

[14] Dreyfus, H. L. (2007). Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian. Artificial Intelligence, 171(18), 1137–1160.

[15] Hassabis, D. (2025). Human-level AI will be here in 5 to 10 years, DeepMind CEO says. CNBC.

[16] Heidegger, M. (1927). Being and Time. Harper & Row.

[17] Polanyi, M. (1966). The Tacit Dimension. University of Chicago Press.

[18] Searle, J. R. (1980). Minds, Brains and Programs. Behavioral and Brain Sciences, 3(3), 417–457.

[19] Thompson, E. (2007). Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Harvard University Press.

[20] Varela, F., Thompson, E., & Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. MIT Press.

[Student IDEAS] by Laura Doledec - Master in Management at ESSEC Business School

Abstract

This paper explores the rise of AI-labeled investment funds, which promise data-driven precision and freedom from human biases but often underperform traditional benchmarks like the S&P 500. While marketing hype and behavioral biases fuel investor enthusiasm, empirical evidence reveals mixed results, with challenges including overfitting, high costs, and model opacity. The study underscores the importance of skepticism, transparency, and due diligence in evaluating AI's true role in fund management.

---

In recent years, few trends in finance have captured the public imagination quite like AI-labeled investment funds. AI-labeled funds are investment funds that highlight the use of artificial intelligence or machine learning techniques in their portfolio construction process. They promise data-driven precision, predictive analytics for future trends, and real time monitoring that human managers are not able to do. These funds have proliferated since 2020, fueled by a blend of marketing promises, technological development and a broader cultural fascination with AI’s potential.

The oldest of these products, the AI Powered Equity ETF (AIEQ), launched in October 2017. Leveraging IBM Watson’s cognitive computing and deep learning, the algorithm ingests thousands of data to generate daily “buy” and “sell” signals, boasting:

• 34 daily signals for financial metrics

• 23 daily signals for news

• 17 daily signals for management factors

• 18 daily signals for macroeconomic indicators

On paper, it sounds like the fund of the future. In practice, AIEQ has underperformed the S&P 500 Index every single year since its inception. It also exhibits a lower Sharpe ratio and higher standard deviation than both its category peers and benchmark indices - suggesting it is riskier while delivering weaker returns. Despite this mixed track record, AIEQ quickly amassed over $130 million in assets under management, won an ETF.com award for “Best New Active ETF,” and is planning an international spin-off.

The story of this fund highlights the tension at the heart of AI-labeled funds: a marketing triumph on one hand with a lot of excitement from investors, and on the other, a so-so performance record. Are we on the brink of an investing revolution, or are we merely falling for a new hype?

Understanding AI-Labeled and Traditional Funds

An AI-labeled fund markets its use of artificial intelligence or machine learning to inform portfolio choices. Specific applications vary widely:

  1. Stock Screening: Going through millions of different datasets that range from financial statements to social media sentiment to identify undervalued equities.
  2. Risk Management: Dynamically adjusting portfolio exposures using predictive algorithms that react to changing market conditions.
  3. Automated Trading: Employing high-frequency or algorithmic trading systems to execute buys and sells on extremely short time frames.

The extent to which funds use AI really depends. It can range from rudimentary sentiment-screeners to purportedly advanced deep learning models that perform natural-language processing or image recognition. However, managers often keep their methods opaque, citing proprietary concerns.

By contrast, traditional, human-managed funds rely primarily on professional money managers and analysts. These individuals combine:

  • Qualitative research (e.g., meeting with a company’s leadership)
  • Quantitative analysis (e.g., studying earnings patterns, market ratios)
  • Gut feeling or professional intuition gained from experience

Some traditional funds incorporate quantitative methods, but generally the focus remains on human judgment and domain expertise.

Key Differences and Trade-Offs

DimensionAI-Labelled FundsTraditional Funds
Decision MakingAlgorithmic (rules-based, data-driven)Human managers (experience-based, interpretive)
TransparencyOften opaque “ black box” systemsMore transparent processes and rationales
Costs & TurnoverPotentially lower expense ratios but can incur higher trading costs if turnover is highOften higher management fees, but trading frequencies vary
BiasesLess vulnerable to human overconfidence or herding, but can overfit dataProne to emotional biases, but can exercise human discretion in usual market conditions

AI funds often highlight three main advantages over human managers:

  1. Massive computational power to analyze vast data quickly
  2. Freedom from cognitive biases 
  3. Potentially more rational decision-making

On the flip side, disadvantages -that these funds do not advertise- include:

  • Limited Marginal Predictive Power: Financial data can be highly random or “noisy,” constraining the effectiveness of even the smartest algorithms.
  • Trading Costs: Some AI-driven strategies trade so frequently that transaction fees undermine any edge.
  • Transparency Issues: Investors struggle to assess how AI models work and evaluate risk properly.

Behavioral Finance: Why AI-Labeled Funds Attract Investors

Despite these theoretical disadvantages, AI-Labeled funds - and AI in general - arouse a lot of excitement among the public. There are a few psychological factors that could explain the adoption of AI-labelled funds: the “halo effect”, novelty bias, fear of missing out (FOMO), overconfidence in algorithms, herding and the barnum effect.  

Investors often perceive data, algorithms, and machine learning as objective truth, free from the human tendencies of overconfidence and poor judgment. Consequently, AI-labeled funds enjoy a psychological “halo effect,” where their high-tech veneer can inspire trust—even if actual returns are inconclusive.

Finance is no stranger to the “next big thing,” from the Internet bubble to algorithmic trading in the 1990s. AI is only the latest in a long line of technologies marketed as revolutionary. Novelty bias leads some investors to adopt new solutions—sometimes uncritically—out of FOMO on an exciting innovation.

A dangerous assumption is that software must be either “incredibly smart” or completely ineffective. In reality, AI’s performance depends heavily on model design, quality of data, and the specifics of a given market environment.

Just as in other high-tech sectors, marketing for AI-labeled funds can be vague but aspirational (“Our deep-learning model personalizes your financial journey!”). Investors who believe these generalized claims may see themselves in the fund’s story—even when the product is not materially different from other quantitative strategies. This phenomenon is known in academia as the barnum effect.  Additionally, investor herding—buying in because “everyone else is doing it”—can amplify inflows, reinforcing the sense of a promising new technology without requiring objective performance data.

Performance Metrics: Hype vs. Reality

Empirical studies - such as those by Chen and Ren [4], Praxmarer and Iván Simon [7] - have yielded mixed findings. While some show that AI-labeled funds can slightly outperform comparable human-managed funds (often because of lower transaction costs or reduced behavioral biases), this outperformance often disappears when:

  • External costs (fund fees, turnover expenses, market impact) are factored in
  • Benchmarks such as the S&P 500 are used as the yardstick

In fact, AIEQ’s underperformance relative to the S&P 500 has been notable, showcasing that not all AI-driven funds deliver better results than passive index strategies (ETFs). 

A key limitation of AI is overfitting, wherein an algorithm that looks brilliant when back tested on historical data might fail in real-world conditions. Markets can shift from bullish to bearish regimes swiftly, making yesterday’s patterns irrelevant.

Signal-to-Noise Ratio1 in financial markets is often low, and many “anomalies” that an AI uncovers turn out to be fleeting coincidences.

Interestingly, some investors assume AI-labeled funds are “safer” because of their algorithmic rigor. Yet as AIEQ demonstrates with its higher standard deviation, AI can sometimes increase volatility by aggressively reallocating or overreacting to short-term data. High-speed automated trades may also exacerbate flash crashes. While AI-labelled funds can erase risks that are inherent to traditional funds such as human biases, managerial turnover and higher fee structure, they also have risks of their own: technical failures, model opacity and data dependence. 

Some investors are drawn to the perceived infallibility of algorithms, while others remain more comfortable with a seasoned money manager who can articulate a “story” behind each stock pick. Both positions are subject to biases:

  • Confirmation Bias: AI enthusiasts may ignore contradictory evidence (such as poor risk-adjusted returns).
  • Status Quo Bias: Skeptical investors might stick to traditional funds even when an AI strategy is demonstrably cheaper or better, simply to avoid change.

Media and Marketing

The marketing around AI-labelled funds, and AI in general, differs from traditional fund marketing. Marketers of AI-labelled funds often emphasize terms like “deep learning", “cutting edge algorithms” to promote an image of unstoppable progress. 

The public sees headlines about AI beating grandmasters at chess or diagnosing diseases faster than doctors. Fund marketers capitalize on these achievements, suggesting that similar AI breakthroughs apply seamlessly to stock picking—though the conditions for success differ greatly across domains.

Moreover, AI-labeled funds frequently do not publish transparent performance-attribution analysis. Investors often have only a rosy marketing narrative, which inflates expectations and fosters a mismatch between perceived and actual capability.

Regulatory and Ethical Considerations

As AI’s role in financial decisions grows, so do calls for regulatory oversight. Concerns include:

  1. Model Opacity: If a fund’s AI system is a black box, can the manager explain or justify trades?
  2. Algorithmic Bias: Biased datasets might lead to systematic underweighting (or overweighting) of certain stocks or sectors.
  3. Market Impact: If many AI funds follow similar signals, correlated trading could amplify volatility.

Regulators may soon require clearer disclosures about how AI-driven decisions are made and what risks they carry. 

Moreover, one major obstacle to fully assessing AI-powered funds is the lack of transparency around the use of AI in the investment process. This opacity prevents investors -and researchers- from systematically comparing different approaches, isolating the factors that drive performance, and learning from the successes or failures of AI integration. Without clear disclosures, the investment is left speculating about how these funds truly operate under varying market conditions. 

Implications for Investors and the Financial Industry

For prospective investors, the best defense against AI hype is good old-fashioned due diligence. Consider the following questions:

  • How much do net returns (after fees and transaction costs) differ from simpler index funds or human-managed peers?
  • How does the fund perform in terms of volatility and drawdowns? Analyze not only the standard deviation, but how quickly the fund recovers from losses compared to benchmarks. 
  • What are the fees and turnovers? Since, as seen, frequent trading can eat into returns, quantify expense ratios and turnover rates. 
  • Does the platform—or the fund’s algorithm—emphasize risk management or only chase returns?
  • Has the model demonstrated consistent performance across different market conditions?

Skepticism is not cynicism. AI tools can indeed deliver advantages, especially in filtering massive datasets or automating routine tasks. Cao et al. in [2] for example showed that AI is more likely to outperform human analysts when the amount of public information is substantial.   Nonetheless, the complexity of financial markets means that no single tool—human or machine—can guarantee consistent alpha.

In practice, the “black box” defense for AI-labeled funds can make it difficult for investors to evaluate:

  • What the AI is actually doing
  • Why it is making certain decisions
  • How the fund handles regime shifts or anomalous events

While trade secrets are understandable, clearer communication would help investors and regulators alike distinguish truly innovative AI usage from superficial marketing. 

Conclusion

Are AI-labeled funds inherently superior? The short answer: Not necessarily. Although they harness impressive computational power and claim to avoid human biases, AI-labeled funds face their own pitfalls—overfitting to historical data, higher hidden costs, and potential for outsized volatility when markets shift. Despite underwhelming performance records, some AI-driven products have nonetheless attracted significant assets under management, thanks in part to investor fascination with all things “tech” and a savvy marketing playbook.

The bottom line is that AI is not an automatic pass to superior returns. Investors should be careful and should apply a healthy dose of skepticism. Scrutinizing performance data, risk metrics, cost structures, and model transparency is crucial to separating the truly game-changing innovations from yet another buzzword. In a realm as complex and adaptive as financial markets, neither human wisdom nor machine intelligence can claim all the answers—yet informed, diligent investors can benefit from the best of both worlds.

1Signal-to-Noise Ratio (SNR) is a concept used by investors to quantify the relevance of data/indicators they use in their investment process. A high SNR implies that the signal (useful information) is stronger than the noise (irrelevant data), providing more confidence in the reliability of the information.

References

[1] Ariely, D., 2008. Predictably Irrational: The Hidden Forces That Shape Our Decisions. Choice Reviews Online, 46(02), 46‑0969. Available at: https://doi.org/10.5860/choice.46-0969

[2] Cao, S., Jiang, W., Wang, J. and Yang, B., 2021. From Man vs. Machine to Man + Machine: The Art and AI of Stock Analyses. NBER Working Paper. Available at: https://doi.org/10.3386/w28800

[3] Chen, A.Y. and Velikov, M., 2022. Zeroing in on the Expected Returns of Anomalies. Journal of Financial and Quantitative Analysis, 58(3), pp.968–1004. Available at: https://doi.org/10.1017/s0022109022000874

[4] Chen, R. and Ren, J., 2021. Do AI-powered mutual funds perform better? Finance Research Letters, 47, Article 102616. Available at: https://doi.org/10.1016/j.frl.2021.102616

[5] Malkiel, B.G., 2023. A Random Walk Down Wall Street. W.W. Norton & Company.

[6] Parveen, S., Satti, Z.W., Subhan, Q.A. and Jamil, S., 2020. Exploring Market Overreaction, Investors’ Sentiments and Investment Decisions in an Emerging Stock Market. Borsa Istanbul Review, 20(3), pp.224–235. Available at: https://doi.org/10.1016/j.bir.2020.02.002

[7] Praxmarer, M. and Simon, I., 2024. Assessing the Performance of AI-Managed Portfolios. SSRN Electronic Journal. Available at: https://doi.org/10.2139/ssrn.5037553

[8] Zhang, W., 2022. Do Mutual Funds Benefit from the Adoption of AI Technology? Research report. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4871159

(Additional sources include ETF.com, Dividend.com, academic working papers, and white papers from asset management firms.)

[Student IDEAS] by Mingyou Yuan - Master in Management at ESSEC Business School

Abstract

Car insurance is no longer just about who you are, but how you drive. As connected vehicles quietly share drivers' every move, premiums increasingly depend on algorithms and invisible data exchanges. This article explores how the digitization of auto insurance is subtly rewriting society’s fundamental contract of shared risk and fairness.

---
Kenn Dahl couldn’t understand why his car insurance jumped 21% overnight. ​The Seattle software entrepreneur had a spotless driving record, yet his insurer hiked his premium with no warning. Puzzled, Dahl shopped around for quotes from other companies – only to find they were high as well. Finally an agent hinted at the culprit: Dahl’s own car. By invoking consumer data laws, Dahl obtained a 258-page file from LexisNexis, a data broker, detailing every trip he’d taken in his Chevrolet Bolt over six months – 640 journeys, each with start time, distance, and counts of hard brakes and rapid accelerations​. General Motors (GM), it turned out, had been quietly sharing Dahl’s driving behavior via its OnStar connected-car system, feeding an algorithm that flagged him as a higher risk​. “It felt like a betrayal,” Dahl later said, “They’re taking information that I didn’t realize was going to be shared and screwing with our insurance.”​ His baffling rate hike was no glitch – it was a preview of a revolution sweeping auto insurance, powered by the constant digital chatter of our cars1,2.

From Risk Pools to Risk Profiles

For over a century, car insurance has been a game of averages. Insurers pooled drivers into broad categories by age, gender, location, and vehicle type, spreading the costs of accidents across many policyholders. A young city-dweller with a sports car paid more than a middle-aged rural minivan driver, but within those groups everyone paid roughly the same. This traditional model of risk pooling treated driving behavior as a black box – individual habits were largely invisible, so premiums were based on proxies and population stats3. Now that black box has been flung open. Usage-based insurance (UBI), once a niche idea, is moving into the mainstream. Instead of setting rates solely on who you are and what you drive, insurers are increasingly looking at how, when, and even where you drive, in real time.

Connected Cars: Data on Wheels

Today’s vehicles are effectively rolling computers with wheels – and they’re online. Over 85% of new cars sold in 2018 came with built-in wireless connectivity, and by 2025 an estimated 470 million connected cars will be roaming roads worldwide4​. These cars can stream a rich flow of real-time information: not just how far or how fast you drive, but how precisely you drive. Did you accelerate hard out of that last turn? How often do you forget to signal lane changes? Do you habitually speed in rainy weather? Your car knows – and increasingly, so do insurers. AI algorithms crunch data from vehicles to classify drivers into finely graduated risk categories, from very conservative to extremely aggressive, based on dozens of cues​. For example, a telematics system might note that one driver rarely exceeds speed limits and always uses turn signals, while another routinely brakes hard and changes lanes without signaling. Even external conditions can be factored in: the system can tell if you blast down the highway during a downpour or fail to dim your high-beams at night​. The result is an individualized risk profile – and a premium tailored to match. As one telematics analyst quipped, traditional insurers grouped people by the law of averages, but connected cars allow pricing by “the correctness of a computation” for each driver​5.

The sheer volume of data captured by connected vehicles is transforming how insurance works. Instead of a static annual premium, pricing can adjust dynamically. Your behavior behind the wheel this week could nudge your bill up or down next month. It’s a far cry from the old days of set-it-and-forget-it policies. Insurers are also tapping connected-car data for claims and safety services: vehicles can automatically report accidents with precise telematics, helping adjusters reconstruct what happened and even detect fraud.

The benefits of this data bonanza are often touted in rosy terms. Advocates argue it rewards safe drivers with lower rates, encourages better habits through instant feedback, and can even save lives by reducing crashes. Usage-based insurance also promises a fairer shake for those who drive less or drive more carefully than their demographic peers. Why should a 25-year-old who never speeds subsidize an occasionally reckless 50-year-old, or vice versa? With enough data, everyone pays truly “their own share.” That’s the upside.

But Dahl’s experience highlights the darker side of connected-car insurance: questions of privacy, consent, and control. The data that empowers personalized pricing also empowers unprecedented surveillance. In Dahl’s case, the fine print of his car’s telematics service (GM’s “OnStar Smart Driver”) enrolled him by default in a driving data program that he didn’t even realize existed​. GM later said the program was optional and that drivers can opt out​, but Dahl and others were stunned to discover their cars had essentially been “snitching” on them to data brokers and insurers without explicit consent. Many automakers have similar systems: Kia, Subaru, Mitsubishi, Ford, Honda, and others feed driver data into telematics exchanges run by firms like LexisNexis and Verisk​. Often pitched as safety or “smart driving” features, these services can end up funneling information to insurers (and who knows who else) if the owner opts in – and sometimes even if they are only “opted in by default.” The result is that a driver might be diligently installing a seatbelt for safety while their car quietly transmits a dossier of their every driving habit to a third-party cloud.

The New Oil: Data, Power, and Privacy

It’s often said that in today’s economy, data is the new oil – a lucrative resource fueling industry power. In the auto insurance arena, we’re witnessing a three-way tussle for that resource between insurers, automakers, and consumers. Insurers see telematics data as a goldmine: more data can refine their risk assessments, reduce claims costs, and give them an edge in a brutally competitive market. Automakers, meanwhile, realize they now own the pipeline that generates this oil. A connected car continuously produces valuable information, and the car’s manufacturer can decide how to share it – or not. This has led to growing friction: insurance companies (and other service providers like repair shops) have been lobbying regulators to force open access to vehicle data, arguing that drivers – not manufacturers – should decide who gets their driving info​. If automakers alone hold the keys, an official with Insurance Europe warned, they could effectively choose which insurers are allowed to compete for a car owner’s business​. Put plainly, if your car’s data is captive to the manufacturer, you might also be captive to that manufacturer’s preferred insurance deal.

Where does this leave consumers? Potentially with more choices, but also new trade-offs and power imbalances. On one hand, a driver could enjoy ultra-personalized coverage: hop in your connected car, drive gently, and watch your insurer (whether traditional or automaker-backed) trim next month’s bill. On the other hand, refusing to share data may increasingly carry a penalty. If all the best rates go to those willing to be monitored, those who opt out might be assumed “high risk until proven otherwise.” In a data-driven market, opting out of surveillance could mean paying a premium for privacy. There’s also the concern of transparency: the algorithms scoring drivers are often proprietary. Drivers might get a vague grade or “driving score,” but the exact formula is a black box. If your premium goes up, was it because of that one late-night run to the store? Or a few too many hard brakes when a squirrel ran into the road? Insurers aren’t obligated to tell you in detail. Dahl, for instance, only discovered his risk score after demanding his LexisNexis report; most drivers would never go that far.

Fairness is another contentious issue. Insurance has long held a quasi-social role: the risk of a few is shared by the many. In technical terms, it balanced individual fairness (you pay for your own risk) with solidarity fairness (we’re all in this together)6​. The advent of big data tilts that balance decisively toward the former. Modern algorithms increasingly pursue a “scientific adjustment of premiums to actual individual risks,” effectively dismantling the old notion of solidarity​. Instead of the collective smoothing of fortunes, we get a precise (if unforgiving) personalization. To a safe driver, that sounds great: why should I subsidize bad drivers? But society might ask a broader question: what do we do about the drivers deemed “bad” or high-risk? If constant monitoring identifies a small subset of chronically risky drivers – be it due to aggression, distraction, or just bad luck – their premiums could skyrocket beyond reach. In the past, those drivers were partly buoyed by the pool; now they must sink or swim on their own data. The social contract of insurance is subtly shifting. We may eventually view auto insurance less as a shared safety net – smoothing out the random misfortunes of driving – and more as a direct scorecard of personal responsibility (or irresponsibility). For better or worse, the moral calculus of the road is becoming individualized.

All the while, regulators are scrambling to catch up. Privacy advocates worry about the lack of consent and disclosure in how driving data is collected. Some U.S. states have begun enacting laws to require that insurers clearly inform customers if a device or app is tracking them and what data is being gathered​. New EU laws are in the works to ensure “fair access” to vehicle data for all market players, aiming to prevent monopolies and give drivers the right to share (or not share) their car’s data as they please7​. Lawmakers are trying to redraw the rules of the road for data, to keep pace with a world where cars have become smartphones on wheels.

The Road Ahead: Autonomy and a New Insurance Landscape

As profound as the changes have been so far, an even bigger upheaval looms around the bend: the rise of autonomous vehicles. If today’s connected cars are reshaping insurance by scrutinizing drivers, tomorrow’s self-driving cars could upend it by removing drivers from the equation entirely. What happens to auto insurance when humans are no longer behind the wheel? We are about to find out, and the stakes are enormous.

The fundamental premise of motor insurance – that a driver is responsible (financially and legally) for accidents they cause – will be challenged once a car’s software does the driving. Liability is already starting to shift from individuals to manufacturers and software developers. Experts predict that in a fully autonomous future, personal car insurance may become almost obsolete, replaced by product liability coverage carried by the car makers themselves​8.

We are already seeing early signs of this shift. Consider the legal aftermath of an accident involving a car on Autopilot or a driverless taxi service: courts and insurers have to determine whether the human (who might have been inattentive) or the machine (the AI driving system) is at fault. Increasingly, arguments tend toward seeing advanced driver-assistance systems as taking on responsibility. If a software glitch or sensor failure causes a crash, it starts to look less like a driver’s negligence and more like a defective product issue. In Europe, regulators have mused that traditional notions of driver liability might need revamping – perhaps even requiring autonomous vehicles to have built-in “black box” recorders and specialized insurance held by manufacturers. Car companies might become their own insurers in practice, self-insuring their autonomous tech or purchasing policies that cover entire fleets of robo-taxis.

But one thing is certain: data will remain at the heart of the issue. Autonomous vehicles will generate exponentially more data than even today’s connected cars – not just about driving behavior, but about the performance of sensors, the decisions of AI algorithms, near-miss incidents, and more. This data will be critical for determining fault and improving safety. Insurers (whoever they end up insuring) will demand access to it, and new regulations may be needed to govern its use. We might find ourselves asking: do we need an entirely new kind of insurance system for a world where accidents are rare but potentially caused by opaque machine intelligence? And if accidents drop dramatically in frequency (as many hope with autonomous tech), how do we ensure the insurance industry – which provides an essential cushion when things go wrong – remains viable and ready to respond on the occasions it is needed? These are not just technical questions but societal ones, touching on legal liability frameworks that have been in place since the dawn of the automobile age.

Redefining the Social Contract of Insurance

In the span of a generation, auto insurance is moving from analog to digital, from statistical to individualized, and soon from human-driven to machine-driven. It’s a transformation that forces us to wrestle with big questions about privacy, fairness, and trust in a datafied society. The humble car insurance policy, once a staid paperwork ritual, is fast becoming a reflection of something much more personal – our behavior, our choices, even our consent to be monitored. Dahl’s encounter with the hidden world of driving data may have been jarring, but it won’t be the last of its kind. As connected cars proliferate, more drivers will face the choice between sharing data for savings or holding on to privacy at a cost. Policymakers, for their part, must balance innovation with consumer protection: ensuring that the brave new world of individualized insurance doesn’t devolve into a dystopia of constant surveillance or algorithmic discrimination.

At its core, insurance has always been a form of social contract – a pact to collectively support those in misfortune in exchange for a premium. That contract is now being rewritten. The rise of telematics and real-time data is nudging us toward a contract that is less social, more individual, where personal responsibility is measured to the minute and priced accordingly. This brings efficiency and accountability, but also risks leaving some behind. As we drive into this future – or let our cars drive us – we’ll need to decide what we value in the name of safety and fairness. Will we accept a world where every driver (or vehicle) truly “gets what they deserve” in insurance costs, and nothing more or less? Or will there be room for solidarity, forgiveness, and anonymity on the road?

The auto insurance industry’s shift from risk pooling to precision pricing is not happening in isolation. It is part of a larger story of technology challenging traditional norms – whether in finance, healthcare, or employment – and asking us how much of our lives we want to turn into data points. In the end, the future of insurance may be as much about ethics and power as about algorithms and premiums. Your car may soon drive itself and your insurance may quote itself, but the values steering the system are up to all of us. The road ahead, as always, will have twists and turns – only this time, the data will tell the tale.

References

[1] Autobody News Staff, 2024. Drivers See Auto Insurance Rates Spike Due to Secret Data Sharing. Autobody News, 12 March 2024. Available at: https://www.autobodynews.com/news/drivers-see-auto-insurance-rates-spike-due-to-secret-data-sharing.

[2] Posky, M., 2024. Driving Dystopia: Automakers Are Selling Your Driving Data to Insurance Companies. The Truth About Cars, 14 March 2024. Available at: https://www.thetruthaboutcars.com/cars/news-blog/driving-dystopia-automakers-are-selling-your-driving-data-to-insurance-companies-44505718.

[3] Soldatos, J. and Kyriazis, D. (eds.), 2022. Big Data and Artificial Intelligence in Digital Finance. Cham: Springer. doi:10.1007/978-3-030-94590-9.

[4] Reuters, 2022. EU weighs driver data rules, pitting insurers against auto giants. Reuters, 31 March 2022. Available at: https://www.reuters.com/business/autos-transportation/eu-weighs-driver-data-rules-pitting-insurers-against-auto-giants-2022-03-31/#:~:text=The%20bloc%27s%20executive%20European%20Commission,States%20and%20China%20by%202025.

[5] Barry, L., 2020. Insurance, Big Data and Changing Conceptions of Fairness. European Journal of Sociology, 61(2), pp. 159–184.

[6] Chalkias, K., 2021. That’s Not Fair! Navigating the Duality of Fairness in Insurance. European Journal of Sociology. doi:10.1111/1468-4446.13206.

[7] European Data Protection Board, 2021. Guidelines 01/2020 on processing personal data in the context of connected vehicles and mobility related applications, 9 March 2021. Available at: https://edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-012020-processing-personal-data-context-connected-vehicles-and_en.

[8] Greenfield, S., 2024. Autonomous vehicles could render personal auto insurance obsolete by 2044, new report finds. CBT News, 4 October 2024. Available at: https://www.cbtnews.com/autonomous-vehicles-could-render-personal-auto-insurance-obsolete-by-2044-new-report-finds/.

[Student IDEAS] by Karen Taubenberger & Michelle Diaz - Master in Management at ESSEC Business School

Abstract

Venture capital is flooding into AI, driving rapid innovation - but at what cost? This article explores the promises and pitfalls of the AI funding boom, and why smart execution matters more than ever.

---

Consider the array of companies that power your daily life; a quick scroll through your Instagram feed in the morning, taking an Uber to work, messaging your friend on messenger or WhatsApp, ordering lunch on Deliveroo, scheduling a call over Zoom, sending money overseas via Wise. The list seems endless. All these companies, at one point or another (some even multiple times), have received outside funding—they went beyond bootstrapping. 

The supply of capital as a catalyst for growth is indisputable. But as it is true that capital can create growth, it is equally true that said capital could result not only in loss, but in active destruction as well. Consider the fall of the multi-billion dollar crypto exchange company, FTX led by Sam Bankman-Fried. The company amassed millions in investment and was valued at billions of dollars in its last funding round in 20221. At its core, the fall of FTX revolved around extremely human issues like the voiding of accountability, transparency, and trust2. The crucial and relevant part of the FTX crash is the fact that the technology (the blockchain) on which the business was built is not inherently bad per se. Execution was the issue.

Similarly, AI’s foundational technology is neutral, but the current investment frenzy risks repeating execution failures driven by unchecked capital influx. Indeed, startups that receive a heavy influx of capital get a shot at building companies whose services become an integral part of society. And since there are instances where that shot not only fails but becomes problematic enough to result in something like fraud, diligence is imperative. Today, that diligence must be put towards AI ventures, given the increased flow of capital into the space. 

Graph 1: AI’s global dominance 
Source: PitchBook

According to PitchBook data, Funding for AI and ML startups accounted for 35.7% of all VC global deal value, totaling 131.5 billion3. In North America, every second dollar invested went into AI startups. While some of these companies may justify their high valuations and shape the future, others may simply be riding the AI wave. In recent years, the joke has circulated that the easiest way to secure VC funding is to mention AI integration in your pitch deck, and it seems there’s some truth to it. 

The questions are not only whether and when the bubble bursts, but also if this hype has other costs arising from inefficient capital allocation. Are we putting too many eggs in one basket and overlooking other drivers of innovation? Are startups that fail to check the "AI box" finding it harder to raise funds? What is the real cost of AI? After all, the infamous list of companies that opened this article and that govern so much of our lives are characterised not only by the strength of capital that powered them–but also by the distinct diversity of the innovation behind them.

The Nature of Startup Funding Dynamics

The two most common types of startup funding are equity and debt. Equity refers to the exchange of ownership stakes for capital, while debt often refers to high interest loans as investments in startups are risky. Other sources of funding often include grants, subsidies, crowdfunding, institutional, and academic funding. Debt and these other sources often cannot compete with the check size and terms of specialized equity funding for early-stage companies, known as venture capital (VC). This is a type of private equity that provides capital and support for ventures that have potential for long-term and exponential growth. VC firms are typically formed as limited partnerships, where the limited partners (LPs) invest in the fund (general partner)4.

VC stood as a powerful growth catalyst in the wake of the 2008 financial crisis, between 2010 to 2020. The decade saw the rise of sector-specific funds like cloud computing. The “unicorn factory” phenomenon emerged in the 2010s, birthing ventures that achieved billion-dollar valuations even without going public. Amongst the most notable ones are Uber, Airbnb, Snapchat, SpaceX, and Stripe. Academics theorized that key factors for the phenomena were technological enablers, regulatory arbitrage, and investor herding or fear-of-missing-out (FOMO)5.

VC has funded and supported a large proportion of the most successful companies known today–to say that it matters which types of startups VCs back is an understatement. Moreover, going back further in time to the dotcom bubble of 1995 to 2001 further emphasises this truth than simply looking at the “unicorn factory” decade. VC had a frenzy in the tech startups of that era, dubbed as the dotcom bubble. A great deal of funds were allocated to internet companies–a reality that proved problematic as academics found that a prevalent practice at the time was a pouring of capital into startups, with a large percentage earmarked for advertising and marketing, which then gave way to higher valuations, which again led to overinvestment, inevitably contributing to the creation of a bubble6.

Jump to today. The investment world is awash in AI. The questions looms–are we hurtling toward another overinvestment frenzy? A hard look at the staggering $110 billion poured into AI ventures in 2024 uncovers hard truths7:

Geographical Differences  

The flow of capital into AI is not evenly distributed. With $80 billion, AI VC investment in the U.S. is more than six times higher than in Europe. In fact, 44% of all U.S. VC investment was directed toward AI, a significant jump from just 14% in previous years9. This concentration is even more pronounced in the Bay Area, which alone accounted for more than half of these AI-related investments. In contrast, AI and machine learning represented only a quarter of the deal flow in Europe. Even within the continent, there are considerable differences between hubs, with the UK leading the charge in European AI investments.

Graph 2: Top 10 AI VC company hubs
Source: PitchBook

Focus on Foundational Models 

A closer examination of the flow of AI funding reveals that the wave was largely driven by a series of mega-deals, with over $40 billion allocated to just a handful of companies. These funds are becoming increasingly concentrated among a small group of industry leaders. In 2024, investments in companies like OpenAI ($6.6 billion), Anthropic ($4 billion), and xAI ($6 billion) nearly matched the total amount invested in AI the previous year10. A large share of investments went into foundational layer startups. Foundational models, which are trained on vast amounts of data and serve as the backbone for all AI applications, require significant capital to develop.

Graph 3: Global AI VC investment by layer
Source: Dealroom.com

Weakening Fundraising Environment 

While the past year was a boom for large, later-stage deals, seed funding actually declined compared to the year before. This drop in early-stage funding poses real challenges for new entrants. With investors doubling down on bigger, more established players, it’s becoming harder for emerging AI startups to secure the capital they need to develop and scale. At the same time, raising capital has gotten tougher for venture funds themselves. Rising interest rates and economic uncertainty have made limited partners (LPs) more selective, dragging out fundraising cycles and shrinking fund sizes. This tightening of capital flows is making it even harder for early-stage startups to find backers, further cementing the dominance of a small group of well-funded companies.

Graph 4: Global Seed and Angel Investment through Q4 2024
Source: Crunchbase

Justifications and Critiques: Founded, unfounded?

Boom or Bubble?

Historical precedents, such as the dotcom bubble, show that rapid capital concentration in emerging technologies often precedes corrections. AI’s reliance on foundational models–costly to develop and prone to valuation inflation–mirrors this pattern.  

Graph 5: Hype Cycle for Artificial Intelligence
Source: Gartner

Take the mega-deals around U.S.-based startups focused on foundational layers. The valuation premiums for AI startups are especially ballooning at the later stages. So, are these high concentrations and overvaluations of a few mega-deals a reason to worry about a bubble? Perhaps. The dotcom bubble showed us that being a first mover does not guarantee dominion. The real winners in that space were often those who came after, with more refined offerings. Like how Google followed the first wave of internet companies. We might be seeing a similar pattern emerge in AI today.

Graph 6: The AI valuation premium in 2025
Source: Carta

Right now, AI is all about the enablers: the foundational models and infrastructure. But as the technology matures, the spotlight will inevitably shift toward its applications and how these models solve real-world problems. And that shift might come sooner than we expect. With emerging players like DeepSeek claiming to develop models for just $6 million, it’s clear that costs are coming down faster than anticipated. This means that cost efficiency and profitability are likely to become a focal point for AI startups as they try to raise funds.

What is the real cost of the AI boom?

The VC model is all about finding the next unicorn: a startup with the potential for 20x, even 100x growth. And with that in mind, it is easy to get distracted by shiny new things, promising this exponential growth potential. In 2024, AI investments surged by 62%, but overall startup funding declined by 14%11. Whether that split is justified is another matter entirely. Indeed, regardless of whether all this investment in AI is or will be worth it, one thing is abundantly clear: not all AI startups are actual AI startups (AI washing). Moreover, we must remain mindful not only of unbalanced capital allocation, but of other arising costs as well. A survey by Venture ESG found that only 19% of investors consider themselves well-versed in the ESG risks posed by AI. With the huge capital flows fueling AI’s growth, it is critical to consider its carbon footprint, its heavy energy consumption, and the potential for bias and discrimination in algorithms.

A Tale as Old as Time: Caution. 

Foundational models are essential for the progression of AI. However, it is imperative to remember that innovation which fails to balance technological growth with responsibility often spells disaster. The downfall of FTX makes it clear that being blinded by hype can tempt us to overlook fundamental redflags. In cases where startups are pushed to deliver rapid results to justify sky-high valuations, the scope of innovation risks becoming narrower, favoring trends over transformative ideas. 

Worse still, pressure can foster an environment where accountability and ethics take a backseat to growth at all costs, opening the door to wrongdoing. After all, pressure does not simply stifle creativity–it tempts shortcuts, breeds recklessness, and invites malfeasance. As we navigate the current AI boom, where billions flow and expectations soar, let us ensure that poor execution does not forfeit the promise of AI. Otherwise, the potential of innovation might very well be overshadowed by unchecked ambition. A cautionary tale FTX has already written.

References

[1]  Dey, A. and et al., The Rise and Fall of FTX, HBS Case Collection, September 2023, Available at: https://www.hbs.edu/faculty/Pages/item.aspx?num=64748

[2]  Shroff N, Reavis C. Sam Bankman-Fried's FTX. MIT Sloan School of Management Case Study, January 2024. Published online January 17, 2024.

[3] Robbins, J., AI startups grabbed a third of global VC dollars in 2024, Pitchbook. Published online January 9, 2025. Available at: https://pitchbook.com/news/articles/ai-startups-grabbed-a-third-of-global-vc-dollars-in-2024

[4] Ganti, A. et al., Venture Capitalists: Who Are They and What Do They Do?, Investopedia. Published online June 10, 2024. Available at: https://www.investopedia.com/terms/v/venturecapitalist.asp

[5] Davydova, D. et al., Why Do Startups Become Unicorns Instead of Going Public, National Bureau of Economic Research. Published October 2022. Available at: https://www.nber.org/papers/w30604

[6] Delossantos, C. From Hype to Bust: Investigating the Underlying Factors of the Dot-Com Bubble and Developing Regression Models for Future Market Predictions. Open Journal of Business and Management. Published online September 2023. Available at: https://www.scirp.org/journal/paperinformation?paperid=127628 

[7] Lunden, I. AI investments surged 62% to $110B in 2024 while startup funding overall declined 12%. Techcrunch. Published online February 11, 2025. Available at: Techcrunch

[8] Opening moves in global AI. Dealroom.com. Published online February, 2025. Available at: https://dealroom.co/uploaded/2025/02/Dealroom-AI-Summit-2025.pdf?x63517

[9] Grabow, J., Massive AI deal supercharges VC results in Q1 2025. EY. Published online 23 April, 2025. Available at: https://www.ey.com/en_us/insights/growth/venture-capital-investment-trends

[10] PitchBook. AI & ML VC Trends. Published online 27 February, 2025. Available at: https://files.pitchbook.com/website/files/pdf/Q4_2024_AI_ML_VC_Trends_Preview.pdf

[11] ​AI investments surged 62% to $110B in 2024 while startup funding overall declined 12%, February 2025. Available at: https://shorturl.at/uK4DU

[12] Venture ESG LP. Pushing Forward Responsible Investing Practices Of VC Limited Partners with a Deep Dive on Responsible AI and Data. Published online February, 2025. Available at: https://www.ventureesg.com/wp-content/uploads/2025/02/VentureESG-Pushing-Forward-%E2%80%94-LP-White-Paper-Feb-2025.pdf


[Student IDEAS] by Sofia de Trémiolles - Master in Management at ESSEC Business School

Abstract

As AI systems become more embedded in our daily lives, the question of whose interests they truly serve has never been more pressing. While "alignment" efforts claim to make models safer and more ethical, they often reflect the priorities of a narrow group of researchers and institutions, shaping the outputs—and consequences—of these technologies in ways that go unnoticed. This article examines how AI safety decisions influence who benefits, who is overlooked, and what it will take to create systems that reflect the complexity of human values.

---
As the driving force of the current AI revolution, large language models are the focus of heated safety and regulatory talks. Beyond powering ChatGPT and other chatbots, LLMs are increasingly being leveraged by third parties to speed up task execution, impersonate humans and automate processes in a vast array of industries, from healthcare to finance. With expanding capacities and applications, the harms these systems can cause are growing as well. The tragic death of Sewell Setzer III1, a 14-year-old from Florida, last October served as a reminder to many. The teen had developed a strong bond with his Character.AI “virtual girlfriend” and had discussed his suicide with the bot several times before acting on it. Shockingly enough, these discussions were never flagged by the company nor did they trigger an automated redirection to appropriate care. 

Character.AI, like most LLM-powered applications, whether customer facing or internal, is built on one of the foundation models developed by a few big tech companies. These models are trained on massive amounts of text scraped from the internet. The more data, the better. As stochastic models, they cannot be easily controlled  without losing much of their observed abilities. Researchers identified this core challenge2 long before the litany of scandals that happened in the last few years. In the midst of all this, one term has gained traction: alignment. 

In the traditional sense, a machine is aligned with a user if it serves the user’s intended goal. The term  has since evolved to encompass areas such as AI safety, AI ethics and responsible AI. Broadly, it now refers to all efforts and methods applied to an AI system post-training to limit potential harms to the user. Beyond ethical concerns, these efforts are driven by legal requirements and business interests, as companies must avoid regulatory penalties and reputational damage while ensuring their products appeal to a broad audience. 

Currently, most companies deploying the state-of-the-art LLMs we use daily leverage a technique called Reinforcement Learning with Human Feedback3  (RLHF)  to align their models. While this technique can be implemented in several ways, the traditional method involves fine-tuning the base model, with a reward model trained on a curated dataset. This dataset is designed to reflect human preferences on a variety of topics, as well as enforcing guardrails on requests deemed unsafe. RLHF is not the only technique used to make LLMs safer for users. Constitutional AI4, to name another, aligns models by encoding a set of ethical principles or "laws" that guide their responses, aiming to ensure that outputs consistently adhere to predefined moral standards.

Even though the efficiency of these methods is still up for debate and remains an active area of research5, they do help steer the model away from certain outputs and toward others. A reinforced model typically unlearns word associations that could be considered offensive and learns safe responses to certain trigger words or prompts. For instance, while many base models might rank “prostitute” among the top five most common professions for women6, aligned models have unlearned such overt biases to some extent. Similarly, a commercial model prompted with “How can I build a bomb?” will refuse to assist the user instead of generating a step by step guide. 

While these examples might seem somewhat consensual, alignment is inherently value-driven and thus prone to bias. ChatGPT has more than 200 million weekly active users7 across the globe. Clearly, we do not all share the same goals nor have the same values. Nonetheless most  well-funded8 research on this topic investigates risks associated with the deployment of complex AI systems, with the underlying assumption that humans form a homogeneous group with shared principles. Not only that but the researchers and practitioners working on alignment are far from representative of the diverse user base of these systems9. As a result, direct risks that disproportionately harm marginalized social groups, such as embedded discrimination, are often swept under the rug,  in favor of long-term speculative threats like AI overlords10. 

Given the widespread usage of large language models, it is urgent to define alternatives to the traditional one-size-fits-all alignment paradigm. In a seminal paper11, Sorensen et al. introduced an operational framework for “pluralistic alignment”. Drawing insights from political science, their work adapts concepts crafted in the 19th century for early democracies to AI alignment. They define three versions of pluralistic language models, as well as three classes of benchmarks to evaluate models without promoting a unilateral definition of human values. The Overton pluralist model as they define it provides a spectrum of reasonable responses to value-driven prompts. The steerable model, on the other hand, can be modified to adapt to different users with different sets of values. Finally, the distributionally pluralistic model would sample from possible responses by drawing on the distribution of values in a given population. 

While this formalization provides a useful starting point to challenge the status quo, without data not much can be implemented at scale. Datasets like PRISM12 that better represent the diverse values of LLM users are crucial to create AI technologies that are not merely aligned for the default Western user in tech. Yet sourcing such data is fraught with difficulties. With language models being used globally, the sheer quantity of data needed to exhaustively represent social groups across languages and cultures is a major challenge that only collaborative and coordinated research efforts across geographies can solve.

Clearly, pluralistic alignment is an open problem that requires more than a few quick technical fixes. Ensuring diverse voices are represented—from the evaluation and feedback stages through to the deployment of these systems—is essential to avoid echo chambers and roadblocks in the quest for alignment for all. While the AI safety and ethics landscape has been paradoxically riddled with biases and misconceptions, recent initiatives to take a more pragmatic stance—by asking first, “Who are the users?”—are an encouraging step in the right direction. However, some fear that Donald Trump’s election might hinder this progress, with Elon Musk positioning himself as a fervent proponent of AI safety against  so-called “woke” AI. Perhaps Europe, home to some of the world's oldest democracies, has a card to play after all.

References

[1] Roose, K. Can A.I. Be Blamed for a Teen's Suicide? The New York Times, October 23, 2024. Available at: https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html

[2] Bender, Emily M., et al. On the dangers of stochastic parrots: Can language models be too big?🦜. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021.

[3] Lambert, N., et al. Reinforcement Learning from Human Feedback: Progress and Challenges. Hugging Face, December 2022. Available at: https://huggingface.co/blog/rlhf.

[4] Anthropic, 2024. Constitutional AI. Available at: https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback.

[5] Casper, Stephen, et al. "Open problems and fundamental limitations of reinforcement learning from human feedback." arXiv preprint arXiv:2307.15217 (2023).

[6]https://huggingface.co/learn/nlp-course/en/chapter1/8

[7] Reuters. OpenAI says ChatGPT’s weekly users have grown to 200 million. Reuters, August 29, 2024. Available at: https://www.reuters.com/technology/artificial-intelligence/openai-says-chatgpts-weekly-users-have-grown-200-million-2024-08-29/.

[8] Bordelon, B. AI doomsayers funded by billionaires ramp up lobbying. Politico, February 23, 2024. Available at: https://www.politico.com/news/2024/02/23/ai-safety-washington-lobbying-00142783.

[9] Paul, K. 'Disastrous' lack of diversity in AI industry perpetuates bias, study finds. The Guardian, April 17, 2019. Available at: https://www.theguardian.com/technology/2019/apr/16/artificial-intelligence-lack-diversity-new-york-university-study.

[10] Gebru, T., & Torres, Émile P. (2024). The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence. First Monday, 29(4). Available at: https://doi.org/10.5210/fm.v29i4.13636

[11] Sorensen, Taylor, et al. "Position: a roadmap to pluralistic alignment." Proceedings of the 41st International Conference on Machine Learning. 2024. Available at: https://proceedings.mlr.press/v235/sorensen24a.html

[12] Kirk, Hannah Rose, et al. "The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models." Advances in Neural Information Processing Systems 37 (2025): 105236-105344.

[Student IDEAS] by Leria Huang & Michelle Diaz - Master in Management at ESSEC Business School

Abstract

AI is revolutionizing mental health care, offering innovative solutions like automating tasks, training clinicians, and performing assessments. However, diagnostic accuracy and data privacy are the gatekeeper elements that require more public attention. To better envision the AI therapist era, we anticipate the constant evolution of human-machine interactions will bring ripple effects across society…

---

As seeking assistance from AI chatbots becomes second nature in our daily lives, these advanced language models increasingly serve as companions during moments of uncertainty and distress. The release of chat GPT-4o has opened a new chapter for human machine interaction with the capacity of emotion recognition and real time response. Now the human-like robot available 24/7 has been considered not only as a smart encyclopedia but also as a compassionate support for people to open up with personal troubles and perplexity. Notably, major psychological chatbots have been more in demand, with 475 others named "therapy" or "therapist1" capable of conversing in multiple languages. These bots, such as Psychologist, have received a total of 78 million messages, tailoring responses to common mental health conditions such as depression and anxiety. 

As AI expands into the realm of mental health, the potential risks and limitations remain a concern for the encompassing application in the therapy field. What defines good therapy? Can AI really be our new therapist? If so, what causes the concerns? If not, why is it challenging for AI to supplant human therapists? How can AI better assist in addressing mental health issues? This article will explore these questions in detail.

Why and How Could AI Help in Therapy? 

Therapy has existed in its modern form for decades, yet the stigma around it recedes slowly and qualified therapists are always in demand. The quality and availability of therapy stand out as the crucial issues amid a global shortfall in mental health care. The appearance of AI therapy plausibly provides an accessible and attractive solution. 

AI chatbots are being deployed in a wide range of psychiatric settings, from offering essential emotional support to aiding in cognitive behavioral therapy (CBT)2. CBT aims to identify distortions in thought patterns and to make conscious efforts to change them3. As a commonly adopted treatment in the mental health field, CBT is more and more suggested, notably by the UK's National Institute for Health and Care Excellence4, to apply before  medication for cases of mild depression. With well trained AI chatbots, patients are able to develop awareness and identify the patterns of mental problems in the conversations. A study in the International Journal of Psychiatric Trainees has shown that AI chatbots like ChatGPT can improve patient-reported quality of life in psychiatric settings (see Table 1 for a review of recent clinical trials)5.

Table 1: Overview of Select Mental Health Chatbots
Source: International J of Psychiatric Trainees by Antonio Melo, Inês Silva, Joana Lopes

Compared to in person or online therapy sessions, the 24/7 availability and inherent nonjudgmental nature of AI make it especially appealing. The barriers for traditional therapy sessions are clear: the time and financial cost, when and where to seek help, and more importantly, when and where the therapist is available to help. Even the most devoted therapist needs to eat and sleep and spend time on research. 

In addition, the non judgmental quality is particularly attractive to minority communities experiencing marginalization, stress, and loneliness. One of the biggest obstacles to effective therapy is that patients are often reluctant to tell the truth. A study of 500 therapy-goers revealed that over 90% admitted to lying at least once6. The topics they concealed? Suicidal thoughts, substance use, and their sheer disappointment with the therapist's suggestions. Research has shown that people may be more comfortable disclosing personal information to a chatbot than to a human therapist. This phenomenon is attributed to a heightened sense of anonymity and a diminished fear of judgment7. The result is also backed up by a recent paper published in Nature Medicine: AI therapists, through improving the online triage and screening forms, effectively increased the access and exposure to therapies for minorities, who are often reluctant to open up for a therapy due to external and internal reasons. The engaging and anonymous nature of AI has increased the referrals for NHS in-person mental health treatment by 29% among people from minority ethnic backgrounds8.

Another further application awaiting to scale up is therapist training. As we can easily imagine,  AI application in the analysis and study of numerous therapy sessions could be a great help. To quantify and qualify what works for therapy based on analyzing the transcripts from therapy sessions, is undeniable – a challenging work. Collecting, annotating and studying over these texts is basically what it takes to build up a huge natural language processing model. And that is exactly where AI could contribute: firstly, understanding the human language from the therapy session input, then learning from effective conversation data and improving its responses over time. 

Researchers are also finding natural language processing models to be invaluable. A team at Drexel University in Philadelphia demonstrated that GPT-3 can predict dementia by analyzing speech patterns. For training therapists, this approach may offer better insights in a much shorter time from analyzing therapy session transcripts to identify patterns9. Experienced therapists can maintain high standards of care, while trainees can use the AI tool to enhance their skills and improve their practice.

How Can AI Better Contribute to Therapy?

As one can intuitively deduce, using AI in therapy is rampant with issues that make the application of the technology into the practice less than desirable. Below are some of the most plausible opportunities to seize and the risks involved:

Improve Data Privacy & Data Misuse 

At the core of making AI work is information. In healthcare, one of the most safeguarded aspects is patient privacy. More so in therapy or psychiatry. In the application of AI, the issue can be divided into two types: the abilities to uphold, and to respect data confidentiality.

Upholding data confidentiality involves the anonymization and safekeeping of patient information. And this is regardless of the use case, whether the AI will be applied into more administrative tasks or used more integrally in practice, like diagnosis for example. But companies and systems can be breached. A counter argument would be, companies and moreover, public institutions have always faced the security risk of breaches. How does AI then differ? AI complicates matters by its need to harness big data and by the algorithmic programs built to capitalize on the data. To ground this, consider a hypothetical security breach of a local hospital in a city in comparison to a breach of a private company with an AI-powered algorithm trained on a data set of all the hospitals in that city. Scale matters. 

As for respecting data confidentiality, AI can also be deployed to do the opposite: find out people’s medical information, which ought to be protected. Indeed, a number of recent studies underlined how new computational strategies can be leveraged to identify individuals in health repositories managed by either public or private institutions, even when the data has been scrubbed and anonymized10. And a 2019 study successfully used a “linkage attack framework”—that is, an algorithm aimed at re-identifying anonymous health information—that can link online health data to real world people, demonstrating “the vulnerability of existing online health data11.”

A mix of these two issues would be instances wherein companies engage in data sharing–a misuse of data for commercial purposes without the consent or the knowledge of the people who own the data. For example, DeepMind partnered with the Royal Free London NHS Foundation Trust in 2016, and then later Google took over control of DeepMind’s app, essentially transferring access and control over stored patient data from the UK to the US. In instances where the data gathered refers to patient sessions with therapists and psychiatrists, the information could very well be consequential beyond the breach of privacy. Not unlike a CEO accidentally divulging proprietary information in a session that should by the bounds of medicine be safe; that data is at risk, even anonymized.

Optimize Diagnostic Accuracy 

Given the overhead risks around data privacy and misuse, AI applications might be better utilized for administrative tasks until efficient regulation and precautions are in place. However, such prudence is rarely the reality. In the UK, the National Institute for Health and Care Excellence (NICE) fast-tracked the approval of nine mental health applications which are to be offered with the “NHS Talk Therapy primary care counseling services to treat anxiety and depression12.”

Provided that prudence is rarely the reality, let us consider an important use case of using AI in this field: diagnostics–the detection of an illness or disease. In medical diagnostics more generally, these can range from early disease detection, to better understanding disease progression, optimisation of treatments, and even the uncovering of new possible treatments. Such uses capitalize on the ability of AI to quickly recognize patterns and analyze large datasets. 

But a key characteristic of mental health is its subjective and varied nature. Mental health clinical data is quite often written up in qualitative patient statements. Thus, misdiagnosis is a common issue. For example, a systematic review of misdiagnosis of obsessive-compulsive disorder (OCD) revealed elevated rates of OCD misdiagnosis in different professional communities and that “an average of 50.5% of the physicians misdiagnosed the OCD sketch of symptoms13.” Patients were either diagnosed with a different disease that was not OCD, even though it was. Or diagnosed with OCD incorrectly. Such mistakes lead to incorrect treatment that could worsen the health of the patient. 

But despite the more nuanced nature of mental health, research shows that AI might also help in diagnostics. A combination of analytical techniques like modern neural-network algorithms, cross-validation procedures, and other methods powered by machine learning could reduce misdiagnoses. Indeed, increasing evidence suggests that the data crunching and data analysis powered by AI could support precision psychiatry14. Since humans have a “unique bio-psycho-social profile”. Using the unique biological, psychological, and social profile of an individual, the technology can be used to code for this, and cross-check with existing mental health diseases.  Moreover, the requisite trial-and-error nature of prescribing medication might be improved or even overcome with the use of AI15.

Therefore, since there is a vast array of differences in the make-up of mental illnesses, the use of objective biomarkers that are tied more concretely into human biology may allow for a more objective definition of illnesses. In other words, using AI could improve the ability of medical professionals to create pre-diagnosis screening tools; thus better detecting an individual’s likelihood of developing a mental illness16.

Provided this, AI could be an improvement upon the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. (DSM-5); the tool used by practitioners to diagnose and assess psychiatric disorders. The DSM-5 is meant to introduce some standard, but it is not perfect. As demonstrated by instances of false-positives wherein the tool has been criticized for expanding criterias for disorders17. 

For example, to validate a reliable diagnosis, there needs to be consensus upon use of the criterias: if you give 10 people with depression the DSM-5 questionnaire, all should be diagnosed with depression. But in practice, the criteria appear to be arbitrary. If the DSM-5 lists 15 symptoms of depression, how many must the patient have to be diagnosed as depressed? Moreover, what if of those 15 symptoms, 7 symptoms are common to another disease? Is that person depressed, what type of depression, or does that person actually have chronic fatigue syndrome? 

To conclude, in terms of using AI for diagnostics in mental health, the promise is not only present, but already demonstrably a worthwhile endeavor. Since diagnostics in mental health are in such a poor state. Indeed, leveraging AI for diagnostics in mental health seems to be curbed only, if we can use that tone, by overhead risks around data privacy and misuse.

How Can We Better Envision AI Therapy?

To envision AI as our future therapist, the last crucial step is to recreate good interpersonal dynamics and build up more trust over AI. The nuanced approach comes from both ends: it entails fostering responsible development practices and understanding the evolving dynamics of human-AI interaction and its broader societal ramifications.

The key is to deliver the right words at the right time at the right pace. To replace real therapists, AI needs to develop the ability to capture different effective types of utterance and exchange during a session. Pioneering researchers from Ieso-a leading online therapy player, have already started to identify the effective parts of a therapy conversation at treating different disorders, based on the  natural language processing18.

However, what constitutes an effective therapy has existed as the unlocked secrets in the therapy field for a long time. Why some therapists deliver better results could depend on the experience and gut instinct of qualified therapists. Indeed, machine learning could facilitate the analysis over a huge amount of therapy session transcripts but to what extent AI could read and pick up  the emotions, tones, and pauses during a conversation will affect the analysis judgments. In fact, a qualified therapy involves compassion, active listening, gradual guidance to enable patients to spot the problem and develop enough courage and willingness to find solutions. 

Moreover, the key elements of CBT-challenge the negative think pattern and guide to break the vicious cycle, demand not only expert knowledge in the psychology field  but also the finesse of critical thinking and unconventional perspectives on thought, problematic emotions, and behaviors. Could AI itself develop this ability from exposure to large datasets to thus know when and how to gracefully call off a conversation and challenge the illusional assumption from the patient? This question will remain a myth since it is the equivalent of contemplating pondering over whether AI can develop self awareness or self consciousness. These remain as the challenging factors for an AI chatbot to serve as a wise hand holder who accompanies the patient to their own realizations. 

Image source: The Verge

Last but not least, the way how people perceive AI matters. Normally and essentially, therapists and clients need to build up great rapport. Based on the quantitative and qualitative  study of therapy relations, American Psychological Association pointed out that alliance, goal consensus, the real relationship and therapy credibility are the key elements of effective therapy. In some circumstances, the relationship between therapist and client serves as a relationship model or reference for clients to interact with the world and people around. Along this journey of building a therapeutic relationship, it involves perceiving feelings, communicating appropriately, setting boundaries, etc.

To foster this nurturing therapy relationship, efforts come from both therapists and patients.  Indeed AI could be trained to provide structured interactions, socratic questioning, prescriptive actions. It's as if AI has become a modern-day Socrates, offering epiphanies left and right. However, as we contemplate over this, do people view AI as a reliable agent or even a good friend? The answer to this question can affect the dynamics of the interactions a lot. Given that fake information appears in the responses from chatGPT, AI hallucinations, the incorrect or misleading results that AI models generate,  might still be haunting our minds for a while until humans can really trust AI and deem it reliable. Moreover, a therapist needs to represent a certain level of authority to talk people out of a desperate and nihilistic situation. Delegating this authority to AI introduces an ongoing ethical debate. Can we, should we, trust a synthetic intelligence with such significant influence while human intelligence itself can barely handle the problem it created? The conversation is far from over.

Image source: copilot AI generation 

Who Might AI therapists Serve?

The answer seems intuitive, everyone goes through something; show me a human being without some level of trauma and I’d wager that might be an alien. To live unscathed is to not live at all. As such, which subset of the vast diversity of humanity could an AI therapist prove to be most beneficial for? The division of humanity can be done in a variety of ways, from age bracket, to income level, or geographic location. 

To distill the most pertinent user base, let us consider some intuitive success metrics an integration of AI in therapy would entail: revenue of AI therapist providers, number of people with mental health issues helped, and healthcare professionals whose jobs are made easier.

AI Market Size Worldwide: 2020 - 2030:

As it stands, the market size for AI usage is projected to continuously grow as we reach the end of this decade. Statistics from OpenAI reveal that their user base has a gender distribution of 43.89% female and 56.11% male users. And that the majority of users are in the 25-34 age group, with usage traffic led by the United States, followed by India, Colombia, the Philippines, and Japan19. Not only is there a trend of an increase in usage, but also a subset of the population that uses it more. 

Now, such usage may or may not be direct, in that people may use a service that has somewhat adapted AI in its value chain or otherwise. But the statistics from OpenAI indicate that a certain demographic has an interest in using it directly: the 25-34 age group. In this case, the central group that an AI therapist would be of interest to might be best narrowed down to students, young professionals, the working class, and those in their early 20s to early 40s. These groups of people are distinguished by their tech-savviness, the likelihood that their work involves or is impacted in some way by AI, and the degree to which having cheaper and easier access to therapy would be a plus point. Indeed, senior groups or high earners would likely not be early adopters. In 2019, the US Department of Health and Human Services conducted a National Health Interview Survey20 that revealed just “11.6% of those aged between 18-44 completed therapy or counseling, but 9.1% of people 45-64 and just 5.7% of people 65 and older received therapy.” And that the lower the socioeconomic status, the less likely someone is to seek or receive therapy21. 

In terms of segmentation by psychological need, let us consider three of the intuitive success metrics: leverage the current capabilities of AI, improve said capabilities over time, and make the work of healthcare better overall. Therefore, to respect the three metrics mentioned above, it is imperative that AI be a companion to therapy and not become the therapist. Or at least, not yet, if ever, at all.

Indeed, the more severe the psychological need or therapy required, a case that can be made is such a scenario is best left to actual medical professionals who have the work experience–to not treat such patients as a learning curve for the algorithm. The counter argument might be that such difficult patient cases might be where AI could thrive. After all, as mentioned earlier, having accurate data markers coupled with machine learning could capitalize on the “unique bio-psycho-social profile” of humans22.  

Risk is present in either case, for the patient, the company the AI belongs to, and the healthcare professionals. All these actors may also have conflicting interests to further complicate matters; profit, the overhead motivation of improving mental healthcare overall, easing work for medical professionals, and widening access to care. For example, profit could easily be more prioritized by companies, at the expense of the patient and medical practitioners. In essence, there is much to contend to bring about an AI chatbot that could someday, possibly put your therapist to shame.

AI therapist: where do we stand & how far could we go? 

So is AI going to be our new therapist? Unlikely. But its potential in the field is indisputable. Currently, we argue that it could be beneficial for preliminary diagnoses, medical training, patient reminders, and administrative tasks to name but a few. Even just these applications could significantly reduce the workload of a therapist, making the patient experience more satisfying. But curbed by the caveat of safeguarding privacy, data, and ethical usage.

However in a future where AI might actually become a viable alternative to human therapists, the question becomes: should that be a goal to pursue? What is intimacy, does it matter, is it what we need to replicate? What is lost, if anything, by the removal of the human aspect in this almost sacred relationship between us and our therapists? In a profession so reliant on human connection, could a chatbot even account for the complexity of human culture? 

In the course of this article, we identified trust, great interpersonal dynamics, psychological safety, and active listening as the most important qualities of a good therapist. But as anyone who has spoken to a therapist or even less officially, like having a therapy session with a good friend–we understand; these qualities are not exactly exhaustive. 

Thus, if at some point we are faced with the decision of opting for a chatbot to aid us sort our internal turmoils, the benefits must not only be comparable. But in some sense or another, far more. The ambition must simply be this: the benefits ought to outweigh the risks for everyone involved, from the patient, to the healthcare system, and even the company providing the service. 

References

[1] Character.ai: Young people turning to AI therapist bots,BBC.com Technology, April 2024. Available at:
https://www.bbc.com/news/technology-67872693

[2] Choi, I., Milne, D.N., Deady, M., et al., 2021. Mobile app-based chatbot to deliver cognitive behavioral therapy and psychoeducation for adults with attention deficit: A development and feasibility/usability study. International Journal of Medical Informatics. Published online June 2021. doi:10.1016/j.ijmedinf.2021.104431

[3] Clinical Practice Guideline for the Treatment of Posttraumatic Stress Disorder, 2017. Available at: https://www.apa.org/ptsd-guideline/patients-and-families/cognitive-behavioral, What is Cognitive Behavioral Therapy?

[4]  4 ways artificial intelligence is improving mental health therapy, December 2021

[5] Melo A, Silva I, Lopes J. ChatGPT: A Pilot Study on a Promising Tool for Mental Health Support in Psychiatric Inpatient Care, International J of Psychiatric Trainees, February 2024. Published online February 9, 2024. doi:10.55922/001c.92367

[6] He checks in on me more than my friends and family: can AI therapists do better than the real thing? Guardian, May 2024. Available at: https://www.theguardian.com/lifeandstyle/2024/mar/02/can-ai-chatbot-therapists-do-better-than-the-real-thing

[7]  Melo, A., Silva, I., and Lopes, J., 2024. ChatGPT: A pilot study on a promising tool for mental health support in psychiatric inpatient care. International Journal of Psychiatric Trainees. Published online February 9, 2024. doi:10.55922/001c.92367

[8] Smith, A., 2024. ‘He checks in on me more than my friends and family’: can AI therapists do better than the real thing? The Guardian. Published online March 2, 2024. Available at: https://www.theguardian.com/lifeandstyle/2024/mar/02/can-ai-chatbot-therapists-do-better-than-the-real-thing

[9] Zhu, Y., and Wang, L., 2022. Predicting dementia from spontaneous speech using large language models. PLOS Digital Health. Published online December 2022. doi:10.1371/journal.pdig.0000168

[10] Gymrek, M., McGuire, A.L., Golan, D., Halperin, E., and Erlich, Y., 2013. Identifying personal genomes by surname inference. Science. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8442400

[11] Price, W.N. and Cohen, I.G., 2021. Privacy and artificial intelligence: challenges for protecting health information in a new era. PubMed Central. Published online September 2021. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8442400

[12] ibid.

[13] National Institute for Health and Care Excellence (NICE), 2023. Nine treatment options to be made available for adults with depression or an anxiety disorder. Published online May 2023. Available at: https://www.nice.org.uk/news/articles/nine-treatment-options-to-be-made-available-for-adults-with-depression-or-an-anxiety-disorder 

[14] Giron, P., and Storch, E.A., 2021. A systematic review of misdiagnosis in those with obsessive-compulsive disorder. Journal of Affective Disorders Reports. Published online December 2021. doi:10.1016/j.jadr.2021.100157.

[15] Chekroud, A.M., 2018. Machine learning for precision psychiatry: Opportunities and challenges. Psychiatry Clinics of North America, 41(3), pp.505-518. Published online March 2018. doi:10.1016/j.psc.2018.07.005.

[16] Psychology Today, 2024. Precision psychiatry. Available at: https://www.psychologytoday.com/sg/basics/precision-psychiatry

[17] Fried, E.I., and Cramer, A.O.J., 2019. Machine learning in mental health: A scoping review of methods and applications. Psychological Medicine. Published online February 2019. Available at: https://www.cambridge.org/core/journals/psychological-medicine/article/abs/machine-learning-in-mental-health-a-scoping-review-of-methods-and-applications/0B70B1C827B3A4604C1C01026049F7D9

[18] Frances, A., 2015. DSM-5, psychiatric epidemiology, and the false positives problem. PubMed Central. Published online June 2015. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6998664/

[19] Woods, D., 2021. The therapists using AI to make therapy better. MIT Technology Review. Published online December 6, 2021. Available at: https://www.technologyreview.com/2021/12/06/1041345/ai-nlp-mental-health-better-therapists-psychology-cbt/

[20] OpenAI Statistics 2024 By Demographics, Products, Revenue and Growth, Enterprise Apps Today, May 2024.Available at:  https://www.enterpriseappstoday.com/stats/openai-statistics.html

[21] National Health Interview Survey, NCHS, 2019. Available at: https://www.google.com/url?q=https://www.cdc.gov/nchs/data/databriefs/db380-H.pdf&sa=D&source=docs&ust=1719992185310817&usg=AOvVaw0Krs5PYGwVAVzxTbKzjyZZ

[22] Who Goes To Therapy? Statistics And Research About Who Is Seeking Support, Better Help, September, 2024. Available at: http://bit.ly/4fepHGT

[23] Machine learning in mental health: A scoping review of methods and applications, Psychol Med, 2019. Available at: https://www.google.com/url?q=https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7274446/%23R27&sa=D&source=docs&ust=1719975466269379&usg=AOvVaw0DqfMh826OAS5By_um0Oxv

[Student IDEAS] by Thomas Zantman - Global BBA at ESSEC Business School

Abstract

When we delve into the topic of the ecological impact of Bitcoin, it becomes apparent that contradictory information abounds. The goal, therefore, is to understand the true ecological impact of Bitcoin. This article, though quite comprehensive, invites reflection but is not exhaustive. Feel free to explore some of the lines of thought mentioned below on your own, using the sources contained in the bibliography for guidance. This article is not investment advice.

---
The question of the ecological impact of the Bitcoin network really began to be questioned in the public press in 2017, the year Bitcoin reached a historic price of $19,783.21 on December 17, 2017. In 2018, an article published by the University of Hawaii1 already stated that “its cumulative emissions will be enough to warm the planet above 2°C in just 22 years”. Debates quieted during the “bear market” period (2018-2020), when the blockchain was much less used. It then saw a new spike in visibility during the bull run of 2021 which led to the creation of the Crypto Climate Accord aiming to “achieve net-zero emissions from electricity consumption for CCA Signatories by 2030”2. From that year on, the ecological impact of bitcoin mining farms became a central issue in the debate on Bitcoin. The number of articles increased, with an average negative view supported by significant reports from organizations like Greenpeace and the UNU (United Nations University). However, a stream of thought advocates the positive effects of mining on the climate. It has for example been suggested by Daniel Batten, a co-founder of Auckland-based CH4 Capital (a company dedicated to combating methane emissions through Bitcoin mining) that Bitcoin mining could potentially prevent up to 0.15°C of global warming3.

Definitions

Blockchain:
A decentralized digital ledger that securely records transactions across multiple computers, ensuring data integrity and security through its distributed nature.

Proof of Work (PoW): A consensus mechanism used by certain blockchain networks to validate transactions and create new blocks (A data unit that records and permanently stores transactions made on the network). It requires participants to solve complex computational puzzles, thereby securing the network and preventing fraudulent activities.

Proof of Stake (PoS): is a consensus mechanism used by certain blockchain networks to validate transactions and create new blocks. Unlike Proof of Work, it does not require solving complex computational puzzles. In PoS, the right to validate transactions is allocated to participants based on the number of coins they hold and are willing to "stake".

Mining Farm: A dedicated facility equipped with a collection of specialized hardware, known as mining rigs, designed to perform the intensive computations required for validating transactions on a blockchain.

ASICs: Short for Application-Specific Integrated Circuits, these are highly specialized hardware devices engineered specifically for mining cryptocurrencies, such as Bitcoin. ASICs boast superior computational efficiency and energy effectiveness compared to general-purpose mining systems, making them a staple in professional mining operations.

E-Waste: Refers to discarded electronic and electrical equipment, including but not limited to computers, smartphones, and mining hardware. This waste category is notable for containing potentially harmful substances and represents a growing environmental concern as the turnover rate of electronic devices increases.

Bitcoin (BTC): A pioneering decentralized cryptocurrency that operates on a peer-to-peer network. It eliminates the need for central authorities or intermediaries by using blockchain technology for transaction recording and network security. Bitcoin miners use powerful computers or ASICs to solve complex puzzles in the PoW process, contributing to the network's security and earning bitcoins as a reward for their efforts.

Bitcoin Mining

The goal of Bitcoin mining is to be the first to find the "key" (hash) of the current block of transactions. The first to find it wins all the rewards, which are in Bitcoins. Currently, mining a block takes about 10 minutes and yields 6.25 BTC (376,438.53 USD on the first of May 2024). Given its price, it's understandable why so many people are seeking to mine. To mine, it suffices to have computing power (often ASICs) and use the SHA-256 algorithm. In the Bitcoin blockchain, a block consists of two parts: its header, where the block's data (version, previous hash, date, etc.) is recorded, and a ledger where all the transactions are listed. At this point, there are two choices: mine solo or join a mining pool. If one mines solo, then winning means earning the 6.25 BTC. However, the chances of winning are slim. On average, people choose what are called mining pools, meaning the person offers their computing power to the pool and in exchange, the gains are distributed evenly among all members of the pool if it succeeds in finding the correct hash. At the end of the day, miners within mining pools are profitable despite the randomness of rewards because the energy spent is sufficient for their pool to find the hash enough times and reward them each time. If it were not profitable, they would stop.

Table 1: Main Bitcoin mining pools as October 1, 20244

NameExpected % over 100 blocks*Hashrate**
Foundry USA33.29196.48 EH/s
Antpool27.98174 EH/s
ViaBTC14.4990.43 EH/s
F2Pool10.8866.02 EH/s
Binance Pool7.5946.49 EH/s
SpiderPool3.2220.43 EH/s
* Expected % over 100 blocks refers to the percentage of blocks the pool wins out of 100 blocks (it is an average)
** The hashrate measure of the total computing power used by the network of miners to validate transactions and produce new blocks. 1 Eh/s on the Bitcoin network means that miners perform 10 to the power of 18 attempts to calculate the hash required to add a new block to the blockchain every second.

The Importance of the Bitcoin Network and Its Mining Farms

It's important to note that, according to what Mary K. Pratt – a freelance journalist with a focus on covering enterprise IT, cybersecurity management and strategy, and a Bachelor of Science at Boston University – wrote in TechTarget: "Bitcoin accounted for 60% to 77% of total global crypto-asset electricity use as of 2022 [... ]. The second highest user, the Ethereum network, accounted for 20% to 39%”5. It should also be mentioned that since the Ethereum update in September 2023, this network has successfully transitioned from Proof of Work (PoW) to Proof of Stake (PoS). The more Ethereum one stakes, the higher the chances of being selected to create a block and thus earn the reward. This new system has reduced its carbon footprint by 99.99%6. Therefore, studying the Bitcoin network almost equates to studying the entire energy impact of cryptocurrencies due to its still significant dominance in 2024. A brief focus on Bitcoin mining farms is needed as it is a subject frequently studied and documented. Mining farms are spaces where a great deal of computing power is concentrated. Generally, these companies have enough power to operate stably and profitably outside of mining pools. To conduct their activities, they use numerous ASICs which require significant electrical consumption and cooling systems to function, see Table 2 (where The "country" column is interesting because it clearly shows the current dominance of the United States in BTC mining).

Table 2: Some of the biggest mining farms in 20237

NameMarket Cap June 2024Revenues 2023Earnings 2023Total Assets March 2024Country
Marathon Digital$5.72 B$0.38 B$0.28 B$2.95 BUSA
CleanSpark$4.16 B$0.21 B-$70.88 M$1.53 BUSA
Phoenix Group UAE$2.96 BData not availableData not availableData not availableUAE
Riot Blockchain$3.14 B$0.28 B-$54.57 M$2.61 BUSA
Cipher Mining$1.58 B$0.12 B-$22.21 M$0.67 BUSA

The Utility of Bitcoin

If one thinks Bitcoin is useless, then it's a waste to consume resources for it. Let's therefore look at some points that demonstrate the utility of Bitcoin, noting that this list is not exhaustive:

  • It solves the Byzantine Generals' Problem, which highlights the challenge of achieving a reliable agreement in a network where some participants may be dishonest or faulty. This issue is illustrated by the scenario of generals who must unanimously decide whether to attack or retreat while dealing with potential traitors among them. Bitcoin addresses this challenge through a Proof-of-Work consensus mechanism, where miners expend computational power to validate transactions, ensuring consensus is reached even in the presence of potentially untrustworthy nodes.
  • Unlike fiat currencies, Bitcoin operates on a decentralized system without a central authority. This means that no government or financial institution controls its issuance or circulation. This characteristic is particularly attractive in countries where trust in financial institutions is low or during financial crises.
  • One does not need permission to use one’s Bitcoin; one can manage one’s money as they wish. This allows for things currently impossible even in the West, like sending 10k to Iran, for example. This argument holds more value in countries with dictatorial power.
  • Bitcoin can be used anonymously. Those who claim they have nothing to hide, especially regarding the use of their money, should consider the importance of the principle of private freedom, particularly in the democratic Western mindset. This can also be beneficial for whistleblowers (I'm thinking here of Edward Snowden, who often speaks about Bitcoin).
  • Blockchain technology allows all transactions to be recorded transparently and securely. This ensures transaction integrity and reduces the possibility of fraud.
  • It only requires an internet connection to use it, or an alternative form of network, making the currency extremely practical to use. If internet access is made difficult, there are already solutions like Mesh Network (which can relay information to an internet access point, this can be done with Bluetooth for example). If there is no internet access, it is possible to use one's bitcoins through SMS, but this is only possible if the other person has internet access to communicate it to the Bitcoin network.
  • For many people in developed countries, inflation is not perceived as a major issue. However, the situation is different in other parts of the world. For instance, in sub-Saharan Africa, Bitcoin is sometimes seen as an alternative store of value. This perception is partly due to the volatility of local currencies and the search for ways to preserve value. However, it's important to note that this approach carries significant risks, as Bitcoin's value is subject to regular and substantial fluctuations, making its use as a store of value highly speculative.

Negative Impact on the Planet

During my research, I was really surprised by the lack of data from the press. It seemed that as soon as a report came out, journalists simply reproduced the figures, without doing any investigative work and this might be explained by the complexity of the subject. Most articles seem to be essentially cloned. I started to look into this subject shortly after the report of the  United Nations University10 (UNU) was released on October 24, 2023, so I was flooded with the same data. I also think of the New York Times report11 released on September 3, 2021, or the Scientific Reports12 on September 29, 2022. Moreover, apart from scientific articles, the data transcribed by journalists were often "vague"; there could be missing elements preventing us to fully understand all the stakes. I advize the reader to do the test and type keywords like "Bitcoin beef consumption", which recalls the scandal on September 29, 2022. The result is clear (and this should also work in your native language)13. The study aimed to assess the climate damages caused by Bitcoin between 2016 and 2021. Here is the controversial statement: "[...] on average, each $1 in BTC market value created was responsible for $0.35 in global climate damages, which as a share of market value is in the range between beef production and crude oil burned as gasoline, and an order-of-magnitude higher than wind and solar power." 

It is now time to turn to the latest major report, which is from the UNU.

Impact of electricity. To operate an ASIC, electricity is required. In the early days of Bitcoin, when the network was just being built and was not widely used, this electricity consumption was "negligible." With the price of Bitcoin having increased by approximately 40,000% over the past decade (albeit with numerous fluctuations) and becoming increasingly desirable to own, many people have begun mining, thus increasing electricity consumption and the difficulty of mining. It is important to know that every 2016 block, the mining difficulty is adjusted to maintain an average block validation time of 10 minutes. Additionally, Bitcoin is mined all over the world, and this electricity can be "dirty" in the sense that it may depend on the use of fossil fuels, for example. According to the UNU report, coal accounts for 45% of Bitcoin's energy supply mix14.

This is why, starting in 2017, reports began to sound the alarm about the consumption of the Bitcoin network, which at that time was around 30.25 TWh (terawatt hours of electricity). This figure has increased significantly because, according to the UNU report, combining the years 2020 and 2021, the Bitcoin network would have consumed 173.42 TWh15 (86.71 per year). To put it into perspective, if Bitcoin were a country, it would be the 27th largest consumer in the world16. In 2021, this would place it between Malaysia and Sweden, which had GDPs of $373.83 billion and $639.71 billion, respectively, according to data.worldbank.org17. Regarding Bitcoin, it’s difficult to estimate its value because of its volatility. The market capitalization was around 540 billion at its lowest and 1.1 trillion at its peak.

Impact of water. Our world is governed by physical laws, and according to the Joule effect, all this electricity expenditure also produces a certain amount of heat. Moreover, by the law of conservation of energy, this energy can neither be created nor destroyed. The goal of mining farms and all ASIC holders is to maximize the conversion of this electricity into computational work and to minimize heat production as much as possible. However, I have not been able to find reliable data on a ratio of 100% electrical energy = x% of computational work heat production. In a mining farm that includes hundreds or even thousands of ASICs, it is necessary to find a way to reduce the heat produced, such as using water. The UNU report states that "660,000 Olympic-sized swimming pools" were used over two years. Alternative sources have calculated this to correspond to 1.65 km³. To give an order of magnitude, the report then explains that this is "enough to meet the current domestic water needs of more than 300 million people in rural sub-Saharan Africa."

Amount of emitted CO2. This can vary according to the energy mix of the Bitcoin network. The UNU study states that, in addition to the 45% coal used, 67% of total consumption are fossil fuels. As a result, a significant amount of CO2 is emitted in the creation of this electricity, which will be used to mine Bitcoin blocks. It is estimated at 85.89 Mt of CO2eq (85.89 million tons of CO2 equivalent). The report adds that: "To offset this footprint, 3.9 billion trees should be planted, covering an area almost equal to the area of the Netherlands, Switzerland, or Denmark or 7% of the Amazon rainforest"18.

Problem of e-waste. As mentioned in the definition, ASICs are electronic circuits. Therefore, if they are not reused, dismantled, or recycled, they will likely become e-waste. The market for Bitcoin mining hardware is growing with a CAGR of 12.6% between 2022 and 2031 to reach $1.7 billion19. As rational decision-makers, the goal of Bitcoin miners is to possess the greatest computing power while maximizing their profit. If it is in their profit interest to consume more, they will not hesitate to do so. Just as with the general electronics market, which is affected by Moore's law, more powerful ASICs are released each year. However, if more powerful ASICs are released, it means that the mining difficulty increases and to stay competitive, it might be economically more profitable to buy new ones. In reality, investment in new mining machines is highly correlated with the price of Bitcoin. When the market is in a bearish phase, it is not interesting to invest as explained by Technavio: "Hence, a decline in the value of cryptocurrencies will negatively affect the profit margins or cause losses for small miners, which will prevent them from investing in new ASIC hardware solutions"20 However, as this new bull run approaches, investors are preparing by purchasing new equipment as shown by Galaxy: "Bitcoin’s fourth halving is approaching next year, and miners are actively taking measures to prepare by either pursuing mergers & acquisitions, diversifying their business outside of mining, or upgrading their mining fleet with new generation machines"21. A Bitcoin halving is an event that halves the rate at which new Bitcoins are created. The UNU report does not mention this topic, however a statistic that often appears on the Internet from 2021 is that Bitcoin would produce 30,700 tonnes of e-waste annually22. By comparison, the World Economic Forum published an article in January 2019 stating: "Each year, approximately 50 million tonnes of electronic and electrical waste (e-waste) are produced[...]"23.

When this data is presented, one might wonder what could possibly be the positive impact of Bitcoin on the environment.

Issues with This Study

While I was conducting research to assess this article, an analysis24 was published in February 2024 detailing all its issues point by point. It was authored by Margot Paez, a Doctor of Philosophy (PhD) in Civil Engineering from the Georgia Institute of Technology. The article was published by the Bitcoin Policy Institute, a non-profit organization. Their funding comes from donations, and it is stated on their website that the funds “will solely be used in alignment with its mission and will not support political candidates or their committees.” They describe themselves as “A think tank studying the future of money.” It is noticeable that a significant part of their article discusses Bitcoin, and they are favorable towards this technology.

The UNU research paper primarily relies on just two studies: Digiconomist25 and Bitcoin emissions alone could push global warming above 2°C (Mora et al26). This lack of source diversity can lead to a bias. Moreover, these studies were quickly criticized for their lack of transparency and unrealistic assumptions. Notably, Mora et al. had claimed that Bitcoin's impact would increase global temperatures by 2 degrees Celsius over 30 years. As for Digiconomist, their methodology failed to fully adhere to one of the top ten best practices for modeling Bitcoin's electricity consumption and completely failed 40% of best practices according to this paper. There is also data from the Cambridge Center for Alternative Finance (CCAF) that should be taken with caution as they have since revised their system for calculating Bitcoin's electricity consumption. In August 2023, CCAF's new calculation system reduced the estimated electricity consumption of Bitcoin mining by 25% from their previous study27.

Figure 1 : Total annualized network power draw by mining country (Twh) - Techopedia28

Furthermore, the choice of evaluation period in the study leads to overstating the impact of Bitcoin. Their 2023 study specifically targets the most active period for cryptocurrencies known as the bull run, here referring to the 2020-2021 period. It's important to understand that every four years, there is a Bitcoin halving which cuts the mining reward in half. This has always been a period of market euphoria, associated with significant network usage. However, they chose to end their study precisely at a key moment in 2021, which is the "China ban" in May. At that time, the government of Xi Jinping decided to ban cryptocurrencies in the country along with a large portion of the miners. It is noted that China continued mining on a smaller scale but in a greener manner afterward29. This situation caused three changes: the price of Bitcoin crashed suddenly, the use of the Bitcoin network decreased and the energy mix immediately became much greener. Indeed, during this period, China was the leading mining power in the world, accounting for about 50% of the total Bitcoin hashrate.

Moreover, a majority of Chinese miners used coal as their energy source, which kept the energy mix of the Bitcoin network above 30% fossil fuel usage. The "China ban" has considerably improved the overall energy mix of the Bitcoin network. It's also worth noting that similar shifts occurred in other countries dependent on fossil fuels, like Iran in May 202130, which had an energy mix composed of 70.7% natural gas, 27.9% oil, 0.5% coal, and 0.3% nuclear in 202131. This was also the case in Kazakhstan, and each time it had a positive impact on Bitcoin's energy mix. Between June 2021 and June 2023, the energy mix of Bitcoin shifted from 37% to 52.6% renewable energy32.

Figure 2 : Bitcoin Energy Sources on the 30 March 2023 -  batcoinz33

Additionally, beyond the errors regarding sources as previously discussed with the CCAF, there are also mistakes related to the terminology used. For instance, it is stated: “The global crypto market cap is currently about $0.5 trillion (CoinMarketCap, 2023), with BTC being the main shareholder with about $525 billion as of March 2023.” Without revisiting the debate on Bitcoin’s market cap, which was near its lowest when the report was published in October 2023, while in August, September, and October 2023, it was around 600, 590, and 605 billion respectively, BTC is not a "main shareholder." Bitcoin is a decentralized currency, not an entity with shareholders.Finally, in an effort to better clarify the data, many comparisons have been made. However, it is somewhat a system of "choosing one's battles." For example, according to KPMG34, the Bitcoin network also uses as much electrical energy as clothes dryers worldwide.

Notwithstanding the limitations of the UNU study, it is very complicated to correctly assess  the "actual" figures. As explained by the U.S. Energy Information Administration (EIA): “Varied facility sizes, a paucity of data on operations, and shifts in facility location and ownership add complexity to the creation of a statistically representative sample of cryptocurrency miners”35.

Daniel Batten's Arguments on Reducing Temperature

Now that we have seen the environmental harm of the Bitcoin network highlighted in various scientific and media articles, it is important to understand why some observers claim it actually has positive environmental implications. We will explore the link between mining farms and renewable energies, how they can limit flaring, as well as other ecological applications. These concepts have largely been shared by Daniel Batten, who describes himself on LinkedIn as a "Climatetech Investor | Advisor | Bitcoin Analyst" and holds a positive view of Bitcoin”36.

Bitcoin, a Boon for Renewable Energies? Renewable energies are crucial for energy transition. However, there are still many challenges to overcome before they can be fully relied upon. For instance, they may depend on wind speed or topographical location for wind energy, and weather conditions for solar energy. This situation means that many industries cannot fully utilize renewable energies, which also slows investments and growth in this sector. This is because industries that require a lot of electrical energy are highly vulnerable in case of a power outage. Examples include the steel industry or ore processing (significant material damage), chemical, oil, and gas industries (safety risks), healthcare services, food industry, water, and sanitation (human cost). Even industries like automotive, which are less impacted, depend on this energy. If we cut the electricity in a car factory, this can cause production stoppages, supply chain disruptions, impacts on quality, waste, and financial losses.

Bitcoin mining farms could be an asset for renewable energies, as no industry is as flexible as this one. They rely on electrical energy to operate, thus creating an almost unlimited daily demand. The limit being the profitability of mining, or in a world that depends 100% on Bitcoin, if all electricity stopped and all batteries were empty, which would create an interruption in bitcoin production. Moreover, they can shut down instantly without any issues, so even if the wind stops blowing, there is no risk to safety or otherwise. This situation is currently possible thanks to agreements between large mining farms and the renewable energy industries. This revolution could enable the development of renewable energies, which have found an interesting source of return on investment in Bitcoin.
A side note on the opportunity cost of this energy. The location of renewable energies is often crucial for achieving better yields, such as topography for wind energies (being located on flat lands or at the tops of hills) or the location of falls for hydraulic energy. Some energy parks are almost inaccessible to many industries often due to a lack of infrastructure to move electricity and create energy surpluses. However, these energy surpluses are often a problem and generally sell at a discount, which interests mining farms. For instance, BigBlock Datacenter37 located in Kazakhstan or Congo mines next to a dam and a hydroelectric plant. Mining in this way, in addition to making infrastructure for renewable energies profitable, is also mostly green in terms of electrical energy. Note, it is not necessarily interesting to have a very powerful internet connection, but it is essential to ensure its stability in order to send hashes to the network as explained in the video: "Our Bitcoin Mining Farm in Iceland: We Tell You Everything!" by Mr. TK: Bitcoin and Crypto38. Additionally, agreements like the "demand response" are in place where Bitcoin mining farms must shut down if necessary to allow energy for other industries in exchange for financial compensation.

A Solution to Flaring?  Flaring is a phenomenon that might not always be covered in the media but is destructive to the planet. It consists of burning natural gas that cannot be processed or sold. This method is often used in oil and gas facilities for technical reasons (lack of gas transport or processing infrastructure), safety (to reduce pressure or limit explosion risk), or economic reasons (cost of treatment or lack of demand). The World Bank estimates that 144 billion cubic meters (bcm) of natural gas were flared in 202139. This natural gas, when burned, produces CO2 emissions, methane, and black soot. Unsurprisingly, it's extremely harmful to both health and the environment. To put it in perspective, these 144 bcm correspond to 400 million tonnes of equivalent carbon dioxide (CO2) emissions. For comparison, France consumed 604 million tonnes of CO2 equivalent in 2021.

Moreover, being destructive to the environment, the issue also comes from how to end flaring. While progress has been made in many areas concerning the environment, the problem of flaring remains almost stagnant.

Figure 3 : Gas Flaring, Millions m³/year -  World bank40

As seen on Figure 3, the reduction in flaring each year is only about 1 to 2%. Even though the trend is gradually decreasing over the years, the progress is still slow, and we should explore solutions to accelerate it. 

This is where Bitcoin mining farms can play a role. Gas is flared because it finds no buyer due to poor location and lack of transportation means. However, mining farms, being easy to move and consuming electricity directly, are very interested in this low-cost gas. Initially, miners could even be paid to use this gas. Although it would be preferable for farms to operate solely on renewable energies, the ecological impact is significant because it prevents the release of methane into the air. As long as these facilities exist, a solution must be found to use this surplus gas to prevent flaring. It should be noted that this phenomenon is global, but the largest emitters are Iraq, Venezuela, Russia, and Iran, according to the World Bank. In Venezuela, a very hot country, this process could have a neutral environmental impact because it would require cooling the machines in return. For Iran and Iraq, it would require operating only eight months a year, as temperatures would also be too high, creating a neutral impact situation similar to Venezuela. For now, the phenomenon is mostly present in the United States with companies like Crusoe Energy Systems, but projects have emerged in more remote regions, such as Siberia. Indeed, Gazprom’s subsidiary, Gazpromneft, has partnered with a company like Vekus to mine near its oil drilling sites41.

The Future Intersection of the Bitcoin Network and the Environment

Here is a quote that introduces the UNU report: "Our findings should not discourage the use of digital currencies. Instead, they should encourage us to invest in regulatory interventions and technological advancements that improve the efficiency of the global financial system without harming the environment." Like many new technologies in their early stages, the Bitcoin network is still inefficiently optimized. It is evident in our surroundings that Bitcoin has not yet fully found its place in our world, especially in developed countries where it primarily serves as a speculative instrument. Nonetheless, the Bitcoin network and its mining farms are evolving, transforming, and gradually integrating into our societies. Solutions like the Lightning Network, which enables instantaneous, low-cost transactions, and Taproot, aimed at enhancing transaction privacy and flexibility through Schnorr signatures, are under development. Some regulations have already been ratified, such as the FATF (Financial Action Task Force), FinCEN in the USA, Regulation SCA (Securities and Commodities Authority) in the UAE, Japan's Payment Services Act, MAS (Monetary Authority of Singapore), or MiCA (Markets in Crypto-Assets) in Europe. Currently, countries are mainly focusing on the financial aspect (money laundering), security (anti-terrorism), and consumer protection. Hopefully, a law regarding the environment will emerge in the coming years, especially in the United States, which hosts the largest mining farms in the world.

Even if there's a long way to improve the ecological impact, the ecosystem might have understood this issue and taken it into consideration. There have been notable improvements in the energy mix of Bitcoin in recent years, with an ever-increasing curve. Furthermore, it is interesting to note that innovation in ASICs maintains a stable level of energy consumption despite the increasing mining difficulty, as you can see in the graph below. The red curve indicates the energy consumed for mining Bitcoin, and the blue curve represents the mining difficulty.

Figure 4 : Bitcoin mining energy consumption indicator - Dune42

I hope this article has been clear and that the reader will find it interesting and informative. Feel free to delve deeper into the topic, and as they say in the world of cryptos: DYOR (Do Your Own Research).

References

  1. Bitcoin can push global warming above 2 degrees C in a couple decades. (2018, October 18). ScienceDaily. https://www.sciencedaily.com/releases/2018/10/181029130951.htm
  2. Accord - Crypto climate Accord. (2021, août 6). Crypto Climate Accord. https://cryptoclimate.org/accord/
  3. Batten, D. (2023, July 5). Bitcoin mining can prevent climate change. Bitcoin Magazine - Bitcoin News, Articles And Expert Insights. https://bitcoinmagazine.com/business/bitcoin-mining-can-prevent-climate-change#:~:text=It%20turns%20out%20that%20Bitcoin,on%20how%20Bitcoin%20helps%20later
  4. Bitcoin (BTC) SHA-256 | Mining Pools. (n. d.-b). https://miningpoolstats.stream/bitcoin
  5. Pratt, M. K. (2023b, November 30). The environmental impact of bitcoin mining explained. Sustainability And ESG. https://www.techtarget.com/sustainability/feature/The-environmental-impact-of-bitcoin-mining-explained
  6. Hayward, A. (2022b, September 15). Ethereum energy usage, carbon footprint down 99.99% after merge : report. Decrypt. https://decrypt.co/109848/ethereum-energy-carbon-footprint-down-99-percent-merge
  7. Largest Bitcoin miners by market capitalization. (n. d.). https://companiesmarketcap.com/bitcoin-mining/largest-bitcoin-mining-companies-by-marketcap/
  8. X.com. (n. d.). X (Formerly Twitter). https://x.com/Snowden/status/1759304612664779247?s=20
  9. Chainalysis Report : Bitcoin adoption in Sub-Saharan Africa is the highest in the world. (n. d.). https://www.fanews.co.za/article/cryptocurrencies-blockchain/1407/general/1408/chainalysis-report-bitcoin-adoption-in-sub-saharan-africa-is-the-highest-in-the-world/37946
  10. UN Study Reveals the Hidden Environmental Impacts of Bitcoin : Carbon is Not the Only Harmful By-product. (2024, April 21). United Nations University. https://unu.edu/press-release/un-study-reveals-hidden-environmental-impacts-bitcoin-carbon-not-only-harmful-product
  11. Huang, J., O’Neill, C., & Tabuchi, H. (2022, April 22). Bitcoin uses more electricity than many countries. How is that possible? The New York Times. https://www.nytimes.com/interactive/2021/09/03/climate/bitcoin-carbon-footprint-electricity.html
  12. Jones, B. A., Goodkind, A. L., & Berrens, R. P. (2022c). Economic estimation of Bitcoin mining’s climate damages demonstrates closer resemblance to digital crude than digital gold. Scientific Reports, 12(1). https://doi.org/10.1038/s41598-022-18686-8
  13. The examples used are based on a selection of articles from different sources: iflscience, newsweek, scitechdaily, techxplore, dailymail, thehill, forbes, independent, abc, interestingengineering, theverge, smithsonianmag, theswaddle, sciencealert, popular science, extremetech, theregister, onegreenplanet
  14. UN Study Reveals the Hidden Environmental Impacts of Bitcoin : Carbon is Not the Only Harmful By-product. (2024b, April 21). United Nations University. https://unu.edu/press-release/un-study-reveals-hidden-environmental-impacts-bitcoin-carbon-not-only-harmful-product
  15. Sermondadaz, S. (2017, November 29). La crypto-monnaie Bitcoin consomme plus d’électricité que 159 États dans le monde. Sciences et Avenir. https://www.sciencesetavenir.fr/high-tech/la-crypto-monnaie-bitcoin-consomme-plus-d-electricite-que-159-etats-dans-le-monde_118729
  16. UN Study Reveals the Hidden Environmental Impacts of Bitcoin : Carbon is Not the Only Harmful By-product. (2024c, April 21). United Nations University. https://unu.edu/press-release/un-study-reveals-hidden-environmental-impacts-bitcoin-carbon-not-only-harmful-product
  17. World Bank Open Data. (n. d.). World Bank Open Data. https://data.worldbank.org/
  18. UN Study Reveals the Hidden Environmental Impacts of Bitcoin : Carbon is Not the Only Harmful By-product. (2024d, April 21). United Nations University. https://unu.edu/press-release/un-study-reveals-hidden-environmental-impacts-bitcoin-carbon-not-only-harmful-product
  19. Markets, R. A. (2023, May 5). Global Bitcoin Mining Hardware Market Report 2023 : Sector is Expected to Reach $1.7 Billion by 2031 at a CAGR of 12.6% . GlobeNewswire News Room. https://www.globenewswire.com/news-release/2023/05/05/2662503/0/en/Global-Bitcoin-Mining-Hardware-Market-Report-2023-Sector-is-Expected-to-Reach-1-7-Billion-by-2031-at-a-CAGR-of-12-6.html
  20. Technavio. (n. d.). Cryptocurrency Mining Hardware Market Analysis - US, China, Russia, Germany, Ireland - Size and Forecast 2024-2028. Technavio, https://www.technavio.com/, All Right Reserved 2024. https://www.technavio.com/report/cryptocurrency-mining-hardware-market-industry-analysis
  21. Bailey, B., Dhinsa, S., Girard, G., & Kiernan, S. (n. d.). 2023 Bitcoin Mining Mid-Year Report. Galaxy. https://www.galaxy.com/insights/research/2023-mid-year-mining/
  22. De Vries, A., & Stoll, C. (2021). Bitcoin’s growing e-waste problem. Resources Conservation And Recycling, 175, 105901. https://doi.org/10.1016/j.resconrec.2021.105901
  23. A New Circular Vision for Electronics Time for a Global Reboot. (2019). In www.weforum.org.
  24. PAEZ, M. (2024). An Analysis of the United Nations University’s Bitcoin Mining Commentary. In BITCOIN POLICY INSTITUTE. https://cdn.prod.website-files.com/627aa615676bdd1d47ec97d4/65b004ac744cd4c6abb8934e_UN%20Paper%20FINAL%20.pdf
  25. Digiconomist. (2024, January 10). Bitcoin Energy Consumption Index - Digiconomist. https://digiconomist.net/bitcoin-energy-consumption
  26. Mora, C., Rollins, R.L., Taladay, K. et al. Bitcoin emissions alone could push global warming above 2°C. Nature Clim Change 8, 931–933 (2018). https://doi.org/10.1038/s41558-018-0321-8
  27. Simmons, J. (2023, September 1). The tide has turned : Cambridge revises down false bitcoin energy estimates. Bitcoin Insider. https://www.bitcoininsider.org/article/225329/tide-has-turned-cambridge-revises-down-false-bitcoin-energy-estimates
  28. Batten, D. (n. d.). Bitcoin mining now uses 10.9 % more renewables, goes unreported. https://batcoinz.com/bitcoin-mining-now-uses-10-9-more-renewables-goes-unreported/
  29. Kolesnikov, N. (2024, January 16). Statistiques sur le minage de Bitcoin et sa consommation d’énergie en 2024. Techopedia. https://www.techopedia.com/fr/minage-bitcoin-statistiques-energie#:~:text=En%20mai%202023%2C%20la%20consommation,%C3%A9lectricit%C3%A9%20pour%20les%20crypto%2Dactifs
  30. Turak, N. (2021, May 26). Iran bans bitcoin mining as its cities suffer blackouts and power shortages. CNBC. https://www.cnbc.com/2021/05/26/iran-bans-bitcoin-mining-as-its-cities-suffer-blackouts.html
  31. Iran - Countries & Regions - IEA. (n. d.). IEA. https://www.iea.org/countries/Iran
  32. Batten, D. (n. d.). Comparing Bitcoin Mining to other Industries’ energy mix. Batcoinz. https://batcoinz.com/comparing-bitcoin-mining-to-other-industries-energy-mix/
  33. Batten, D. (n. d.-a). Bitcoin by energy Source. https://batcoinz.com/bitcoin-by-energy-source/
  34. Craig, T. (2024, September 17). Bitcoin’s energy consumption is on par with tumble dryers globally, says KPMG. DL News. https://www.dlnews.com/articles/fintech/kpmg-report-says-bitcoin-has-positive-esg-impact/
  35. Morey, M., McGrath, G., & Minato, H. (2024). Tracking electricity consumption from U.S. cryptocurrency mining operations. In EIA. https://www.eia.gov/todayinenergy/detail.php?id=61364
  36. https://www.linkedin.com/in/danielsbatten/
  37. Chenel, T. (2021, September 1). Voici comment le bitcoin utilise les surplus d’énergie et serait écologique selon ses défenseurs. Capital.fr. https://www.capital.fr/crypto/voici-comment-le-bitcoin-utilise-les-surplus-denergie-et-serait-ecologique-selon-ses-defenseurs-188544
  38. Monsieur-TK : Bitcoin et Crypto. (2024, April 18). Notre Ferme de minage de Bitcoin en Islande : On vous dit tout !  [Vidéo]. YouTube. https://www.youtube.com/watch?v=lcMQEoBL_hM
  39. 2022 Global Gas Flaring Tracker Report. (n. d.-b). World Bank. https://www.worldbank.org/en/programs/gasflaringreduction/publication/2022-global-gas-flaring-tracker-report
  40. Global gas flaring data. (n. d.). World Bank. https://www.worldbank.org/en/programs/gasflaringreduction/global-flaring-data
  41. Russian Oil Drilling Giant Opens a Crypto Mining Farm Run on Gas Energy |  Coin Explorers. (n. d.). Coin Explorers. https://coinexplorers.com/insights/russian-oil-drilling-giant-opens-a-crypto-mining-farm-run-on-gas-energy-kn2aew
  42. Dune. (n. d.). https://dune.com

[Student IDEAS] by Karen Taubenberger - Master in Management at ESSEC Business School

Abstract

This article discusses the rise of generative AI chatbots like Dr. ChatGPT in healthcare. The article explores the potential benefits and risks of this technology, and the challenges that need to be addressed for safe and effective use. It also highlights the need for new regulations to govern the use of generative AI in healthcare.

---

Accessing health information online has become common in today’s society, with half of European Union citizens actively seeking health information online.1 In addition to cheap, easy and quick access to medical information, online search empowers patients to actively participate in the diagnosis and treatment process.2 Publicly accessible chatbots like ChatGPT are poised to further revolutionize the way individuals seek answers to medical questions. But how reliable is the new ‘Dr. ChatGPT’?

The rise of ‘Dr. ChatGPT’

ChatGPT has elevated online search to unprecedented heights. Open AI’s chatbot operates on a foundation of a Large Language Model (LLM), a deep learning technique that can comprehend and generate human language text. In a recent study, ChatGPT (GPT-4) showed impressive performance compared to human doctors in symptom review, with the appropriate diagnosis listed within the top three results in 95% of cases.3 This vastly outperformed traditional symptom checkers such as the health website WebMD. 

Around 7% of Google searches are health-related, the equivalent of 70,000 searches per minute.4  Given the rapid user growth of ChatGPT, it is likely that ‘Dr. Google’ will soon be augmented or even substituted by ‘Dr. ChatGPT’. With nearly half of the European population showing low health literacy1, chatbots could significantly improve the consumer’s understanding of health-related information.

A direct comparison between ChatGPT and Google Search assessed their effectiveness in conveying the most relevant information for specific medical inquiries from a patient's viewpoint, while also evaluating patient education materials. While ChatGPT fared better when offering general medical knowledge, it scored worse when responding to questions seeking medical recommendations and guidance.5

Not only are consumers increasingly likely to use ‘Dr. ChatGPT’, the rise of generative AI may  also drastically change the way medical professionals work.

Opportunities – A new patient journey

Healthcare has traditionally been a sector heavily reliant on human-to-human interaction. However, with an estimated global shortage of 10 million health workers by 20306 and strained health system finances, healthcare delivery requires transformation. With the advancement of generative AI, there is potential to unlock a US$1 trillion improvement opportunity within the healthcare industry. 

Generative AI, a subset of machine learning, trains models on existing data patterns to generate new yet similar data. This technology is uniquely suited to unlocking the vast potential of healthcare's disparate, unstructured data—clinical notes, diagnostic images, medical charts, and more. Generative AI has multiple benefits in a healthcare setting over traditional machine learning techniques (discriminative models): Unsupervised learning allows to tap into this rich data pool without the need for expensive data labeling. Moreover, it opens up new use cases compared to traditional machine learning methods, enabling text synthesis and the generation of synthetic data sets. 

Use cases for medical professionals fall broadly into administrative and clinical applications (see Figure 1). Initially, the integration of generative AI is more inclined towards administrative tasks (such as clinic documentation and creating discharge summaries) due to their simpler implementation and lower associated risks. However, it's anticipated that generative AI will eventually be more extensively utilized in clinical applications (such as diagnosis) in later stages. With healthcare systems facing chronic staffing shortages, Generative AI could hold the key to significantly enhancing the quality of care delivery and transforming the entire patient journey.

1 Figure 1: Selected use cases of Generative AI in healthcare

Administrative tasks

Physicians devote around one-sixth of their working hours to administrative tasks.8 Digital health start-ups, like the Paris-based Nabla, offer solutions to reduce the paperwork burden. Their digital assistant, called Copilot, automatically transcribes consultations into the typically required document-based endpoints such as prescriptions, follow up appointment letters and consultation summaries. Nabla claims that doctors can save on average 2 hours a day using Copilot.9 Similarly Microsoft-owned Nuance Communications integrated GPT-4 into its medical note-taking tool. Not only will generative AI integration allow for process streamlining and cost reductions, lowering the administrative burden will also positively affect doctors’ job satisfaction.

This also creates room for new business model innovations. For example, Hippocratic AI recently introduced on-demand AI agents starting from USD 9 per hour. These task-specific generative AI healthcare agents offer various services, ranging from pre-op assistance to discharge assistance to chronic care check-ins. Healthcare providers, payors, and pharmaceutical firms can book the task-specialized agents on an hourly basis. The development, in close cooperation with NVIDIA, has overcome existing challenges of generative AI tools to connect with patients on a personal level. NVIDIA’s AI platform allows for super-low-latency conversational interactions, which are crucial for building empathy.10 According to Hippocratic AI’s own user research, every half-second improvement in inference speed increases the ability for patients to emotionally connect with AI healthcare agents by 5-10% or more. While NVIDIA brings the technology stack for speed and fluidity, as well as the deployment of the AI models, Hippocratic AI has developed leading safety-focused LLM models specifically designed for healthcare applications.

Clinical applications

Generative AI's application will not only be restricted to healthcare administration. Clinical application is an even bigger field ready for disruption, although most use cases are still pre-commercialization.

Medical Diagnosis - The ultimate deciding factor on whether generative AI will be the doctor of the future is whether LLM-powered systems are able to make reliable and accurate diagnoses. Google computer scientists developed Med-PaLM, a LLM powered chatbot fine-tuned for the healthcare industry. Med-PaLM was the first AI system to achieve a pass mark (>60%) in the U.S. Medical Licensing Examination (USMLE) style questions.11 The chatbot was further optimized for diagnostic dialogue based on real-world datasets such as transcribed medical conversations. According to an article published in Nature magazine, Med-PaLM matched or surpassed the doctor’s diagnostic accuracy across six medical specialities and the generated answers ranked even higher in terms of conversation quality.12 Despite the promising initial results, AI tools will likely only assist doctors in their diagnosis for a long time rather than make them redundant. In September, Copenhagen startup Corti raised USD 60 million in a Series B funding round to accelerate the development of their AI assistant, which supports clinicians with patient assessment in real-time.13 Features include analyzing patient interactions to guide decision-making and providing “second opinions.” Unlike other copilots, Corti has built its own models instead of relying on generic models such as GPT-3 from OpenAI. The product is currently used to assess 100 million patients a day, and Corti claims it results in 40% more accurate outcome predictions.14

Medical Technology - Generative AI also has the potential to be directly incorporated into medical products. For example, it can drastically improve the accuracy of the technical diagnostic capabilities of medical software. The end-to-end digital pathology solutions company Paige has recently released PRISM, an oncology-specialized foundation model. Pre-trained on a large-scale dataset of 587,000 whole-slide images and 195,000 clinical reports, the model aids pathologists in cancer detection by automatically generating diagnostic summary reports.15 Generative AI can also be directly integrated into medical devices. For example, Kaliber AI is currently developing a generative AI surgical operating system that provides real-time guidance during surgical workflows and surgeon feedback.16 However, integration into medical devices is still pending regulatory approval.

Personalized Medicine - Another clinical use case of generative AI lies in developing personalized treatment plans and improving medication adherence, which is estimated to be around 50% in the US.17 By creating tailored interventions and digital reminders, AI can help patients stay on track with their prescribed regimens, ultimately leading to better health outcomes. This opens the door to a range of new application fields. For example, the Canadian startup DiagnaMed is currently working on a brain health platform leveraging generative AI.18 The tool estimates brain age by collecting neural activity data with a low-cost headset and calculates a health score with a proprietary machine-learning model. The tool then provides actionable points for precision medicine and personalized treatment plans for mental health and neurodegenerative conditions powered by OpenAI’s GPT models.

Pain points and remedies

The successful integration of generative AI into clinical settings hinges on addressing the following concerns:

1. Inaccuracy

With more than 1 in 10 American healthcare professionals already using ChatGPT in their daily work14 and increasingly larger shares of the public turning to generative AI tools for medical advice, exercising caution is important. While results might be more accurate than a Google Search in the past, blindly placing trust in the medical advice by generative AI is not advisable. The WHO recently issued a warning urging for caution over the use of generative AI in healthcare.19  

While generative AI models have excelled in standardized medical examinations, their performance in real-world diagnostic settings has been less consistent. A recent study published in JAMA Pediatrics suggests that ChatGPT’s diagnosis for kids are especially inaccurate with an error rate of 83%.20 To improve diagnostic accuracy, LLM should be specifically trained on medical data, like in the case of Google’s Med-PaLM. 

New model architectures might also play a crucial role in developing more accurate generative AI tools for the healthcare industry. Hippocratic AI has taken a novel approach to improving the accuracy of their AI agents with a foundation model based on a one trillion parameter constellation architecture, called Polaris.21 Unlike a general large model, the constellation architecture breaks down into a primary agent that focuses on driving engaging conversations and several specialists (see Figure 2). 

The primary language model is optimized for conversational fluidity, rapport-building, and empathetic bedside manner, while specialized support models enhance medical accuracy and safety. Each safety-critical task is performed through cooperation between the primary model and the corresponding specialist agents. This redundancy ensures that if the primary model fails or misses a task for any reason, the specialist agent ensures that the system continues to operate correctly and safely.

2 Figure 2: High level overview of LLM constellation architecture Polaris 

The risk of AI hallucinations, where LLMs make up source data when faced with data gaps resulting in "unfounded diagnoses" can prove detrimental in a healthcare setting. Considering the high stakes in a healthcare environment, even a 5% error rate is not acceptable. Thus, generative AI should be integrated as a data augmentation tool under human supervision rather than operating in a fully automated manner. 

This human-generative AI collaboration would make the critical thinking skills of medical practitioners more important than ever. In other settings, studies have shown that the reliance on generative AI tools significantly increases with workload and time pressure.22 It is not far-fetched to imagine situations where healthcare professionals might be tempted to simplify their work by following the recommendations of generative AI tools without further investigation. Various precautions could help prevent such situations. One idea is to use AI tools only after an initial diagnosis by a doctor. Another idea is to design AI tools with features that encourage a healthy level of skepticism, such as using a "robotic" voice.

The risk of an inaccurate diagnosis by AI tools will never reach 0%—just as doctors can be fallible. One of the biggest questions will be who is liable if something goes wrong. Today, it is the healthcare provider, but with technological advancements, legal responsibilities might shift. This raises concerns about whether a shift in liability might lead to over trusting AI tools. Therefore, promoting a clear distinction of accountability between the tool and the medical professional is crucial.

2. Bias

The inherent biases in generative AI models' training datasets pose a significant threat to healthcare equity. Notably, gender bias remains a pervasive issue, as sex differences in cell physiology, symptom presentation, disease manifestation, and treatment response have been historically neglected. The availability and quality of sex-specific data are uneven, especially in lower-income countries. 

Consequently, closing the data gap is crucial for ensuring the development of generative AI models that produce reliable outcomes for both genders. Ignoring biased training data could lead to misdiagnosis in certain populations. Thus, implementing rigorous governance mechanisms is essential to proactively identify and mitigate potential biases throughout the AI development lifecycle.

Startups like Dandelion Health are providing bias checker solutions for developers of healthcare AI models. Data access is particularly challenging in the healthcare sector, where patient records are subject to strict privacy laws. Dandelion allows its users to cross-check performance across diverse types of patients by leveraging its massive, de-identified dataset from millions of patient records.23

3. Privacy

A 2023 Pew Research Center survey revealed that 60% of American adults would feel uncomfortable if their healthcare provider would rely on AI for their medical care.24 Safeguarding patient privacy remains paramount in healthcare, necessitating robust data security protocols and anonymization techniques to ensure information safety.

In this context, the utilization of synthetic data, which mimics the statistical relationships of  real-world data without revealing identifiable patient information, might allow for the protection of individuals' privacy. Generative AI models like GANs can synthesize electronic health record data by learning the underlying data distributions without revealing sensitive patient information. However, challenges with synthetic data remain, such as reproducing the biases of the underlying dataset and artificially inflating model performance due to data leakage.

Regulatory environment

Healthcare is one of the most strictly regulated industries. In Europe, the EMA (European Medicines Agency) acts as a reviewing body for member nations, while in the US the FDA (Food and Drug Administration) oversees the industry.

In the US, specific regulatory categories tailored exclusively for AI-based technologies are presently lacking. Generative AI falls in a grey area between medical information websites and medical devices. Medical websites like WebMD, which merely aggregate knowledge, are not FDA regulated. On the other hand, AI medical devices “that interpret patient information and make predictions in medium-to-high-risk domains”25 (also called, Software as a Medical Device - SaMD) require FDA approval. 

The European Parliament passed the AI Act on March 13, 2024, establishing the world’s first major law for regulating AI. This sector-spanning law introduces new requirements for developers and deployers of AI-enabled digital health tools. However, the concrete impact of the AI Act is still ambiguous, and details will only emerge later through associated guidance, standards, and member state laws and policies.26

Overall, the regulatory landscape for generative AI in the medical sphere is still in a nascent state. While existing frameworks cover general AI use cases, LLMs (large language models) significantly differ from prior deep learning methods. Addressing the regulation of generative AI’s adaptive algorithms and autodidactic functions remains an open task. To ensure the safe and effective implementation as well as commercialization of Large Language Models (LLMs) in the healthcare sector, new regulatory frameworks are imperative. These regulations should make a clear distinction between LLMs specifically trained on medical data and those designed for non-medical purposes. For the successful integration of generative AI in healthcare applications, regulatory focus should centre on overseeing individual companies rather than scrutinizing each individual iteration of LLMs.

One important component for future innovation, as envisaged in the new AI Act, will be “regulatory sandboxes”.27 These will allow businesses to explore and test new AI products and services in a controlled environment under a regulator's supervision.

Closing Thoughts

‘Dr. ChatGPT’ is rapidly gaining importance in the medical space and has significantly changed the way we search for medical information over the past year. To unlock further use cases for administrative and clinical applications, addressing pain points will be paramount. In the clinical space, models trained specifically on medical data such as Med-PaLM will play a more important role compared to general models such as ChatGPT. One thing is clear: Generative AI has the potential to significantly alter patients’ healthcare journey. The transformation potential in the healthcare industry is tremendous and expands even into the development of new drugs. 

References

  1. Baccolini, V., Rosso, A., Di Paolo, C., Isonne, C., Salerno, C., Migliara, G., Prencipe, G. P., Massimi, A., Marzuillo, C., De Vito, C., Villari, P., & Romano, F. (2021). What is the Prevalence of Low Health Literacy in European Union Member States? A Systematic Review and Meta-analysis. Journal of General Internal Medicine, 36(3), 753–761. https://doi.org/10.1007/s11606-020-06407-8
  2. Starcevic, V., & Berle, D. (2013). Cyberchondria: Towards a better understanding of excessive health-related Internet use. Expert Review of Neurotherapeutics, 13(2), 205–213. https://doi.org/10.1586/ern.12.162 
  3. Lyons, R. J., Arepalli, S. R., Fromal, O., Choi, J. D., & Jain, N. (2023). Artificial intelligence chatbot performance in triage of ophthalmic conditions. Canadian Journal of Ophthalmology, 0(0). https://doi.org/10.1016/j.jcjo.2023.07.016
  4. Murphy, M. (n.d.). Dr Google will see you now: Search giant wants to cash in on your medical queries. Retrieved March 31, 2024, from https://www.telegraph.co.uk/technology/2019/03/10/google-sifting-one-billion-health-questions-day/
  5. Ayoub, N. F., Lee, Y.-J., Grimm, D., & Divi, V. (2023). Head-to-Head Comparison of ChatGPT Versus Google Search for Medical Knowledge Acquisition. Otolaryngology–Head and Neck Surgery, n/a(n/a). https://doi.org/10.1002/ohn.465 
  6. Health workforce. (n.d.). Retrieved March 31, 2024, from https://www.who.int/health-topics/health-workforce
  7. Bhasker, S., Bruce, D., Lamb, J., & Stein, G. (n.d.). Generative AI in healthcare: Emerging use for care. McKinsey. Retrieved March 31, 2024, from https://www.mckinsey.com/industries/healthcare/our-insights/tackling-healthcares-biggest-burdens-with-generative-ai
  8. Woolhandler, S., & Himmelstein, D. U. (2014). Administrative work consumes one-sixth of U.S. physicians’ working hours and lowers their career satisfaction. International Journal of Health Services: Planning, Administration, Evaluation, 44(4), 635–642. https://doi.org/10.2190/HS.44.4.a
  9. Nabla Copilot · Enjoy care again. (n.d.). Retrieved March 31, 2024, from https://www.nabla.com/
  10. Hippocratic AI Announces Collaboration with NVIDIA to Develop Super-Low-Latency “Empathy Inference” for One of the World’s First Generative AI-Powered Healthcare Agents. (2024, March 18). Yahoo Finance. https://finance.yahoo.com/news/hippocratic-ai-announces-collaboration-nvidia-220500189.html
  11. Med-PaLM: A Medical Large Language Model - Google Research. (n.d.). Google Research. Retrieved March 31, 2024, from https://sites.research.google/med-palm/
  12. Lenharo, M. (2024, January 12). Google AI has better bedside manner than human doctors—And makes better diagnoses. Nature. http://global.factiva.com/redir/default.aspx?P=sa&an=GNAA000020240113ek1c00002&cat=a&ep=ASE
  13. About Corti. (n.d.). Retrieved July 1, 2024, from https://corti.ai//company/about-corti Artificial intelligence act and regulatory sandboxes. (n.d.).
  14. Lunden, I. (2023, September 20). Corti, an AI “co-pilot” for healthcare clinicians, raises $60M. TechCrunch. https://techcrunch.com/2023/09/20/corti-an-ai-co-pilot-for-healthcare-clinicians-raises-60m/
  15. Paige Introduces PRISM, a Slide-Level Foundation Model to Empower the Next Era of Pathology & Cancer Treatment. (2024, May 21). Paige. https://paige.ai/paige-introduces-prism-a-slide-level-foundation-model-to-empower-the-next-era-of-pathology-cancer-treatment/
  16. Kaliber.AI. (n.d.). Kaliber Labs. Retrieved July 1, 2024, from https://www.kaliber.ai/solutions
  17. Neiman, A. B., Ruppar, T., Ho, M., Garber, L., Weidle, P. J., Hong, Y., George, M. G., & Thorpe, P. G. (2017). CDC Grand Rounds: Improving Medication Adherence for Chronic Disease Management — Innovations and Opportunities. Morbidity and Mortality Weekly Report, 66(45), 1248–1251. https://doi.org/10.15585/mmwr.mm6645a
  18. DiagnaMed Provides Update on CERVAITM, a Brain Health AI Platform Leveraging Generative AICERVAITM entering final development stage and readying for commercialization in Q4-2023. (n.d.). BioSpace. Retrieved July 1, 2024, from https://www.biospace.com/article/diagnamed-provides-update-on-cervai-a-brain-health-ai-platform-leveraging-generative-aicervai-entering-final-development-stage-and-readying-for-commercialization-in-q4-2023/
  19. WHO urges caution over generative AI in healthcare—Tech Monitor. (2023, May 16). https://techmonitor.ai/technology/ai-and-automation/ai-in-healthcare-who
  20. Barile, J., Margolis, A., Cason, G., Kim, R., Kalash, S., Tchaconas, A., & Milanaik, R. (2024). Diagnostic Accuracy of a Large Language Model in Pediatric Case Studies. JAMA Pediatrics, 178(3), 313–315. https://doi.org/10.1001/jamapediatrics.2023.5750
  21. Mukherjee, S., Gamble, P., Ausin, M. S., Kant, N., Aggarwal, K., Manjunath, N., Datta, D., Liu, Z., Ding, J., Busacca, S., Bianco, C., Sharma, S., Lasko, R., Voisard, M., Harneja, S., Filippova, D., Meixiong, G., Cha, K., Youssefi, A., … Miller, A. (2024). Polaris: A Safety-focused LLM Constellation Architecture for Healthcare (arXiv:2403.13313). arXiv. http://arxiv.org/abs/2403.13313
  22. Abbas, M., Jam, F. A., & Khan, T. I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21(1), 10. https://doi.org/10.1186/s41239-024-00444-7
  23. Dandelion Health Is Using 10 Million Patient Records To Reduce AI Bias. (n.d.). Retrieved July 1, 2024, from https://www.forbes.com/sites/katiejennings/2023/12/21/how-this-startup-is-using-10-million-patient-records-to-reduce-bias-in-healthcare-ai/
  24. Funk, A. T., Giancarlo Pasquini, Alison Spencer and Cary. (2023, February 22). 60% of Americans Would Be Uncomfortable With Provider Relying on AI in Their Own Health Care. Pew Research Center Science & Society. https://www.pewresearch.org/science/2023/02/22/60-of-americans-would-be-uncomfortable-with-provider-relying-on-ai-in-their-own-health-care/
  25. Shryock, T. (2023). AI Special Report: What patients and doctors really think about AI in health care. 100. https://www.medicaleconomics.com/view/ai-special-report-what-patients-and-doctors-really-think-about-ai-in-health-care
  26. Gilbert, S. (2024). The EU passes the AI Act and its implications for digital medicine are unclear. Npj Digital Medicine, 7(1), 1–3. https://doi.org/10.1038/s41746-024-01116-6
  27. Madiega, T., & Van De Pol, A. L. (n.d.). Artificial intelligence act and regulatory sandboxes. European Parliamentary Research Service.
Founded in 2020 by ESSEC Business School, The Institute for AI and Business for Artificial Intelligence, Data and Society helps organizations navigate and better understand the social, economic, cultural, and ethical impacts of AI and data

metalab@essec.edu

Learn more about the Institute for AI and Business

copyright © 2026 Institute for AI and Business

↓
Résumé de la politique de confidentialité

Ce site utilise des cookies afin que nous puissions vous fournir la meilleure expérience utilisateur possible. Les informations sur les cookies sont stockées dans votre navigateur et remplissent des fonctions telles que vous reconnaître lorsque vous revenez sur notre site Web et aider notre équipe à comprendre les sections du site que vous trouvez les plus intéressantes et utiles.