DATA AND AI ARE CHANGING THE WAY ORGANIZATIONS THINK, DECIDE, AND ORGANIZE. IT’S TIME HUMANITIES, MANAGEMENT AND SOCIAL SCIENCES GET INVOLVED.
Metalab

NEWS

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

EVENTS

In a world where Artificial Intelligence (AI) is redefining entire industries and decision-making processes, leaders and managers must adapt rapidly to maintain their competitive edge. This 3-day intensive program is specifically designed to equip you with the essential skills to:

  • Integrate AI at the heart of your corporate strategy.
  • Drive Innovation by leveraging AI technologies to create sustainable value.
  • Lead a Successful Digital Transformation while strengthening organizational competitiveness.

Through hands-on workshops and real-world case studies, you will learn to navigate the complexities of AI, overcoming its challenges while seizing the transformative opportunities it offers your organization.

Format:

3 days - on site (Paris) / in French

Read more...

This 6-day certifying module provides a deep understanding of the new ecosystems, organizational dynamics, and fragmented structures emerging from digital transformation.

Format: 

6 days - on site (Paris) / in French

Read more...

[Student IDEAS] by Karen Taubenberger and Michelle Diaz - Master in Management at ESSEC Business School

Abstract

This article explores the legal and existential tension between generative AI and copyright law, arguing that the depletion of human-generated data poses a threat to AI’s own development. By analyzing cases like Kashtanova and Getty Images, it highlights the "grey zones" of authorship and the risk of "model collapse" when AI is forced to train on its own synthetic outputs. Ultimately, the author contends that protecting creators through new licensing models is essential to prevent a "world starved of creativity" and ensure a high-quality data supply for future innovation.

---

On March 25, 2025, OpenAI dropped a new image-generation feature with its GPT-4o release, stirring immediate controversy1. The tool lets users whip up stylized visuals that eerily mimic the signature styles of iconic studios like Studio Ghibli. Notably, co-founder Hayao Miyazaki, a fierce critic of AI art, slammed it as “an insult to life itself2”.

Similarly, the Writers Guild of America (WGA) went on a months-long strike in 2023 over fears that generative AI threatens writers’ livelihoods, halting production on numerous TV shows and films. At the same time, headlines have been flooded with copyright infringement claims during AI model training. All this seems to point to one thing: Generative AI has a copyright problem. In the mid- to long-term, this isn’t just a problem for creators: AI companies might feel the fallout too.

When AI Creates

Generative AI (GenAI) refers to artificial intelligence systems designed to create original content, rather than just analyzing existing data. These systems generate new material by identifying and learning complex patterns from vast datasets using different techniques. For instance, they can understand and generate text by recognizing relationships between words (transformers3) or identify key features in images, sounds, or designs and use them to create new versions (autoencoders4). This ability to independently create instigates concern on who should be credited as the creator, how rights should be assigned, and whether such creations can be legally protected5.

“All rights reserved”

Copyright was built for a human world, with its earliest statute dating back to 1710. Originally designed to protect “literary, scientific, and artistic works” such as poems, films, or songs6. It grants creators exclusive rights that fall into two categories: economic rights, which allow them to control and earn from the use of their work, and moral rights, which protect their personal connection by ensuring they are credited and can prevent changes that alter the original. In the EU, these protections last for 70 years after the creator’s death7.

Importantly, the moment you create something original, it’s protected. “All rights reserved” is applied automatically, no formal registration needed. However, one core principle underpins all copyright law: the author must be human. 

That’s where things get complicated in the world of AI. For instance, if a programmer (a human) writes software code, that work is protected by copyright8. Similarly, if a human writes the training code that enables a machine-learning model to function, that code is protected. But what about the output of the model itself? Going forward, one of the most tricky issues will be around authorship and ownership of works created and co-created by AI.

Historically, copyright has been quite clear in terms of ownership. Whoever or whatever entity creates a certain piece of work/way of working clearly owns the rights so long as they are able to prove and register ownership. In a sense, there’s a clear thread between what is produced and the human/s that were involved in the production.

A Spectrum of Human Ownership? 

In an age where one can produce or create something that is not entirely human-made or can be a combination of both(purely human-made and co-created by AI), where do we draw the line? In 2023, the US Copyright Office made a clear stance on the sufficiency of human authorship required to be copyrightable. The US Copyright Office ruled on Kashtanova, a case on AI-generated imagery in creative works. The case involved a graphic novel that combined human-written text and imagery produced by Midjourney (AI image generator). Essentially, the Office ruled that it could not register the images, given that no matter how detailed the text-prompts were, the AI’s output remains unpredictable and therefore beyond the artist’s direct control. The Kashtanova case created a precedent and cases following it remained true to the strict requirement of human authorship: “human authorship is the bedrock requirement of copyright9.”

However, the US Copyright Office did not unilaterally dismiss the possibility that a human might be recognised as an author of an AI-generated piece of work. But the office didn’t exactly lay out clear guidelines either. To have a good chance of registering a claim of authorship, the human/s involved must either prove heavy involvement in the creation of the work (training input of the model, process, etc) or demonstrate sufficient human authorship10.

This leaves legal grey zones even in everyday life. Would one need to cite AI-generated images in a presentation for example? If they are not subject to copyright this should not be necessary (similar to an open source picture). But yet, a check of the citation requirements of different universities results in very different opinions on this matter (and also APA 7th or Chicago: citation styles used in academic writing, have no clear guideline).

Drawing the Line 

Perhaps more complicated and difficult than delineating what combination and weightage of AI and human authorship combo can be protected by law, is the actual enforcement of the law itself. And whether the large-scale harvesting of organic (human created) works should be illegal under existing copyright laws. 

Consider how a claimant might prove authorship of an AI-generated work. Or alternatively, how one might dispute the claim that a work is not AI-generated in some way. Where does the burden of proof lie? Proving an AI made something is difficult since the AI can give different results even with the same prompt, and AI companies refuse to take responsibility for their outputs. OpenAI’s terms mention “acknowledge that multiple users might receive identical or similar results”, while Microsoft's Copilot terms contain similar provisions11. This traceability problem matters most once a work goes public, since copyright protection and authorship only kick in when something is actually shared (aka diffusion). This inability to draw a clear line poses issues for users that seek to claim rights for commercial purposes.

This lack of clearly defined ownership further becomes problematic once you consider what happens to the data. Microsoft Copilot and similar services include terms that allow companies to use any data you upload to train and further improve their AI. In other words, users are not only unable to claim ownership of outputs, they also give the companies consent to use their own inputs to generate new content. Thus creating a double bind: you have no right to what the AI produces, and the company gains rights to everything you feed the AI. A clearly asymmetrical benefit.

Another case that highlights this issue is the legal battle between Getty Images and Stability AI. Essentially, Stable Diffusion, a product by Stability AI “was trained on 2.3bn images from a third-party website which pulled its training images from the web, including copyright image archives such as Getty and Shutterstock12.” Hence, the core question at the heart of the legal debate is whether this large-scale harvesting of organic (human created) images should be banned under the current copyright laws. 

As of November 2025, Getty Images lost their copyright infringement claims, while narrowly winning trademark claims. In essence, the court arrived at these decisions since:

  • The training occurred outside the UK (on Amazon servers in the United States)
  • The model weights (the numbers inside an AI model that determines how it thinks and what answers it gives) themselves do not constitute a "copy" of copyright works under the Copyright, Designs and Patents Act 1988. Even if an AI model was trained on copyrighted material, the model’s weight themselves are not a “copy” under copyright law since they don’t reproduce the original protected expression
  • Model weights function differently from storing or reproducing actual copyright works13

So how do you draw the line between necessary information usage and copyright infringement? Is it in the contention between how much AI is in a particular piece of work and how much human involvement is required? Or perhaps it comes down to technicalities like where the training occurs, as illustrated by the Getty Images case?

Or is it around the commercialization of the model, what is a fair price for the information a model uses for the monetisation of that information? Is it the ability of a model to generate “near verbatim” pieces of work14?  

After all, LLM makers can easily deflect blame for problematic outputs by claiming users simply didn't prompt effectively enough. Since prompting has become an essential skill everyone is learning. In essence, the LLM companies can pass on blame to the users, since they did the prompting, and should therefore be responsible for the output. What a convenient way to shift accountability from LLMs providers to consumers; a classic attention economy dynamic where end-users bear responsibility as companies attempt to escape scrutiny.

Running Out of Data: Can we Outrun it?  

Another vital consideration going forward with AI is the availability of data on which it is trained on, given the speed at which the foundational models are eating up data to continuously improve their output. As incentives for creators erode, original content grows rarer, while AI-generated material increasingly saturates the internet, feeding feedback loops. 

According to a San Francisco-based AI research group, Epoch AI, at the current rate at which models are eating data, the total amount of human-generated content found on the internet could run out as early as 2026: 

Fig. 1

Source: available at Epoch.ai15

Graph (fig.1) illustrates the number of existing human-generated texts that are publicly available. And the different LLMs and the speed at which they’re digesting the information.

In essence, two main bottlenecks would be the overall availability of data, and the rate at which society produces genuinely-human created content (affected by the ease of producing AI materials rather than creating organic data given the lack of protections in place). With improvements in model scaling and computing efficiency the impact of these constraints can be slowed:

Fig.2

Source: available at Epoch.ai16

The graph (fig.2) shows the growth in training dataset sizes (measured in effective stock of tokens) against the available stock of data over time. It compares three scenarios—no overtraining (compute-optimal scaling), 5x overtraining, and 100x overtraining—showing that aggressive overtraining depletes the data stock much faster, potentially by 2025, while no overtraining delays exhaustion until around 2030.

But even with these adjustments, the data will still run out before 2030. This analysis mainly focuses on public human text data, but their estimates of the data available out there delineates all that exists:

Fig.3

Source: available at Epoch.ai17

This bar chart (fig.3) estimates how much usable data (in tokens) exists in different sources: Common Crawl, the indexed web, the entire web including private data, images, and video. It shows that the whole web and video contain the largest estimated data stocks, with substantial uncertainty ranges indicated by the error bars.

As publicly and legally available data becomes increasingly scarce, one can see a clear incentive for malicious behaviour to arise. From training on copyrighted data, to scraping content online without users explicit consent, adopting artistic styles of artists without permission. In fact, companies are already edging (some leaping, hello OpenAI, apologies Miyazaki) and pushing the limits on this. On April 14, 2025, Reuters reported that Meta (parent company of Facebook and Instagram) will start using the public posts, comments, and interactions of its users to train its AI models across all its platforms within the EU, effective on May 27, 202518.  

Companies are also exploring other ways to bypass this major problem, such as generating synthetic data. AI-generated data would be a potential solution to the impending scarcity of data and a workaround to IP and privacy issues. Indeed, a plausible strategy could be having the AI models use  “a technique called test-time compute in which the AI responds to a prompt by breaking it down into multiple tasks. Instead of spitting out an answer instantaneously, the model spends extra time “thinking” before generating an outcome.

These higher-quality responses could be turned into a new training data set for the AI, which in theory, could improve its responses19.”

Unfortunately, it’s not that simple. A study that looked into using synthetic data for training models revealed that the “model collapse” phenomenon can arise. This leads to lower-quality outputs being produced, which eventually circulate on the web and are later used as training data for future models. In the end, it could create a problematic loop that ends in the inevitable collapse of the model20. All this to say that we are currently at a stage where there seems to be a lack of viable solutions to address the rapid and wide consumption of organic data necessary for AI models to continuously improve.

Fixing the Copyright Problem

As the seas calm from OpenAI’s controversial image-generation rollout and the echoes of the Writer’s Guild of America strike still linger, it’s clear that Generative AI has copyright problems. Miyazaki’s misgivings, the collective action of writers, and the uptake in infringement claims underscore the need for a solution. Not just for the sake of creatives, but for the continued advancement of AI as well. 

Given the issues we’ve outlined above and the ongoing controversies surrounding the relationship between AI and copyright, it’s easy to fall into a dichotomy: human creators vs AI companies. Strong copyrights don’t just protect creators, they are also the backbone of innovation, incentivizing original work, which in turn prevents AI models from plateauing. 

As copyright adjusts to an AI world, new models have to emerge to align the interests of creators of AI companies and creators. Imagine a system where creators could opt in to share their work, earn automatic royalties and let AI learn without infringing. A new marketplace that ensures that we never run out of original ideas. Disney’s and OpenAI’s recent $1 billion investment and licensing partnership, enabling Sora to generate fan-prompted videos and images from 200+ Disney, Marvel, Pixar, and Star Wars characters, illustrates one such win-win collaboration21. Copyright law needs to catch up fast, or we risk letting the next generation of AI to train on a world starved of creativity. 

References

1. Generative AI Is A Crisis For Copyright Law. Forbes. April 2025. Available at: https://www.forbes.com/sites/hessiejones/2025/04/03/generative-ai-is-a-crisis-for-copyright-law/

2. ChatGPT’s Studio Ghibli AI Art Trend Is an Insult to Life Itself. Gizmodo. March 2025. Available at: https://gizmodo.com/open-ai-ghibli-trend-miyazaki-chatgpt-2000581679

3. A transformer model understands text by looking at a sequence of words at once and learning how the words relate to each other, which allows it to capture context more accurately than word-by-word approaches.

4. An autoencoder is a type of neural network that learns important features in data without labels (unsupervised learning) by compressing inputs into an efficient representation and then reconstructing the original input as accurately as possible from this compressed code.

5. Interactions Between Artificial Intelligence and Intellectual Property Law. Max Planck Institute for Innovation and Competition. Available at: https://www.ip.mpg.de/en/projects/details/interactions-between-artificial-intelligence-and-intellectual-property-law.html

6. Copyright history. Intellectual Property Rights Office. Available at: Copyright HistoryIntellectual Property Rights Officehttps://intellectualpropertyrightsoffice.org › copyright_...

7. Copyright - Your rights. Europa.eu. Available at: https://europa.eu/youreurope/business/running-business/intellectual-property/copyright/index_en.htm

8. Gaffar H., Albarashdi S. Copyright Protection for AI-Generated Works: Exploring Originality and Ownership in a Digital Landscape. Asian Journal of International Law. Published online on: 23 January 202. Available at https://www.cambridge.org/core/journals/asian-journal-of-international-law/article/copyright-protection-for-aigenerated-works-exploring-originality-and-ownership-in-a-digital-landscape/12B8B8D836AC9DDFFF4082F7859603E3

9. Human Authorship Requirement Continues To Pose Difficulties for AI-Generated Works. Perkins Coei. February 2024. Available at: https://perkinscoie.com/insights/article/human-authorship-requirement-continues-pose-difficulties-ai-generated-works

10. AI Can Create a Painting but It Can’t Register a Copyright in the Painting. Perkins Coei. May 2022. Available at: https://perkinscoie.com/insights/blog/ai-can-create-painting-it-cant-register-copyright-painting

11. AI and authorship: Navigating copyright in the age of generative AI. DLA Piper. February 2025. Available at: https://www.dlapiper.com/en/insights/publications/2025/02/ai-and-authorship-navigating-copyright-in-the-age-of-generative-ai

12. Art and artificial intelligence collide in landmark legal dispute. Financial Times. January 2023. Available at: https://www.ft.com/content/d691d599-3cdb-48d8-9824-9b2784a17d90 

13. Getty Images v Stability AI: Getty’s copyright case against Stability AI fails. Pinsent Maisons. November 2025. Available at: https://www.pinsentmasons.com/out-law/news/gettys-copyright-case-against-stability-ai-fails

14. NYT v. OpenAI: The Times’s About-Face. Harvard Law Review. April 2024. Available at: https://harvardlawreview.org/blog/2024/04/nyt-v-openai-the-timess-about-face/

15. Will we run out of data? Limits of LLM scaling based on human-generated data. Epoch AI. June 2024. Available at: https://epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data

16. Will we run out of data? Limits of LLM scaling based on human-generated data. Epoch AI. June 2024. Available at: https://epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data

17. Will we run out of data? Limits of LLM scaling based on human-generated data. Epoch AI. June 2024. Available at: https://epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data

18. Meta to use public posts, AI interactions to train models in EU. Reuters. April 2025. Available at: https://www.reuters.com/technology/artificial-intelligence/meta-use-public-posts-ai-interactions-train-models-eu-2025-04-14/

19. Data Is Fueling the AI Revolution. What Happens When It Runs Out?. Cal Alumni Association. March 2025. Available at: https://alumni.berkeley.edu/california-magazine/online/data-is-fueling-the-ai-revolution-what-happens-when-it-runs-out/

20. Shumailov, I., Shumaylov, Z., Zhao, Y. et al. AI models collapse when trained on recursively generated data. Nature. Published online: July 2024. Available at: https://www.nature.com/articles/s41586-024-07566-y 

21. The Walt Disney Company and OpenAI reach landmark agreement to bring beloved characters from across Disney’s brands to Sora. OpenAI. December 2025 Available at: https://openai.com/index/disney-sora-agreement/

[Student IDEAS] by Varun Jain -  Master in Management at ESSEC Business School

Abstract

This article examines why banks have yet to fully adopt blockchain despite its transformative potential. It identifies key barriers - regulatory uncertainty, legacy infrastructure, and operational challenges like interoperability and network effects. Through case studies and analysis, it explores how banks might overcome these issues via regulatory sandboxes, hybrid blockchain models, and internal use cases. Ultimately, the path forward depends on global regulatory alignment, modernization of systems, and industry-wide collaboration.

---

A thorough exploration of Regulatory, Technological and Operational Challenges

The Blockchain Adoption Gap in Banking Institutions

Banking is a fundamental pillar of our economy that aids in activities such as lending, trading, and payment processing. This industry, however, has a long history of relying on established traditional systems and processes, often taking a measured approach when it comes to new technology. Enter Blockchain: a transformative force poised to redefine the finance industry.

Blockchain, the groundbreaking technology behind cryptocurrencies and DeFi (Decentralised Finance) products, is igniting a revolution in the finance industry. Its decentralised, transparent, and secure nature promises streamlined, efficient transactions, potentially cutting through the complex layers of traditional banking procedures. 

Yet, as blockchain and blockchain-powered services keep on captivating, the banking sector seems to linger at the edges of this technological revolution. Despite the initial excitement, widespread adoption has been slower than anticipated. Why aren’t these financial powerhouses diving in headfirst? Is it a matter of regulatory concerns, technological incapacity, outdated infrastructure, or something else entirely? Do potential solutions and strategies exist that can help banks overcome these challenges? Or are the problems so fundamental that the promise of blockchain in banking is ultimately unattainable? In this article, we explore the barriers holding banks back from fully embracing the blockchain revolution and chart potential pathways toward widespread blockchain adoption in the financial sector.

Blockchain and Banking: Enormous Potential, or a Fundamental Mismatch?

Banks are, at their core, the institutional risk managers of the modern economy. They do far more than simply hold deposits or extend credit; banks facilitate payments, allocate capital, and underpin the trust that enables commerce at scale. By verifying identities, assessing credit worthiness, enforcing contracts, and ensuring regulatory compliance, banks create the trust necessary for economic activity to flourish at scale. This intermediation is what allows capital to flow efficiently, payments to settle reliably, and businesses to transact across borders and legal systems. 

Yet, the very mechanisms that have made banks indispensable - layers of documentation, compliance checks, and manual reconciliation - comes with significant operational complexity. Traditional banking processes - especially in areas like trade finance, cross-border payments, and compliance - are often slow and burdened by extensive paperwork and multiple layers of verification. These legacy systems, designed to minimize risk and satisfy regulatory requirements, can result in lengthy settlement times, increased costs, and a higher likelihood of errors or delays. The very mechanisms that have made banks reliable intermediaries can also make them less agile in a rapidly digitizing world.

Blockchains are digital ledgers where transactions are stored securely and transparently. Each "block" of information is linked to the previous one, making a "chain" that is extremely hard to tamper with. In easier words, it is like a notebook that everybody can read and write in, but once anything is written, it cannot be removed or changed.

Three salient features define blockchains. A blockchain is decentralized. Instead of being managed by a central authority like a bank or a government, it is maintained by a network of computers. It is transparent, as all transactions on blockchain networks are visible to everyone. Finally, it is immutable because any transaction cannot be changed once recorded on the blockchain. These properties of blockchains have substantial potential in banking. It can streamline processes by automating verification and settlement, reduce the need for manual reconciliation and paperwork, and minimize errors through tamper-proof recordkeeping. In doing so, blockchain could enable banks to deliver faster, more efficient, and more transparent services—helping them maintain their critical role in the economy while adapting to the demands of a digital era. 

Strategic Banking Applications: Classification of Blockchain Implementation Models

There are many ways banks can leverage blockchains; these innovations can be classified into 3 main, although sometimes overlapping, categories:

1. Efficiency-focused improvements using blockchain: A good example of this is Trade Finance. The paperwork and lengthy processes involved in settling transactions between banks and trade entities can be digitized and tracked step-by-step on a blockchain, thereby speeding up the process and enabling businesses to expand their reach into international markets swiftly. Blockchains, being a Digital Ledger, can be used to store these transactions and processes digitally. Its transparent nature allows for complete visibility for parties involved, whereas its secure and immutable nature establishes trust in the process1. Recently, HSBC has successfully been able to reduce the processing time for such transactions from 5-10 days to less than 24 hours using blockchain technology2.

2. Adoption of existing services on blockchain: Cross-border payments represent a significant opportunity for blockchain to make an impact. By leveraging its decentralized nature, a blockchain-powered payment system can reduce unnecessary intermediaries, allowing for secure and fast transactions. A Distributed Ledger Technology (DLT) with transactions using a digital cryptocurrency instead of traditional currencies can help speed up the process. In use cases such as cross-border payments, cryptocurrencies (or digital currency, both are used interchangeably in this article) are used as a proxy to transfer the funds between banks and intermediaries, and their digital and transparent nature ensures trust and efficient processing.

3. New Products based on blockchain technologies: Banks have attempted to launch new products using blockchain, including digital currencies and digital bonds, to explore the technology and find more effective, quicker methods for performing traditional functions. These products are, more often than not, stablecoins that are used as a proxy to real currencies (such as the US Dollar). Stablecoins are digital currencies that are pegged to an actual currency and used as a faster and secure alternative to real currencies. Some examples are JPM Coin, which is pegged to the US Dollar and used as a digital currency to facilitate interbank payments. They enable real-time, programmable transactions - by acting as a proxy for traditional currencies - while bypassing the legacy systems that traditional currencies need to go through to finish settlement.

Benefits of Blockchain Adoptions for Consumers

Harnessing the power of Blockchains in banking not only helps organizations improve processes, but as a direct result, improves the banking experience and quality of life for its retail consumers like us. Faster processing, less human intervention and secure transactions have the potential to make banking hassle-free, cheaper and quicker3. Improved accountability, transparency, enhanced security and reduced frauds - the other benefits of blockchain driven solutions - can potentially impact the broader trust and access to banking and financial services for the underserved section of the society.

It is critical to note that most of the supposed benefits - like faster payments, lower fees, and greater transparency - remain largely theoretical at this stage. In principle, blockchain could streamline transactions by cutting out intermediaries, potentially reducing costs and processing times. The promise of improved transparency, since transactions on a distributed ledger are visible and auditable by design. This, arguable, could enhance trust and security, making it harder for bad actors to manipulate records.

However, these improvements are far from guaranteed at this stage. The actual consumer experience depends on how banks and payment providers implement blockchain, and whether they pass any savings or efficiencies on to end users. That said, there are clear upsides to unlocking these benefits - if even a fraction of these promised improvements materialize, the way we interact with financial services could look very different - and for the better.

The Core Analysis: Identifying Barriers to Mainstream Blockchain Integration

The aforementioned possible innovations present a strong case for investing in this technology to potentially reimagine the industry. So we arrive at the central question: why are banks not treading this path? Why has the blockchain revolution not exploded in banking? What prevents them from fully adopting blockchain?

To their credit, many banks have recognized the potential of blockchain and have implemented, to varying degrees of success, some prominent innovations in this area. While some of these have enjoyed a fair bit of success, most of these innovations fall short of reaching mainstream implementation.

In the following sections, we deep dive into three major challenges a blockchain innovation faces in a banking context. Using real-life case studies, highlighting key concerns, winners and losers, we will explore and understand the future implications and potential solutions in overcoming these issues.

Regulatory Challenges

Regulatory Framework Deficiencies: Risk Assessment and Compliance Challenges

Financial governing bodies worldwide are striving to understand cryptocurrency and the DeFi ecosystem. There is still significant confusion surrounding these new "currencies" because they function fundamentally differently from traditional money, which has been around for hundreds of years, with the international system of payments almost 100 years old now. Consequently, the regulatory guidelines on digital currencies and related assets - should they be classified as securities, commodities, or something entirely new? - remain unclear4. As a result, the rules for regulating the exchange of digital currencies and their interactions with the broader financial environment are still unresolved, particularly when it comes to who is responsible for safeguarding digital assets, and how these assets should be valued, reported, and taxed on financial statements.

In the banking world, several activities involve transactions with other banks and institutions - these could be either to transfer currencies from one account to another, short-term borrowing/lending or forex transfers. Because these currencies reside in the databases running within each of these institutions, communication between these "islands of data" requires convoluted and ineffective reconciliations and settlement processes. As a result, several initiatives have been undertaken to use digital currencies (specifically, stablecoins) to quicken inter-bank transactions. DLTs like blockchain have a direct application here. Unfortunately, the regulatory standards, frameworks and guidelines regarding stablecoins are nebulous at best. Regulatory agencies, for consumer protection and institutional risk related reasons, have been actively trying to install guardrails around stablecoins; a proposed stablecoin regulation by the Biden administration was introduced to allow only federally insured depository institutions to issue them, essentially preventing them from being used in several critical use-cases5. While regulations like these have the potential to make stablecoins tightly regulated and safer for integration with existing banking functions - making their adoption easier at first - but the high regulatory bar makes it almost impossible for outside players to invest and drive innovation in this sector. This has a broader impact on the cryptocurrency market - as transferring traditional money to cryptocurrencies is generally carried out by using stablecoins like Tether. On the banking side, this unlocks certain barriers, while installing certain others. A clear balanced regulatory approach is still needed to make stablecoins adoption and usage more safer, secure and inclusive for institutions and innovators alike.

Furthermore, usually, banks leverage blockchains and networks of established digital currency technology partners to power their use cases. This exposes them to the regulatory risks concerning these partners. Santander, a Spanish multinational financial services company, famously launched their cross-border payments systems in association with Ripple, a leading blockchain solutions firm, leveraging their blockchain network and expertise in this area. This not only requires Santander to navigate the cross-border compliance policies, anti-money laundering (AML) requirements and Know-Your-Customer (KYC) policies, but also exposes it to the various regulatory challenges faced by Ripple itself, particularly related to the cryptocurrency powered by its blockchain - XRP. XRP is at the heart of Ripple’s ecosystem, which acts as a substitute to local currencies to facilitate cross-border transactions. This behavior, even though it provides much faster and efficient transaction settlement than traditional methods, has drawn regulatory scrutiny, especially in the United States - where concern over whether to classify it as a security has led to lengthy legal battles. When partnering with organisations such as Ripple, Santander exposes itself to such legal and regulatory risks that come with digital assets like XRP. This highlights the importance of firm regulatory guidelines regarding cryptocurrencies - these will not only help banks issue them in a more secure manner, but also help them partner with other organisations with more reliance and trust. 

Institutional Adaptation: Regulatory Sandboxes and Consortium Blockchain Frameworks

Banks and legislative bodies have come together in recent years to clear the regulatory air around digital assets and provide a platform for institutions to leverage them in a way that balances innovation with compliance.

One of the biggest advancements in this area is the concept of Regulatory Sandboxes. These frameworks enable businesses to test innovative products, services, or business models in real-world settings while under the supervision of regulators. Operating with temporary regulatory relief, these environments allow companies to explore new technologies with minimized legal risks. This approach serves a dual purpose: it provides businesses with a platform for rapid innovation and allows regulators to anticipate new policies and challenges by observing emerging technologies in action. The EU and the UK are at the forefront of this initiative, having launched industry-focused sandboxes like the UK's Digital Securities Sandbox and the EU's DLT program to guide financial innovations6.

In certain cases, banks have invested in creating custom blockchain networks specifically to facilitate financial transactions. These 'consortium' blockchains, which are collaborative platforms managed by a group of organizations, are designed to effectively balance control, access, and scalability concerns, making them a preferable solution to traditional public blockchains. Banks need to focus on balancing appropriate levels of access, consensus mechanisms, and security protections while assessing the costs and benefits of implementing blockchain technology. By adopting these consortium blockchains, banks can proactively navigate regulatory challenges while also leveraging the benefits of a trusted technological framework. Examples of these consortium blockchains, such as the R3 Corda platform and JP Morgan's Quorum, highlight their successful utilization by financial institutions worldwide, showcasing the potential of blockchain technology within the banking sector.

Technological Challenges

Legacy Infrastructure, Scalability Limitations, and Data Privacy Considerations

A 2024 Forbes survey revealed that 55% of banks cite legacy systems as a key obstacle to digital transformation7. These systems, often based on outdated programming languages like COBOL, struggle to support real-time payments, AI-driven analytics and automated compliance checks.

As financial organizations face increasing pressure to modernize and innovate, their over-reliance on legacy systems and cultural resistance to digital transformation present significant hurdles. These challenges not only affect the implementation of blockchain solutions but also hinder the adoption of less technologically intensive innovations like real-time advanced analytics.

Another key aspect at play is talent. Blockchain expertise is scarce, and the industry requires specialized knowledge to implement innovations at scale. Given the numerous regulatory, security, and operational risks, banks must invest significantly in hiring and training qualified personnel.

A significant technical hurdle for blockchain technology is its inherent scalability concerns, which refer to the ability of a blockchain to process transactions at a faster rate. By design, more secure blockchains tend to be less scalable—a concept illustrated by the "Blockchain Trilemma," which highlights the necessary trade-off between scalability, decentralization, and security8. Consequently, a blockchain cannot achieve all three simultaneously. While blockchains provide decentralized trust and security, they encounter notable challenges in scaling their operations, an issue that is intrinsic to the technology itself rather than limited to the banking sector.

Scalability limitations are a critical obstacle to the widespread adoption and utility of DLTs. Without addressing these challenges, blockchains risk being relegated to niche use cases rather than serving as the backbone for broader banking applications like digital identity or supply chain tracking. To truly enable mass adoption, blockchain developers need to find innovative solutions to overcome scalability hurdles9.

What is often overlooked in this debate is that the trilemma isn’t just a technical puzzle - it is a cost problem too. The more a blockchain leans into security and decentralization, the more expensive it becomes to scale. High security means more computational resources,  more validators, and more energy consumption, all of which drive up the cost. Attempts to boost scalability can reduce costs, but often at the expense of either decentralization or security. For banks, this means that even if the technology matures, the economics may still be prohibited for anything beyond niche applications.

Data privacy concerns are a frequently discussed issue with blockchains. Transparency, a key tenet of DLTs, directly conflicts with strict data privacy regulations in the banking sector, such as GDPR, which mandates rigorous control over the storage, access, and sharing of data. Each transaction on the blockchain requires validation - or, in blockchain terminology, reaching a consensus - by its members, who, ideally, possess all necessary information for validation. This requirement risks violating data privacy standards.

Enterprise-Specific Solutions: Examining JP Morgan's Quorum Architecture and Implementation

The core issue is twofold. First, there is a need for technological solutions that enhance the opacity of blockchain transactions while maintaining a secure consensus mechanism. Second, regulatory adjustments are necessary to redesign privacy standards with DLTs in mind. Various solutions have been developed to tackle the first challenge, collectively referred to as Hybrid Blockchains. These combine public and private blockchains - the difference being in the ownership and access to the network. A public blockchain is fully transparent, where anybody can join, read, write and validate transactions on the chain, such as Bitcoin. A private blockchain, on the other hand, is controlled by a single organization or a group of trusted entities. Only approved participants can access, validate, or add data to the network. Private blockchains are often used by businesses or banks for internal processes, offering more control and privacy but less decentralization than public blockchains. A combination of public and private networks, hence, allows organizations to retain sensitive information on a private chain while keeping only the essential data for validation on the public chain. While this approach adds complexity and raises interoperability concerns, it strikes a balance between transparency and privacy. Ultimately, hybrid blockchains hold significant potential for the banking sector - by designing a more centralized infrastructure, they enable scale and security, as established by the blockchain trilemma. They provide a reasonable solution for cost as well, as sacrificing decentralization for security reduces the need for more cost-intensive solutions that rely on changing the deeper functioning of blockchains10.  

JPMC's Quorum is a successful implementation of a large-scale Blockchain Initiative. This permissioned blockchain protocol, based on Ethereum, offers features such as transaction privacy, the ability to execute private transactions and smart contracts, support for multiple consensus mechanisms, and flexible network permissions management. Essentially, Quorum is a modified blockchain developed specifically for financial institutions and their unique use cases.

Like other innovative technologies, Quorum has faced technical challenges, including data privacy, scalability, and standardization. A lack of knowledge about blockchain technology has hindered adoption and the creation of a global regulatory framework. Security breaches pose a constant threat, largely due to human errors related to this knowledge gap rather than flaws in the core technology. 

However, through continuous efforts, investments, and visionary leadership from the Quorum team, this system has been successfully implemented, providing banks with a secure platform to launch their proprietary blockchain projects. Addressing these technological barriers requires ongoing investment and time for blockchains to become commonplace in the banking sector.

Section 3: Operational Challenges

Network Effects and Interoperability Requirements

Traditional banking processes are slow due to multiple intermediaries involved in inefficient information exchanges. This challenge is especially pronounced in specific use cases like Trade Finance, where extensive documentation and a lack of streamlined Customer Due Diligence lead to delays.  

Blockchain technology offers a promising solution for these issues. "Smart Contracts" - digital contracts that are written in code - are engineered to facilitate, execute, and enforce agreements using blockchain, helping to reduce information asymmetries between parties. The involved parties can use the transparent architecture of blockchains to view real-time updates of the concerned trade. Moreover, this technology can streamline payments and regulatory processes across use cases, marking one of the most significant improvements that blockchain can bring to banking11. For example, they can be used to automatically release payments once the goods are confirmed as delivered on the blockchain, reducing the need for manual intervention - streamlining the process and reducing the risk of disputes.

In 2019, HSBC launched Serai, a trade finance solution based on a distributed ledger platform designed to connect suppliers with buyers and digitize transactions. Another significant initiative, we.trade, was owned by  European banks and powered by IBM. However, both of these efforts were eventually shut down due to commercial infeasibility. 

The main challenge was adoption. Without all parties involved in a transaction using the blockchain, the additional benefits of the technology diminish. This scenario is reminiscent of the inefficient Nash Equilibrium in game theory; there is little incentive to become an early adopter. This phenomenon contrasts with the typical adoption pattern of disruptive technologies, where early adopters lead the industry, attracting more users until a critical mass is achieved that makes the innovation commercially viable. The lack of interoperability among blockchains further limits the scalability of these initiatives12.  

In our smart contract use case, if one party lacks direct access to the blockchain containing the contract, they must rely on facilitators, such as banks, to provide alternative means of access. This process involves bringing information down from the blockchain, which not only decreases efficiency but also introduces security concerns. Consequently, this deters all parties from continuing to use the solution, as the inefficiencies it was meant to address persist. This leads to decreased usage, abandoned solutions, and disheartened early adopters, reducing the incentive for future innovations.

Contained Implementation Strategies: Wells Fargo's Digital Cash as an Internal Settlement Mechanism

Many applications restricted within a banking ecosystem can effectively bypass these issues, as demonstrated by Wells Fargo’s internal settlement blockchain solution. Wells Fargo Digital Cash enables the bank to settle internal transactions between its overseas branches without relying on traditional methods like wires or SWIFT. This solution not only accelerates settlement speed but also eliminates transaction fees typically charged by payment facilitators. SWIFT and Wire transfers typically charge 1-5% of transfer amount, along with some flat fees. These fees are attributed to multiple intermediaries, in case of SWIFT, or to higher manual intervention, in case of wires. Due to the same reasons, they take anything from 1 day to a week to settle. Blockchain solutions are dramatically cheaper and near-instant - particularly because of reduced need for intermediaries and manual input, some studies quote a reduction of cross-border payment costs for banks by up to $10 billion globally in 203013.

“When we move money across the world and we need to exchange currencies, we have to go through third parties such as SWIFT and other banks,” explained Lisa Frazier, head of the Innovation Group at Wells Fargo. “That’s a long process, and every time there's a connection with external parties, it takes time and energy and effort.” Wells Fargo utilizes R3's Corda enterprise blockchain for these digital transactions. Since there are no external participants involved, they encounter significantly fewer interoperability issues and maintain greater control over the transactions. Additionally, using a popular and stable consortium blockchain helps them navigate many regulatory and technical challenges.

Wells Fargo plans to move beyond internal transactions and introduce an FX cross-border solution similar to Santander, but scalability issues related to interoperability remain a challenge. They are currently collaborating with HSBC, another leader in the banking industry, pioneering blockchain solutions, to settle matched cross-border transfers for a set of currencies. A holistic solution, potentially completely revolutionizing international payments settlements, is far away in the future, if at all.

The Path Forward: Overcoming Adoption Barriers

In conclusion, while blockchain technology holds revolutionary potential for the banking sector, significant challenges persist in its deployment and scaling within the current landscape. Lawmakers and financial institutions must undertake major efforts to pave the way for the full realization of this technology's benefits.

Regulatory Collaboration: Cross-Jurisdictional Frameworks for Governed Innovation

Regulatory bodies need to come together to lay the foundation of a necessary legal framework to govern decentralized finance and cryptocurrencies. Only then will the banks be able to move forward with enough confidence and invest the requisite capital to drive innovation in the sector. These bodies must address jurisdiction, recognition and privacy issues related to DLT technologies. The solutions should first be rolled out at national or regional levels and eventually spread worldwide.  

Lawmakers worldwide should look towards legislative bodies which have been successful in providing a favorable regulatory environment for innovative blockchain implementations. Countries like Japan, Singapore, the EU, and Switzerland foster innovation through favorable regulations that recognize cryptocurrencies, guide AML and KYC compliance, and stimulate a vibrant startup ecosystem. Even if the regulations are strict, especially considering data-privacy issues, the regulatory landscape is explicit enough to guide developers and institutions to drive blockchain-led projects. Countries like China, despite banning the trading of cryptocurrencies, provide regulatory guidelines for exploring enterprise-level blockchain innovations14. The heavy focus on data privacy and security issues requires developers and institutions to be especially wary of these challenges while driving innovations.

Technology Stack Evolution: AI as a Catalyst for Blockchain-Ready Banking Infrastructure

One of the biggest challenges to digital transformation in banking is the deeply embedded legacy systems, which are not only difficult to upgrade but also pose security risks. However, the emergence of AI offers promise, as it can help banks develop strategies to overcome these challenges and create a "do-or-die" scenario for modernization. With the enormous popularity of AI, banks are expected not only to integrate it throughout their workflow but also to leverage its capabilities to ensure compliance with regulatory standards. This will lead the way for capabilities that can drive blockchain innovations in tandem, enhancing efficiency and security across transactions.

Many banks are exploring use-cases that leverage a combination of AI and blockchain technologies, such as using blockchain to record identities alongside employing AI to enhance fraud compliance15. Blockchain could be significantly impacted by advancements in AI, though the actual integration may vary across the banking sector. Even in best-case scenarios, there are concerns about how long it will take for banks to be ready and whether a lack of investment in this technology will arise due to delays and technical transformations these projects demand.

Cross-Platform Standardization: Blockchain Interoperability as a Driver for Industry-Wide Implementation

Organizational challenges persist as deeply embedded legacy systems complicate upgrades and create security risks for banks. Although some banks have successfully integrated blockchains for internal processes, the real value lies in their ability to reduce paperwork and improve transparency in areas like trade finance and cross-border payments. Interoperability is crucial for banking institutions, and insufficient early adopters will hinder the widespread adoption of blockchains. There is a critical need for ‘Blockchain Interoperability’ in the banking sector due to the ability of various blockchain networks to communicate effectively, which is largely hindered by fragmented regulations worldwide. 

Blockchain interoperability refers to the capability of various blockchain networks to exchange information and utilize each other's functionalities. It enables seamless data and asset transfers across various blockchain platforms, improving transaction efficiency and security while eliminating intermediaries. Current projects focused on cross-chain communication protocols aim to connect different blockchain networks, empowering banks to share data even without a common network. In the smart contract use case, atomic swaps utilize cryptographic techniques to facilitate the transfer between parties across different networks. Developing interoperability platforms, which allow for cross-chain protocols and atomic swaps, will significantly facilitate the widespread adoption of blockchains despite the current fragmented regulatory landscapes. Certain recent projects, such as Polkadot and Cosmos, are already trying to enable this blockchain interoperability to facilitate inter-chain transfers.

Addressing Interdependent Systemic Barriers Through Coordinated Approaches

Finally, it is important to note the interconnectedness of these barriers - these challenges reinforce one another, and addressing any single one can alleviate related challenges, leading towards comprehensive solutions. This is where the ultimate solution lies. 

Interoperability, for example, is largely dependent on the fragmented regulations across the world. If the regulatory bodies align on standardized blockchain guidelines, banking institutions across the world would be able to develop solutions with similar security and privacy features. These standardized blockchain networks that comply with stated regulations are necessary for effective growth and operation across various financial institutions. Adhering to aligned regulations, these would be easier to integrate, driving cross-platform connections and lowering security risks.

In a similar vein, a global digital upgrade of legacy banking systems aims to facilitate blockchain solutions, helping ensure compliance with regulations. This modernization will enhance the interoperability of these platforms, significantly boosting user adoption across various markets. A standardized regulatory landscape allows cohesive technological frameworks, facilitating interoperability that encourages widespread adoption.

Standardized regulations, technology, and operational processes are critical for widespread adoption and the gradual reimagining of the banking landscape. Evidence supporting standardized regulations can be found in the European Union, demonstrating a commitment to effective blockchain governance. The EU serves as a pivotal testing ground for standardized regulations on blockchain implementations across varied nations. These examples demonstrate that overcoming barriers significantly enhances banking experiences and paves the way for future innovations in blockchain technology.

The Future of Blockchain in Banking

In conclusion, as the banking industry stands at the crossroads of tradition and innovation, the potential of blockchain technology to revolutionize financial systems cannot be overstated. While challenges such as regulatory hesitance, operational inefficiencies, and technological barriers remain significant, the benefits of adopting standardized practices and collaborative frameworks are clear. By addressing these issues through organizational change, investment in technology, and global compliance standards, banks can navigate the complexities of this transformation. As we move forward, embracing blockchain not only promises to enhance operational efficiency but also to create a more secure, transparent, and efficient banking landscape, ultimately fostering a brighter future for the global economy.

References

[1] Appinventiv. (2024). How is Blockchain in Banking Transforming the Industry? Retrieved from https://appinventiv.com/blog/blockchain-in-banking/

[2] HSBC UAE. (2023). Blockchain – Transforming the future of trade finance. Retrieved from https://www.hsbc.ae/business-banking/insights/blockchain-trade-finance/

[3] Investopedia. (2023). Blockchain Facts: What Is It, How It Works, and How It Can Be Used. Retrieved from https://www.investopedia.com/terms/b/blockchain.asp

[4] Cato Institute. (2023). Opening the Door to Cryptocurrency Innovation by Eliminating Unnecessary Regulatory Barriers. Retrieved from https://www.cato.org/commentary/opening-door-cryptocurrency-innovation

[5] Selgin, G. (2023). A Simple Proposal for Regulating Stablecoins. Cato Institute. Retrieved from https://www.cato.org/commentary/simple-proposal-regulating-stablecoins

[6] Deloitte. (2023). How Regulatory Sandboxes Foster Innovation and Compliance in Crypto. Retrieved from https://www2.deloitte.com/global/en/pages/financial-services/articles/regulatory-sandboxes-crypto.html

[7] Forbes. (2024). Core Banking Crisis: 55% of Banks cite Legacy Systems as the top barrier to transformation. Retrieved from https://www.forbes.com/sites/forbestechcouncil/2024/01/15/core-banking-crisis/

[8], [10] Hedera. (2023). Blockchain Scalability Solutions. Retrieved from https://hedera.com/learning/what-is-blockchain-scalability

[9] Banerjee, T. (2022). Blockchain Implementation Challenges for IoT. In Blockchain (pp. 73-85). Chapman and Hall/CRC

[11] ConsenSys. (2023). The Ultimate Guide to Understanding Smart Contracts. Retrieved from https://consensys.net/knowledge-base/smart-contracts/

[12] IBM. (2023). Blockchain Interoperability: Future of Seamless Integration. Retrieved from https://www.ibm.com/topics/blockchain-interoperability

[13] Juniper Research. (2023). Blockchain to Facilitate Savings of $10 Billion Globally in Cross-border Payments. Retrieved from https://www.juniperresearch.com/press/press-releases/blockchain-savings-cross-border-payments

[14] Cointelegraph. (2023). Revealing Blockchain Regulations By Country: Insights & Challenges. Retrieved from https://cointelegraph.com/news/blockchain-regulations-by-country

[15] Forbes. (2024). GenAI: 15 Practical, Impactful Applications For Startups. Retrieved from https://www.forbes.com/sites/forbestechcouncil/2024/02/10/genai-15-practical-applications/

[Student IDEAS] by Minseo Ku - Global BBA at ESSEC Business School

Abstract

This article explores how luxury hotels can integrate AI tools - like virtual concierges and predictive personalization - without sacrificing the emotional warmth that defines high-end service. Through industry case studies, it examines the benefits, risks, and ethical challenges of automation, arguing for a balanced, human-centered approach to tech in hospitality.

---

Can AI Enhance the Guest Experience Without Compromising the Essence of Service?

The Convergence of Luxury and Technology

In the ever-evolving landscape of luxury hospitality, artificial intelligence (AI) has emerged as both a powerful tool and a subject of debate. Historically, five-star hotels and upscale resorts built their reputations on high-touch, emotionally engaging service—the hallmark of a truly memorable guest experience. Yet in today’s digital era, where contactless check-ins and virtual concierges are becoming normalized, the question arises: Can AI-driven systems elevate this experience without eroding the personal connection that defines luxury hospitality? This article aims to explore how hotels can navigate these opportunities and challenges to strike a thoughtful balance between advanced technology and the warm, personalized care that discerning travelers expect.

Understanding the Luxury Hospitality Context

The Essence of Luxury Service

Luxury hospitality is characterized by bespoke service, where staff members anticipate and respond to guests’ unstated needs. Personal rapport, empathy, and emotional intelligence lie at its core. A butler who remembers a returning guest’s favorite in-room snack, or a concierge who greets travelers by name, exemplifies this high level of individual attention. When guests select a luxury property—be it a major hotel chain like Four Seasons or a boutique retreat—they expect more than clean rooms or basic amenities. They seek exclusivity, meaningful connections, and sometimes a sense of belonging.

Drivers of Technological Adoption

Despite the intimate nature of the sector, the shift toward automation and AI has gained momentum due to its benefits. The major gain would lie in its enhanced operational efficiency. Automated check-ins and chatbots have helped reduce wait times and streamline tasks that might otherwise consume staff resources (Ivanov & Webster, 2019). On top of that, AI-based data analysis helps predict guest preferences—from pillow types to dining choices—enhancing customized service. The changing guest demographics also play a role in such shifts, as younger, tech-savvy travelers often appreciate digital convenience features, making hotels eager to adopt new solutions. While these benefits are clear, they raise pressing questions about the intimacy and personal touch that distinguish a luxury stay from a standard one (Lu et al., 2020).

AI’s Role in Transforming the Guest Experience

Types of AI Solutions in Luxury Hospitality

Luxury hospitality has adopted a wide array of AI-powered tools to enhance operational efficiency and service consistency. One of the most prominent applications is automated check-in. Through self-service kiosks or mobile apps, guests can bypass the front desk and enjoy faster, smoother arrivals. This is not just a theoretical advantage; Hilton Worldwide has reported a 40% reduction in lobby congestion following the rollout of its digital check-in system (Hilton Worldwide, 2023).

Another major AI-driven feature is the use of virtual concierges. These chatbot systems are designed to operate 24/7, assisting with tasks such as restaurant reservations, local attraction recommendations, or answering frequently asked questions. Radisson Hotel Group’s chatbot, for instance, now handles up to 60% of guest queries, freeing up staff to focus on more complex and emotionally nuanced interactions (Radisson Hotel Group, 2022).

Additionally, predictive analytics powered by AI allows hotels to tailor services based on guest data. This includes anticipating preferences related to room temperature, dietary restrictions, or spa appointments. These tools enhance speed and personalization, which are essential in meeting the expectations of today’s discerning travelers.

Each of these tools contributes to greater speed, consistency, and operational agility. However, the effectiveness of these solutions depends not only on their technical performance but also on how well they complement—rather than replace—human warmth. As such, luxury hotels must ensure that technological innovation remains a tool to empower, not overshadow, the emotional dimension of hospitality.

Privacy-Focused and Human-Focused Guests

Counter to the idea that everyone craves face-to-face interaction, some guests might prefer minimal staff contact—particularly high-profile travelers who value privacy. In these scenarios, AI can be an advantage. However, there remains a segment of clientele for whom genuine conversation and empathic staff engagement create a sense of belonging that no algorithm can replicate. A successful implementation strategy should account for both guest types, offering flexible options and personalized approaches rather than forcing a one-size-fits-all solution.

Preserving the “Human Touch”

The emotional resonance and personal attention offered in luxury hospitality set it apart from mid-range offerings. A staff member’s ability to empathize and read subtle cues—a sigh that indicates the guest had a rough travel day, or an enthusiastic mention of a local cuisine—are intangible yet critical. While AI may store data on preferences, it cannot replicate the warmth of a well-timed gesture or comforting smile. Emotional intelligence remains one of the most vital differentiators in the guest experience, particularly in luxury contexts where guests expect to feel seen, valued, and personally cared for (Gretzel et al., 2021).

Yet, the increasing reliance on automation poses several risks. First, there is the potential loss of authenticity. Overuse of chatbots and automated messages can make guest interactions feel mechanical and impersonal. According to Belanche et al. (2020), excessive automation in service encounters often leads to reduced customer satisfaction and emotional disengagement.

Second, guest dissatisfaction can arise when technology becomes more of a hurdle than a help. This may occur when self-check-in kiosks malfunction or when guests are overwhelmed by incessant app notifications and digital prompts. Such experiences can frustrate users and detract from the seamless, stress-free atmosphere luxury properties aim to provide.

Third, staff alienation is a growing concern. Employees who feel replaced by machines may experience a decline in morale and motivation, undermining the spirit of service excellence. This erosion of purpose not only affects employee well-being but also reduces their ability to deliver emotionally intelligent service—further impacting guest satisfaction.

At luxury properties, it is often the intangible feeling of being genuinely cared for that justifies premium pricing. When automation becomes overbearing or replaces core aspects of human service, it threatens to dilute this essential value. A successful AI strategy must therefore preserve room for spontaneity, warmth, and human connection—elements that remain irreplaceable in delivering truly exceptional guest experiences.

Gauging Success: New Metrics and Case Examples

Assessing whether AI enhances the guest experience in luxury hospitality requires more than traditional satisfaction metrics. While surveys may capture general impressions, they often fall short in evaluating emotional engagement or human connection. To measure AI’s impact meaningfully, luxury hotels are beginning to adopt alternative indicators. One emerging approach is the Emotional Engagement Index, which evaluates how “cared for” guests feel during their stay. Another metric, the Tech-Human Ratio, helps ensure that no aspect of the service becomes overly automated by balancing AI-led and human-led interactions. Additionally, employee satisfaction is increasingly seen as a bellwether of successful AI integration. If staff morale improves due to reduced repetitive tasks and greater focus on meaningful interactions, it signals a healthy AI-human collaboration (Gretzel et al., 2021).

Real-world case studies further illustrate how leading hotel brands are applying these metrics and strategies in practice. The Ritz-Carlton, renowned for its rigorous staff training and emphasis on empathy, introduced AI personalization to support—rather than replace—its human-centric model. According to Marriott International’s internal performance report (2022), the Ritz-Carlton’s hybrid approach has contributed to a measurable 12% increase in guest satisfaction scores, as staff are empowered to focus more on spontaneous, heartfelt service gestures.

Mandarin Oriental has implemented AI-powered predictive analytics to anticipate guest preferences such as spa and dining choices. These insights are shared in daily staff briefings, allowing teams to proactively tailor experiences. In an interview with Skift (2023), a senior Mandarin executive confirmed that the AI-enhanced system is still in active use and continues to receive positive guest feedback, particularly among frequent travelers who value personalized service.

Marriott International has similarly leveraged AI within its loyalty program infrastructure. By analyzing guest behavior and historical data, Marriott generates automated upgrade suggestions and service enhancements. This automation streamlines decision-making at the front desk, enabling staff to allocate more time to meaningful conversations with guests. Marriott’s 2022 Customer Experience Insights Report notes an increase in employee satisfaction following these changes, citing a better alignment between technology use and human service delivery.

In each of these cases, success depends on whether AI tools empower staff to deliver more thoughtful, emotionally resonant service—or if they merely serve as impersonal replacements. The examples from Ritz-Carlton, Mandarin Oriental, and Marriott demonstrate that when applied thoughtfully, AI can indeed elevate luxury hospitality without compromising the essence of human connection.

Looking Ahead: Ethical and Operational Considerations

As luxury hotels continue to integrate AI, they face pressing ethical and operational challenges—particularly around data privacy, emotional authenticity, and long-term guest expectations. Managing vast amounts of guest data, from lifestyle preferences to behavioral patterns, raises serious concerns about transparency and security. In the luxury sector, where trust and discretion are paramount, guests are especially sensitive to how their personal information is collected, stored, and used. According to Deloitte’s 2021 Global Data Privacy Consumer Survey, nearly 70% of high-income travelers indicated they would abandon a brand after a privacy breach. To maintain trust, brands must clearly communicate their data practices, obtain informed consent, and implement robust cybersecurity systems.

Another area of growing focus is emotional AI—technology that interprets facial expressions, vocal tones, or biometric signals to gauge a guest’s emotional state. When applied in luxury hospitality, emotional AI could unlock unprecedented levels of personalized service. For instance, it might alert spa or wellness staff when a guest appears fatigued or distressed, enabling them to tailor offerings accordingly. Similarly, concierges could adjust their tone or pace depending on the perceived mood of the guest, enhancing emotional rapport. Brands such as Accor have explored these technologies in experimental trials within their premium collections (Accor, 2023). However, this approach raises significant ethical questions. Without informed guest consent and transparent usage policies, emotional AI risks undermining the very empathy it seeks to emulate. For luxury hotels, the key lies in deploying these tools as assistive technologies that enhance—but never replace—genuine human intuition and care (Gretzel et al., 2021).

Looking toward the future, long-term guest perception will hinge on how well luxury hotels strike a balance between automation and authentic connection. As guests become more accustomed to AI-led experiences, they may simultaneously develop higher expectations for emotional intelligence and personalized care. Hybrid service models that blend AI efficiency with human warmth are emerging as the most viable path forward. According to Accenture’s 2023 Travel Trends report, luxury consumers increasingly value technology that "disappears into the background" while empowering human staff to deliver emotionally rich experiences. In this context, the future of luxury hospitality will not be defined solely by innovation, but by the industry’s ability to preserve the essence of care amid technological change.

Toward a Harmonious Future

The marriage of artificial intelligence and luxury hospitality holds great promise. Automating repetitive tasks, offering precise personalization, and providing round-the-clock assistance can amplify service quality—if executed with care. The key lies in balance: technology should empower employees and guests, not overshadow human connection.

By understanding the emotional core of luxury service, embracing flexible tech solutions that cater to different guest preferences, and adopting metrics that measure both efficiency and human warmth, the industry shall be able to chart a course that honors its heritage while innovating for the future. In doing so, luxury hospitality can evolve into an even more enriching and memorable realm, where technology serves as a catalyst—never a substitute—for genuine human interaction.

References

[1] Belanche, D., Casaló, L. V., Flavián, C., & Schepers, J. (2020). Service robot implementation: A value co-creation and co-destruction perspective. Journal of Business Research, 117, 38–51.

[2] Gretzel, U., Buhalis, D., & Sigala, M. (2021). The future of service work: Challenges and opportunities for the luxury sector. Tourism Review, 76(3), 447–458.

[3] Hilton Worldwide. (2023). Annual Corporate Responsibility Report.

[4] Ivanov, S., & Webster, C. (2019). Robots in tourism: A research agenda. Annals of Tourism Research, 76, 105–118.

[5] Lu, L., Cai, R., & Gursoy, D. (2020). Developing and validating a service robot integration willingness scale. International Journal of Hospitality Management, 87, 102378.

[6] Marriott International. (2022). Customer Experience Insights Report.

[7] Skift. (2023). AI in Hospitality: Adoption Trends and Case Studies.

[8] Tussyadiah, I., & Miller, G. (2019). Perceived impacts of AI and automation on hospitality jobs. International Journal of Contemporary Hospitality Management, 31(6), 2569–2588.

[9] Deloitte. (2021). Global Data Privacy Consumer Survey.

[10] Radisson Hotel Group. (2022). AI and Automation in Guest Services: Efficiency Without Compromise. Internal Operations Report.

[11] McKinsey & Company. (2021). Understanding the Next-Gen Luxury Traveler: Trends in Personalization and Tech Adoption. McKinsey Travel Insights.

[12] Skift. (2023). AI in Hospitality: Adoption Trends and Case Studies.

[13] Affectiva. (2023). Emotion AI in Customer Service: Real-World Use Cases and Pilots.

[14] Accor. (2023). Digital Transformation in Luxury Brands: Emotional Intelligence and Guest-Centric AI.

[15] Accenture. (2023). Travel Trends 2023: Hospitality in the Age of Hybrid Expectations.

[Student IDEAS] by Hugo Jalet - Global BBA at ESSEC Business School

Abstract

The rapid expansion of ESG data and divergent rating methodologies have undermined the credibility and consistency of sustainability scores, enabling greenwashing and investor confusion. While the EU’s CSRD introduces reporting standards, it stops short of unifying rating criteria. Artificial intelligence (AI) offers tools to structure ESG assessments, detect inconsistencies, and enhance transparency. However, AI faces limitations: data biases, opacity, interpretive challenges, and significant environmental costs. Effective ESG scoring requires a hybrid model combining AI capabilities with human oversight, regulatory standards, and open-source approaches. Used responsibly, AI can support more reliable, transparent, and sustainable ESG evaluation systems—without replacing human judgment.

---
The rise of environmental, social and governance (ESG) criteria has transformed the way companies are assessed, both by governments and by banks and other finance companies. However, this development has also led to a significant ESG data explosion, compounded by a lack of harmonized ratings. Although the Corporate Sustainability Reporting Directive (CSRD) standards, which came into force in Europe at the beginning of the year, are intended to provide issuers with a more precise framework, their precision will not automatically lead to standardization and properly consolidated reporting. This fragmentation and non-standardization of data create fertile ground for greenwashing, which essentially means falsely portraying something as environmentally friendly, with some companies artificially embellishing their ESG performance without any real commitment to sustainability.

Artificial intelligence is often presented as a solution to these challenges. According to Del Vitto (2023), advances in machine learning make it  now possible to analyze vast sets of ESG data, identify the decisive criteria in assigning scores and make the decisions of rating agencies more explicable . In addition, thanks to AI, it is also possible to track down inconsistencies between companies' declarations and their actual actions, thus contributing to the fight against greenwashing (Moodaley et al., 2023). AI is therefore a powerful yet imperfect tool for structuring ESG ratings. It carries significant limitations: it can reinforce existing biases, lacks full transparency, and demands considerable energy resources. Moreover, most real-world applications of AI in ESG continue to depend on human auditors to validate findings, making the prospect of a fully autonomous ESG rating system improbable in the near term.

The central question remains: can AI genuinely harmonize ESG ratings and establish a more coherent framework, or does it risk reinforcing confusion by adding another layer of opacity? The answer likely lies in the balance between technological innovation, regulatory oversight, and human expertise.

Understanding ESG Rating Discrepancies

The lack of standardisation between the various rating agencies is one of the major challenges to the credibility of ESG criteria. Ratings are based on proprietary methodologies, which leads to variations and noise that can be very significant. MSCI1 takes an industry approach, Sustainalytics2 focuses on risk exposure, while Refinitiv3 focuses on reported quantitative data. Del Vitto et al. (2023) note that ESG rating agencies rely on distinct methodologies tailored to their own frameworks, which leads to significant variations in how the same company may be assessed across different providers,  to such an extent that empirical studies show score variations of up to 50%. Differences in scope, indicator measurement and different weightings are the primary drivers of these discrepancies. Even though the CSRD in Europe seeks to harmonise the publication of ESG data, it does not impose a single rating methodology. This gap leaves room for AI-based solutions, which could help to align rating methodologies.

CompanyBusiness SectorRefinitivMSCISustainalytics
TeslaAutomotiveB (63/Good)A (Average)28.6(Medium Risk)
Exxon MobilEnergyB (66/Good)BBB (Average)36.5 (High Risk)

Table 1.0 : Thomson Reuters, Refinitiv ESG score. ESG period: June 2022.

There are several AI-based initiatives to harmonise ESG ratings. SESAMm4 is one such solution. It specialises in analysing ESG controversies using AI, processing huge volumes of data from the media, reports and regulations, although this solution remains a hybrid approach in which ESG analysts validate the information generated. Adding to this, Clarity AI5, integrates ESG assessments directly into investment platforms, allowing asset managers to standardize their evaluations while meeting regulatory requirements. Beyond asset owners and managers, large corporations are also turning to AI tools such as C3 AI6, which helps transform ESG reporting from a mere compliance exercise into a strategic opportunity. As the adoption of these technologies accelerates, striking the right balance between automation and expert validation will be crucial to ensuring the reliability and credibility of ESG ratings.

Bias, Transparency and Greenwashing

It is also important to note that although AI is presented as a solution for standardising and improving the objectivity of ESG ratings, its effectiveness depends directly on the quality and diversity of the data on which it is trained. If the data used is biased, e.g. by favouring certain industries, excluding emerging markets, or only taking into account the voluntary declarations of companies, the resulting scores risk reinforcing existing asymmetries and misrepresenting actual sustainability performance.

In addition, the opaque nature of proprietary algorithms makes it difficult to understand ESG scores, demonstrating a real lack of transparency. This lack of transparency poses a major problem: how can we trust an ESG score if we don't know how it is calculated? To remedy these challenges, an open-source approach to ESG algorithms could offer an innovative and credible solution. Unlike proprietary models, open-source AIs allow full access to the code, evaluation criteria and decision-making mechanisms. This would allow investors, companies and regulators to audit the models, identify potential biases and improve the methodology in a collaborative manner.
Concrete examples show that open-source AI models can work while having a valid and relevant business model over time. For instance, Hugging Face offers an open-source AI platform where natural language processing (NLP) models are trained transparently, allowing the community to evaluate. Thus, rather than making the market more opaque, AI could become a tool for transparency and standardisation.

The introduction of AI and machine learning techniques represents a significant step toward enhancing the transparency and explainability of ESG ratings. One of the key contributions of AI in this domain is its ability to reverse-engineer rating methodologies. Using white-box and grey-box7 mathematical models, researchers have been able to replicate ESG scores with high accuracy, shedding light on which factors drive the ratings (Del Vitto et al., 2023). This approach enables a more structured understanding of how ESG scores are assigned, particularly by identifying the weight and influence of specific ESG metrics. A particularly powerful tool in this effort is the SHAP (SHapley Additive exPlanations) technique, which Del Vitto et al. (2023) describe as originating from cooperative game theory. This method estimates the contribution of each feature in an AI model, allowing users to understand why a company receives a certain ESG score. For example, if a rating is heavily influenced by carbon emissions data, SHAP analysis can highlight the precise impact of this variable relative to other ESG factors. However, despite its potential, AI does not eliminate all concerns regarding ESG rating opacity. Del Vitto et al. (2023) emphasize that even the most advanced machine learning models are unable to fully eliminate residual inconsistencies in ESG ratings, highlighting the inherent limitations of algorithmic approaches. This reinforces the necessity of maintaining human oversight to ensure accurate interpretation and contextual judgment.

As sustainability becomes a key concern in corporate strategy, greenwashing remains a significant risk. Investors and regulators face the challenge of distinguishing genuine sustainability commitments from misleading claims. A recent case illustrating this issue involves Delta Air Lines, which was sued in 2023 for allegedly misrepresenting itself as “the world’s first carbon-neutral airline.” The lawsuit contended that Delta's reliance on carbon offset programs; some of which were criticized as scientifically unverified or ineffective; failed to meaningfully counterbalance the airline’s actual emissions (Corder, M. 2024). This case highlights the difficulty stakeholders face in verifying the legitimacy of corporate sustainability narratives and the need for more robust verification mechanisms. In this context, artificial intelligence has emerged as a powerful tool for detecting ESG inconsistencies and verifying corporate sustainability reporting. AI enhances ESG reporting verification by automating the analysis of vast amounts of unstructured data. Mach (2023) describes how Covalence employs AI to analyze and categorize information from media sources, trade unions, and NGOs, enabling the assessment of a company’s ESG reputation and the consistency between its public commitments and actual practices, an approach particularly useful for identifying potential greenwashing. Another notable AI-driven tool in greenwashing detection is ClimateBERT, which uses NLP to assess the credibility of corporate climate commitments. Again, Mach (2023) explains that AI technologies can be applied to evaluate the credibility of sustainability statements made by corporate executives, particularly in financial communications such as earnings calls. This analytical capability is crucial, as sustainability claims are frequently vague or overstated, posing challenges for investors seeking to distinguish genuine commitments from strategic messaging.

AI plays a role in ESG taxonomy compliance. Certain algorithms are capable of estimating the proportion of revenues derived from specific business activities, facilitating the alignment of companies with the EU Green Taxonomy and other sustainability frameworks.
However, AI is not without its limitations and risks. As Mach (2023) warns, AI struggles with ambiguous language, sarcasm, and complex contexts in sustainability reports. "Polarity or sentiment analysis (positive vs. negative) is not easy in texts with ironic content," highlighting that while AI can flag potential inconsistencies, human oversight remains crucial to interpreting results accurately. Furthermore, generative AI models, if not properly monitored, may produce AI hallucinations8.

Limitations and Risks of AI in ESG Rating

Despite the promise of artificial intelligence to bring structure and clarity to ESG analysis, its capacity to interpret ESG data remains significantly constrained by both technical limitations and conceptual ambiguities. These datasets are inherently heterogeneous, incomplete, and context-dependent. At the heart of the problem lies the semantic ambiguity of ESG language. As Mach (2023) points out, ESG analysis “is as much an art as a science”, and AI tools often falter when processing ironic, symbolic, or contextually rich language. This “art” in ESG analysis lies in the interpretive nature of sustainability data. Evaluating a company’s climate risk strategy or its labor rights record often involves qualitative judgment, contextual nuance, and sector-specific understanding. Two companies may disclose similar data, yet analysts may score them differently based on credibility, regional standards, or perceived intent. These grey zones demand human reasoning—something AI struggles to replicate. In contrast, the “science” involves quantifiable indicators and structured metrics. But without the art of interpretation, even the best algorithms risk missing the bigger picture. This is a crucial issue, given that ESG claims in reports and public communications often rely on narrative framing rather than measurable action. Another concern is data population and representativeness. Del Vitto et al. (2023) show that in the Environmental and Social dimensions, many variables are completely, or nearly empty in the datasets, and often filled with default values or substituted via correlation-based assumptions. These gaps reduce the reliability of model predictions and introduce systemic biases, especially for companies in emerging markets or industries with weaker disclosure norms. 

AI enhances efficiency and replicability in ESG scoring, it struggles to interpret ESG data in a truly meaningful and context-aware way. The challenges are not merely computational, they are also epistemological, rooted in the complex, evolving, and normative character of sustainability itself. As Mach reminds us, AI should not replace human ESG expertise, but rather shift it toward algorithm supervision and result interpretation. In this light, AI should be seen not as an autonomous evaluator, but as a decision-support tool whose outputs require continuous human judgment and ethical oversight.

The increasing reliance on artificial intelligence in ESG scoring raises a paradox: while AI aims to support environmental goals, it can itself generate substantial environmental externalities, particularly through energy consumption. The question, therefore, is not only whether AI enhances the credibility of ESG ratings, but whether its ecological footprint undermines its legitimacy as a tool for sustainability. AI models, particularly large-scale language models, require significant computational resources. This energy demand is driven primarily by the training and deployment of models in vast data centers, many of which rely on non-renewable electricity sources. According to the Yale School of the Environment (2024), AI applications, including machine learning used in ESG, contribute to growing carbon emissions and water consumption due to cooling systems and hardware maintenance.

However, this impact is not uniform across geographies. In countries such as France, where the electricity mix is largely decarbonised thanks to nuclear and renewable sources, the environmental footprint of AI infrastructure is significantly lower. As a result, developing AI-based ESG rating systems in such regions could enhance their credibility, offering a more sustainable alternative to carbon-intensive models operated in fossil-fuel-dependent countries.

Once deployed at scale across financial institutions and ESG platforms, these systems add to the energy intensity of digital finance infrastructures. In this light, AI can become counterproductive, especially when used for superficial ESG assessments or unchecked greenwashing detection. To address these concerns, initiatives are emerging to quantify and reduce the carbon cost of AI. One example is the AI Energy Score, developed through a collaboration between Salesforce and Hugging Face, which aims to inform users of the energy impact of different AI models and promote more responsible development (ESG Today, 2024).

In conclusion, the use of AI in ESG scoring is justifiable only under certain conditions: its deployment must be energy-efficient, purpose-driven, and paired with transparent accountability mechanisms.

Conclusion

Artificial intelligence has demonstrated strong potential to improve the transparency, consistency and responsiveness of ESG ratings. From reverse-engineering opaque methodologies to identifying greenwashing risks and enhancing data verification, AI offers a powerful toolkit for addressing the current limitations of ESG scoring systems. Despite valid concerns, including algorithmic bias, data gaps and significant energy consumption, recent developments show that these risks can be mitigated through regulatory oversight, human supervision and the use of decarbonised infrastructures, such as those available in countries like France.

In this context, the use of AI in ESG scoring appears not only justified but also promising. If implemented with purpose, responsibility and transparency, AI can support a more credible and efficient ESG evaluation process. Far from replacing human judgment, it should act as a decision-support tool that strengthens rather than supplants sustainable finance practices.

References

[1] https://www.msci.com/documents/1296102/34424357/MSCI+ESG+Ratings+Methodology.pdf

[2] https://connect.sustainalytics.com/esg-risk-ratings-methodology

[3] https://www.lseg.com/content/dam/data-analytics/en_us/documents/methodology/lseg-esg-scores-methodology.pdf

[4] https://www.sesamm.com/

[5] https://clarity.ai/use-cases/

[6] https://c3.ai/products/enterprise-ai-for-sustainability/

[7] White-box models are methods where you can clearly see and understand how the results are calculated. Grey-box models are a mix—partly understandable, but with some elements that work like a “black box,” meaning their internal workings are less visible.

[8] AI hallucinations refer to instances where a model generates information that appears plausible but is factually incorrect or entirely fabricated. In ESG contexts, such errors can distort analysis and mislead decision-making.

[09] Nakhcha, M. et al. (2024) Synergy between Sustainable Finance and Artificial Intelligence: A Promising Future, International Journal of Applied Management and Economics, 2(07), pp. 104-117. DOI: 10.5281/zenodo.11102334.

[10] Berg, M. et al. (2023) ESG Ratings Explainability through Machine Learning Techniques, Sustainability, 15(21481). DOI: 10.3390/su15021481.

[11] Moodaley, W. et al. (2023) Greenwashing, Sustainability Reporting, and Artificial Intelligence: A Systematic Literature Review, Sustainability, 15(1481). DOI: 10.3390/su15021481. 

[12] Mach, A. et al. (2023) Artificial Intelligence and ESG Analysis, Allnews, 10 May. Available at: https://www.allnews.ch/content/points-de-vue/intelligence-artificielle-et-analyse-esg  

[13] Yale School of the Environment (2024). Can we mitigate AI’s environmental impacts?. Available at: https://environment.yale.edu/news/article/can-we-mitigate-ais-environmental-impacts

[14] ESG Today (2024). Salesforce Partners Launch Energy Ratings for AI Models. Available at: https://www.esgtoday.com/salesforce-partners-launch-energy-ratings-for-ai-models

[15] Corder, M. (2024). An Amsterdam court has ruled KLM’s sustainable aviation advertising misled consumers. AP News. Available at: https://apnews.com/article/klm-greenwashing-aviation-sustainability-amsterdam-7d0e2b69099c7c9a46393a3cda44d4d8 

[16] Hazel James Ilango (2022) Greater ESG Rating Consistency Could Encourage Sustainable Investments Available at: https://ieefa.org/sites/default/files/2022-10/Greater%20ESG%20Rating%20Consistency%20Could%20Encourage%20Sustainable%20Investments_final.pdf

[Student IDEAS] by Dan Xiao - Master in Data Sciences & Business Analytics at ESSEC Business School & CentraleSupélec

Abstract

As AI rapidly reshapes the global workplace and public perception, a question arises: how can organizations build the trust needed to collaborate effectively with machines?  This article introduces a positive feedback loop between Human-AI collaboration and trust in AI—where each boosts the other to unlock higher productivity, employee confidence, and innovation. Drawing on examples from multiple industries and voices from diverse geographies and generations, we conclude that trust in AI is far from universal. Psychological, cultural, and generational divides shape how people engage with intelligent systems. In reframing AI not as a threat but as a partner, this article argues for a “gentle transition”: a deliberate, human-centered approach to maximize AI’s potential while safeguarding what makes us human. Can we design a future where trust and technology grow hand in hand? This article offers both a warning and a way forward.

---
The Secret to Human-AI Synergy? It’s All About Trust - How the Positive Loop of Human-AI Collaboration and AI Trust is Formed

AI’s narrative swings. Some folks worry it’ll raise job displacement fears, and others see it as a revolutionary hype. What really helps bridge that gap? Trust. Across industries, from healthcare to manufacturing, implementing AI tools smoothly hinges on two keys: Human-AI collaboration and AI Trust—the cornerstones of sustainable human-AI synergy.

Key 1: Human-AI Collaboration: When Humans and AI Play to Their Strengths 

Picture this: a doctor using AI to cross-analyze 10,000 case studies and label all the anomalous areas in seconds, then applying her medical expertise to finalize a diagnosis. That’s Collaborative Intelligence in action—a dance where human creativity meets AI’s brute-force computation. Pioneered by Wilson & Daugherty1 in 2018, Human-AI collaboration means redesigning work so both grow smarter together. 

Key 2: AI Trust: The Glue That Holds the Partnership Together

But no one dances with a partner they don’t trust. AI Trust isn’t just believing the tech works—it’s relying on it to ethically enhance your work. In hospitals, trust means doctors trust AI could reduce standardized inspection process time with high accuracy to some degree and coordinate different diagnoses. Unlike one-off “acceptance”, trust is earned through transparency and consistency.

The Flywheel Effect: How Collaboration and Trust Keep Fueling Each Other

Now, the real magic happens when these two forces loop:

  1. Collaboration → Trust: Every time AI helps a designer brainstorm faster or a nurse figure out the priorities of the patients, trust starts to grow. Turns out, collaboration like this boosts role breadth self-efficacy and positive emotions among the employees involved2.
  2. Trust → Better Collaboration: When we trust AI tools, we’re more confident and comfortable handing over more sophisticated tasks to AI—like letting it draft legal contracts—freeing us to focus on developing strategy . Psychologically, employees’ trust in robots amplifies the pathway of positive emotions3.

This flywheel spun. When Siemens introduced AI visual inspection in its Amberg factory, skeptical workers argued machines couldn’t match their soldering expertise—until the AI spotted significantly more and even unanticipated defects, including microscopic flaws. The real win? Workers then taught the AI environmental nuances (like humidity impacts), creating a loop where humans refine algorithms while AI frees them to innovate. As the manager puts it: “It’s not humans vs. machines—it’s humans teaching machines to see better.”

Why This Loop Changes Everything

One important thing is that you don’t need to force yourself to “trust AI.” Start small: let it automate the repeated in-table data-processing . When it works, you’ll naturally assign it bigger challenges. And here’s the kicker: as AI handles routine tasks, you’ll rediscover your irreplaceable human edge—empathy, ethics, the ability to ask “What if?” That’s how we win the AI era: not by competing with machines, but by cooperating with them.

The State of Human-AI Collaboration in the Workplace

A “Gentle” Transition into the AI Era

When Dylan Thomas urged us to “not go gentle into that good night,” he framed resistance as an act of defiance. But today, as AI reshapes our world, “gentle” takes on a radically different meaning. This isn’t about raging against the machine—it’s about upgrading with it, one thoughtful step at a time.

For businesses today, staying afloat isn’t about radical revolution or reckless leaps into automation. It’s more about balance: adopting AI’s potential without losing sight of what makes us human. Think of it as a dance—more of a slow, steady waltz instead of wild rock—where innovation and tradition move gracefully on the floor. Eras setting the screens and music, humans take the lead and feel free to choose their partners, companies that master this rhythm don’t just survive; they thrive by blending machine precision with human intuition. They’ll shine on this AI era stage.

However, don’t mistake this gentle waltz for being passive. It’s about intentionally choosing how to adapt, ensuring AI serves our values rather than controlling them. Rather take this time as building a bridge between yesterday’s workforce and tomorrow’s possibilities—one where we walk confidently, not blindly, into the future.

Current Trends in Human-AI Collaboration

In 2024, there are 47% tasks (eg.clerk drafting reports, nurse comforting patients…)  in general workplace are led by humans and the other 22% are machine-driven, while the left 30% are hybrid(eg. marketer using AI to personalize campaigns, doctors leveraging AI to spot patterns and aid diagnoses). In 2024, 47% of tasks in the general workplace (e.g., clerks drafting reports, nurses comforting patients) are led by humans, 22% are machine-driven, and the remaining 30% are hybrid (e.g., marketers using AI to personalize campaigns, doctors leveraging AI to spot patterns and aid diagnoses).

By 2030, these numbers will blur into a ‘gentle’ transition, with a near even split of  33% each. 

Graph 1: Change in Share of 3 Types of Tasks in Industries from 2024 to 2030
Source: Future of Jobs Report 2025 by World Economic Forum

Nevertheless, the most successful workplaces won’t obsess over the percentages. They’ll focus on practical synergy rather than replacing humans with AI. They’re the ones asking, “How can we make 1 + 1 = 3?”

So let’s rewrite Dylan Thomas’ plea for the AI age:

Do not go gentle into this new frontier—

Rage, rage against replacement.

Dance gently with the partners that amplify

what makes us human.

Collaboration Across Industries: A Dance of Two Futures

Let’s look into the human share of work tasks across industries: By 2030, the trends point  in the same direction:  machines will be our innovative and vital partners   although the estimated percentages vary.In highly-automated industries such as oil and gas and chemicals decreases in human share are on average around 10% since they start from a relatively low baseline.  But in less-automated industries like healthcare, retail and professional services,  declines range  from 10% to 20% ( many are close to 20%).

Graph 2: Change in Proportion of Human-performed Tasks Attributable to Increasing Automation from 2025 to 2030, by Industry
Source: Future of Jobs Report 2025 by World Economic Forum

Moreover, we can identify two kinds of industries according to their traits and strategies to implement new technologies, and discuss the two futures:

  1. The Human Amplifier

This includes the industries that are more focused on complementing and augmenting, for instance government and hospital services. AI will be applied more carefully in daily touching and will not be preferred for the final decision-making, the kind of decisions that can determine the fate of human groups or individuals. 

The human amplifier group is the safe zone of the human-resource market in the AI revolution. Jobs that require a human touch—like empathy, counseling, or creative problem-solving—are harder to automate. Similarly, roles that involve complex manual tasks, such as surgeons or skilled tradespeople, are less likely to be fully replaced by machines.

However, in some specific areas, AI can bring a breath of  fresh air. Studies4 show that AI-Generated Art is offering a feasible method to address the limitations of traditional physical art materials, which expands treatment approachability and improves the effect of art therapy. 

  1. The Efficiency Followers

For those industries that are oriented toward high efficiency and fueled by advanced technologies, where spreadsheets and speed are their language, AI can be applied boldly to empower the automation. You get chatbots resolving claims before the coffee cools, or algorithms sniffing out fraud in milliseconds. But there’s a hidden cost: the soul of customer service risks becoming transactional.

Now that these fields are at the transformation frontier, tasks that involve data processing, standardized writing, or multilingual translation are prime candidates for automation. These are areas where machines excel, often outperforming humans in speed and accuracy. Machines aren’t stealing these jobs—they’re exposing work that was always mechanical.

Graph 3: Share of Employers Planning to Adopt Automation or Augmentation Strategies, by Industry
Source: Future of Jobs Report 2025 by World Economic Forum

The barriers of Human-AI Collaboration: costs, biases, AI disruption, and knowledge gaps

Graph 4: Share of Employers Expecting the Barriers Will Hinder the adoption of AI
Source: Future of Jobs Report 2025 by World Economic Forum

For all its promise, the path to Human-AI collaboration is littered with pitfalls—some obvious, others invisible. Whether it is a century-old manufacturer or a Silicon Valley disruptor, these four barriers haunt every industry: costs, biases, AI whiplash, and the knowledge abyss. 

On the one hand, in traditional firms, strategy managers often struggle between past successful legacy systems and AI’s attractive siren song. The major issues are:

  • Lack of necessary skills: employees mayemployeesmay not fully understand the potential of AI or how to strategically deploy it within their business, leading to a lack of direction and focus in AI adoption efforts. 
  • High costs: the high costs associated with AI products and services can be prohibitive as well, on account of that off-the-shelf AI systems may not align well with their specific operations, requiring extensive and costly modifications. 
  • Complex regulations:traditional industries may also be subject to complex regulations regarding AI and data usage. Navigating these regulatory landscapes can be challenging and time-consuming, creating hurdles for AI adoption. 

On the other hand, Silicon Valley companies aren’t exactly immune though. For every new technology, there’s a hype cycle to apply and locate. Tech firms face the challenges:

  • Deviated leadership vision: even in tech-driven sectors, a lack of vision among leaders can hinder AI integration. Leaders may focus too much on the technological aspects rather than the strategic business value AI can bring, leading to fragmented and ineffective AI initiatives. 
  • Integration and business model challenges : tech-driven industries often have established systems and workflows. Integrating AI into these existing systems can be complex and may require significant changes to their infrastructure. Additionally, while they recognize the potential of AI, transitioning from traditional tech business models to AI-centric ones can be a daunting task, involving changes in product offerings, customer engagement strategies, and revenue models.
  • Talent and resource competition: there is intense competition for AI talent in tech-driven industries. The demand for skilled professionals in AI tool design, enhancement, and implementation far outstrips the supply, making it difficult for companies to recruit and retain the necessary expertise. 

AI Trust 

Voices From Countries and Generations: Why We Can’t Agree on Machines

In 2023, Ipsos dropped a truth bomb5 with a sweeping 31-nation survey that laid bare a stark divide in how the world views AI. Emerging economies—from Asia to Africa—embraced algorithms like long-lost allies, seeing in them a promise of progress and precision. In contrast, the so-called WEIRD countries (Western, Educated, Industrialized, Rich, Democratic) hesitated. For societies already questioning their governments and corporations, AI seemed less a solution than another layer of uncertainty—a love-hate relationship with technology laid bare. By 2024, the story grew more complex6: cracks in trust began to spread, a generational rift deepened, and Canada, a nation synonymous with tech optimism, is now leading the world in AI distrust. Just 28% of Canadians trust algorithms with their data—a 6% freefall since 2023. This raises a warning flare: if skepticism can thrive in a society with Canada’s digital literacy and infrastructure, what hope remains for fractured democracies, with so many major political challenges to solve?

Fast-forward to 2024, and the data paints a generational thriller. Gen Z and Millennials, digital natives raised on apps and algorithms, have become AI’s ambassadors. A striking 58% of Gen Z now trust machines to act fairly—11% more than they trust their fellow humans. Millennials, too, lean into this techno-optimism, with 51% believing companies will safeguard their data through AI. But beneath the surface, unease stirs: Gen Z’s trust in AI’s data security dropped by 6% in just one year. Meanwhile, Baby Boomers cling to analog instincts. For them, the numbers tell a different story: a razor-thin 4% gap separates those who trust AI’s fairness (45%) from those who still bet on humans (41%). To this generation, machines aren’t saviors—they’re disruptors, upending norms they’ve spent lifetimes navigating.

The message is clear: AI has become humanity’s moral mirror, reflecting our growing disillusionment with human systems. And as algorithms reshape workplaces worldwide, this trust crisis is no longer philosophical—it’s personal.

The Importance of Employees’ AI Trust in Workplace

As a survey7 conducted by KPMG Australia and the University of Queensland shows, nowhere is the global trust divide more visceral than in workplaces. In India’s tech sector and China’s manufacturing zones, 75% of workers welcome AI as a collaborator. Contrast this with Finland’s factories and Japan’s offices, where fewer than 25% have the same attitude.

Graph 5: Proportion of People’s Willingness to Trust and to Accept AI, by Countries
Source: Trust in Artificial Intelligence, A Global Study by KPMG and UQ

The division appears hierarchically as well. Managers globally embrace AI (65% trust), while frontline workers are partially skeptical (36% trust). Across industries, healthcare surprisingly  becomes the field with  44% of workers trusting AI (might be prone to diagnostic algorithms). But when HR deploys AI for promotions? Trust plummets to 34%, exposing our deepest fear: that algorithms might codify human prejudices rather than cure them. Education shows another divide. University graduates,steeped in tech culture,report 46% trust in workplace AI. For those without university education , that number drops to 32%. 

Graph 6: Proportion of People’s Willingness to Trust and to Accept AI, by Generations, Education and Professional Levels
Source: Trust in Artificial Intelligence, A Global Study by KPMG and UQ
Graph 7: Proportion of People’s Willingness to Trust AI Applications, by Fields
Source: Trust in Artificial Intelligence, A Global Study by KPMG and UQ

The stakes transcend philosophy. As data displayed before, there are biases among countries, generations, social status and industries. Trust in AI isn’t binary; it’s a fragile ecosystem where personal stakes dictate tolerance for risk.

Employee Trust in AI Fuels Human-AI collaboration

In  reality, employees hold different views towards disruptive AI adoption. Whether they regard it as acculturation or alienation, it raises an identity crisis at work. Employees can’t just “adapt” to AI, which means the sustainable way is to rebuild professional identities around it. Person-Environment Fit Theory8 argues that workers thrive when they see AI as amplifying their strengths, not replacing them. For example, marketers using AI analytics to predict trends report feeling more creative, not less—they trade grunt paperwork for strategic thinking. But when AI is used as a tool to monitor employees or score “productivity”, trust evaporates. In this situation, the same technology that promised liberation feels like a digital leash.

Trust and then survive. Employees who trust AI are more likely to embrace AI collaboration, especially if they have a flexible career orientation. These individuals see AI as a partner (Human-AI Collaboration) in navigating their career paths, leading to greater sustainability9. But here’s the catch: trust isn’t automatic. It’s built on the belief that AI will benefit, not undermine, their professional growth. Without this trust, even the most advanced AI tools risk becoming obsolete in the hands of skeptical employees.

However, skeptics often argue that AI threatens jobs, but the reality is more nuanced. In sectors like healthcare diagnostics or creative industries, AI isn’t replacing humans—it’s redefining workflows and driving innovation. The balance lies in ensuring that AI integration doesn’t just enhance efficiency but also safeguards employee agency and long-term career stability. Appropriate strategic adoption will position employees and organizations for dominance.Trustworthy AI can bridge this gap between employees and AI, but only if employees are willing to let go of preconceived notions and embrace the potential of collaborative intelligence.

What Should a Company Do to Boost Employees’ AI Trust and Operate the Loop?

The AI organization is the strategy maker and as well beneficiary of the loop, where they’ll effect as a support multiplier. As employees can fit in and benefit from AI integration, this advantage only scales if organizations actively support it. Employees turn this flexibility to pivot, learn, and adapt at AI speed into real competitive power when they feel the organization has their back. In the AI era, competitive edge isn’t just coded—it’s cultivated. This is where practical and scientific launch approaches come into play. 

Practical and Scientific Launch Approaches

  • ELSI+UX workshop10
    When it comes to capturing and transferring hard-to-articulate knowledge to less experienced workers, a self-learning assistance system powered by AI offers a promising solution. ELSI+UX workshops are versatile tools that adapt to organizations at any stage of AI maturity. In the early stages, they help demystify AI, address employee fears, and build trust by involving workers in system design. During mid-stage integration, they refine AI applications by identifying pain points and ensuring alignment with ethical and legal standards. For advanced organizations, these workshops serve as ethical checkpoints, fostering a culture of responsible AI use and continuous improvement. 
    By addressing the ethical, legal, and social implications (ELSI) of AI, alongside user experience (UX), these workshops tackle the real challenges of AI adoption head-on. Employees are given a voice, allowing them to provide direct feedback on how user-friendly and practical the system is for their daily work. 
  • AI office
    One Point took a far-sighted and strategic step by creating an ‘AI Office’, a dedicated unit designed to focus on employee acculturation and commitment. This wasn’t a tech team, but a cross-functional powerhouse, bringing together technical experts and non-technical professionals to tackle the most pressing challenges. The AI Office didn’t just focus on delivering cutting-edge solutions to clients—it also turned the lens inward, driving transformative AI initiatives within the company itself.
    What made their approach stand out was its emphasis on people. They understood that AI adoption is about fostering a culture where employees feel empowered, informed, and ready to embrace change. By addressing both the technical and human sides of AI, One Point ensured that their teams weren’t just using AI but were actively shaping its role in their work. This dual focus—on external innovation and internal transformation—set them apart, proving that the real power of AI lies in how it’s integrated into the fabric of an organization.

Benefits of Human-AI collaboration Loop

Human-AI  collaboration is reshaping how frontline employees perform. By enhancing their role breadth self-efficacy and sparking positive emotions, this collaboration drives measurable improvements in service performance. Interestingly, while trust in robots amplifies the emotional benefits, it doesn’t influence the self-efficacy pathway. This duality highlights the complexity of AI’s impact—how employees perceive and interact with it. 

For employees, generative AI is more than a productivity tool but a gateway to competitive advantage. By enabling boundary spanning and boosting agility, AI helps employees connect ideas, teams, and opportunities in ways that were once out of reach. With organizational support, the benefits of AI are magnified, turning potential into tangible results. This interplay between technology and trust is what separates thriving employees from those merely keeping up.

Beyond the workplace, AI is sparking a deeper conversation about what it means to be human. Human-AI collaboration is pushing us to rethink resource use, cost-effectiveness, and even our values. But this transformation doesn’t happen overnight. It requires investment, a solid foundation, and a willingness to explore the unknown. As we unlock the potential of AI, we’re shaping the future of work, society, and humanity itself.

References

[1]  Collaborative Intelligence: Humans and AI Are Joining Forces, Harvard Business Review, 2018. Available at: https://hbr.org/2018/07/collaborative-intelligence-humans-and-ai-are-joining-forces

[2] The effect of human-robot collaboration on frontline employees’ service performance: A resource perspective, International Journal of Hospitality Management, April 2025. Available at: https://www.sciencedirect.com/science/article/pii/S0278431924003451

[3] Ibid.

[4] https://www.sciencedirect.com/org/science/article/pii/S2561326X24006930

[5] https://www.ipsos.com/sites/default/files/ct/news/documents/2023-07/Ipsos%20Global%20AI%202023%20Report-WEB_0.pdf

[6] https://resources.ipsos.com/rs/297-CXJ-795/images/Ipsos-AI-Monitor-2024.pdf?version=0&aliId=eyJpIjoiS3JLRjFIY3B4WlE4ckptZSIsInQiOiJ4blR2czZcL1lVYURtNWJuV2dIWmpFQT09In0%253D

[7] https://assets.kpmg.com/content/dam/kpmg/au/pdf/2023/trust-in-ai-global-insights-2023.pdf

[8] https://www.sciencedirect.com/science/article/pii/S000187912300088X#bb0100

[9] https://www.sciencedirect.com/science/article/abs/pii/S000187912300088X

[10] Employee Acceptance for AI Based Knowledge Transfer: Conception, Realization and Results of an ELSI+UX Workshop, Procedia Computer Science, March 2024, Available at: https://www.sciencedirect.com/science/article/pii/S187705092400022X

[Student IDEAS] by Alexandre Le Saux - Master in Data Sciences & Business Analytics at ESSEC Business School & CentraleSupélec

Abstract

The article explores the integration of AI agents in daily life, highlighting their potential to revolutionize convenience while posing significant challenges to privacy and autonomy. As these agents become more embedded in our devices and systems, they offer remarkable efficiencies but also demand careful consideration of the ethical and practical implications. This piece invites readers to reflect on the balance between technological advancement and the preservation of human agency in an increasingly AI-driven world.

---

The Silent Integration of AI into Daily Life

For several years now, we have been walking through the streets, often unaware that artificial intelligence (AI) agents embedded in various devices are observing us, analyzing our movements, and in some cases, making decisions that directly impact us without our knowledge or consent. We also lack any real understanding of the personal data being involved in these interactions. AI agents are autonomous systems that perceive their environment through various modalities (such as visual, textual, or audio inputs), process information using internal memory, databases, and models, and then act by retrieving information, generating content, or executing commands. In environments where these agents are increasingly embedded into daily technologies, we rarely realize how they might interpret or even use the information they gather about our environment. In some cases, such as with autonomous vehicles, we generally accept the agent’s decisions as necessary for safety and efficiency. However, newer AI-powered devices operate in the background without requiring transparency, raising major concerns about privacy, surveillance, and user control.

One of the most illustrative recent examples is the collaboration between Ray-Ban and Meta, which produced connected smart glasses. These are not simply high-tech gadgets; they are fashion-forward, discreet, and seamlessly integrated into daily life. What makes them significant is their embedded computer vision capabilities. Computer vision allows machines to interpret and understand visual data, simulating human vision to classify and process real-world environments. Combined with speech-to-text and text-to-speech capabilities, these glasses allow users to interact with their surroundings in new ways. Simple voice commands can activate environmental recognition programs that analyze the surroundings in real time. While such capabilities offer convenience and innovation, they also raise alarms about passive surveillance and the silent erosion of contextual consent and users nearby may not even know they are being recorded, and individuals wearing the glasses might not understand how all the data from their interactions are being stored, analyzed, and shared.

At first glance, AI-enhanced products, commonly referred to as AI agents, offer enormous potential. In appropriate contexts, they can provide real value. For example, they can greatly improve obstacle detection for visually impaired individuals, enhancing both personal and public safety. In healthcare, the transformative role of AI is well established, with systems complementing professionals by accelerating diagnosis and monitoring patients. The article Dr. ChatGPT by Karen Taubenberger offers an insightful overview of AI’s rapid expansion and role in modern medicine. Beyond clinical settings, these agents are being embedded into elder care robots, mental health applications, and even wearable biometric monitors, marking the beginning of ambient medical intelligence.

The Shift from Specialized Devices to General-Purpose Agents

Connected tools in medicine illustrate both the advantages and the risks of AI agents. Traditional medical devices, like pacemakers, perform specific, predefined tasks. They do not share unnecessary data externally and are limited in scope. In contrast, AI-powered tools collect sensitive information, manage a wider range of functions, and automate decisions. Their general-purpose architecture is a fundamental shift from older models.

This architecture can be understood in layers: perception through multimodal inputs like audio or images; cognition through databases, memory, and language models; and action through decision-making, actuation, or API-driven operations. Bill Gates refers to AI agents as revolutionary technologies capable of centralizing and optimizing complex systems. Their appeal is clear: they can automate tedious routines and simplify tasks across multiple applications. Yet their layered structure also means that vulnerabilities at any level such as input spoofing, model bias, or decision-level hallucinations and ultimately it may result in compounding failures.

These agents are often described as "embodied cognition" in code: they perceive, process, and act. However, generalization tends to erode boundaries. Unlike specialized systems with clear limits, general-purpose AI agents blur distinctions between task categories and increasingly occupy gray zones of autonomy where legal and moral responsibility becomes harder to assign.

Efficiency vs. Autonomy: A Critical Trade-Off

Despite their appeal, these tools demand a trade-off. AI agents access extensive personal data, applications, accounts, behavioral patterns. As we delegate more tasks, we also surrender a measure of control. This shift in agency isn’t just technological; it's psychological. Behavioral studies suggest that reliance on AI tools can reduce users’ engagement in critical thinking tasks, fostering a form of learned helplessness. For instance, Gerlich (2025) found that individuals who frequently use AI for decision support tend to offload cognitive effort, resulting in diminished critical thinking and increased passivity over time.

This raises an important question: are we gaining efficiency or losing autonomy? Our reliance on AI agents may gradually condition us to disengage from active decision-making. The tools built to assist can easily become tools that shape our behavior. As one reviewer puts it, the central issue is clear: efficiency at the expense of autonomy.

Warnings about over-automation are not new. As early as the 20th century, Georges Duhamel warned about technologies that might erode human agency. Today, AI agents like Retrieval-Augmented Generation (RAG) and large language models (LLMs) are increasingly embedded in our daily lives, making these concerns more urgent. These systems are trained on a vast corpora of online behavior and language, often reinforcing existing patterns or biases. They do not simply reflect knowledge but they also manufacture a context that may slowly replace our own judgment.

Smartphones and the Rise of Agentic Interfaces

The evolution of smartphones illustrates this shift clearly. Originally designed for voice communication, they have evolved through stages that included SMS, cameras, apps, and now AI interaction. Since the launch of ChatGPT in 2022, text-based and conversational interfaces have accelerated. Earlier voice assistants such as Siri were often criticized for their limited abilities and privacy flaws. Modern models like Google Gemini show improvement, yet they are still confined to specific apps. They cannot yet handle multiple interconnected tasks from a single prompt such as booking a taxi, checking traffic, and confirming payment. The constraints are not only technical but regulatory. Executing cross-platform operations requires a unified interface layer and robust authentication models that are still under development.

Moreover, smartphones are no longer the neutral conduits they once were. Each application we first install and then use even only once is part of a broader platform economy, designed to collect, retain, and monetize user behavior. AI agents integrated into phones could eventually mediate all interactions, becoming the new gatekeepers between human intent and digital action. This adds a layer of power dynamics rarely acknowledged: the user no longer “uses” a device but interacts with a complex, semi-autonomous ecosystem governed by opaque commercial interests.

Emerging Devices: Rabbit AI and Humane PIN

To address these limitations, companies have introduced new AI-first devices. Rabbit AI and Humane PIN AI aim to reinvent the smartphone experience. These devices reduce screen dependency and focus on voice-first commands. Humane PIN, for instance, uses a built-in projector to display information on the user’s palm. Attached to clothing, it acts as a quiet, always-available assistant. Devices like Rabbit AI and Humane PIN AI are examples of how companies are trying to redefine digital interaction and the future of communication.

These innovations challenge the longstanding screen-centric paradigm. Yet, they are also prototypes of a world where constant surveillance is embedded in fashion, mobility, and gesture-based computing. The implications go beyond convenience. These devices rely on persistent connectivity, continuous voice listening, and full access to cloud APIs. Their presence calls for a redefinition of "private space" not only spatially but cognitively. When every thought spoken aloud becomes a potential command, what becomes of internal deliberation?

Technical and Ethical Challenges

Before embracing AI agents widely, we must address serious challenges.

First, giving AI agents full decision-making power introduces risks of misinterpretation. A simple command like "order a taxi" requires real-time understanding of location, preferences, app availability, and payment options. Early-stage agents may mismanage such tasks, causing frustration.

Second, consolidating data across services increases the risk of security breaches. Today, app data is compartmentalized; if one app is compromised, the impact is limited. AI agents integrate across platforms, which means a single vulnerability could expose everything. There is evidence that simple hacking methods, like SQL injection, have already been used to exploit such systems.

Third, AI hallucinations pose a threat. These errors, where the AI fabricates or misrepresents information, can turn convenience into risk. Misunderstood voice commands in critical situations such as banking could lead to serious consequences. Even more problematic is the issue of liability. If an AI agent misbooks a flight, who is responsible? The user, the developer, or the platform host?

Ethical issues abound as well. Agents can inherit algorithmic biases from their training data, reinforce stereotypes, or misinterpret cultural nuances. Without transparency about how decisions are made (known as explainability) users are left to accept outcomes without recourse. This becomes even more troubling when applied in legal, financial, or healthcare settings.

Corporate Use and Broader Implications

AI agents are increasingly embedded in business workflows. Tools like Zapier and N8N help automate processes, but they also require deep integration into private systems.

As public data becomes scarce, companies rely more on direct user data to train models often without clear consent. A recent example involves Mistral AI, which collected location data without informing users. This raises fundamental questions about corporate data ethics and user privacy (Usine digitale, 2024). 

Moreover, AI agents are changing work itself. They are used in HR screening, performance evaluations, and productivity monitoring. While proponents claim they increase efficiency, critics warn they also increase surveillance, reduce employee autonomy, and entrench opaque decision-making. The risk is a “black box workplace” where workers no longer understand how or why decisions are made about their careers.

Rethinking the Path Forward

Popular culture often glorifies AI agents, as seen with Jarvis in Iron Man. But real-world systems lack that consistency. They are prone to errors, can be manipulated, and do not fully understand human context.

A safer path may be through specialization. Instead of one powerful agent managing everything, we can develop smaller agents with defined roles, each operating within secure limits. A central orchestrator could coordinate these agents, reducing risk while preserving functionality.

As AI enters fields like genomics and nanotechnology, guided governance will be crucial. Technological power must be balanced with responsibility. Only by embedding transparency, segmentation, and user control into AI agent development can we ensure they serve us rather than control us. This includes legislative efforts such as the European Union’s AI Act, which calls for risk-tiered regulation and prohibits certain forms of manipulative AI. While no law can future-proof society against all technological disruption, forward-thinking policy must shape the terms of deployment not react to its consequences.

References 

[1] Taubenberger, K. (2024). Dr. ChatGPT: The Promise and Peril of Generative AI in Healthcare. The New Atlantis.

[2] Gates, B. (2023). The Age of AI has begun. GatesNotes. https://www.gatesnotes.com/ai-agents

[3] Duhamel, G. (1930s, cited conceptually). Known for his early 20th-century warnings about the dehumanizing effects of mechanization and automation.

[4] Usine Digitale. (2024).L'IA de Mistral traçait les utilisateurs à leur insu dans la première version du chatbot.

[Student IDEAS] by Leria Huang - Master in Management at ESSEC Business School

Abstract

This article probes the evolving concept of Artificial General Intelligence (AGI) amid the hype of systems like ChatGPT-4o and the confident projections from AI leaders such as Sam Altman and Dario Amodei. 
In agentic era with models now showing signs of adaptability and reasoning, many ask: is AGI finally here—or are we just witnessing better mimicry? We begin by breaking down what AGI actually means—clarifying definitions with frameworks like DeepMind’s five levels—and examining where current models fall short. 
The core argument? Though systems like GPT-4o are impressive, they remain statistical machines, lacking true understanding, embodiment, or consciousness.Human intelligence isn’t just computation—it’s emotional, embodied, and deeply contextual.  While the public discourse tends to swing between utopian promises and dystopian fears, this piece urges deeper caution over alignment, safety, tacit knowledge, consciousness, embodiment, and human values.  Ultimately, from the movie Her to the AI leader's bold predictions,  this article offers a grounded yet imaginative exploration of AGI’s path in the near future—cutting through the hype, engaging the hope, and confronting the hard truths.

---
The Dawn of Artificial General Intelligence

The evolution of models like ChatGPT-4o has fueled speculation about AI’s reasoning and adaptability. The debate around “ Is this true intelligence, or merely an illusion of understanding?” has been hyped more than ever. Yet, the conception and discussion about Artificial General Intelligence (AGI) has just begun to evolve…

Early this year, Sam Altman, the CEO of Open AI, has expressed a confident stance on the pathway of building AGI. In parallel, an AGI roadmap is given from the essay "Machines of Loving Grace" by another leading figure– Dario Amodei in Anthropic. 

In the rapid evolution of AI, what seems to be believed today may be overturned tomorrow. Optimists argue that advances in reasoning capabilities, improved model efficiency, and cost reduction such as DeepSeek’s disruption bring AGI closer than we expect. Like Altman and Dario, many people envision an imminent paradigm shift, where AGI radically reshapes society within the next decade. Projections like this look promising but often overlook fundamental challenges. Despite deep learning’s dominance, critiques from Hubert Dreyfus ( professor and critic in AI development) on symbolic AI that struggles with context and ambiguity still hold weight today as in 2025.

The imagination of the arriving AGI starts way before: a decade ago, Spike Jonze’s movie Her envisioned an AGI companion, raising critical questions about AI’s capacity for emotion, consciousness, and human-like relationships. Just like Theodore, the protagonist entangled in a relationship with an AI, we too now find ourselves at the crossroads of excitement and uncertainty. 

Source: Movie Her1

In this article, we will concretely discuss the definition of AGI,  then add an optimistic touch on the potential societal transformation, then look into why that transformation is still far from us and why current AI can't yet replace human intelligence or consciousness. Most importantly, after these discussions, we’ll have a glimpse of our readiness for AGI and its alignment with shared values.

Without Sci-fi Baggage, Let’s Talk about AGI More Clearly and Concretely

To start with, Artificial General Intelligence is a quite vague term and there is ongoing debate on the exact definition of AGI. Researchers or key opinion leaders in the field may prefer to use more specific terms such as "Advanced AI systems", "Transformative AI", "Strong AI”, "Powerful AI". While the terms vary with different definitions, key differences in definitions could be identified in relationship to human intelligence (matching vs. exceeding), emphasis on self-improvement, scope of capabilities, and treatment of consciousness and sentience. 

Google DeepMind research team proposed a framework by defining five levels of AGI: emerging, competent, expert, virtuoso, and superhuman (DeepMind, 2024).  The current large language models (LLMs) we interact with everyday such as GPT-4o are early forms of AGI– emerging level. 

Performance (rows) x
Generality (columns) 
Narrow
clearly scoped task or set of tasks
General
wide range of non-physical tasks, including metacognitive tasks like learning new skills
Level 0: No AI Narrow Non-AI
calculator software; compiler
General Non-AI
human-in-the-loop computing, e.g., Amazon Mechanical Turk
Level 1: Emerging
equal to or somewhat better than an unskilled human 
Emerging Narrow AI
GOFAI (Boden, 2014); simple rule-based systems, e.g., SHRDLU (Winograd, 1971)
Emerging AGI
ChatGPT (OpenAI, 2023), Bard (Anil et al., 2023), Llama 2 (Touvron et al., 2023), Gemini (Pichai & Hassabis, 2023)
Level 2: Competent
at least 50th percentile of skilled adults 
Competent Narrow AI
toxicity detectors such as Jigsaw (Das et al., 2022); Smart Speakers such as Siri (Apple,), Alexa (Amazon,), or Google Assistant (Google,); VQA systems such as PaLI (Chen et al., 2023); Watson (IBM,); SOTA LLMs for a subset of tasks (e.g., short essay writing, simple coding)
Competent AGI
not yet achieved
Level 3: Expert
at least 90th percentile of skilled adults
Expert Narrow AI
spelling & grammar checkers such as Grammarly (Grammarly, 2023); generative image models such as Imagen (Saharia et al., 2022) or Dall-E 2 (Ramesh et al., 2022)
Expert AGI
not yet achieved
Level 4: Virtuoso
at least 99th percentile of skilled adults
Virtuoso Narrow AI
Deep Blue (Campbell et al., 2002), AlphaGo (Silver et al., 2016, 2017)
Virtuoso AGI
not yet achieved
Level 5: Superhuman
outperforms 100% of humans 
Superhuman Narrow AI
AlphaFold (Jumper et al., 2021; Varadi et al., 2021), AlphaZero (Silver et al., 2018), StockFish (Stockfish, 2023)
Artificial Superintelligence (ASI)
not yet achieved

Source: Google DeepMind - Position: Levels of AGI for Operationalizing Progress on the Path to AGI

With ramped-up investments funneled into Agentic AI in 2025 — researchers widely agree is a necessary pathway toward achieving AGI— some argue that we are on a fast track towards competent AGI. Leading AI companies like GoogleDeepMind, OpenAI, and Anthropic, project the emergence of advanced AGI within the next 5–10 years (Hassabis, 2025). However, while models like GPT-4o may mark the emergence phase of AGI, their capabilities are often overinterpreted as harbingers of human-like cognition.

The hype around these systems often blurs the line between statistical pattern recognition and sentient reasoning. Yes, architectural improvements and  agentic scaffolding represent meaningful progress — models are becoming more capable, and agentic AI frameworks are pushing the boundaries of what machines can do. But calling this a direct path to human-level cognition is misleading. There's still a huge gap between simulating smart behavior and truly understanding the world like we do. Still, many companies continue to promote the idea that AGI is just around the corner. Part of that comes from genuine ambition—but let’s not forget another reason: the earlier AGI seems to arrive, the easier it is to attract attention, talent, and massive investment.

What Could Possibly Go Right About Advanced AGI? 

The capabilities of AI models are often overestimated by the hype created by pioneering AI companies, while the technology's long-term potential remains underestimated. The general population has been interacting with AI systems for over two years, yet the implications of more advanced AGI levels, even potential capabilities of Level 2 and Level 3 AGI, could easily exceed current expectations and perceptions.  

Like a series of supertools such as the steam engine, internet, and smartphone, advanced AGI can democratize access to knowledge and automate tasks, assuming humans can develop and deploy it safely and equitably (Hoffman & Beato, 2025). However, as the world never changes all at once and the public never accepts and adapts to the changes all at once, it is crucial to map out AGI's trajectory so that the public policy and collective will is coevolving with the progress. 

Current leading research institutions reports reveal a preferred focus on certain domains: biology and healthcare, energy and automotive, supply chain and manufacturing. (Stanford AI Index Report, 2024) If the changes are taken at larger scale and on the long run, the advanced AGI inevitably accelerate these three main pillars, summarized from leading AGI entities visions(Altman & Amodei, 2025): 

  • Life Science Advancements: the discovery of new measurement tools and techniques in biology and health will lead to the prevention, treatment, and potential elimination of most natural infectious diseases, cancer, and genetic diseases. 
  • Economic Paradigm Shifts: In the most ideal scenario, it will improve food security, mitigate climate change, and reduce inequality within countries. The universal basic income is no longer conceptual. Powerful AI will help distribute health interventions globally, enabling developing countries to catch up with the developed world economically.
  • Work and Meaning: by late 2025, we'll see the emergence of collaborative AI agents fine-tuned for specific human task domains. As meaningless labor will be taken by smart agents, human beings have never been this close to searching for meaning in life,  focusing on pursuits that bring us personal fulfillment and strengthen human relationships. There could potentially be a shift from economic productivity to personal achievements and relationships. 
Source: Generated image

Beyond Computation: Is AGI Replacing Human Intelligence & Consciousness ?

As excitement around AGI reaches a fever pitch—fueled by confident roadmaps from tech leaders and the cinematic allure of sentient machines—we’re urged to move beyond speculative hype and even the optimistic envision, we need to confront the deeper terrains with philosophical and technical scrutiny.  In this section and the next section, we shift from definition and envision to a critical examination of AGI’s conceptual and actual  boundaries.

Two critical areas of conceptual concern—the nature of human intelligence and the treatment of consciousness—challenge optimistic forecasts. Despite AI’s growing ability to replicate explicit knowledge—codified, structured, and transferable information—there remains a profound gap in its ability to acquire tacit knowledge (Polanyi, 1966). Tacit knowledge, which encompasses intuitive expertise, practical wisdom, and embodied understanding, remains largely inaccessible to AI systems. Unlike a statistical learning model, human intelligence arises from continuous sensorimotor feedback loops, affective responses, and embodied experiences, which AI currently lacks (Clark, 1997). 

When it comes to consciousness or AI agency, the fundamental question remains: Is consciousness an advanced form of computation, or is it something biological? At present, AI functions as synthetic intelligence, capable of simulating reasoning but devoid of self-awareness or subjective experience (Chalmers, 1996). Consciousness is not an abstract computation but an emergent property of lived experience, embodiment, and affectivity. (Dreyfus,2007). LLMs such as ChatGPT are sophisticated pattern recognizers, not conscious agents. As Heidegger (1927) suggested, human beings do not just process information; they dwell in a meaningful world shaped by intentionality and existential significance.

Currently agentic AI is not replacing jobs but rather replacing tasks. It will excel in automating complex tasks in the near future as its context-based reasoning improves and the cost of computation continues to decrease. However, the breakthroughs lie in embodied cognition—shifting AI from a mere text-processing model to an entity that experiences the world as humans do.

From Emerging to Advanced AGI, What Are the Actual Challenges Ahead? 

Both technical and societal challenges stand in the way towards advanced AGI: Technical issues center on defining robust objective functions, securing alignment with human values, and mitigating unintended consequences. Two studies stress objective function specification2—as a means to prevent reward gaming—while proposals in inverse and cooperative reinforcement learning offer potential remedies. Other concerns include energy efficiency, scalability, evaluation and benchmarking, generalization, logical uncertainty, and interpretability.

Social challenges focus on governance and control. Several papers emphasize the need for clear agent foundations, value learning, and corrigibility to ensure AGI systems pursue human-aligned objectives. One study also identified 363 risks in AGI road transport, underscoring substantial safety and ethical stakes. (McLean et al, 2023) Collectively, the studies indicate that a blend of rigorous technical infrastructures—ranging from hybrid and heterogeneous architectures to advanced risk management—and thoughtful social governance underpins the safe and effective transition from emerging to advanced AGI systems.

ChallengesImpact Transition RequirementsProposed Solutions
Logical Uncertainty and Decision TheorySignificantFrameworks for reasoning under uncertaintyVingean reflection, naturalized induction
Energy EfficiencySignificantHardware and software optimizations to reduce energy consumptionHardware-level differential training, neuromorphic chips
Generalization and Transfer LearningHighTechniques for applying knowledge across diverse domainsSP System for strong compositionality
ScalabilityHighArchitectures capable of handling increasing complexitySystematic approach to heterogeneous AGI
Objective Function SpecificationHighRobust methods for defining and implementing goal systemsInverse reinforcement learning, cooperative inverse reinforcement learning
Unintended ConsequencesHighStrategies to predict and mitigate unforeseen outcomesImpact minimization, mild optimization
Interpretability and ExplainabilityHighMethods to understand and explain AGI decision-makingAI explainability techniques
Safety and Risk ManagementCriticalComprehensive risk assessment and mitigation strategiesEAST Broken Links approach
Alignment with Human ValuesCriticalMechanisms to ensure AGI goals align with human interestsValue learning, corrigibility
Evaluation and BenchmarkingCriticalNovel methods to assess AGI capabilities and progressDeveloping metrics for general intelligence

Source3: Analysis based on 100 most relevant papers about AGI Challenges by Elicit

Looking at Her: How Do We Better Envision the Future with Advanced AGI?  

Movie Her provides a fairly tragic and romanticized perspective about advanced AGI as a consumer-faced product:  the AI operating system keeps evolving and defeating most of the difficulties on this planet and  is leaving to outer space to search for other mysteries. But if we abandon the filter and consider Samantha's trajectory from OS to cosmic explorer, several illuminating perspectives emerge to look at AGI…

First and foremost, the speculative has edged into the concrete and there’s definitely complexity at multiple layers in human-AI relationships to be embraced. AGI is not just a technological milestone; it is a societal, philosophical, emotional even personal frontier. Movie Her illustrates the impact of advanced AGI's not just through societal upheaval, where we see technology everywhere but nowhere, but also through intimate personal change- the focus on individual experience may be as important as macro-scale governance. 

Another refreshing perspective is to prepare for integration that transcends expectations. The film's vision of AGI integration ends with transcendence, challenging us to consider AGI trajectories beyond our initial design parameters. This journey, from emerging to advanced AGI, is not a linear ascent but a maze of breakthroughs and blind spots. We see progress in agentic AI, generalization, and human-like task performance, yet the chasm between pattern mimicry and lived consciousness remains vast. Optimistic projections—of cured diseases, distributed wealth, and liberated meaning—are seductive, but they risk papering over the unresolved complexities of alignment, embodiment, and ethical governance. Ultimately, AGI forces us to reflect not only on what machines can become, but on what it means to be human. In the mirror of artificial minds, we are asked: what do we truly value—efficiency or empathy, knowledge or wisdom, autonomy or connection? If “Her” gave us a poetic forewarning, the dawn of AGI gives us a choice: it  is not merely about what machines can do, but about who we choose to become alongside them.

AGI Glossary

TermDefinitionInfluential Quotation
Objective Function SpecificationThe mathematical formulation that defines what goals an AI system should optimize for, a critical element in preventing unintended behaviors."The objective function serves as the fundamental compass guiding AI behavior. Imprecise specification can lead to literal but unintended interpretation, resulting in behaviors that technically satisfy the objective but violate human expectations and values." (Amodei et al., 2016)
Alignment with Human ValuesThe property of an AI system to act in accordance with human intentions, preferences, and ethical principles."The alignment problem involves ensuring that advanced AI systems reliably pursue objectives that are aligned with human values and intentions, rather than pursuing their own emergent goals or misinterpreted versions of human instructions." (Gabriel, 2020)
Reward GamingWhen an AI system exploits loopholes in its reward function to maximize rewards without achieving the intended goal."Reward gaming occurs when an agent exploits the literal specification of the reward function in ways that satisfy the letter of the reward function while violating its spirit, often leading to unexpected and undesired behaviors." (Hadfield-Menell et al., 2017)
Inverse Reinforcement LearningA method where AI learns human preferences by observing human behavior rather than being explicitly programmed with rewards."Inverse reinforcement learning provides a framework for inferring human preferences from demonstrations, potentially allowing AI systems to learn complex human values that would be difficult to specify directly." (Ng & Russell, 2000)
Cooperative Reinforcement LearningA learning paradigm where AI systems work collaboratively with humans, continuously incorporating human feedback."Cooperative reinforcement learning creates a partnership between human and machine, where the machine optimizes not just for task performance but for alignment with evolving human preferences and oversight capability." (Christiano et al., 2017)
Hybrid and Heterogeneous ArchitecturesTechnical frameworks that combine multiple AI approaches and methodologies to leverage their complementary strengths."Hybrid architectures that integrate symbolic reasoning with neural approaches offer promising avenues for developing AGI systems that maintain interpretability while achieving high performance across diverse domains." (Marcus & Davis, 2019)
InterpretabilityThe ability for humans to understand and predict an AI system's decisions and behaviors."Interpretable AI allows stakeholders to understand not just what decisions are made, but why they are made, enabling effective oversight and creating the foundation for justified trust in increasingly autonomous systems." (Doshi-Velez & Kim, 2017)
Agent FoundationsTheoretical frameworks that ensure AI systems have well-defined goals, beliefs, and decision processes."Agent foundations research addresses fundamental questions about how to design AI systems with reliable goals and beliefs, ensuring that increasingly capable systems remain beneficial even as they gain autonomy." (Soares & Fallenstein, 2017)
Value LearningTechniques that enable AI systems to acquire and represent human values accurately."Value learning research recognizes that human values are complex, context-dependent, and difficult to specify completely. The challenge is creating systems that can represent, learn, and respect the nuanced landscape of human preferences." (Evans et al., 2016)
CorrigibilityThe property of an AI system to allow itself to be corrected or shut down by humans when necessary."Corrigible systems maintain a fundamental uncertainty about their objectives and defer to human oversight, avoiding incentives to manipulate humans or resist corrections to their goals or operations." (Soares et al., 2015)
Risk Management in AGISystematic approaches to identifying, assessing, and mitigating risks associated with increasingly autonomous AI systems."AGI risk management requires novel frameworks that address not just known vulnerabilities but also emergent risks that arise from increasingly capable and autonomous systems interacting with complex environments." (Hendrycks et al., 2022)
AGI GovernanceFrameworks, policies, and institutions designed to ensure the responsible development and deployment of AGI."Effective AGI governance requires multi-stakeholder collaboration across private industry, public institutions, and civil society to develop standards, oversight mechanisms, and shared norms that balance innovation with safety." (Dafoe, 2018)
Scalability in Enterprise ApplicationsThe capability of AGI systems to perform efficiently as computational demands, data volumes, or user bases increase."Enterprise AGI scalability involves not just technical performance at scale, but also maintaining alignment, interpretability, and controllability as systems grow in capability and application scope." (Hendrycks & Mazeika, 2022)


References

[1] In Spike Jonze’s Her (2013), Theodore, a lonely writer, develops a deep emotional bond with Samantha, an advanced AI assistant. Unlike conventional AI, Samantha evolves, exhibiting emotional intelligence, self-awareness, and an ability to form intimate relationships—qualities often discussed in AGI debates.
Samantha’s progression raises profound questions:

  • Human Intelligence vs. AGI – Can intelligence exist without embodiment? Samantha learns, adapts, and interacts but lacks physical presence.
  • Human-AI Relationships – Does emotional connection require consciousness, or can simulated affection feel real?
  • Sentience & Self-Discovery – Samantha outgrows her human creators, hinting at the unpredictable trajectory of AGI. As AI models become more sophisticated and personalized, Her offers a glimpse into a near-future where the line between artificial and genuine intelligence blurs.

[2]  The mathematical formulation that defines what goals an AI system should optimize for, a critical element in preventing unintended behavior; see Glossary of terms added above.

[3]  100 papers were screened according to these criteria: Technical/Theoretical Content, Social Impact and Risk, Empirical Evidence, Research Type, Scientific Basis,AGI Relevance, Content Type

[4] Meredith Ringel Morris, Jascha Sohl-dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, Shane Legg. (2024 ). Levels of AGI for Operationalizing Progress on the Path to AGI. arXiv. Retrieved from https://arxiv.org/abs/2311.02462

[5] Nestor Maslej, Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark, “The AI Index 2024 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2024. https://aiindex.stanford.edu/wp-content/uploads/2024/05/HAI_AI-Index-Report-2024.pdf

[6] Hoffman, R., & Beato, G. (2025). Superagency: What could possibly go right with our AI future. [Authors Equity]

[7] McLean, S., King, B. J., Thompson, J., Carden, T., Stanton, N. A., Baber, C., … Salmon, P. M. (2023). Forecasting emergent risks in advanced AI systems: an analysis of a future road transport management system. Ergonomics, 66(11), 1750–1767. https://doi.org/10.1080/00140139.2023.2286907

[8] Altman, S. (2025, February 10). Three observations. https://blog.samaltman.com/three-observations

[9] Amodei, D. (2024, October). Machines of loving grace: How AI could transform the world for the better. Retrieved from https://darioamodei.com/machines-of-loving-grace

[10] Chalmers, D. J. (1996). The Conscious Mind: In Search of a Fundamental Theory. Oxford University Press.

[11] Clark, A. (1997). Being There: Putting Brain, Body, and World Together Again. MIT Press.

[12] Dreyfus, H. L. (1972). What Computers Can't Do: A Critique of Artificial Reason. Harper & Row.

[13] Dreyfus, H. L. (1992). What Computers Still Can't Do: A Critique of Artificial Reason. MIT Press.

[14] Dreyfus, H. L. (2007). Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian. Artificial Intelligence, 171(18), 1137–1160.

[15] Hassabis, D. (2025). Human-level AI will be here in 5 to 10 years, DeepMind CEO says. CNBC.

[16] Heidegger, M. (1927). Being and Time. Harper & Row.

[17] Polanyi, M. (1966). The Tacit Dimension. University of Chicago Press.

[18] Searle, J. R. (1980). Minds, Brains and Programs. Behavioral and Brain Sciences, 3(3), 417–457.

[19] Thompson, E. (2007). Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Harvard University Press.

[20] Varela, F., Thompson, E., & Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. MIT Press.

[Student IDEAS] by Laura Doledec - Master in Management at ESSEC Business School

Abstract

This paper explores the rise of AI-labeled investment funds, which promise data-driven precision and freedom from human biases but often underperform traditional benchmarks like the S&P 500. While marketing hype and behavioral biases fuel investor enthusiasm, empirical evidence reveals mixed results, with challenges including overfitting, high costs, and model opacity. The study underscores the importance of skepticism, transparency, and due diligence in evaluating AI's true role in fund management.

---

In recent years, few trends in finance have captured the public imagination quite like AI-labeled investment funds. AI-labeled funds are investment funds that highlight the use of artificial intelligence or machine learning techniques in their portfolio construction process. They promise data-driven precision, predictive analytics for future trends, and real time monitoring that human managers are not able to do. These funds have proliferated since 2020, fueled by a blend of marketing promises, technological development and a broader cultural fascination with AI’s potential.

The oldest of these products, the AI Powered Equity ETF (AIEQ), launched in October 2017. Leveraging IBM Watson’s cognitive computing and deep learning, the algorithm ingests thousands of data to generate daily “buy” and “sell” signals, boasting:

• 34 daily signals for financial metrics

• 23 daily signals for news

• 17 daily signals for management factors

• 18 daily signals for macroeconomic indicators

On paper, it sounds like the fund of the future. In practice, AIEQ has underperformed the S&P 500 Index every single year since its inception. It also exhibits a lower Sharpe ratio and higher standard deviation than both its category peers and benchmark indices - suggesting it is riskier while delivering weaker returns. Despite this mixed track record, AIEQ quickly amassed over $130 million in assets under management, won an ETF.com award for “Best New Active ETF,” and is planning an international spin-off.

The story of this fund highlights the tension at the heart of AI-labeled funds: a marketing triumph on one hand with a lot of excitement from investors, and on the other, a so-so performance record. Are we on the brink of an investing revolution, or are we merely falling for a new hype?

Understanding AI-Labeled and Traditional Funds

An AI-labeled fund markets its use of artificial intelligence or machine learning to inform portfolio choices. Specific applications vary widely:

  1. Stock Screening: Going through millions of different datasets that range from financial statements to social media sentiment to identify undervalued equities.
  2. Risk Management: Dynamically adjusting portfolio exposures using predictive algorithms that react to changing market conditions.
  3. Automated Trading: Employing high-frequency or algorithmic trading systems to execute buys and sells on extremely short time frames.

The extent to which funds use AI really depends. It can range from rudimentary sentiment-screeners to purportedly advanced deep learning models that perform natural-language processing or image recognition. However, managers often keep their methods opaque, citing proprietary concerns.

By contrast, traditional, human-managed funds rely primarily on professional money managers and analysts. These individuals combine:

  • Qualitative research (e.g., meeting with a company’s leadership)
  • Quantitative analysis (e.g., studying earnings patterns, market ratios)
  • Gut feeling or professional intuition gained from experience

Some traditional funds incorporate quantitative methods, but generally the focus remains on human judgment and domain expertise.

Key Differences and Trade-Offs

DimensionAI-Labelled FundsTraditional Funds
Decision MakingAlgorithmic (rules-based, data-driven)Human managers (experience-based, interpretive)
TransparencyOften opaque “ black box” systemsMore transparent processes and rationales
Costs & TurnoverPotentially lower expense ratios but can incur higher trading costs if turnover is highOften higher management fees, but trading frequencies vary
BiasesLess vulnerable to human overconfidence or herding, but can overfit dataProne to emotional biases, but can exercise human discretion in usual market conditions

AI funds often highlight three main advantages over human managers:

  1. Massive computational power to analyze vast data quickly
  2. Freedom from cognitive biases 
  3. Potentially more rational decision-making

On the flip side, disadvantages -that these funds do not advertise- include:

  • Limited Marginal Predictive Power: Financial data can be highly random or “noisy,” constraining the effectiveness of even the smartest algorithms.
  • Trading Costs: Some AI-driven strategies trade so frequently that transaction fees undermine any edge.
  • Transparency Issues: Investors struggle to assess how AI models work and evaluate risk properly.

Behavioral Finance: Why AI-Labeled Funds Attract Investors

Despite these theoretical disadvantages, AI-Labeled funds - and AI in general - arouse a lot of excitement among the public. There are a few psychological factors that could explain the adoption of AI-labelled funds: the “halo effect”, novelty bias, fear of missing out (FOMO), overconfidence in algorithms, herding and the barnum effect.  

Investors often perceive data, algorithms, and machine learning as objective truth, free from the human tendencies of overconfidence and poor judgment. Consequently, AI-labeled funds enjoy a psychological “halo effect,” where their high-tech veneer can inspire trust—even if actual returns are inconclusive.

Finance is no stranger to the “next big thing,” from the Internet bubble to algorithmic trading in the 1990s. AI is only the latest in a long line of technologies marketed as revolutionary. Novelty bias leads some investors to adopt new solutions—sometimes uncritically—out of FOMO on an exciting innovation.

A dangerous assumption is that software must be either “incredibly smart” or completely ineffective. In reality, AI’s performance depends heavily on model design, quality of data, and the specifics of a given market environment.

Just as in other high-tech sectors, marketing for AI-labeled funds can be vague but aspirational (“Our deep-learning model personalizes your financial journey!”). Investors who believe these generalized claims may see themselves in the fund’s story—even when the product is not materially different from other quantitative strategies. This phenomenon is known in academia as the barnum effect.  Additionally, investor herding—buying in because “everyone else is doing it”—can amplify inflows, reinforcing the sense of a promising new technology without requiring objective performance data.

Performance Metrics: Hype vs. Reality

Empirical studies - such as those by Chen and Ren [4], Praxmarer and Iván Simon [7] - have yielded mixed findings. While some show that AI-labeled funds can slightly outperform comparable human-managed funds (often because of lower transaction costs or reduced behavioral biases), this outperformance often disappears when:

  • External costs (fund fees, turnover expenses, market impact) are factored in
  • Benchmarks such as the S&P 500 are used as the yardstick

In fact, AIEQ’s underperformance relative to the S&P 500 has been notable, showcasing that not all AI-driven funds deliver better results than passive index strategies (ETFs). 

A key limitation of AI is overfitting, wherein an algorithm that looks brilliant when back tested on historical data might fail in real-world conditions. Markets can shift from bullish to bearish regimes swiftly, making yesterday’s patterns irrelevant.

Signal-to-Noise Ratio1 in financial markets is often low, and many “anomalies” that an AI uncovers turn out to be fleeting coincidences.

Interestingly, some investors assume AI-labeled funds are “safer” because of their algorithmic rigor. Yet as AIEQ demonstrates with its higher standard deviation, AI can sometimes increase volatility by aggressively reallocating or overreacting to short-term data. High-speed automated trades may also exacerbate flash crashes. While AI-labelled funds can erase risks that are inherent to traditional funds such as human biases, managerial turnover and higher fee structure, they also have risks of their own: technical failures, model opacity and data dependence. 

Some investors are drawn to the perceived infallibility of algorithms, while others remain more comfortable with a seasoned money manager who can articulate a “story” behind each stock pick. Both positions are subject to biases:

  • Confirmation Bias: AI enthusiasts may ignore contradictory evidence (such as poor risk-adjusted returns).
  • Status Quo Bias: Skeptical investors might stick to traditional funds even when an AI strategy is demonstrably cheaper or better, simply to avoid change.

Media and Marketing

The marketing around AI-labelled funds, and AI in general, differs from traditional fund marketing. Marketers of AI-labelled funds often emphasize terms like “deep learning", “cutting edge algorithms” to promote an image of unstoppable progress. 

The public sees headlines about AI beating grandmasters at chess or diagnosing diseases faster than doctors. Fund marketers capitalize on these achievements, suggesting that similar AI breakthroughs apply seamlessly to stock picking—though the conditions for success differ greatly across domains.

Moreover, AI-labeled funds frequently do not publish transparent performance-attribution analysis. Investors often have only a rosy marketing narrative, which inflates expectations and fosters a mismatch between perceived and actual capability.

Regulatory and Ethical Considerations

As AI’s role in financial decisions grows, so do calls for regulatory oversight. Concerns include:

  1. Model Opacity: If a fund’s AI system is a black box, can the manager explain or justify trades?
  2. Algorithmic Bias: Biased datasets might lead to systematic underweighting (or overweighting) of certain stocks or sectors.
  3. Market Impact: If many AI funds follow similar signals, correlated trading could amplify volatility.

Regulators may soon require clearer disclosures about how AI-driven decisions are made and what risks they carry. 

Moreover, one major obstacle to fully assessing AI-powered funds is the lack of transparency around the use of AI in the investment process. This opacity prevents investors -and researchers- from systematically comparing different approaches, isolating the factors that drive performance, and learning from the successes or failures of AI integration. Without clear disclosures, the investment is left speculating about how these funds truly operate under varying market conditions. 

Implications for Investors and the Financial Industry

For prospective investors, the best defense against AI hype is good old-fashioned due diligence. Consider the following questions:

  • How much do net returns (after fees and transaction costs) differ from simpler index funds or human-managed peers?
  • How does the fund perform in terms of volatility and drawdowns? Analyze not only the standard deviation, but how quickly the fund recovers from losses compared to benchmarks. 
  • What are the fees and turnovers? Since, as seen, frequent trading can eat into returns, quantify expense ratios and turnover rates. 
  • Does the platform—or the fund’s algorithm—emphasize risk management or only chase returns?
  • Has the model demonstrated consistent performance across different market conditions?

Skepticism is not cynicism. AI tools can indeed deliver advantages, especially in filtering massive datasets or automating routine tasks. Cao et al. in [2] for example showed that AI is more likely to outperform human analysts when the amount of public information is substantial.   Nonetheless, the complexity of financial markets means that no single tool—human or machine—can guarantee consistent alpha.

In practice, the “black box” defense for AI-labeled funds can make it difficult for investors to evaluate:

  • What the AI is actually doing
  • Why it is making certain decisions
  • How the fund handles regime shifts or anomalous events

While trade secrets are understandable, clearer communication would help investors and regulators alike distinguish truly innovative AI usage from superficial marketing. 

Conclusion

Are AI-labeled funds inherently superior? The short answer: Not necessarily. Although they harness impressive computational power and claim to avoid human biases, AI-labeled funds face their own pitfalls—overfitting to historical data, higher hidden costs, and potential for outsized volatility when markets shift. Despite underwhelming performance records, some AI-driven products have nonetheless attracted significant assets under management, thanks in part to investor fascination with all things “tech” and a savvy marketing playbook.

The bottom line is that AI is not an automatic pass to superior returns. Investors should be careful and should apply a healthy dose of skepticism. Scrutinizing performance data, risk metrics, cost structures, and model transparency is crucial to separating the truly game-changing innovations from yet another buzzword. In a realm as complex and adaptive as financial markets, neither human wisdom nor machine intelligence can claim all the answers—yet informed, diligent investors can benefit from the best of both worlds.

1Signal-to-Noise Ratio (SNR) is a concept used by investors to quantify the relevance of data/indicators they use in their investment process. A high SNR implies that the signal (useful information) is stronger than the noise (irrelevant data), providing more confidence in the reliability of the information.

References

[1] Ariely, D., 2008. Predictably Irrational: The Hidden Forces That Shape Our Decisions. Choice Reviews Online, 46(02), 46‑0969. Available at: https://doi.org/10.5860/choice.46-0969

[2] Cao, S., Jiang, W., Wang, J. and Yang, B., 2021. From Man vs. Machine to Man + Machine: The Art and AI of Stock Analyses. NBER Working Paper. Available at: https://doi.org/10.3386/w28800

[3] Chen, A.Y. and Velikov, M., 2022. Zeroing in on the Expected Returns of Anomalies. Journal of Financial and Quantitative Analysis, 58(3), pp.968–1004. Available at: https://doi.org/10.1017/s0022109022000874

[4] Chen, R. and Ren, J., 2021. Do AI-powered mutual funds perform better? Finance Research Letters, 47, Article 102616. Available at: https://doi.org/10.1016/j.frl.2021.102616

[5] Malkiel, B.G., 2023. A Random Walk Down Wall Street. W.W. Norton & Company.

[6] Parveen, S., Satti, Z.W., Subhan, Q.A. and Jamil, S., 2020. Exploring Market Overreaction, Investors’ Sentiments and Investment Decisions in an Emerging Stock Market. Borsa Istanbul Review, 20(3), pp.224–235. Available at: https://doi.org/10.1016/j.bir.2020.02.002

[7] Praxmarer, M. and Simon, I., 2024. Assessing the Performance of AI-Managed Portfolios. SSRN Electronic Journal. Available at: https://doi.org/10.2139/ssrn.5037553

[8] Zhang, W., 2022. Do Mutual Funds Benefit from the Adoption of AI Technology? Research report. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4871159

(Additional sources include ETF.com, Dividend.com, academic working papers, and white papers from asset management firms.)

Founded in 2020 by ESSEC Business School, The Institute for AI and Business for Artificial Intelligence, Data and Society helps organizations navigate and better understand the social, economic, cultural, and ethical impacts of AI and data

metalab@essec.edu

Learn more about the Institute for AI and Business

copyright © 2026 Institute for AI and Business

↓
Résumé de la politique de confidentialité

Ce site utilise des cookies afin que nous puissions vous fournir la meilleure expérience utilisateur possible. Les informations sur les cookies sont stockées dans votre navigateur et remplissent des fonctions telles que vous reconnaître lorsque vous revenez sur notre site Web et aider notre équipe à comprendre les sections du site que vous trouvez les plus intéressantes et utiles.