Seizing the AI opportunity in Europe

AI’s risks for business

Generative AI models create two broad areas of legal risk for the companies that deploy them. The first relates to the expectation of errors (the so-called “black box” problem). Here, there is a likelihood that the models “hallucinate” and give incorrect responses that, in certain contexts, could lead to legal liability for tort, breach of advisory duties, consumer harm and/or regulatory violations.

Hallucinations could be a function of incorrect or out of date data in the model’s training set, inaccurate mathematical predictions based on the weighting of sources of randomisation, or historical bias in the information used to develop the model. They are also simply a product of the way the technology works. The output generated by any AI system is nothing more than a prediction, and no prediction will be 100% accurate. The models underpinning generative AI systems are no different, save that the risks are likely amplified in practice given their general-purpose nature and their wide range of potential use cases.

At the same time, the outputs of generative AI models are inconsistent and unpredictable, making it extremely difficult to ensure standards of quality and accountability are met. The same questions will produce different answers, and where AI models are deployed to deliver (or assist in delivering) financial advice for example, this can lead to variances in outcomes for consumers.

The second risk derives from the fact that AI models take human-generated content and account for it in a mathematical response. This – coupled with the fact that AI developers are incentivised to access as much data as possible to train their models – raises the possibility that someone else’s data may be used without permission or credit, which in turn creates real risks for both the developer and the AI user, and raises the possibility that the user may not be able to assert ownership over the model’s output. Crucially, the model may also automatically retain and learn from a user’s own IP. Questions in relation to data privacy and protection also arise in instances where the model has been trained using personal data (which is often the case for large language models) or where users input personal data in their prompts.

• If an AI model has been fed with illegally scraped information, not only would the AI developer be likely to infringe third party IP rights (most likely copyright) at the point of training the model, but so would the user of the AI model at the point of use. This is because AI models “memorise” their training data and there is a risk that they reproduce a substantial part of an individual copyright work in an output. These issues are currently being tested through litigation, with the likes of OpenAI, Microsoft and Stability AI (the developer of AI image generator Stable Diffusion) being sued for copyright infringement, among other things, in various actions in the UK and the U.S.
• There are various tools to mitigate these risks, from internal governance to operational controls and contract terms. Some developers have created what they refer to as “IP safe” AI by training their models only on licensed content, proprietary IP and rights-free information in the public domain, and are offering to indemnify users against any IP claims linked to content created by their tools. However whether this will become standard practice remains to be seen.
• Another risk is that in in most jurisdictions, the outputs of AI systems do not benefit from copyright protection. As a result, any user looking to protect the commercial value of the outputs of an AI model will need to consider alternative forms of protection, such as trade secrets. The test for whether something qualifies as a trade secret is both evidentiary and practical, and any trade secret strategy requires careful consideration across multiple stakeholders in a business. Who has access? How is that access controlled? What security protections and encryption protocols are deployed? Are the appropriate non-disclosure agreements in place? 
• Investors will also need to assess open-source software risk given that the model may have been trained using publicly available source code repositories such as Github. A proposed class action lawsuit has been launched against major AI developers in the U.S. alleging licence breaches, fraud, negligence, “unjust enrichment”, unfair competition and privacy violations linked to the use of open source code to train large language models (LLMs). Open source software risk is particularly important to consider where the AI is being used to generate software code, as the output from the model may reproduce parts of the open source software from the training data set which in turn can raise broader IP risks for the business.
• Appropriate governance can help reduce the IP risks once the model is in use, for example by designing controls to ensure users avoid prompting the model with an instruction to copy, or via reference to any known trademarks or individuals. Likewise, clearly labelling outputs as the products of generative AI can guard against them being deployed for purposes beyond their intended use case. Other issues to consider include whether the outputs are for public or internal consumption, and whether they can be used verbatim or “for inspiration”. 

Deploying AI: three risk management pillars

Use case+

Businesses deploying AI must articulate clearly and exactly what the model is to be used for, and, given the sweeping abilities of LLMs and the risk that they can be deployed for other purposes, implement strict governance controls to keep the system’s use within the original design. This means the use case needs to be reinforced with playbooks, training, system settings and working practices to reduce the likelihood that they are used in ways that were not intended (eg it is not enough to implement a contractual restriction designed to protect trade secrets if operational steps such as encryption aren’t also introduced).

Operational

Businesses must also implement operational measures to integrate generative AI safely into their operations. Here, legal functions need to work closely with information security and technology teams. This includes in relation to security measures, the configuration of the model, and the use of privacy enhancing technologies such as homomorphic encryption and differential privacy, a process which adds “statistical noise” to a data set so AI models can still look for patterns without breaching privacy rules. The interdependence between legal, operational and security stakeholders is higher in generative AI rollouts than for other types of IT project.

Contractual

Various terms help mitigate legal risk, both in contracts between the company deploying the AI and the model’s developer, and between the deployer and any end user (where generative AI is built into consumer-facing products or services). Businesses deploying AI systems will need to adapt their contracts with developers around sector-specific requirements and conduct pre-contract due diligence – for example exploring how the model was trained and what data was used to quantify the nature and extent of any IP infringement risk. The market continues to evolve in novel areas of contracts negotiation.