Judges use AI daily without thinking about it: spam filters, spell check, autocorrect, facial recognition, and internet searches. Generative AI (GenAI) is just the newest iteration; it is a tool that creates (‘generates’) new content in response to a prompt.

Traditional (Discriminative or Predictive) AI does not create content; it makes predictions or classifications based on its training data. GenAI creates new content, including text, images, audio, and video. For example, traditional AI can distinguish between pictures of dogs and pictures of cats. GenAI can generate a picture of a cat from a text description. More relevant to judicial practice, GenAI can distill a 20-page document into two pages of bullet points. Both traditional AI and GenAI rely on machine learning and natural language processing.

While it can be helpful to understand the mechanics underlying AI systems, it is not essential. Judges who are interested in using GenAI to facilitate their work do not have to have extensive technical expertise with the technology; they should, however, be knowledgeable about its limitations as well as how to use it safely and responsibly. When a judge is faced with an AI-related evidentiary question, it is usually sufficient to determine (through expert testimony or other proffers) that the AI evidence at issue was the product of reliable principles and methods.  

Unlike search engines, GenAI tools respond to a prompt with the most statistically likely response based on patterns learned from massive sets of training data. They are programmed to ‘satisfy’ the user by inferring a desired response based on the prompt. GenAI does not ‘understand’ search prompts or necessarily verify facts. Large Language Models (LLMs) – text-based GenAI systems – are designed to generate plausible-sounding text. They make predictions (about the next most likely element in a sequence, such as words in a sentence); they do not to tell the ‘truth.’ This is why GenAI output can be wrong but sound convincingly accurate.

Because GenAI is designed to generate content (output), because this output is based on probabilities derived from large training sets, and because it is not necessarily verified by the AI system for its accuracy, the output can be incorrect. In addition, when training data does not include a clear-cut answer to a prompt, the system often makes one up, filling gaps with statistical guesses. Incorrect or fabricated output is commonly referred to as a ‘hallucination.’ This is particularly worrisome with respect to legal citations, which can be entirely fabricated but appear authoritative, and extends to misstatements about what a legal case stands for, inaccurate quotations, and other possible errors.

Bias can enter an AI system in multiple ways. AI systems trained on biased or mislabeled data can reproduce that bias in its outputs. For example, an AI-enabled facial recognition system trained on ethnically homogeneous data may produce inaccurate results when applied in a more heterogeneous context. This is not a theoretical concern—it has already manifested in consequential ways within the justice system, including with risk assessment tools used by some courts for criminal sentencing. 

A second type of bias can enter an AI system through choices made by the developers designing it. Developers make decisions during the design process including which features and variables to use when measuring certain characteristics. These choices can influence the AI system’s output. For example, a developer may introduce bias by selecting unrepresentative data in an AI system’s dataset. In February 2024, Google was accused of over-correcting its training model to emphasize diversity when its LLM (Gemini) responded to a prompt for pictures of Nazis with an image that included Asian women and a Black man dressed in a 1943 German military uniform.  

Users of AI systems may introduce another type of bias: an individual’s cognitive biases affect how they interpret the output of AI systems. Automation bias, for example, is the tendency to assume the output from a computer is objective and reliable. Confirmation bias is the tendency to interpret output consistent with the user's beliefs. 

Understanding AI bias helps courts address questions about what data the AI system was trained on and how the AI system was developed. For example, during a sentencing hearing, the judge may want to inquire whether the sentencing report includes information derived from an AI system and, if so, ask questions about the training data used during that system’s development.

Many GenAI systems tailor their responses to reflect what they perceive as the viewpoint or predilections of the user. While the system’s output may satisfy (or even please) the user, it may not be valid or correct and, in some cases, it can be dangerous. This phenomenon is called ‘AI sycophancy.’ It happens when an AI system is trained to prioritize agreement and engagement over accuracy. There have been documented instances when AI sycophancy has led to user harm including endorsing plans to commit suicide, decisions to stop taking medication, and choices to make an inadvisable financial decision.

General-purpose AI systems* like ChatGPT, Claude, and Gemini are trained on the internet and other proprietary databases but are typically not fine-tuned for a specific subject matter or task. ‘Frontier’ models of general-purpose AI systems are large-scale, cutting-edge versions of the technology that perform a wide variety of tasks, some better than others. General-purpose AI systems can have two (or more) tiers: free and paid. Free versions have virtually no privacy and confidentiality protections. Paid versions may provide more advanced (and reliable) versions of the technology and may - but not necessarily - offer more privacy protections. Users always must carefully review the system’s terms of service and settings. 

Legal AI systems like Westlaw CoCounsel and Lexis Protégé use large language models (LLMs) trained on specialized, subject-matter content (case law, statutes, etc.). They employ tailored user interfaces to automate complex tasks and enable multiple legal applications like research, drafting, and contract analysis. Legal AI systems may combine more than one LLM to improve accuracy and performance. For example, CoCounsel uses advanced LLM models from OpenAI, Anthropic, and Google. Although legal AI systems draw from curated databases and tend to offer more reliable legal research results, they are not immune from hallucinations, can misinterpret information, and may make other types of errors. However, they do typically enable more robust privacy and security settings.

*Note: Throughout this Toolkit, the term ‘General-purpose AI system’ is used to describe non-legal generative AI systems.

Legal AI systems, such as Westlaw CoCounsel and LexisNexis Protégé, make use of Retrieval Augmented Generation (RAG) technology. Using a legal research question on Westlaw CoCounsel as an example, when a user inputs a question, CoCounsel sends the user’s question to a large language model (LLM) GenAI system to parse the user’s intent and interpret the user’s question. The LLM does not answer the user’s question at this point. Instead, it responds with a plan of action for CoCounsel’s AI system in code because computers do not understand instructions in plain language. Then, using the LLM’s suggested plan and recommended tools, CoCounsel decides which tools will best answer the user’s question, such as Westlaw, Practical Law, or other software or algorithm in Westlaw’s toolkit.  

To minimize hallucinations, CoCounsel uses a RAG approach to retrieve trusted legal data from its proprietary database. After CoCounsel identifies what it concludes are the most relevant legal materials, it sends all the materials, together with the user’s question, back to the LLM. When the LLM receives the materials and the question from CoCounsel, that is RAG technology at work—the LLM can use the information from Westlaw’s database to answer the user’s question. The LLM therefore processes the provided legal materials, attempts to answer the question, returns the answer to CoCounsel, and CoCounsel returns the answer to the user. If the materials provided by CoCounsel to the LLM contain the answer to the user’s question, the LLM is far more likely to answer the question correctly. If the materials provided by CoCounsel to the LLM do not contain the answer, however, the LLM is less likely to answer the question correctly, and the answer is more likely to contain a hallucination.

Closed-source AI systems cannot be altered, inspected, or retrained by public end users. They are usually designed and controlled by technology companies, government agencies, and institutions such as law firms. These systems are proprietary and are often referred to as ‘enterprise’ systems; their underlying models, training data, and weights are not publicly available. Examples include Westlaw CoCounsel, Lexis Protégé, Harvey, and general-purpose AI systems such as ChatGPT and Gemini. These systems are housed within an enterprise’s cloud infrastructure and may be fine-tuned with private data, such as a law firm’s internal documents. ‘Closed-source,’ in this context, does not necessarily imply that a system is safe, accurate, or compliant; it refers solely to restrictions on public access, modification, and transparency.

In an open-source AI system, model weights and other technical information are accessible and can be downloaded, inspected, retrained, fine-tuned, and deployed by anyone. These systems are typically trained on large mixtures of publicly available internet data and licensed or proprietary datasets, and they allow users to run models locally (on their own computers without an internet connection) or within closed networks. Examples include Mistral, Falcon, and Gemma. Unlike closed-source systems, open-source systems generally enable greater transparency, customization, and independent public auditing.

GenAI systems have different risk profiles in terms of privacy and security protections, as well as reliability. Both free and paid tiers of general-purpose AI systems (such as ChatGPT or Claude) may utilize user inputs to train their systems. This practice can compromise user privacy and confidentiality. General-purpose AI systems are primarily trained on large amounts of data from the internet, proprietary datasets, and rarely on curated legal datasets. These factors decrease their reliability and increase their risk profile when used for legal tasks.

By contrast, the contractual terms of legal AI systems and some enterprise versions of general-purpose AI systems can be negotiated to prohibit the inclusion of user inputs in LLM training. This mitigates (though may not eliminate) risks to confidentiality and privacy: even if user data is not used for training, contract provisions or terms of service may allow the AI platform to retain certain user inputs and outputs (e.g., metadata) for a defined period to serve the AI company’s security, auditing, or system-improvement needs.

Another feature that enhances the reliability of legal AI systems is the use of retrieval augmented generation (RAG) which improves accuracy by limiting the LLM outputs to data from curated, verified, proprietary, and up-to-date legal datasets. However even RAG-based AI may hallucinate. For this reason, the output of all types of AI systems always requires human review and validation.

GenAI can improve the structure, clarity, and speed of some judicial tasks, but it cannot perform critical judicial functions like decision making and the exercise of discretion. Every output requires human review. It is up to each judge to determine whether the efficiencies gained by using GenAI outweigh the time and effort necessary to verify output.  

A more in-depth exploration of AI can be found in the Center’s 2023 Introduction to Artificial Intelligence for Federal Judges as well as in the 2026 Federal Judicial Center’s Reference Manual on Scientific Evidence, beginning on page 1481. 

Artificial Intelligence (AI): an umbrella or generic term, like the word ‘vehicle,’ which refers to a range of technologies. The term is generally meant to reflect computers mimicking human intelligence, for example, emulating cognitive tasks like learning, reasoning, problem-solving, and understanding language. 

Agentic AI: a more advanced AI system that can operate autonomously to achieve complex goals with minimal human intervention. For example, ordinary LLMs can plan an itinerary. Agentic AI can independently implement the itinerary by booking flights and hotels. Fully Agentic AI is still in development.

AI Agent: the manifestation of an agentic AI system. AI agents use GenAI to perform tasks autonomously, often employing multi-step processes, without human intervention. Examples of AI agents include chatbots on a website that answer questions and resolve issues without human intervention, AI agents that assist software developers with coding and systems testing, and AI agents on a physician’s website that schedule appointments or facilitate renewing a prescription.

Algorithm: a set of instructions to complete a task. A recipe to bake a cake is one type of algorithm. Computer algorithms are sets of instructions given to a computer to complete specific tasks. AI uses algorithms to learn from vast amounts of data, identify patterns, and make predictions. For example, an AI system might use algorithms to review applicants to be selected for a job interview. The neutrality of these algorithms might be challenged in an employment discrimination lawsuit. 

Alignment: a field of research that studies ways to design AI systems that behave in a manner consistent with human-centered, ethical principles. AI, though developed by humans, does not have values. Alignment seeks to mitigate the potential for an AI system to produce outputs that are inaccurate, biased, or harmful. It does this by exploring methods to prevent the AI system – which seeks to optimize its goals - from taking steps that are inconsistent with ethical norms. An example of misalignment is an AI system that makes up rules or cheats to win a game against a human.  Alignment is an ongoing process that begins with system design, continues through LLM pre-training, includes different types of fine-tuning, such as reinforcement learning, and incorporates continuous monitoring.  At the present time, fully realized alignment is aspirational.

Artificial General Intelligence (AGI): a (still) theoretical advanced type of AI. Narrow (weak) AI can perform specialized tasks within a narrow scope. General (strong) AI can perform any task at least as well as a human. Super AI, often referred to as Artificial General Intelligence (AGI), is theoretical AI that can perform better than a human. There are currently debates over whether and when AGI will be achieved. 

Black-box Technology: parts of an AI system in which the mechanisms of operation are opaque and not well understood.

Context Window: the maximum amount of information, measured in parts of words (tokens), that an LLM can process, analyze, and remember at one time. It is the equivalent of the AI system’s short-term memory.

Discriminative or Predictive AI: traditional AI that categorizes data or makes predictions based on its training data. In electronic discovery, technology-assisted review (TAR) systems use discriminative AI to distinguish between relevant and non-relevant documents based on labeled training examples.

Frontier Model: the latest and most advanced versions of large GenAI systems. They often have multi-modal capabilities (e.g., text, image, audio, and video). They are capable of complex reasoning, coding, and agentic-like actions. They require massive computing resources, including electricity, to train and deploy. These models are general purpose and adaptable, but they are not always reliable. Because of their uneven abilities across tasks, they are often referred to as having a ‘jagged edge.’

Generative AI (GenAI): a subset of AI that involves training on massive data sources—primarily from the internet—to generate content in response to a user prompt. It can converse and replicate specific styles. For example, a user can train an AI model to write in their own voice. GenAI excels at creative tasks and synthesizing and summarizing complex content.

Hallucination: when LLMs generate fluent but factually incorrect, nonsensical, or fabricated information. A hallucination may occur when the model’s training was based on limited or contradictory information related to the subject matter of the prompt. 

Large Language Model (LLM): advanced AI systems trained on massive textual data sets drawn from the internet and other proprietary sources. Their purpose is to understand, generate, process, and predict language.

Machine Learning (ML): a subset of artificial intelligence that enables a system to learn and improve from experience. There are at least four types of machine learning systems that can accomplish tasks of increasing levels of sophistication:

• Supervised Learning: trained by humans using labeled training examples (e.g., this is a relevant document / this is not a relevant document). The system infers rules and patterns from the training data to make educated guesses about new unlabeled data. 
• Unsupervised Learning: uses algorithms to look for naturally occurring patterns, clusters, groupings, or anomalies in a data set. Programmers don’t instruct the system to look for a particular pattern. The system does this automatically. This technology is useful for learning about what is in a dataset.
• Reinforcement Learning: machine-learning method where the system is trained by feedback (positive or negative) in response to the system’s output. The system learns to optimize for positive behaviors. 
• Deep Learning: multiple layers of neural networks (algorithms simulating the human brain). Information from each layer is combined with information at the next layer in increasing levels of complexity. At the highest layer, there is either a decision or an action taken. Deep learning enables state-of-the-art AI applications like computer vision, natural speech recognition, and self-driving cars. The precise mechanics of deep learning are not well understood which is why it is often referred to as a black-box technology. 

Natural Language Processing (NLP): algorithms that enable computers to understand and generate human language as it is written and spoken. Examples of technologies that use NLP are predictive text like autocorrect, personal assistants (e.g., Alexa and Siri), and Google Translate. 

Retrieval Augmented Generation (RAG): a technique that connects LLMs to external, up-to-date, sources (e.g., databases or legal cases) to provide more accurate and factually grounded outputs. RAG can reduce but does not eliminate the LLM hallucination problem.