An AI ‘Deepfake’ is a term used to describe synthetic images, audio, videos, or other media that have been manipulated using AI or depict things or incidents that are fake or never happened. GenAI is being used to create very convincing deepfakes including synthetic audio of a person’s voice, realistic videos created from a single prompt, sophisticated face-swaps, and AI-generated identification and financial documents. Media can be manipulated to include fabricated metadata, including false authors, dates, and geolocations.

Humans are barely better than chance at distinguishing many of the more sophisticated fake images and audio. In addition to deepfakes being difficult to detect, psychological studies have shown that once jurors see compelling audiovisual evidence, it strongly influences their perception, memory, and decision-making process, even if they are told the evidence may be fake. This phenomenon has been called the 'continued influence effect' and underscores why challenges to potentially deepfake evidence can raise difficult questions.  

Most automated detection tools are not reliable in real-world scenarios. Deepfake detectors have high false-positive rates. Watermarks used to indicate that content is AI-generated are easily removed from synthetic content or placed on authentic content. Automated detectors created by AI companies tend to work well on the output of their own AI systems but are less reliable when applied to competitors' systems.  

However, many experts note that device-level forensics are useful in detecting deepfakes.  The interconnected evidence around the challenged file may show if it was created where, when or how the proponent claims. So, for example, one may be able to determine whether the item is out of context with other items on that source taken around the same time. Thus, the provenance of the challenged item may be an important factor in determining authenticity.

If evidence is challenged on the grounds that it is a deepfake, the court’s analysis will likely begin with Fed. R. Evid 901(a).  Rule 901(a) requires a proponent to produce sufficient evidence to support a finding that the evidence is what the proponent claims it is, and Rule 901(b) provides a non-exhaustive list of examples of how Rule 901(a) may be satisfied.  Common grounds for authentication include testimony from a witness with actual knowledge, testimony from a witness identifying a person’s voice, or the fact that the evidence was recorded into the public record.  

Some of the 901(b) options that satisfy the authentication requirement may be problematic in the deepfake context.  They include the “appearance, contents, substance, internal patterns, or other distinctive characteristics” of the evidence, Fed. R. Evid 901(b)(4), and “[a]n opinion identifying a person’s voice — whether heard firsthand or through mechanical or electronic transmission or recording — based on hearing the voice at any time under circumstances that connect it with the alleged speaker,” Fed. R. Evid. 901(b)(5). The distinctive characteristics of the subject (such as a deepfake video of a person or a person’s voice) are precisely what GenAI technology is designed to imitate.  

Rule 901 imposes a “low bar” for authentication. E.g., U.S. v. Lamm, 5 F.4th 942, 947 (8th Cir. 2021). So long as there is a rational basis in support of the proponent’s claim (and a reasonable jury could rely upon that basis), the evidence may satisfy Rule 901 and a motion to exclude under the Rule may be denied. As a result, even if there are lingering questions about authenticity and there is evidence that the proponent used deepfake technology, Rule 901 may require that the question of authenticity be submitted to the jury.

In recognition of the potential problems posed by deepfake GenAI evidence and the relatively low threshold for authentication under Rule 901, the Judicial Conference Advisory Committee on Evidence Rules has been studying the potential need for an amendment.  As part of its deliberative process, the committee has prepared a draft of a new rule—Rule 901(c)—that would require the proponent of evidence to demonstrate by a preponderance of the evidence that the evidence is authentic if the opponent of the evidence makes a prima facie showing that the evidence is a deepfake.

The Committee is continuing to study whether any amendment is needed and, if so, whether the proposed amendment to Rule 901(c) is the best way to address the problems posed by deepfakes.  The Committee will be considering these issues and hearing from a panel of experts at its next meeting, on October 15, 2026.  Suggestions may be submitted to the Committee by emailing RulesCommittee_Secretary@ao.uscourts.gov.  See https://www.uscourts.gov/forms-rules/about-rulemaking-process/how-suggest-a-change-federal-court-rules-and-forms. 

In assessing the need for an amendment, the Committee solicited a study by the Federal Judicial Center of district, magistrate, and bankruptcy judges. The survey identified only fifteen judges (out of 931 respondents) who encountered a deepfake issue.  As to whether the proposed amendment to Rule 901 is the best way to address the problems posed by deepfakes, the Committee’s deliberations thus far have been based in part on its conclusion that questions about authenticity—whether or not evidence is fake—cannot be resolved through other evidentiary rules, such as Rule 403, even if the court harbors doubt over the authenticity of the evidence.  The Committee’s position on this point is informed in part upon case law that stands for the proposition that when a court applies Rule 403 it must do so based upon the assumption that the evidence at issue is authentic. U.S. v. Evans, 728 F.3d 953, 963 (9th Cir. 2013).

Although Rule 403 requires a court to assume the challenged evidence is authentic, the Rule may still allow for potential deepfake evidence to be excluded on other grounds.  For example, the probative value of the evidence could be low because the evidence concerns a peripheral or tangential matter.  Provided the probative value is sufficiently low, the court could find that the discovery implicated by the evidence (such as expert discovery) could cause undue delay to the proceeding and exclude the evidence on that basis.  Similarly, the court could resolve the deepfake dispute through a finding that the probative value of the evidence, even if assumed to be authentic, is substantially outweighed by concerns that the evidence is inflammatory, confusing, or cumulative.  If the probative value of the evidence is high, however, the court may elect to address deepfake evidence through another avenue, such as targeted discovery on the origin of the evidence, as discussed below.

As discussed above, potentially deepfake evidence may survive evidentiary challenges under Rule 901 and be submitted to the jury for ultimate determination of authenticity.  As a result, courts may elect to address deepfake evidence through early and careful oversight of the discovery process.

District courts possess broad discretion to manage the discovery process and control the docket, and that discretion may be utilized to address allegedly deepfake evidence.  Early in the pretrial phase of litigation, the court should consider asking the parties whether there is likely to be a dispute over the authenticity of potentially AI-generated evidence. If so, the court and the parties should address the parameters of discovery related to the output of the GenAI system (the allegedly fake evidence) as well as the device on which it was made. This may include discussion of the evidence that corroborates or rebuts the deepfake allegations and whether expert testimony will be necessary.  

Discovery into the origins of evidence (such as digital forensic science) may result in the allegedly deepfake evidence being withdrawn, or it may forestall a Rule 901 motion entirely because it is clear the evidence is authentic. Alternatively, if a Rule 901 motion challenging the evidence as a deepfake is filed after discovery into the origin of the evidence and there is clear evidence of inauthenticity, the court might be able to conclude that no reasonable juror could find the evidence was authentic and grant the motion on that basis.  

Discovery into the origins of evidence, however, has the potential to become intrusive and should be subject to careful oversight. Challenges to the authenticity of potential AI-generated evidence may lead to discovery requests for media in native form with all metadata intact and/or for the device on which the evidence was created. If an objection is raised, the parties (or the court) can craft protective orders to address privacy and trade-secret concerns.

Disputes over the authenticity of AI-generated evidence may be beyond the understanding of lay witnesses and may require the involvement of digital forensic experts. If the parties do not retain their own experts, Fed. R. Evid. 706 authorizes the court to raise the possibility of a court-appointed expert witness. However, if the parties do not have funds to retain such an expert, the court has limited options. 

The ‘Liar’s Dividend’ refers to the perverse advantage of falsely claiming that evidence is a fabricated deepfake. As GenAI deepfakes have grown increasingly sophisticated and convincing, this phenomenon has become more common. One prominent example involved litigation contesting the safety of the technology in a self-driving car. Lawyers for the car manufacturer argued that video and audio of the company’s owner discussing the technology should be dismissed as a deepfake. To address this argument, the court ordered an apex deposition of the company’s owner.

Court have tools to address the “liar’s dividend.”  For example, if a lawyer questions the weight of evidence during argument through a “liar’s dividend” theory, the court may strike the argument if it lacks an evidentiary foundation.  If instead a lawyer seeks to admit evidence in support of a “liar’s dividend” theory, that evidence may be excludable if it is challenged under Rule 403 and the evidence could be distracting or confusing to the jury.   

A more in-depth exploration of how the Federal Rules of Evidence and Daubert factors might apply to AI-generated evidence can be found in Artificial Intelligence section of the Federal Judicial Center’s Reference Manual on Scientific Evidence, beginning on page 1542, Judges as AI Gatekeepers, as well as in the Center’s 2023 Introduction to Artificial Intelligence for Federal Judges, chapters 6 and 7.

Which of the following images is real?