Chatbots that can answer questions and write legal briefs?
Three-quarters of judges responding to our survey say they have never tried the technology. That doesn’t mean they don’t have strong opinions about it.
Our latest Question of the Month* survey emailed to NJC alumni earlier this month went like this:
“Increasing numbers of lawyers and even some judges are turning to ChatGPT and other forms of artificial intelligence for help with research and other legal tasks. What’s your experience with the technology?”
Of the 332 judges who responded, nearly 76 percent said they hadn’t tried it. About 7 percent said they had tried it and don’t like it. About 17 percent said had tried it and do like it.
Among the 68 judges who left comments, a common refrain was that the technology isn’t reliable enough. Several mentioned reports of AI “hallucinating” and producing misinformation. One anonymous judge reported that the technology had invented a nonexistent opinion from the judge’s own court.
“I have had the experience of informing a real lawyer that his participation was an element of that hallucination,” the judge wrote.
J. Jeremiah Mahoney, chief administrative law judge for the U.S. Department of Housing and Urban Development, said he now notifies new litigants that they are required to disclose to the Court “any use of generative AI in all filings and … certify corroboration of assertions of fact and law in such filings by identified human attorneys or paralegals.”
Judges who indicated that they had tried out the technology gave it mixed reviews.
“I’ve tried it for non-work-related use; hallucination possibilities are too scary right now for official writing/communicating,” an anonymous judge wrote.
A couple of judges lauded AI’s potential to increase access to justice. One compared it to the technologies that have enabled remote hearings, which have greatly benefitted rural areas.
But far more judges voiced concerns, even “grave” concerns.
“As a legal research tool, I think that it may prove to be revolutionary,” wrote Associate Justice David Deakin of Boston Superior Court. “As a substitute for judicial analysis and exercise of discretion, I think that it is both inadequate and potentially quite dangerous.”
“(The) reason I don’t like it,” an anonymous judge wrote, “is that it’s blah at best and wrong at worst. Sure, it is totally amazing that this technology can spin out what it does in a matter of seconds. But what comes out isn’t distinguished or even clever, just middle-of-the-road analysis. I have found it to be wrong at times, enough to make me not rely on it.”
That was nitpicking, compared with another unnamed judge who accused AI of “degrading our society/social connections.”
“At the rate we’re going,” the judge wrote, “‘chatbots’ will replace judges and decisions will be made solely on algorithms. When people’s lives hang in the balance, it is beyond scary to think that the human factor may be removed entirely!” Limiting AI to merely “research and other legal tasks,” as mentioned in the poll question, “is only the beginning to a disastrous end,” the judge predicted.
Retired Alabama Circuit Court Judge N. Pride Thompkins sounded a note of resignation, writing, “It’s here. Get used to it.”
and at least one judge seemed inclined to do just that.
“One day, just as a ‘trial’ run,” the anonymous judge mused, “I’d like to input transcripts from a hearing and perpetuation depositions and ask ChatGPT to summarize the evidence from the proceeding and compare it to my own work in that regard.”
Chess-playing computers eventually beat grand champions. It remains to be seen whether artificial intelligence can do a better job of judging.
* Each month the College emails an informal, non-scientific, one-question survey to its more than 12,000 judicial alumni in the United States and abroad. The results, summarized in the NJC’s Judicial Edge Today, are not intended to be characterized as conclusive research findings. The NJC is a nonpartisan, nonpolitical organization and these monthly polls are designed to engage our alumni in thought-provoking dialogue.
After 22 years of teaching judges, Tennessee Senior Judge Don Ash will retire as a regular faculty member a...
This month’s one-question survey* of NJC alumni asked, “How is 2024 shaping up for you and your court?...
RENO, NV (PNS) – As they eye their inaugural football season this fall, the Gaveliers have question marks...
RENO, Nev. (March 8, 2024) — In what may reflect a devastating blow to the morale of the judiciary, 9 out...
In what may reflect a devastating blow to the morale of the judiciary, 9 out of 10 judges believe the publi...