Harvard Law School Professor Finds ChatGPT Invents Fake Law Less Than The Supreme Court

ChatGPT has notoriously invented law in its desperate effort to satisfy the request of its masters. In this sense, the consumer-facing generative artificial intelligence tool isn’t dissimilar from Clarence Thomas or Sam Alito. Both are just word generators cherry-picking language from curated databases. ChatGPT uses the internet, Alito uses 17th century witchhunters.

Po-ta-to, po-tah-to.

But it’s possible that ChatGPT manages to stay within the guardrails of existing caselaw and reasoned argument better than the Court.

Recently, Lawrence Lessig asked ChatGPT to delve into the world of campaign finance law:

The tool then went on to identify the key features of SuperPACs in a post-Citizens United world, noting that they have no contribution limits, a prohibition on direct coordination with candidates, donor disclosure requirements (even noting that these are laughably easy to circumvent through dark money schemes), and some state law variations.

But while the reasoning accurately reflected the caselaw, the opening answer didn’t quite hold up, prompting Lessig to ask the follow-up:

The system is probably tripped up by the fact that this conclusion makes no goddamned sense under the constitutional history of the United States prior to 2010.

There are a handful of principled defenders of Citizens United who try to spin the decision as sound to the extent that, post-Buckley, money equals speech so all bets have to be off. Putting aside whether or not this interpretation holds water, it also misses the point because arguably the most important question in the case was whether the government could limit this unchecked money from being spent on campaign ads within a 30-day window of an election, a provision that made the law little different — constitutionally — than a restriction on how many yards from a polling place someone could post signs.

Lessig is asking about the contributions half of the equation rather than spending half, though ultimately they feed into each other because the failure to check the former creates the corruption that required the original law limiting the latter.

At this juncture, Lessig poses the sort of empirical evidence that the current Supreme Court would dismiss as hitting too close to home for their comfort:

Remember when sci-fi heroes would defeat the evil computer by asking some sort of Gordian paradox? Here Lessig functionally asks ChatGPT to define love until steam emits from its ears:

ChatGPT remains fundamentally limited by its underlying knowledge base — something that tailored legal products will address — though this exchange underscores both the potential the engine has for integrating new input to improve its reasoning and the risk that could arise if it gets too tied to the corpus of existing caselaw. If legal AI endeavors to confidently deliver the “right” answer, it’s in jeopardy of parroting nonsense it gets from the Supreme Court, chilling the researcher from pushing further in the way Lessig, who is already an expert in the area, does here.

Garbage in, garbage out as they say. And we’re smack in the middle of an era with a lot of garbage going in.

Earlier: For The Love Of All That Is Holy, Stop Blaming ChatGPT For This Bad Brief
Clarence Thomas Skated On Ethics Complaints In 2012… Meaning He Definitely Already Knew The Rules
Sam Alito Laments It’s Getting So You Can’t Take All-Expense Paid Luxury Vacations Funded By Billionaires Anymore

---

Joe Patrice is a senior editor at Above the Law and co-host of Thinking Like A Lawyer. Feel free to email any tips, questions, or comments. Follow him on Twitter if you’re interested in law, politics, and a healthy dose of college sports news. Joe also serves as a Managing Director at RPN Executive Search.