Among the three draft issue memoranda the Legislative Research Council presented to the Legislative Executive Board Wednesday is a research brief on “Artificial Intelligence in Government”. Two sentences in that brief tell us why artificial intelligence should not be in government:
A Stanford University study found that general-purpose chatbots hallucinated “between fifty-eight and eighty-two percent of the time on legal queries.” Researchers at the Stanford University Institute for Human-Centered Artificial Intelligence found that even custom AI legal tools made for specific purposes, like those developed by Westlaw and Lexis Nexis for legal research, still hallucinated more than thirty-four percent and seventeen percent of the time, respectively [link added; Legislative Research Council, Draft Issue Memorandum: “Artificial Intelligence in Government,” presented to Legislative Executive Board 2026.08.26, p. 2].
If you did your job wrong 17%, 34%, 58%, or 82% of the time, you’d lose your job. If you’re a legislator using chatbots to get information about laws and to write the speeches you make about laws you are voting on, you’re giving your colleagues and constituents lots of bogus information. If you’re a public servant in any agency using artificial intelligence to give citizens information about the requirements of federal, state, and local law, you’re giving your fellow Americans lots of bad guidance.
If I told you that of the 98 sources that I linked in the 14 blog posts I published over the last five days, I made up 82, or 58, or 33, or 16, or even one, I would lose all credibility, and deservedly so.
An “intelligence” that hallucinates every day is kinda dumb, and certainly dumber than anyone/anything I want helping me understand, compose, and implement law and policy.
* * *
The Legislative Research Council does not call for electing a robot Governor or putting the LRC on chatbot-pilot. This issue memo points to a new Generative AI Policy posted by the South Dakota Bureau of Human Resources that recognizes concerns about accuracy along with security, privacy, and intellectual property and puts a hard clamp on the use of generative AI in government work by state employees, executive board and commission members, contractors, vendors, and volunteers. Crucially:
GenAI-created content must be considered as a starting point. The content must be reviewed for accuracy, clarity, spelling, proper grammar, and contextual appropriateness before relying on it for work purposes. GenAI-created content is not a replacement for a User’s professional judgment and creativity [Bureau of Human Resources, “Statewide Policies: Generative AI Policy,” retrieved 2026.08.29].
LRC notes that individuals subject to this policy may use AI in government work “only after obtaining approval from the individual’s agency leadership.”
This draft issue memorandum provides examples of how other states have tried to put guardrails around the use of this inherently unreliable tool:
- Nevada outlaws the use of AI in “administering care” by school counselors, school psychologists, and school social workers. Nevada also requires that “a natural person—not artificial intelligence—makes final decisions in emergency response planning and resource allocation.” (I have to ask: why limit those restrictions to school care-giving and emergency response? Shouldn’t actual people make every decision in government of the people, by the people, and for the people?)
- North Dakota prohibits law enforcement from deploying robots that can fire lethal weapons autonomously (North Dakota cops have to do their own shooting and killing.)
- Montana’s governmental entities and state officers can’t use AI “for the cognitive behavioral manipulation of a person or group” (but Facebook still can!), to classify people in ways that lead to discrimination, or for malicious or surveillance purposes.
- Utah requires cops to tag AI-generated reports as AI products and certify that they have read and reviewed those products for accuracy (and again, shouldn’t this declare-and-review requirement apply to everyone, public and private sector, posting bot-barf instead of genuine human output?)
The issue memo’s summary of states’ cautious restrictions on AI use contrasts markedly with its summary of federal action. LRC focuses on the Trump Administration’s July 2025 America’s AI Action Plan, which in LRC’s telling seems to be all about building and using more AI everywhere all the time and not at all about limiting the influence of inherently hallucinating bots, algorithms, and the corporations that build and control them.
This draft issue memo concludes by telling legislators that they “will be challenged to ensure that statutory and regulatory efforts will keep pace with, and appropriately address, the rapid advances in this technology.” But the core message legislators and other public servants need to take from LRC’s AI brief lies in the statement of how often AI screws up and that no amount of rapid advances will remove those screw-ups from this technology:
The most serious problem plaguing every LLM is the tendency to hallucinate. An LLM is said to have hallucinated when it gives a user information that is factually incorrect or implausible. Because of the way LLMs work and the math involved, hallucinations are an inevitability: “Numerous studies have demonstrated that hallucinations are an inherent characteristic of LLMs” [link added to LRC cited source; LRC, 2026.08.26, p. 2].
AI hallucinates. AI will always hallucinate. Legislators, your job is not to push for more hallucinatory technology in government. Your job is to limit the effects that inherently malfunctioning technology has on government functions and the people you serve.