Human or AI Lawyers? Which Is Less Harmful?
June 9, 2026
Another dinobaby post. No AI unless it is an image. This dinobaby is not Grandma Moses, just Grandpa Arnold.
Ever since the president of Stanford University, the tree of Silicon Valley, departed because of some intellectual fancy dancing, I have been skeptical about [a] its ethos, [b] its business practices, and [c] its research. I want to comment on some of the findings reported as actual factual in “AI Outperforms Law Professors in Stanford Law Study.” I like that two “law” approach to headlines.
Thanks, Midjourney. Good enough like most AI.
Here’s the snappy subtitle:
In a rigorous blind study, law professors overwhelmingly preferred AI-generated answers to student legal questions over answers written by fellow law professors—and flagged the AI answers as potentially misleading or harmful far less often
The way I read this is as follows: Lawyers make mistakes. Some of those mistakes are “potentially misleading or harmful.” Then the coup de grâce: “far less often.” Therefore, lawyers make mistakes. These mistakes are potentially misleading or harmful. What’s that say about a lawyer graduating from Stanford or another “brand” institution, passing the bar (no, not that bar), and litigating. Yep, mistakes. More mistakes that LLM-centric AI systems known to output fake legal citations, hallucination (yes, that type), and harm.
Let’s hit the write up, shall we. I learned:
The research team took extensive precautions to ensure the study’s validity. [N.B. Those peer-reviewed papers with the bogus bio-art were allegedly afforded “extensive precautions” too.] They calibrated AI responses to match the length and structure of human answers, used multiple evaluation methods, and had professors assess whether responses might mislead or confuse students.
I think the study’s purpose is to provide “Stanford research type proof” that smart software is pretty darned good. None of that Waymo-type fear of wet streets. None of those erroneous cancer analyses one hopes a human radiologist checks before the chemo pumps starts whizzing. I think the study makes it clear that legal education and by extension cranks out humanoids who deliver “potentially misleading or harmful” human outputs. Yeah, harmful.
The write up bangs home its point with this passage:
“Our study evaluates the quality of answers given by AI tools. But how to implement these tools to most effectively improve student learning is still an open question. So we’re not advocating for wholesale adoption of AI tutors,” Nyarko cautioned. “But our data suggests that blanket skepticism may be equally unwarranted. The conversation should shift from whether AI can give accurate, high quality responses to how we can deploy it responsibly to the benefit of our students.”
To me, this is a lawyerly way to say, “Hey, AI is pretty darned capable.” But what about those humanoid lawyers? Yep, they beaver away providing “potentially misleading harmful” outputs an astounding one out of 10 times.
Stephen E Arnold, June 9, 2026
Comments
Got something to say?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
