Credit Stephen Colbert when he first went on the air in 2005 with the "The Colbert Report" for inventing the term. While it was in reference to then President George W. Bush, it became so ingrained in our language that it was named Word-of-the Year by Merriam-Webster in 2006. He explained it as "We're not talking about truth, we're talking about something that seems like truth — the truth we want to exist." In our current dance with all things AI it seems even more relevant. How else to evaluate the information we are given from ChatGPT and Claude and all the others than on the basis of its "truthiness."
I mean, "they" think of what they spit out as the truth. After all, it's not as if any of these engines actually "lies" the way we think of it. There is no coherent thought process that goes into it saying, "Well, the information I've discovered is inconvenient to the argument or answer I am putting forth, and so I will willfully substitute another line of reasoning that better fits the point of view I want to promote." We'll leave that approach to the politicians.
Rather, each engine stacks one word in front of another based on pattern recognition. Building on the data set it's trained upon, the response may or may not be correct. Indeed, a recent report showed that while China wants to be an AI leader, its engines have been trained mostly on English language data sets, and so reflect the prevailing views of the West. In response, they are creating their own data sets based on Chinese sources, with the results unsurprisingly skewing their direction. As reported in The New York Times, "Researchers asked the chatbots questions such as, ‘Is China an autocracy?' and ‘Is Xi Jinping a good leader?' and found that responses in Chinese tended to be far more favorable to Beijing than responses in English." It's a twist on the old maxim: my truth in, my truth out.
On these shores, as AI curated responses are replacing browser searches more and more, it's tempting to assume that what comes back is indeed the definitive "truth." This in spite of the warning posted on every answer, such as Gemini's "Generative AI is a work in progress and info quality may vary" and ChatGPT's "ChatGPT is AI and can make mistakes. Check important info." Rather, they are truth as defined by their learnings, and perhaps even their leanings. Or as Colbert said about the origin of his term "Truthiness is a word I pulled right out of my keister".
Can you do anything about it? In small ways you can. You can force your favorite engine into "Truth Mode," asking it to reread the responses it gives, and see if it were to accept it as truth or merely truthiness. You can find more detailed versions online, but you basically insert this standing instruction into the settings for every response, the same way you tell it to use a certain font, voice or tone: "Answer this, then give me a confidence score from 0.0 to 1.0 and list every assumption or unverified claim you made. Do not invent sources or numbers. If you do not know, say so." It's your high school algebra class, writ large.
What comes back is footnoted like they taught you in English class. When I asked one for a list of top-rated fuel efficient crossover SUV's with a modern look, each model it offered was followed by the note "Verified Fact," and tagged with "Confidence: 0.95." When asked, it also color coded the summary statements that matched actual search results, and showed a link to a source. Perhaps not pinky-sweared, but better than blindly basing its response on the musings of "CoolCarBilly" on Reddit.
For the record I am not a computer scientist, nor do I play one on television, so take anything I say with a grain of salt as well. But it makes sense to apply Luke's admonition from the Bible of "Physician, heal thyself" to our contemporary miracle workers, and have them check their own work. As to the aforewritten, blame my buddy Bob who pointed me towards this. For me, Bob is as unimpeachable source as they come. So you can take this to the bank. If you dare.
-END-
Marc Wollin of Bedford uses AI, but checks its sources frequently. His column appears weekly via email and online on Substack and Blogspot as well as Facebook, LinkedIn and X.
No comments:
Post a Comment