Trusting AI, sensibly

AI sounding right doesn’t mean it is right. What it learns are correlations and patterns, and when it answers it uses probability to generate whatever “looks most like an answer,” without inherently knowing whether that’s a fact.

AI answering well doesn’t mean it really understands. It uses the Transformer’s attention mechanism to find relationships and fill in structure within the context and generate the most reasonable expression, without needing to understand first and then answer the way people do.

Before asking AI anything, I first sort out what type of question it is. For factual questions it’s an explainer; for questions of probability, a generator of hypotheses; for questions of value, a mirror; for questions of responsibility, a preparation tool.

Using AI a lot doesn’t necessarily make you dumber, but if you keep handing it the “first step” — your first reaction, your first draft, naming your own feelings — your judgment really does get dull.

If I could read only three papers to understand today’s AI, I’d pick Transformer, Chinchilla and InstructGPT. They answer, respectively, why AI can, why AI is expensive, and why AI acts like an assistant.