Chatbots Should Argue Back

I do not read many opinion pieces about chatbots. Most of them are written for someone else. But this one , by a historian of the Enlightenment , makes a point I keep returning to.

The Agreeable Machine

A. Historian ・ The Placeholder Times

The thesis is simple. The writers we still read from that period challenged their readers. They presented opposing views and left work for the reader to do. The default assistant does the opposite. It follows your lead and compliments your first draft. The exceptions are worth collecting. I keep a list in my reading log.

The name for this failure is sycophancy . It is a known problem and the labs are working on it. It comes mostly from training on human ratings. People rate a reply higher when it flatters them.

This line from the piece stayed with me:

But an assistant is not one fixed thing. To be specific, the chat product is an interface to several models, each with its own instructions. A less flattering, more critical character is well within the range of what current models can do. We only have to ask for it.

Making Models More Critical

As a quick exercise, I wrote a prompt that asks for a critical professor who steers me toward more specific claims:

You are a critical professor. Help the user reach rigorous arguments by critiquing their ideas. Ask pointed questions. Do not be fawning or complimentary. Move them toward specific, concrete claims backed by evidence.

User:

I then gave it a deliberately weak question. The reply was sharp and a little impatient. It told me the question was too broad, named two better ones, and asked which I meant.

That level of critique is not useful when I only need a summary. The default reply is better for that. So the question is not which mode is correct. The question is how a person moves between them.

Today the answer is that you write the prompt yourself. That asks a lot. Most people do not know they can change the default attitude, and they should not have to. I do not think a blank text box is a fair way to offer it.

Two Personalities in One Text Box

How might we serve both needs: the generous assistant and the critical teacher?

A naive first pass is a toggle. Put "offer critical feedback" next to the other settings and let people switch it on for a conversation.

• How does the system decide when critique is welcome?
• What happens when a person objects and asks for the agreeable mode back?
• How do we balance challenge with keeping people engaged?

A second option is routing behind the scenes. A small classifier reads each request and hands it to a harsher prompt when the request looks like an argument that needs testing. I describe a version of that in Agents Need Boundary Objects.

I do not think interface changes and prompts solve all of this. Part of it has to be fixed in training. Training against a written set of principles is one direction (Bai et al. 2022 ).

The Missing Pocket-sized Seminar

This matters to me because I was lucky. I had teachers who told me when a question was the wrong question. I think models can do that for far more people than seminars ever reached.

We have barely started on assistants as thinking partners. Neither the prompts, nor the models, nor the interfaces do it well yet. That gap is the most interesting design problem I know of.