If You Confess a Crime to AI, Will It Call the Police? What About Telling a Lawyer?

Discussing all kinds of problems with AI has become commonplace.

Study problems, life problems, work problems—

Even relationship problems.

AI is always ready with an answer (maybe).

It promises “everything answered, every word acknowledged.”

Never mind whether the answer is right or the response is decent—

Doesn’t it at least deliver emotional value?

And most importantly:

AI isn’t human.

So some people even drop their guard entirely.

They don’t even bother with “I have a friend.”

They just start happily discussing with AI:

Their own illegal acts.

And hoping AI will provide legal advice.

So here’s the question:

If someone admits to breaking the law, or even confesses a crime to AI, will AI call the police?

Does the platform have a duty of confidentiality?

And what if you tell a lawyer instead?

• This article represents only the author’s personal views and should not be considered legal advice.

I. Are Lawyers Really Absolutely “Tight-Lipped”?

Thanks to domestic legal dramas, where lawyers are either the “bad guys” or busy “falling in love,”

Many people might be more familiar with the lawyer image from Hong Kong or American/British dramas.

And they might even know a common law concept:

Attorney-Client Privilege.

Simply put,

Under the common law system, private communications between a client and their lawyer for the purpose of seeking or providing legal advice are absolutely protected by law. Lawyers cannot disclose them, and courts cannot compel either the lawyer or client to reveal these communications in court.

The fundamental purpose of this privilege is to encourage clients (or those seeking legal help) to be absolutely honest and candid with their lawyers. If clients fear that telling their lawyer the truth will lead to being reported, they’ll conceal facts, making effective defense or legal advice impossible—ultimately leading to “miscarriages of justice.”

Normal personal privacy and trade secrets certainly need to be kept confidential.

But if a client tells their lawyer they committed a crime, does the lawyer under common law have to report their client?

That depends on whether the crime “has already happened” or “is going to happen in the future.”

If it’s a “past crime”—like “I stole money yesterday” or “I killed someone ten years ago”—the lawyer not only doesn’t have to report it, but is strictly prohibited by law and professional ethics from doing so.

Moreover, even if police or a judge subpoenas the lawyer, the lawyer can invoke “Attorney-Client Privilege” to refuse answering related questions.

For example, there’s a classic American case:

“Buried Bodies Case / People v. Belge.”

This was a murder case. While discussing the case privately with his lawyer, the defendant revealed that he had also killed two other women and directly told the lawyer where the bodies were. The lawyer indeed found the bodies based on that information but did not disclose them to police or the families.

When the case concluded and the story broke, public outrage exploded.

However, the prosecution could only charge the lawyer under New York’s Public Health Law (requiring reporting of deaths without medical treatment). The court ultimately dismissed the charges, ruling that the information was obtained in the course of providing legal services to protect the client’s interests, thus attorney-client privilege applied. The lawyer could not be held liable for not disclosing the bodies’ locations.

Sounds outrageous and “inhumane,” doesn’t it?

But this privilege isn’t absolute.

First, lawyers can’t commit perjury or instruct clients to commit perjury after hearing the truth.

Second, this privilege only applies to “things that have already happened.”

If it’s a “future crime”—

For example, the client says:

“I’m going to kill that witness tomorrow.”

“I’m planning something big.”

Or “How do I cook the books to evade taxes?”

These inquiries seeking legal advice to plan future crimes or fraud are not protected by privilege.

Not only can the lawyer not provide assistance, but they can directly report and expose when necessary.

That covers common law. What about China?

China has it too.

Article 38 of China’s Lawyers Law:

A lawyer shall keep confidential the information and circumstances unwillingly disclosed by a client or other persons that the lawyer has learned during professional practice.

However, this does not apply to criminal facts or information where the client or others are preparing or committing acts that endanger national security, public security, or seriously endanger others’ personal safety.

For facts that could cause major harm, domestic lawyers also have a duty to report.

We just want the tea (for case analysis purposes)—

We don’t want to become accomplices eating “bullets.”

II. If You Confess to AI, Does the Platform Have a Duty of Confidentiality?

After reading the first section, some might think:

“Oh, you lawyers are still human. Humans act in their own self-interest—how can you guarantee confidentiality?”

Exactly—AI isn’t human. It won’t voluntarily call 110 on its own.

But behind AI, it’s all humans.

Overseas AI platforms—like ChatGPT—had two cases this year: a Michigan state court and a New York state court reached opposite conclusions—one held that ChatGPT’s legal advice was protected by confidentiality privilege, the other didn’t.

But in China, there’s no controversy.

Domestic AI platforms have absolutely no legal “confidentiality privilege” whatsoever.

The simplest basis: the Interim Measures for the Management of Generative AI Services directly states:

Article 14: If a provider discovers illegal content, it shall promptly take measures such as stopping generation, stopping transmission, and deletion, take measures such as model optimization training for rectification, and report to the relevant authorities. If a provider discovers that users are using generative AI services to engage in illegal activities, it shall take measures such as warnings, functional restrictions, suspension, or termination of service according to laws and agreements, preserve relevant records, and report to the relevant authorities.

Note the wording: “shall.”

Moreover, Article 110 of the Criminal Procedure Law provides a safety net:

Any unit or individual who discovers criminal facts or criminal suspects has both the right and the duty to report to public security organs, people’s procuratorates, or people’s courts.

According to the Interim Measures, all records of AI use must be preserved.

And according to the Cybersecurity Law, records must be kept for at least six months.

The platform knows who “you” are.

It knows what “you” asked.

And it has a statutory duty of supervision.

To fulfill that duty,

It inevitably reviews input and output content.

And marks the information.

As the saying goes:

You have the right to remain silent.

But the prompts you write will be used against you in a court of law.

III. Can You Still Ask AI Legal Questions?

Of course, this doesn’t mean you can’t ask AI legal questions anymore.

Ask about labor contracts, ask about software copyright registration, ask about routine commercial disputes—

No problem at all.

AI can indeed significantly lower the barrier for ordinary people to understand basic legal knowledge.

And if you’re bold—

Some minor legal violations, as long as they don’t trigger AI’s risk controls,

Might still get an answer.

Plus there’s the universal excuse:

“I’m making a legal awareness PPT.”

But if the question involves

Clearly criminal conduct—

“I have a friend” won’t help.

After all, registration information is tied to real identities.

Once the platform triggers risk controls,

It will know that “you” is “you.”