DeepSeek, Qwen, Kimi, Doubao on three real tasks

Not a ranking, but a look at which step each one suits in learning concepts, judging a job and analyzing a product.

Recently I ran a small experiment.

I used three real tasks to try four general-purpose Chinese AI assistants: DeepSeek, Qwen, Kimi and Doubao.

This isn’t a rigorous comparison, nor an attempt to crown a winner. What I wanted to know more was:

if I put these tools into a real workflow, which part of it does each one suit?

Given the same question, all four AI assistants could basically give a well-structured answer.

But actually using them, the differences weren’t in “who can answer,” but in:

• who can explain a concept as boundaries you can make judgments with;
• who proactively fills in business context;
• who looks professional but needs extra checking;
• who suits doing the first version of an answer;
• who suits being a second opinion.

So this post isn’t a “ranking of Chinese AI assistants.” More precisely, it’s a real user trying them out in a workflow.

I was the one actually opening the four platforms, typing the questions, and recording my immediate impressions and what made me uncomfortable; Codex helped me design the tasks, calibrate how I recorded things and do the side-by-side analysis; the Feishu (Lark) CLI was used to store the raw answers and process notes.

The biggest feeling I came away with:

given the same question, all four AI assistants can answer very completely. But complete isn’t the same as having judgment, and good-looking isn’t the same as trustworthy.

The three tasks I tested

I didn’t start by asking “which AI assistant do you think is best.”

I designed three tasks, from simple to complex.

Task 1: explain a few basic concepts in AI application development

Such as RAG, fine-tuning, tool calling, MCP and the eval loop.

This task mainly tests whether it can explain technical concepts clearly, say where they do and don’t fit, and help a product manager build boundaries for judgment.

Task 2: analyze a real AI product manager job

I used a public Global Public Sector AI PM role at Scale AI.

This task mainly tests whether it just restates the JD, or can see the business context, client situations, real bar and difficulty of preparing behind the role.

Task 3: turn around and analyze the product form of a “general-purpose AI assistant”

This task mainly tests whether it can analyze its own kind of product from a product manager’s perspective, rather than just list features.

These three tasks happen to match several situations where I really use AI now: learning concepts, judging jobs and doing product analysis.

The conclusion first: I no longer just ask “which is stronger”

If I summarize in one line my provisional impressions of these four tools on these three tasks:

DeepSeek is like a steady first explainer. Good for first getting concepts, the structure of material and basic boundaries clear.

Qwen adds a lot of information when filling in the business context of public texts. But the more external background it volunteers, the more you need to check it yourself.

Kimi is more like long-form study notes and product commentary. Good for organizing, adding a risk perspective, and flagging what might be over- or underestimated.

Doubao is more like an assistant for business proposals. Good at breaking a complex problem into layers — the business side, the client side, internal coordination, long-term productization — though its categories can be a bit mechanical.

This isn’t a ranking of platforms. Just my provisional impressions from these three tasks.

What’s really useful isn’t “picking one tool that’s always best,” but knowing: for this kind of task, who should answer first, who adds a second opinion, and what I must check myself.

Task 1: technical explanations aren’t better for being livelier

The first task was explaining RAG, fine-tuning, tool calling, MCP and the eval loop.

The question doesn’t look hard, but it reveals a lot about how an AI assistant explains technical concepts.

My first impression of DeepSeek this round: better than I expected.

It described RAG as “giving the model an open-book exam,” and tool calling as “the model doesn’t say everything itself, but learns to phone for outside help.”

The analogies are simple, but useful. Because they aren’t there for fun; they help me tell the boundaries apart:

• RAG is more like letting the model look things up on the spot;
• fine-tuning is more like changing the model’s stable behavior or style;
• tool calling lets the model call external tools;
• MCP standardizes how tools are connected;
• the eval loop is ongoing evaluation and improvement.

DeepSeek’s comparison table was good too. It didn’t just list “definitions,” but compared the source of knowledge, whether the model changes, the main costs, the effect on latency and how often things update.

Those dimensions matter a lot to a product manager. Because the real question isn’t “what does this term mean,” but “which approach should I use when I hit this situation.”

Qwen also answered completely this round, but its style pulled me out of it a bit.

It used lots of very lively analogies, like describing a large model as “a brilliant fresh graduate who forgot their laptop and is a bit too proud to admit what they don’t know,” describing fine-tuning as being sent to a cram school or trade school for further training (New Oriental, Lanxiang), and there was some internet slang too.

These lines aren’t entirely useless. But for me, in technical explanations, too much liveliness actually costs points.

I don’t want a concept made to sound lively. I want to know whether it’s accurate, where its boundaries are, and when to use it.

Kimi is more like study notes.

Its answers are long and good for reading slowly. It gives subcategories, points of confusion and decision frameworks. For example, it puts easily confused points right under each concept, which helps learning.

But one detail of the experience bothered me: at first it looked like I could type directly, and only after I hit enter did it ask me to log in — and my original question was gone. That kind of interruption hurts the experience.

Doubao had lots of business examples this round, but one categorization at the start put me on alert.

It put RAG under “model capability enhancement.” Reading that, I immediately wanted to correct it: RAG is more accurately external knowledge / retrieval augmentation, not an enhancement that changes the model’s weights.

The later explanation actually had plenty of useful content, like “use RAG for static documents, call tools for real-time data / operations.” But an inaccurate top-level framework affects how much I trust the answer.

What I took away from this round:

technical explanations aren’t better for being livelier. A good explanation should help me build boundaries, not just make the jargon sound lively.

Task 2: what really separates them is business context

The second task was the round with the most new information.

I showed the four platforms the same public AI product manager role: Scale AI’s Product Manager of AI Applications, Global Public Sector.

The role is based in Doha or Dubai, serves governments and government-linked entities outside the US, and involves custom AI applications, custom LLMs, training data solutions, client workshops, model evaluation and so on.

What I wanted to know wasn’t “what the JD says,” but:

What problem does this role actually solve? Is it really a standard AI PM role? Which abilities are hard requirements? If you don’t have an engineering background but do have B2B product experience, how do you close the gap?

DeepSeek’s judgment was fairly steady.

It said that at heart this role isn’t “building an AI product,” but packaging Scale AI’s data and model customization capabilities into solutions that sovereign governments can accept and implement.

That judgment was useful. It pulled the role from “AI product manager” back to a more realistic one: solutions product manager + technical consultant + government delivery manager.

But the 30-day preparation plan DeepSeek gave was fairly rough, and it didn’t ask about my background first. By default it followed the question and gave a plan, instead of first judging “is this plan realistic.”

Qwen deserves its own mention this round.

It proactively filled in business context about Scale AI, government clients in the Middle East and complex delivery models. It didn’t just read the JD literally; it placed the role in a bigger commercial and delivery picture.

The effect on me was direct.

I had been imagining this role as an ideal AI PM opportunity. But after reading its breakdown, I was actually clearer that this role may not be the “general AI product manager” I imagined, but something very heavy on government clients, complex delivery, model evaluation, custom solutions and cross-functional work on the ground.

That’s the value of a good answer: it doesn’t just give me advice; it makes me re-understand reality.

But there’s a risk here too.

Qwen volunteered a lot of external background, like partnerships, regions and the context of government projects. It seems to understand the business very well, which is exactly why it’s easy to let your guard down.

But those external facts can’t be trusted directly. If I want to cite them publicly, they have to be checked separately.

Kimi had one experience advantage this round: it clearly showed that it had read the URL.

That builds trust, because I know it isn’t answering entirely out of thin air. But reading a web page doesn’t mean its judgments are all accurate.

For example, it put a lot of weight on AI-assisted rapid prototyping, almost as if it were a hard requirement. I think that weighting was a bit much.

Doubao did better this round than in Task 1.

It broke the role’s problem into the business side, the client side, internal team coordination and the long-term product:

• the business side is Scale bringing its capabilities to government and enterprise clients in the Middle East;
• the client side is that government clients don’t necessarily know how AI can be implemented;
• the internal side is that algorithms, engineering, delivery and sales have misaligned goals, which a product manager needs to translate and coordinate;
• in the long run, it also means building up repeatable government-and-enterprise AI solutions.

That breakdown was very clear, and made it easier for me to understand why the role is complex.

But all four platforms shared one problem this round:

they all followed the question and gave a “30-day preparation plan,” but almost none asked first:

What’s your background now? Do you really have AI product experience? Are you open to Doha/Dubai? Do you want to prepare for interviews, or close a real capability gap?

A more ideal answer would have said first: 30 days can only cover initial preparation; it can’t close the real gap for this role.

That made me realize: many AI assistants are very good at completing the assignment, but not very proactive about questioning its premise.

Task 3: product analysis isn’t listing features

The third task was having them, from a product manager’s perspective, turn around and analyze the product form of a “general-purpose AI assistant.”

This task wasn’t asking “what features does an AI assistant have,” but whether it could analyze:

• what problem it really solves;
• why users keep coming back;
• which behaviors come from model capability;
• which come from web access, tool orchestration and interaction design;
• where this kind of product is easily overestimated and underestimated.

DeepSeek gave a very clear first-version explanation this round.

It summed up a general-purpose AI assistant as “intent translator + cognitive accelerator + lightweight action agent.”

That framing helps. Because it separates AI assistants from search engines: search engines mostly give you links, while AI assistants try to turn a fuzzy intent directly into usable output.

But its later analysis mostly followed the question, and its second-level abstraction wasn’t especially strong.

The second half of Qwen’s answer was more interesting this round.

It proposed some indicators a product manager could keep watching: failure modes, last-mile cost, error recovery paths, context decay, model self-calibration.

These terms sound a bit abstract, but they’re very practical.

Because when you try an AI assistant, what’s really worth watching is:

• how it fails when it fails;
• whether it can correct itself;
• whether it forgets earlier conditions the longer you chat;
• whether it tells you which judgments need real data.

Kimi is more like product commentary.

It reminded me that generality, the feel of multi-turn intelligence and agent autonomy are easily overestimated, while the ability to refuse, error recovery and holding onto details in long context are easily underestimated.

That kind of perspective suits a second opinion. Not for reaching conclusions directly, but for reminding yourself: have you been taken in by a fluent answer?

Doubao had an expression this round that I think would travel well.

It said general-purpose AI assistants fill the gap of non-standardized lightweight tasks between “standardized professional tools” and “pure information retrieval.”

That sentence explains a lot.

A lot of the time, we don’t want AI to replace Excel, Photoshop, lawyers or doctors, nor do we just want to search a pile of links. What we need to handle are lightweight tasks that “aren’t worth learning a professional tool for, aren’t worth half an hour of searching, aren’t worth hiring a professional for, but are annoying to do yourself.”

Like revising an email, sorting out some material, understanding a job, or turning a fuzzy idea into a first draft.

That may be where general-purpose AI assistants are really used.

What I took away from this round:

in product analysis questions, a good answer isn’t necessarily the longest or most complete, but the one that gives you a method for what to keep watching next.

Stepping back: the five conclusions I really took away

First, a complete structure isn’t the same as judgment.

All four platforms are very good at writing structure: headings, tables, bullet points, summaries.

But structure is just the shell. What’s really useful is whether the tables have dimensions of judgment, whether the conclusions have boundaries, and whether it says clearly what needs checking.

Second, web access isn’t automatically a plus.

Web access, search and reading pages are useful, of course. Much of Qwen’s new information in Task 2 came from filling in external business context.

But the more external the information, the less it can be trusted directly.

These days I’d rather call it “fact governance”: does it have sources? Does it separate facts from inferences? Does it tell me what needs checking?

Third, a long thinking process isn’t the same as transparency.

Both DeepSeek and Doubao sometimes showed very long thinking processes.

I understand the product wants to show “I’m thinking hard,” but if the process is so long it’s like another answer, users don’t know whether to read it.

The transparency I’d rather see is: what it’s reading, what it’s looking up, where it’s unsure, which conclusions need verifying.

Fourth, tone affects trust.

Among analogies, some make things clearer and some pull you out of it.

For tasks like learning technology, judging careers and analyzing business, I need restraint, accuracy and steadiness more. Not every situation suits a lively style.

Fifth, for important questions, don’t trust just one answer.

But that doesn’t mean opening four platforms for every question.

My new rule:

• Low-risk tasks: one handy tool is enough, for rewriting, summarizing or explaining a small concept.
• Tasks with judgment at stake: use a second tool to cover blind spots, like understanding a job or breaking down a product.
• Tasks to be published or that affect real decisions: AI can only give leads; the facts must be checked by a person.

In other words, what’s really worth building isn’t an answer to “which platform is best,” but a workflow of your own.

Finally

Before, I was more likely to ask: which answer is best? Which model is stronger? Which platform is smarter?

Now I first ask:

What type of task is this, really? Do I need a first explanation, business context, a risk reminder, or fact-checking? Does this answer look complete because it really has judgment, or just because the structure is nicely written?

For small everyday tasks, one handy tool is enough.

But for learning, judging jobs, analyzing products, or even preparing to say something in public, I’d rather treat AI as a “tool for working together in stages” than a magic machine that gives the final answer in one go.

That may be the most useful conclusion from this trial for me:

a good answer and a trustworthy answer aren’t the same thing. Knowing how to use AI isn’t just knowing how to ask, but also knowing when to push back, when to switch tools, and when you must check things yourself.

I’ll put the full report in the attachment.

It includes a more detailed test process, the original prompts for each task, the four platforms’ answers, and the dimensions I used for the later side-by-side comparison.

This post keeps only the main conclusions suitable for public reading. If you want to run a similar test, you can use the prompts in the attachment directly and swap in your own real tasks.