How an AI Writing Assistant Generates a Suggestion, in Plain Terms

Last Updated: September 14, 2026

  Disclosure: This article currently contains no affiliate links, and no company mentioned here paid for or previewed it. If that changes, I will say so here and mark any affiliate links clearly. See my Affiliate Disclosure and Editorial Policy.

You're typing an email. You pause after "Let's schedule the ___" and a faint gray suggestion appears: "meeting for next week." You hit Tab, and it's inserted like the tool read your mind.

It didn't read your mind. It did something much more mechanical — and once you see the mechanics, the suggestions stop feeling magical and start feeling predictable, which is exactly what makes them useful (and occasionally wrong in very specific, explainable ways).

This piece breaks down, in plain English, what actually happens between the moment you pause typing and the moment a suggestion appears — using the same underlying techniques (tokenization, embeddings, attention, and sampling) that power the writing assistants reviewed on this site, including Grammarly, Jasper, and Copy.ai. No jargon left unexplained, no hand-waving.

The short version

An AI writing assistant doesn't "understand" your sentence and doesn't "know" what a meeting is. It:

1. Breaks your text into small chunks called tokens.
2. Converts each token into a list of numbers called an embedding, which represents how that word tends to relate to other words.
3. Runs those numbers through a transformer network, which weighs how much every earlier word should influence the next one (this weighing step is called attention).
4. Produces a probability score for every possible next token — essentially a ranked list of what's statistically likely to come next.
5. Samples one token from that ranked list (not always the top-scoring one — more on why below), checks it against basic rules, and displays it as your suggestion.

Then it repeats that entire five-step loop for the next token, and the next, until it has a full suggestion. All of this happens in a fraction of a second. Below, each step gets unpacked properly.

Original diagram: saassoftwarereviews.com. Feel free to save or share this image, with credit and a link back to this article.

Step 1: Your text gets chopped into tokens, not words

The first thing an AI writing assistant does with your sentence is break it into tokens — small chunks of text that are not necessarily whole words. "Unbelievable" might become three tokens: "un," "believ," and "able." Common short words like "the" or "and" are usually a single token each.

Why chop words up instead of processing whole words? Two practical reasons:

• Vocabulary size stays manageable. If a model had to store a unique entry for every possible whole word (including misspellings, plurals, and made-up brand names), the vocabulary would be enormous and inefficient. Sub-word tokens let the model handle almost any text — including words it's never seen before — by recombining familiar pieces.
• Rare and new words still work. A brand-new product name the model has never encountered gets broken into recognizable fragments instead of causing a total failure.

This is also why AI writing tools sometimes stumble on invented words, code, or heavily technical jargon — the tokenizer is doing its best to chop up something it has no clean pattern for.

Step 2: Each token becomes a list of numbers (an embedding)

Computers don't process the word "meeting" the way you do. Every token gets converted into an embedding — a list of numbers (in practice, often hundreds of numbers) that represents where that token sits in a mathematical space built from patterns in the model's training data.

The useful property of embeddings is that words used in similar contexts end up numerically close to each other. "Meeting" and "call" might sit near each other in that space, because they show up in similar sentence positions across billions of training examples. This is the mathematical basis for an AI writing tool being able to suggest a near-synonym or a contextually appropriate follow-up phrase — it isn't looking words up in a thesaurus, it's operating on their numerical neighborhood.

Step 3: Attention — deciding which earlier words matter right now

This is the step that actually made modern AI writing assistants possible. Older text-prediction systems (the kind that powered basic phone keyboard autocomplete a decade ago) mostly looked at the last word or two. Modern tools use an architecture called a transformer, and its core mechanism is called attention.

Attention lets the model look back across your entire sentence (or paragraph, or document, depending on the tool's context window) and assign a weight to every earlier token based on how relevant it is to predicting the next one. In the sentence "Let's schedule the meeting Sarah asked about," when the model is about to predict what comes after "the meeting," attention allows it to pull extra weight from "Sarah" and "asked about," not just the word immediately before the blank.

This is also the mechanical reason AI writing tools can maintain tone and stay "on topic" across a longer piece — the attention mechanism keeps referring back to earlier context rather than only reacting to the last few words.

Step 4: Turning attention into a ranked list of next-word candidates

Once the model has weighed the relevant context, it doesn't just pick a next word — it computes a probability score for every token in its entire vocabulary (which can be tens of thousands of possible tokens). Most of those scores will be essentially zero. A handful will be meaningfully high.

For "Let's schedule the meeting ___," a real model's output distribution might look conceptually like this (illustrative numbers, not pulled from any specific vendor's live system):

This ranked list is the raw material every suggestion is built from.

Step 5: Sampling — why the suggestion isn't always the single "best" word

Here's the part most explainers skip, and it's the reason two AI writing tools (or even the same tool on two different days) can suggest different completions for identical text.

The model doesn't always take the single highest-scoring token. Writing assistants apply a sampling strategy, controlled mainly by two settings:

• Temperature — controls how "safe" versus how "creative" the choice is. Low temperature pushes the system toward the highest-probability token almost every time, producing predictable, conservative suggestions. Higher temperature allows lower-probability tokens to occasionally get picked, producing more varied or surprising phrasing.
• Top-p (nucleus sampling) — instead of considering every possible token, the system narrows the field to the smallest group of top candidates whose combined probability crosses a threshold (say, 90%), then samples from just that group. This prevents wildly unlikely tokens from ever being chosen while still allowing some variety among the reasonable options.

This is precisely why a grammar-focused suggestion (correcting "their" to "there") tends to feel deterministic and consistent, while a "rewrite this paragraph" or "generate three headline options" feature tends to give you different results each time you click it — the tool is deliberately using a higher temperature or repeated sampling for open-ended generation tasks, and a much lower one for corrections where there's a single defensible right answer.

The invisible layer: filtering and guardrails before you ever see a suggestion

Before a sampled token (or a full generated suggestion) reaches your screen, most commercial AI writing assistants run it through additional layers that are separate from the core language model:

• Grammar and mechanics rules — many tools blend the probabilistic language model with traditional rule-based grammar checking for things like subject-verb agreement, so a purely "statistically likely" suggestion doesn't override a hard grammar rule.
• Style and brand-voice constraints — tools built for marketing teams layer in a company's specific tone, banned words, or formatting preferences on top of the base model's output, so the same prompt produces different phrasing depending on whose account is asking. Jasper positions this as its core differentiator: general-purpose models like ChatGPT, Claude, and Gemini have no built-in knowledge of a specific brand, so Jasper layers brand voice, style guides, and product knowledge on top of whichever base model it's using.[^3]
• Safety and content filters — a separate check screens generated text for disallowed content before it's shown.
• Context personalization — some tools adapt suggestions to the specific document, app, or intended audience you're writing in. Grammarly describes tailoring its feedback to a user's context, audience, goals, and voice, aiming to let people communicate effectively without breaking flow or switching tools.[^1]

This is why the same underlying model can behave noticeably differently depending on which product wraps it — the raw prediction engine is only one layer of what you actually experience as "a suggestion."

Where the underlying models actually come from

It's worth being direct about something a lot of marketing pages gloss over: most consumer AI writing tools are not training their own frontier language model from scratch. They license, fine-tune, or orchestrate existing large language models and wrap them in a product experience.

• Jasper has publicly described itself as model-agnostic — routing tasks across multiple foundation models rather than relying on one. Jasper's own architecture write-up states that the company maintains a flexible, model-agnostic infrastructure so it can swap the underlying model powering a given feature within 24 hours of a new release, rather than being locked into a single provider.[^4] Jasper has also announced a direct integration with Anthropic's Claude, describing the pairing as combining Claude's reasoning and long-form writing capability with Jasper's own brand-context and governance layer — Claude handles open-ended research and drafting, while Jasper keeps the output aligned to approved brand messaging.[^5]
• Grammarly's generative features, part of its broader AI assistant lineup, are described by the company as built on top of generative AI models and layered with its own computational-linguistics research into the patterns of good writing, drawn from analyzing large research corpora — rather than being a bare, unmodified chatbot.[^2]

The practical takeaway for anyone evaluating these tools: "which AI writing assistant is best" is often less about which company trained the smartest core model (several tools now draw on the same handful of underlying model providers) and more about what's layered on top — the brand-voice tooling, the grammar-rule integration, the workflow it's embedded in, and how tightly the suggestion UI is tuned for the kind of writing you actually do.

A concrete walkthrough, start to finish

Let's trace one real example end-to-end using the mechanics above.

You type: "Thanks for reaching out. I'd be happy to"

1. Tokenize: The system breaks this into roughly a dozen tokens, including punctuation.
2. Embed: Each token is converted to its numeric representation.
3. Attend: The model weighs "Thanks," "reaching out," and "happy" heavily, since those words strongly constrain what a polite, forward-looking reply tends to say next.
4. Score: The model produces probabilities across its vocabulary. Given the training data pattern of polite business replies, tokens like "help," "assist," "discuss," and "schedule" likely score highest, while something like "decline" scores very low (because "happy to decline" is a much rarer real-world pattern than "happy to help").
5. Sample and filter: At low temperature, the system likely picks "help" — the top candidate — then continues the same five-step loop token by token to build out "help with that. Could we set up a quick call this week?" A grammar layer confirms the sentence is well-formed before it's displayed.

Nothing in that process involved the tool "understanding" that you're replying to a work email. It involved a very large, very well-trained pattern-matching system correctly predicting what tends to follow similar text, then presenting the single most statistically defensible continuation.

Why this explanation matters for how you actually use these tools

Understanding the mechanism changes how you should treat the output:

• Suggestions reflect patterns in training data, not facts about your situation. If an AI writing tool suggests a confident-sounding statistic or claim mid-sentence, it's predicting a statistically plausible continuation of that sentence shape — not verifying a fact. Always fact-check anything factual an assistant "completes" for you.
• Low-temperature tasks (grammar, spelling) are far more reliable than high-temperature tasks (open-ended drafting). This is a direct, mechanical reason grammar correction feels near-perfect while "write me a paragraph about X" needs more editing.
• Context window size is not a marketing gimmick — it changes suggestion quality. Because attention only operates over the text the model can currently "see," a tool with a small context window will lose track of instructions or facts you mentioned much earlier in a long document, while a tool with a larger context window can keep referencing them.
• The same base model can feel completely different across two products, because grammar rules, brand-voice layers, and safety filters sit on top of it and reshape what actually reaches your screen.

How to see this mechanism for yourself in about five minutes

You don't need to take any of this on faith. Here's a simple, repeatable way to observe temperature and sampling behavior directly in whichever AI writing tool you're evaluating:

1. Pick a short, ordinary sentence starter with an obvious "default" continuation — something like "I wanted to follow up on" or "Thank you for your patience".
2. Generate a suggestion, note it down, then undo and regenerate the exact same prompt five or six times in a row without changing anything else.
3. Compare the results. If a tool's grammar/autocomplete feature gives you the same completion every time, that feature is likely running at low or zero temperature — it's optimizing for the single most probable, safest continuation. If a "rewrite" or "generate ideas" feature gives you noticeably different phrasing each time, that feature is sampling from a wider pool of candidates on purpose.
4. Test with a longer document. Write two or three unrelated paragraphs, then go back to the top and ask for a suggestion that should logically depend on something you wrote near the end. If the suggestion ignores that later context, you've likely hit the edge of that tool's effective context window — a real, testable limitation, not a bug report you have to take the vendor's word for.
5. Try the same prompt in two different tools. Differences you see are coming from some mix of: a different underlying model, different sampling settings, and different proprietary layers (grammar rules, brand voice, safety filters) — exactly the layers described above.

This five-minute test is genuinely more informative than most marketing pages, because it shows you the tool's actual behavior rather than a vendor's description of it. If you're comparing multiple AI writing assistants for a purchase decision, running this same test across each candidate tool with identical prompts is one of the few ways to get an apples-to-apples read on how conservative or creative each one's suggestions really are by default.

Common myths, corrected

Myth: "The AI reads my sentence and understands my intent." It processes numerical patterns derived from your tokens and prior training data. The output can look like understanding because human language itself is highly patterned — but there's no comprehension step, no model of your goals, and no awareness that "the meeting" refers to a real event on your real calendar.

Myth: "If I get the same prompt twice, I should get the same suggestion." Only true for low-temperature or fully deterministic settings. Most open-ended generation features deliberately sample rather than always picking the top-scoring token, which is why regenerating a headline or paragraph often gives you something different.

Myth: "A bigger, newer model always gives better writing suggestions." Model quality matters, but so does the layer built on top of it — brand-voice tuning, grammar-rule integration, and how well the product's UI surfaces the right suggestion at the right moment. A well-engineered product on a mid-tier model can outperform a raw, unwrapped frontier model on writing tasks specifically because of that extra layer.

FAQ

Does the AI store or "remember" what I type? This depends entirely on the specific product's data policy, not on the mechanism described above — tokenization and embedding happen regardless of whether the text is later stored, used for training, or discarded. Always check the individual tool's privacy policy for its actual data retention and training-use terms rather than assuming based on how the suggestion engine works.

Why does the assistant sometimes suggest something grammatically correct but tonally wrong? Grammar rules and tone/style prediction are frequently handled by different layers. A suggestion can pass every grammar check while still missing your intended tone, because tone is a probabilistic prediction, not a rule-based check.

Can I make suggestions more consistent? Many tools expose a "creativity" or "tone" setting that is, functionally, a temperature or sampling control. Lowering it will generally produce more consistent, conservative suggestions at the cost of variety.

Is this the same technology across every AI writing tool? The core mechanism (tokenize → embed → attend → score → sample) is common to essentially all modern transformer-based writing assistants. What differs — often significantly — is which underlying model(s) a product uses, how much proprietary layering (grammar rules, brand voice, workflow integration) sits on top, and how the product tunes sampling settings for different features.

Sources referenced in this article

[1]: Grammarly, "What Is Generative AI? How It Works and How to Use It" — https://www.grammarly.com/blog/ai/what-is-generative-ai/ 

[2]: Grammarly, "How Grammarly's Product Works" — https://www.grammarly.com/how-grammarly-works

[3]: Jasper, product homepage — https://www.jasper.ai/

 [4]: Jasper Blog, "Gemini 3 Pro in 24 Hours: Inside Jasper's LLM-Optimized Architecture" —  https://www.jasper.ai/blog/llm-optimized-architecture 

[5]: Jasper Blog, "Jasper + Claude: Bringing Marketing Intelligence Into Your Workflow" — https://www.jasper.ai/blog/jasper-claude 

About the Author:

Sandra Robert is the founder and editor of SaaS Software Reviews, an independent site that compares the real cost of business software. She checks prices against vendor pages and linked sources where possible, and each article shows the date its figures were checked. Prices change, so confirm current rates with the vendor before you buy. Corrections? Contact Sandra.

Disclosures

This article explains general AI writing-assistant technology and references publicly available statements from Grammarly and Jasper. Where this page links to Grammarly, Jasper, Copy.ai, or other third-party products, This article currently contains no affiliate links.

  This article currently contains no affiliate links, and no company mentioned here paid for or reviewed it. I am not sponsored by Grammarly or Jasper to write this piece, and no company reviewed or approved this article before publication.

This article was written with the assistance of AI writing tools for drafting and research support; all claims were fact-checked and the final content was reviewed and edited by a human before publication.