Structuring web content so AI can understand it

AI systems often read content in excerpts. An assistant can retrieve an entire web page. But a search system may also deliver only a single section to it. If that section lacks the subject, a prerequisite or an important restriction, even a technically correct text easily becomes misleading.

This guide shows how to structure web pages, help articles and technical documentation so that individual sections remain understandable outside the complete page. You get concrete work steps, examples and a checklist for review.

The most important rule: A section should answer a specific question and contain the context necessary for that answer.

How AI systems read your content

Direct retrieval of a web page

An AI assistant retrieves a page and processes its content. In the process, HTML may be converted into a simplified text representation or Markdown. Design elements may be dropped. For long pages, the content may be truncated.

For this access path, a clear page structure, understandable text and links to further content that are visible early are helpful.

Search in individual text sections

When retrieving from a knowledge base, a page is often broken down into smaller units even before the actual search query. These units are called chunks.

The process can look like this:

1. Web pages or files are ingested.
2. A parser recognizes text and structural elements such as headings, lists and tables.
3. A chunker breaks the content down into smaller sections.
4. The sections are stored in a searchable index, for example using vector embeddings.
5. For a query, the assistant receives matching excerpts and, if applicable, metadata such as page title and URL.

An MCP server can provide such a search as a tool. MCP itself does not split or index content. What matters is the search system behind the tool.

Whether parent headings, neighboring sections or table headers are included depends on the respective implementation.

Step 1: Check your actual influence

Before revising texts, clarify whether you control only the website or also the processing search system.

Important for SEO and GEO: A readable section structure can make it easier to process your content. That is no guarantee of rankings, citations or recommendations in AI answers. In addition, the settings of your own knowledge base cannot simply be transferred to public search engines.

Step 2: Start with ten important pages

First revise a manageable selection. This lets you identify recurring problems before you change the entire website.

For example, choose pages with:

• frequent customer questions,
• important product or service information,
• long instructions,
• extensive tables,
• several operating system or product variants,
• important information that is only in graphics.

For each page, note the biggest comprehension problem and the planned change.

Step 3: Give each section a clear task

Long sections often deal with several questions at once. If they are split automatically, an excerpt may begin in the middle of an explanation.

So structure by specific user questions or tasks.

Too general:

“Setup”

More specific:

• “What prerequisites the setup requires”
• “How to create a new user account”
• “How to assign access rights”
• “How to verify the setup”

Each section should provide a coherent answer. Don't separate a prerequisite or restriction from the action it applies to.

Check: Read only the heading and its section. Are the task, subject and conditions recognizable?

Step 4: Use meaningful, real headings

A heading such as “Restrictions” makes sense within a page. As the start of an isolated search excerpt, it lacks the subject.

Implement headings in the CMS as actual headings. A paragraph formatted in bold does not replace a semantic heading element.

Use a logical hierarchy. For many guides, H2 and H3 below the page title are enough. Deeper levels make sense when the content requires them. Don't flatten the hierarchy just because of a presumed AI preference. With your own search system, check which levels the parser recognizes and the chunker takes into account.

Step 5: Make important statements understandable on their own

Pay particular attention to phrases such as “this setting”, “the procedure”, “as described above” or “as a result”. They can lose their reference if a previous paragraph is missing.

Before:

“This setting overrides the default. It should be adjusted before the next export.”

After:

“The individually set export time replaces the preset export time. Change it before the next scheduled data export if that export should already take place at the new time.”

The goal is sufficient context. Repeat product names and terms where ambiguity would otherwise arise. Mechanical repetition in every sentence makes the text unnecessarily clunky.

Check: Copy a middle section into an empty document. Is it clear without the preceding text what the statements refer to?

Step 6: Base section length on the system used

Chunk sizes are often measured in tokens. Tokens are processing units and do not correspond to a fixed number of words. Language, technical terms, code, URLs and tables affect the ratio.

There is no universally valid word count for AI-friendly sections.

If you control the search system:

1. Determine the size limit actually configured.
2. Check whether headings and other context are counted within this limit.
3. Plan leeway for such additions.
4. Check whether related explanations end up completely in one excerpt.

If you don't control the system, structure long passages by meaningful sub-questions. Don't shorten any necessary explanation just to reach a supposed ideal number.

Step 7: Keep tables understandable even in excerpts

A table can be clear on the website and become incomprehensible after automatic splitting. If the column headings are missing, a value such as “No” or “30” is hardly unambiguous.

Check:

• Are table headers preserved when splitting?
• Are units placed directly next to the values or clearly in the column name?
• Is it clear whether a number denotes a default value, a maximum value or a prerequisite?
• Are exceptions assigned to the correct row?

Split long tables by topic if necessary. A large parameter table can, for example, become separate tables for login, time limits and retries. Each gets its own heading and a complete table header.

Keep the benefit for people in mind: A large filterable table can still make sense. Supplementary detail pages should ideally be generated from the same data source. In your own knowledge base, you can specifically define which representation is indexed so that the same content doesn't appear multiple times unnecessarily.

Step 8: Structure instructions by completed subtasks

An excerpt that begins at step eight often leaves open what has to be done beforehand.

Break long processes down into meaningful stages, for example:

• Create user account
• Confirm identity
• Assign access rights
• Test login

At the start of each stage, state the necessary prerequisites. Keep warnings right next to the affected action.

Unfavorable:

A general warning about data backup comes after a long list of deletion steps.

Better:

The step in question reads: “Before deleting, create a backup of the existing data. Then delete …”

This keeps the prerequisite connected to the action when only this part is retrieved.

Step 9: Preserve the context of code examples

With long code blocks, initialization, the actual call and error handling may end up in different excerpts.

Proceed as follows:

1. Immediately before the code block, state what the example does and which prerequisites apply.
2. Structure longer examples into clearly named subtasks.
3. Mark excerpts that cannot be run on their own.
4. Also link to a complete runnable example if the use case requires it.
5. In your own search system, check whether code blocks are split.

Don't shorten examples so much that necessary dependencies or security checks are missing. A search excerpt can help find the example; the complete code may be required for implementation.

Step 10: Write the key message of graphics into the text

Not every system processes images or SVG diagrams reliably. In a simplified text representation, only the alt text and the image URL may remain.

So describe the technically important message in the surrounding text.

Example:

“The graphic shows three steps: the form submits the inquiry to the CRM. The CRM assigns the inquiry to a contact person. The contact person then receives a notification.”

Add meaningful alt texts. For complex diagrams, a short list of the components and their relationships can be useful.

Also check symbols: if a check mark means “included”, this meaning should also be recognizable in the text. Purely decorative images don't need an additional technical description.

Step 11: Check tabs and hidden content in the text view

Tabs can visually shorten a page. In a text representation, however, their contents may appear one after another. Tab labels may be missing or the overall length may grow considerably.

Typical mistake: Instructions for Windows, macOS and Linux appear directly one after another without each section naming its operating system.

Concrete improvement: Use headings such as “Installation on Windows” within the content. For extensive instructions, separate platform pages can make sense.

Also check accordions: are their contents actually present in the retrieval used? Is it still clear outside the visual interface which question belongs to which answer?

Step 12: Test real excerpts

A visual check of the website is not enough to assess chunking. If possible, test the representation that your search system actually generates.

1. Choose a long text, a long table and an instruction with images or code.
2. Run this content through the real processing.
3. Look at the generated excerpts including metadata.
4. Search for the content with specific user questions.
5. Check whether the excerpts found contain the answer including conditions and exceptions.

Sample questions for the test:

• Which prerequisites apply to the data import?
• Which operating system do the instructions apply to?
• Which services does the described package include?
• What must happen before a record is deleted?

Record errors separately: is the right content not found? Does it lack context? Is it incomplete in itself? These problems require different corrections.

Review prompt for an AI tool

With this prompt, you can carry out an initial editorial review. The prompt cannot reliably reproduce the actual limits of an unknown search system.

Prüfe den folgenden Inhalt darauf, ob einzelne Abschnitte auch außerhalb der vollständigen Seite verständlich sind.

Untersuche:
1. Überschriften, die ihren Gegenstand nicht klar benennen.
2. Abschnitte, die mehrere eigenständige Aufgaben oder Fragen vermischen.
3. Formulierungen, deren Bezug nur aus vorherigen Abschnitten hervorgeht.
4. Voraussetzungen und Warnungen, die zu weit von der betroffenen Handlung entfernt stehen.
5. Tabellen, die ohne Spaltenüberschriften missverständlich wären.
6. Codebeispiele mit unklaren Abhängigkeiten.
7. Informationen, die ausschließlich in Bildern oder Symbolen enthalten sind.
8. Plattform- und Produktvarianten, die nicht ausdrücklich benannt werden.

Gib die Ergebnisse als Tabelle aus:
Textstelle | Verständnisproblem | Konkreter Änderungsvorschlag

Erhalte die fachliche Bedeutung, notwendige Einschränkungen und eine sinnvolle Überschriftenhierarchie. Erfinde keine Produktdetails. Kennzeichne fachliche Rückfragen.

Falls ich Einstellungen des Suchsystems angebe, berücksichtige sie. Falls keine vorliegen, unterstelle weder feste Chunk-Grenzen noch eine allgemeingültige optimale Wortzahl.

Checklist for publication

• Every important heading names its subject.
• Headings are marked up semantically correctly.
• Every section answers a recognizable question or describes a coherent task.
• Important statements are understandable without unclear back-references.
• Prerequisites and restrictions are placed with the related statement.
• Warnings remain connected to the affected action.
• Tables contain unambiguous column names and units.
• Code examples name dependencies and their execution scope.
• The key technical messages of images are also in the text.
• Platform and product variants are explicitly labeled.
• For your own knowledge base, actual search excerpts have been checked.

Start with one page: Revise the headings, resolve unclear back-references and move scattered prerequisites to the right place. Then check three typical user questions using individual excerpts. Use the problems found as a template for the next pages.