Rajeev Pentyala – Technical Blog on Power Platform, Azure and AI

We all use JSON every day. It’s the default way we structure data in APIs, config files, databases, and integrations. A typical JSON looks like this:

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

Simple, right?

But the moment we try to pass this JSON to a Large Language Model (LLM) like GPT or Azure OpenAI, etc, we start to hit a few problems.

What happens when you send JSON to an LLM?

Look closely at the example above.

The keys:

"id", "name", "role"

repeat again and again for every object in the array.

If you have:

• 10 users → keys repeat 10 times
• 100 users → keys repeat 100 times
• 10,000 users → keys repeat 10,000 times

LLMs are token-based models

They don’t read text like humans — they read tokens, which are pieces of text.

More tokens →

• More cost
• More latency
• Less room in the model’s context window
• More chance the model truncates, forgets, or gets confused

And JSON uses lots of tokens:

• Quotes
• Curly braces
• Square brackets
• Colons
• Commas
• Repeated field names

This means JSON is great for APIs… but not very efficient for LLM prompts.

TOON (Token-Oriented Object Notation)

To solve these issues, a new format has been created called TOON — Token-Oriented Object Notation.

TOON keeps the meaning of JSON but removes the unnecessary token-heavy syntax. It’s designed specifically for LLM efficiency.

Think of TOON as: The same structured data, but without repeating keys and punctuation that waste tokens.

The Same JSON Example in TOON

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

How is TOON different?

• Field names are declared once inside {id,name,role}
• Data rows are listed like a simple table
• No repeated quotes, colons, braces, or key names
• Consumes much fewer tokens, making it cheaper and faster for LLMs to process

For LLMs, TOON is easier, cleaner, and more compact.

When NOT to Use TOON

You should stick with JSON for:

• Deeply nested objects
• Irregular or optional fields
• Public REST APIs
• System-to-system integrations

Conclusion

• JSON is amazing, but it wasn’t designed for LLMs.
• TOON offers a lightweight, token-efficient way to send structured data to models.
• TOON is not a replacement for JSON — it is an optimization specifically for LLM prompts.

Refer this documentation for more details.

🙂