🚀 Work 1:1 with a Software Engineer and let AI handle the busywork → https://www.skool.com/ai-academy-with-robby-6849/about
Do Computers Read Like Us?
Hi! I’m Robby. I spend my days building AI systems, and I get asked this question a lot: “Does ChatGPT read a book just like I do?”
The answer is no! When you look at a sentence, your brain sees words. When a computer looks at a sentence, it sees something called tokens.
Think of tokens like tiny Lego bricks. Instead of reading whole words, AI breaks everything down into these small pieces to make sense of the world.
The Secret "Lego Language"
There are about 170,000 words in the English language. That is a lot for a computer to memorize! If an AI tried to learn every single word as one big chunk, it would be very slow and use way too much memory.
Instead, AI engineers use a trick. We break language down into a smaller "vocabulary" of about 50,000 tokens.
How it works:
- Small words: Short words like "cat" might be one token.
- Big words: Long words might be split into two or three tokens. For example, the word "friendship" might be broken into "friend" and "ship."
- Parts of words: Sometimes, even parts of a word (like "ing" or "ed") get their own token.
Why Use Tokens?
By using this Lego-like system, the AI can build almost any sentence you can imagine using just those 50,000 bricks.
This is the secret to why AI is so fast. Because it is working with smaller, compressed pieces of information, it can "read" through the entire internet in a blink of an eye. It doesn't need to struggle with complex grammar like we do—it just learns which Lego bricks usually snap together next.
The Big Picture
Next time you type a question into ChatGPT, remember that you aren't just sending words. You are sending a stack of Lego bricks! The AI quickly pulls apart your bricks, looks at what they mean, and builds a new, smart answer just for you.
Pretty cool, right?