Scanning · Crafting Interpreters
The first step in any compiler or interpreter is scanning. The scanner takes in raw source code as a series of characters and groups it into a series of chunks we call tokens. These are the meaningful “words” and “punctuation” that make up the language’s grammar.
4 Scanning Take big bites. Anything worth doing is worth overdoing. Robert A. Heinlein, Time Enough for Love The first step in any compiler or interpreter is scanning . The scanner takes in raw source code as a series of characters and groups it into a series of chunks we call tokens . These are the meaningful "words" and "punctuation" that make up the language's grammar. This task has been variously called "scanning" and "lexing" (short for "lexical analysis") over the years. Way back when computers were as big as Winnebagos but had less memory than your watch, some people used "scanner" only
Explore this link on the map →related reading
- Let’s Build A Simple Interpreter. Part 1. - Ruslan's Blogruslanspivak.com
- 1. Kaleidoscope: Kaleidoscope Introduction and the Lexer — LLVM 23.0.0git documentationllvm.org
- Lexical analysis - Wikipediaen.wikipedia.org
- Introduction · Crafting Interpreterscraftinginterpreters.com
- Statements and State · Crafting Interpreterscraftinginterpreters.com
- 2. Kaleidoscope: Implementing a Parser and AST — LLVM 23.0.0git documentationllvm.org
- Parsing Expressions · Crafting Interpreterscraftinginterpreters.com
- Reading 12: Regular Expressions & Grammarsweb.mit.edu
- A Map of the Territory · Crafting Interpreterscraftinginterpreters.com
- A Map of the Territory · Crafting Interpreterscraftinginterpreters.com
- Lexical Analysis With Flex, for Flex 2.6.2: Simple Exampleswestes.github.io
- Evaluating Expressions · Crafting Interpreterscraftinginterpreters.com