Website Summary Files Commits Branches @commit @diff @stats
diff options
context:
space:
mode:
Commit81499f10a983bded055029af258084976774910c (patch)
Author
 Dimitar Trendafilov <trendafilov.dn@gmail.com>
2022-10-12 21:09:58 +0300
Committer
 Dimitar Trendafilov <trendafilov.dn@gmail.com>
2022-10-12 21:16:30 +0300
Filesba13df3d809316fb709a6f198c9370b2fca8bc41
IPL-81499f10a983bded055029af258084976774910c.tar
IPL-81499f10a983bded055029af258084976774910c.tar.gz
IPL-81499f10a983bded055029af258084976774910c.zip
Parent4434a0481b2face945bf46c7c5a07b5bca512526 (diff)
Expand on token storage
1 files changed, 12 insertions, 4 deletions
-rw-r--r--website/content/slides/02_lexical_analysis/_index.md16
diff --git a/website/content/slides/02_lexical_analysis/_index.md b/website/content/slides/02_lexical_analysis/_index.md
index 3caa107..da69d31 100644
--- a/website/content/slides/02_lexical_analysis/_index.md
+++ b/website/content/slides/02_lexical_analysis/_index.md
@@ -111,11 +111,19 @@ Typical tokens are:
---
### double + string ~ wasteful
-- `std::variant<double, IPLString` will require *C++17*
-- `union` does't work good with strings
-- `union` with `string_view` -> *C++17*
-- tokens can just keep strings and leave the parser to turn them into numbers
+The token is either a string or a number, but we are keeping both, so it is
+somewhat wasteful.
+---
+#### Keeping data in token
+
+1. `std::variant<double, IPLString` will require *C++17*
+2. `union` does't work good with strings
+ - non-trivial destructor inside the `union`
+3. `union` with `string_view` over the source contents -> *C++17*
+ - this means we have to keep the contents of the source alive as long as
+ any token is still alive
+4. tokens can just keep strings and leave the parser to turn them into numbers
---
## Lexer