diff options
| Commit | 990a8f5e2c2d6e246acfd8b7a99ac1cbf4fb1675 (patch) | |
|---|---|---|
| Author | 2018-11-07 19:34:16 +0200 | |
| Committer | 2018-11-07 19:34:16 +0200 | |
| Files | d7f3f8cdc90ea9cc405b7c353d93893c80f15e4d | |
| IPL-990a8f5e2c2d6e246acfd8b7a99ac1cbf4fb1675.tar IPL-990a8f5e2c2d6e246acfd8b7a99ac1cbf4fb1675.tar.gz IPL-990a8f5e2c2d6e246acfd8b7a99ac1cbf4fb1675.zip | ||
| Parent | 76b849fbaa276566fa8838d745c0e5fbd2f138bc (diff) | |
Предаване на първо домашно ("XXX2html") (#12)
* Initial commit of clex
new file: .gitignore
new file: Setup.hs
new file: app/Main.hs
new file: keyword-strings.txt
new file: package.yaml
new file: sample.c
new file: src/Boilerplate.hs
new file: src/Lex.hs
new file: src/Lib.hs
new file: stack.yaml
new file: test/Spec.hs
Need to implement ctohtml functionality now.
And maybe tests for the lexer.
* Clean-up
modified: app/Main.hs
modified: package.yaml
modified: src/Boilerplate.hs
modified: src/Lex.hs
deleted: src/Lib.hs
Clean-up warnings (unused stuff). Remove unused Lib.hs
* Add swap files to gitignore
modified: .gitignore
* Add comments, fix keyword parsing
modified: src/Lex.hs
Previously keyword parsing would only work if you followed the keyword
with a whitespace. Now we just include all non-alphanumerir characters
as ends of a keyword.
* Fix escaped character parsing in strings
modified: src/Lex.hs
Had previously only fixed it for characters.
* Add explicit exports and comments to Boilerplate.hs
modified: src/Boilerplate.hs
* Remove chars written as numbers
modified: src/Lex.hs
* Rename Boilerplate to Types
modified: Lex.hs
renamed: Boilerplate.hs -> Types.hs
* Fix forgotten shift assigns, implement tokenToBS
modified: src/Types.hs
* Add generic tests, fix a couple of bugs
modified: package.yaml
renamed: src/Lex.hs -> src/Clex/Lex.hs
renamed: src/Types.hs -> src/Clex/Types.hs
modified: stack.yaml
new file: test/Clex/LexSpec.hs
new file: test/Clex/TypesSpec.hs
modified: test/Spec.hs
Added pretty basic tests that generate tokens and then lex to and from
bytestrings.
Fixed a bug where I didn't realise unsigned could be 0 or more
instances, and not a single char.
Fixed a bug where (tokenToBS Not) == "!=", instead of "!".
Fixed a bug where I was parsing Numbers before Floating numbers,
and as a result "0.123" would show up as [Number _, DotFloat _]
instead of [LeadingDot _].
* Remove executable info from clex lib
deleted: app/Main.hs
modified: package.yaml
* Add cToHtml tool
new file: .gitignore
new file: Setup.hs
new file: app/Main.hs
new file: package.yaml
new file: sample.c
new file: src/Html.hs
new file: stack.yaml
* Add instructions and global stack.yaml for ease of use
new file: .gitignore
new file: README.md
new file: stack.yaml
* Add a bit of docs to cToHtml
new file: README.md
modified: package.yaml
* Add comment on comments and the CPP
modified: README.md
* Apply hlint, remove ChangeLog.md, add *.swp
modified: .gitignore
new file: cToHtml/app/.Main.hs.swp
modified: cToHtml/app/Main.hs
modified: cToHtml/package.yaml
modified: clex/src/Clex/Lex.hs
modified: clex/test/Clex/LexSpec.hs
modified: clex/test/Clex/TypesSpec.hs
* Don't need Lucid for lexer
modified: clex/package.yaml
* Fix comma typo
modified: README.md
* Fix typo and misinformation
modified: cToHtml/package.yaml
modified: clex/package.yaml
23 files changed, 1256 insertions, 0 deletions
diff --git a/homework/81248/.gitignore b/homework/81248/.gitignore new file mode 100644 index 0000000..03c5d2f --- /dev/null +++ b/homework/81248/.gitignore @@ -0,0 +1,3 @@ +.stack-work/ +*~ +*.swp diff --git a/homework/81248/README.md b/homework/81248/README.md new file mode 100644 index 0000000..829ece8 --- /dev/null +++ b/homework/81248/README.md @@ -0,0 +1,28 @@ +# Георги Любенов – 81248 + +## Stack +За разработването на `Haskell` често се използва инструментът `Stack`. + +Той се опитва да предостави възпроизведимо компилиране и се грижи за всякакви зависимости. + +В общия случай ще гледам да наглася папките, така че да може да се влезе в корена (демек `81248`), +и да се пусне `stack run X`, за да се пусне програмата `X`. От [тук][Stack] може да свалите инструмента. + +Например: +``` +stack run cToHtml +``` + +Ще пусне `cToHtml` инструмента, който конвертира `C` файл към оцветен `HTML`. +При пускане инструментите ще дават повече информация за тяхното ползване. + +Ако желаете, можете да "инсталирате" съответния инструмент във вашия `PATH`, използвайки `stack install` +(просто го копира в някаква юзърска директория, която се намира в `PATH` стандартно). + +Първото пускане на инструмента/дадена програма ще отнеме повече време (~15m), защото трябва да свали компилатор +и да компилира всичките библиотеки (и съответно програмата), които ще ползва. + +**N.B.** Ако ще инсталирате `Stack` от вашия пакетен мениджър, пуснете `stack upgrade`, защото версийте на +`Stack` в повечето дистрибуции са много стари. (`Stack` поддържа `stack run` командата само от `1.9.1` - скорошна версия) + +[Stack]: https://docs.haskellstack.org/en/stable/README/ diff --git a/homework/81248/cToHtml/.gitignore b/homework/81248/cToHtml/.gitignore new file mode 100644 index 0000000..37d0663 --- /dev/null +++ b/homework/81248/cToHtml/.gitignore @@ -0,0 +1,3 @@ +.stack-work/ +cToHtml.cabal +*~
\ No newline at end of file diff --git a/homework/81248/cToHtml/README.md b/homework/81248/cToHtml/README.md new file mode 100644 index 0000000..d56ce26 --- /dev/null +++ b/homework/81248/cToHtml/README.md @@ -0,0 +1,8 @@ +# cToHtml + +При подаден C файл го конвертира до html файл, в който всичките литерали, +ключови думи и оператори са оцветени в някакъв подаден цвят. Мързеше ме да оцветявам +всичко различно, но принципно става много лесно. + +Не обработвам препроцесорски директиви (те не са част от езика) и коментари +(бих предположил, че препроцесорът ги маха и тях). diff --git a/homework/81248/cToHtml/Setup.hs b/homework/81248/cToHtml/Setup.hs new file mode 100644 index 0000000..9a994af --- /dev/null +++ b/homework/81248/cToHtml/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/homework/81248/cToHtml/app/.Main.hs.swp b/homework/81248/cToHtml/app/.Main.hs.swp Binary files differnew file mode 100644 index 0000000..d6ddc14 --- /dev/null +++ b/homework/81248/cToHtml/app/.Main.hs.swp diff --git a/homework/81248/cToHtml/app/Main.hs b/homework/81248/cToHtml/app/Main.hs new file mode 100644 index 0000000..b504b82 --- /dev/null +++ b/homework/81248/cToHtml/app/Main.hs @@ -0,0 +1,29 @@ +module Main + ( main + ) where + +import qualified Data.ByteString as BS + +import System.Environment (getArgs) + +import Lucid (renderToFile) + +import Clex.Lex (lexC) +import Html (cToHtml) + +main :: IO () +main = do + xs <- getArgs + if length xs /= 3 + then putStrLn usage + else do + let [inFile, outFile, col] = xs + file <- BS.readFile inFile + renderToFile outFile $ cToHtml col $ either fail id $ lexC file + +usage :: String +usage = " \nUsage: cToHtml inFile.c outFile.html <colour>\n\ + \ Converts inFile.c to a colourised version in which all\n\ + \ keywords, identifiers and literals are coloured in <colour>.\n\ + \ We rely on <colour> working in a style tag.\n\ + \ (we basically set \"color:<colour>\" as a style)\n" diff --git a/homework/81248/cToHtml/package.yaml b/homework/81248/cToHtml/package.yaml new file mode 100644 index 0000000..f34147c --- /dev/null +++ b/homework/81248/cToHtml/package.yaml @@ -0,0 +1,34 @@ +name: cToHtml +version: 0.0.0.1 +author: "Georgi Lyubenov" +maintainer: "godzbanebane@gmail.com" + +description: Lex c using clex and then output coloured C. + +extra-source-files: +- README.md + +dependencies: +- base >= 4.7 && < 5 +- clex +- lucid +- bytestring +- text + +library: + source-dirs: src + +ghc-options: +- -Wall +- -Werror + +executables: + cToHtml: + main: Main.hs + source-dirs: app + ghc-options: + - -threaded + - -rtsopts + - -with-rtsopts=-N + dependencies: + - cToHtml diff --git a/homework/81248/cToHtml/sample.c b/homework/81248/cToHtml/sample.c new file mode 100644 index 0000000..dad89ad --- /dev/null +++ b/homework/81248/cToHtml/sample.c @@ -0,0 +1,95 @@ +float xs = 0.5e123213f; +float xss = .5f; + +typedef struct word_struct { + const char* begin; + int length; +} word; + +const word get_word(const char* begin){ + const char* current = begin; + short len = 0; + while( (*current != ' ') && (*current != 0) && (*current != '\n') ) { + current++; + len++; + } + word res = { + .begin = begin, + .length = len + }; + return res; +} + +void print_op(const char symb, FILE* file_write) { + fprintf(file_write, "%c", symb); +} + +const int single_char(char *line, int *count, FILE* file_write) { + char letter = *(line + *count); + switch(letter) { + case ';': print_op(';', file_write); return 1; + case '{': print_op('{', file_write); return 1; + case '}': print_op('}', file_write); return 1; + case ',': print_op(',', file_write); return 1; + case ':': print_op(':', file_write); return 1; + case '=': print_op('=', file_write); return 1; + case '(': print_op('(', file_write); return 1; + case ')': print_op(')', file_write); return 1; + case '[': print_op('[', file_write); return 1; + case ']': print_op(']', file_write); return 1; + case '.': print_op('.', file_write); return 1; + case '&': print_op('&', file_write); return 1; + case '!': print_op('!', file_write); return 1; + case '~': print_op('~', file_write); return 1; + case '-': print_op('-', file_write); return 1; + case '+': print_op('+', file_write); return 1; + case '*': print_op('*', file_write); return 1; + case '/': print_op('/', file_write); return 1; + case '%': print_op('%', file_write); return 1; + case '<': print_op('<', file_write); return 1; + case '>': print_op('>', file_write); return 1; + case '^': print_op('^', file_write); return 1; + case '|': print_op('|', file_write); return 1; + case '?': print_op('?', file_write); return 1; + default: print_op('n', file_write); return 1; + + } +} + +const int number(char *line, int *count, FILE* file_write){ + +} + +int main (void) { + + FILE *file_read; + FILE *file_write; + + file_read = fopen("example_code.c", "r"); + file_write = fopen("output", "w"); + + char *line = NULL; + size_t len = 0; + ssize_t nread; + printf("------------------------------\n\tWRITING\n------------------------------\n"); + while((nread = getline(&line, &len, file_read)) != -1) { + int counter = 0; + while(*(line + counter) != 0 ) { + if(*(line+counter) == '\n') { + print_op('\n', file_write); + } + if(single_char(line, &counter, file_write)) { + counter++; + continue; + } + if(number(line, &counter, file_write)) { + continue; + } + + + + } + } + + +} diff --git a/homework/81248/cToHtml/src/Html.hs b/homework/81248/cToHtml/src/Html.hs new file mode 100644 index 0000000..49c6eb1 --- /dev/null +++ b/homework/81248/cToHtml/src/Html.hs @@ -0,0 +1,30 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE ExtendedDefaultRules #-} + +module Html + ( cToHtml + ) where + +import qualified Data.Text as T + +import Lucid + +import Clex.Types + +cToHtml :: String -> [Token] -> Html () +cToHtml col toks = pre_ $ foldMap (colourize col) toks + +colourize :: String -> Token -> Html () +colourize _ (Whitespace bs) = toHtmlRaw bs +colourize _ (HexNumber bs) = toHtml bs +colourize _ (OctNumber bs) = toHtml bs +colourize _ (Number bs) = toHtml bs +colourize _ (CharLit bs) = toHtml bs +colourize _ (Scientific bs) = toHtml bs +colourize _ (DotFloat bs) = toHtml bs +colourize _ (LeadingDot bs) = toHtml bs +colourize _ (StringLit bs) = toHtml bs +colourize _ (Identifier bs) = toHtml bs +colourize col x = span_ [style_ $ "color:" <> col'] $ toHtml $ tokenToBS x + where col' = T.pack col diff --git a/homework/81248/cToHtml/stack.yaml b/homework/81248/cToHtml/stack.yaml new file mode 100644 index 0000000..80ae9b2 --- /dev/null +++ b/homework/81248/cToHtml/stack.yaml @@ -0,0 +1,5 @@ +resolver: lts-12.13 + +packages: +- . +- ../clex diff --git a/homework/81248/clex/.gitignore b/homework/81248/clex/.gitignore new file mode 100644 index 0000000..c0989f9 --- /dev/null +++ b/homework/81248/clex/.gitignore @@ -0,0 +1,4 @@ +.stack-work/ +clex.cabal +*~ +*.swp diff --git a/homework/81248/clex/Setup.hs b/homework/81248/clex/Setup.hs new file mode 100644 index 0000000..9a994af --- /dev/null +++ b/homework/81248/clex/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/homework/81248/clex/keyword-strings.txt b/homework/81248/clex/keyword-strings.txt new file mode 100644 index 0000000..d6c2157 --- /dev/null +++ b/homework/81248/clex/keyword-strings.txt @@ -0,0 +1,76 @@ +"auto" +"break" +"case" +"char" +"const" +"continue" +"default" +"do" +"double" +"else" +"enum" +"extern" +"float" +"for" +"goto" +"if" +"int" +"long" +"register" +"return" +"short" +"signed" +"sizeof" +"static" +"struct" +"switch" +"typedef" +"union" +"unsigned" +"void" +"volatile" +"while" +".." +"+=" +"-=" +"*=" +"/=" +"%=" +"&=" +"^=" +"|=" +"++" +"--" +"->" +"&&" +"||" +"<=" +">=" +"==" +"!=" +">>" +"<<" +"." +"<" +">" +"&" +"|" +"~" +"!" +";" +"{" +"}" +"," +":" +"=" +"(" +")" +"[" +"]" +"-" +"+" +"*" +"/" +"%" +"^" +"?" diff --git a/homework/81248/clex/package.yaml b/homework/81248/clex/package.yaml new file mode 100644 index 0000000..3bf0dc9 --- /dev/null +++ b/homework/81248/clex/package.yaml @@ -0,0 +1,33 @@ +name: clex +version: 0.0.0.1 +author: "Georgi Lyubenov" +maintainer: "godzbanebane@gmail.com" + +description: Lex C and keep whitespace so I can print it afterwards. + +dependencies: +- base >= 4.7 && < 5 +- bytestring +- attoparsec +- word8 + +library: + source-dirs: src + +ghc-options: +- -Wall +- -Werror + +tests: + clex-test: + main: Spec.hs + source-dirs: test + ghc-options: + - -threaded + - -rtsopts + - -with-rtsopts=-N + dependencies: + - clex + - QuickCheck + - hspec + - bytestring-arbitrary diff --git a/homework/81248/clex/sample.c b/homework/81248/clex/sample.c new file mode 100644 index 0000000..dad89ad --- /dev/null +++ b/homework/81248/clex/sample.c @@ -0,0 +1,95 @@ +float xs = 0.5e123213f; +float xss = .5f; + +typedef struct word_struct { + const char* begin; + int length; +} word; + +const word get_word(const char* begin){ + const char* current = begin; + short len = 0; + while( (*current != ' ') && (*current != 0) && (*current != '\n') ) { + current++; + len++; + } + word res = { + .begin = begin, + .length = len + }; + return res; +} + +void print_op(const char symb, FILE* file_write) { + fprintf(file_write, "%c", symb); +} + +const int single_char(char *line, int *count, FILE* file_write) { + char letter = *(line + *count); + switch(letter) { + case ';': print_op(';', file_write); return 1; + case '{': print_op('{', file_write); return 1; + case '}': print_op('}', file_write); return 1; + case ',': print_op(',', file_write); return 1; + case ':': print_op(':', file_write); return 1; + case '=': print_op('=', file_write); return 1; + case '(': print_op('(', file_write); return 1; + case ')': print_op(')', file_write); return 1; + case '[': print_op('[', file_write); return 1; + case ']': print_op(']', file_write); return 1; + case '.': print_op('.', file_write); return 1; + case '&': print_op('&', file_write); return 1; + case '!': print_op('!', file_write); return 1; + case '~': print_op('~', file_write); return 1; + case '-': print_op('-', file_write); return 1; + case '+': print_op('+', file_write); return 1; + case '*': print_op('*', file_write); return 1; + case '/': print_op('/', file_write); return 1; + case '%': print_op('%', file_write); return 1; + case '<': print_op('<', file_write); return 1; + case '>': print_op('>', file_write); return 1; + case '^': print_op('^', file_write); return 1; + case '|': print_op('|', file_write); return 1; + case '?': print_op('?', file_write); return 1; + default: print_op('n', file_write); return 1; + + } +} + +const int number(char *line, int *count, FILE* file_write){ + +} + +int main (void) { + + FILE *file_read; + FILE *file_write; + + file_read = fopen("example_code.c", "r"); + file_write = fopen("output", "w"); + + char *line = NULL; + size_t len = 0; + ssize_t nread; + printf("------------------------------\n\tWRITING\n------------------------------\n"); + while((nread = getline(&line, &len, file_read)) != -1) { + int counter = 0; + while(*(line + counter) != 0 ) { + if(*(line+counter) == '\n') { + print_op('\n', file_write); + } + if(single_char(line, &counter, file_write)) { + counter++; + continue; + } + if(number(line, &counter, file_write)) { + continue; + } + + + + } + } + + +} diff --git a/homework/81248/clex/src/Clex/Lex.hs b/homework/81248/clex/src/Clex/Lex.hs new file mode 100644 index 0000000..ae78c5d --- /dev/null +++ b/homework/81248/clex/src/Clex/Lex.hs @@ -0,0 +1,181 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Clex.Lex + ( lexC + ) where + + +import Prelude hiding (takeWhile) + +import Data.Foldable (asum) +import Data.Word8 ( Word8, isHexDigit, isOctDigit, isAlpha + , _e, _E, _x, _X,_f, _F, _u, _U, _l, _L, _0 + , _plus, _hyphen, _underscore, _quotesingle + , _quotedbl, _backslash, _period) + +import Data.Attoparsec.ByteString +import Data.Attoparsec.ByteString.Char8 (isDigit_w8, isSpace_w8) + +import Data.ByteString (ByteString) +import qualified Data.ByteString as BS + +import Control.Applicative ((<|>)) + +import Clex.Types + + +lexC :: ByteString -> Either String [Token] +lexC = parseOnly (cParser <* endOfInput) + +cParser :: Parser [Token] +cParser = many' tokenParser + +-- Order is important! For example we need to first check for a floating point number +-- with a leading digit, instead of one without, because the second one will always match the first one. +tokenParser :: Parser Token +tokenParser = asum $ concat + [ [whitespaceParser] + , [ scientificParser, leadingDotParser, dotFloatParser + , hexNumParser, octNumParser, numParser, charParser, stringParser + ] + , map keywordParser allKeyWords + , map operParser allOpers + , [identifierParser]] + +whitespaceParser :: Parser Token +whitespaceParser = Whitespace <$> takeWhile1 isSpace_w8 + +keywordParser :: ByteString -> Parser Token +keywordParser kw = do + parsed <- recognize <$> string kw + -- we need to check if we still have more letters after the keyword + next <- peekWord8 + let parsed' = pure parsed + case next of + Nothing -> parsed' + (Just x) -> if (not . isAlpha) x then parsed' + else fail "more input after keyword" + +operParser :: ByteString -> Parser Token +operParser = (recognize <$>) . string + +isDigit :: Word8 -> Bool +isDigit = isDigit_w8 + +isLetter :: Word8 -> Bool +isLetter b = isAlpha b + || b == _underscore + +exponentParser :: Parser ByteString +exponentParser = do + e <- satisfy (\c -> c == _E || c == _e) + sign <- option "" plusMinus + digits <- takeWhile1 isDigit_w8 + pure $ BS.singleton e <> sign <> digits + where + plusMinus = BS.singleton <$> satisfy (\c -> c == _plus || c == _hyphen) -- plus or minus + +float :: Parser Word8 +float = satisfy $ + \c -> c == _F + || c == _f + || c == _L + || c == _l + +unsigned :: Parser ByteString +unsigned = BS.pack <$> many' (satisfy $ + \c -> c == _U + || c == _u + || c == _L + || c == _l) + +isxX :: Word8 -> Bool +isxX c = c == _X + || c == _x + +singleQuote :: Parser Word8 +singleQuote = word8 _quotesingle + +doubleQuote :: Parser Word8 +doubleQuote = word8 _quotedbl + +charParser :: Parser Token +charParser = do + l <- option "" (BS.singleton <$> word8 _L) + _ <- singleQuote + str <- BS.concat <$> many1' ((BS.snoc <$> string "\\" <*> anyWord8) <|> notInWeirdClass) + sq <- singleQuote + + pure $ CharLit ((l <> (sq `BS.cons` str)) `BS.snoc` sq) + where + notInWeirdClass = BS.singleton <$> + satisfy (\c -> c /= _backslash + && c /= _quotesingle) + +stringParser :: Parser Token +stringParser = do + l <- option "" (BS.singleton <$> word8 _L) + _ <- doubleQuote + str <- BS.concat <$> many' ((BS.snoc <$> string "\\" <*> anyWord8) <|> notInWeirdClass) + dq <- doubleQuote + + pure $ StringLit ((l <> (dq `BS.cons` str)) `BS.snoc` dq) + where + notInWeirdClass = BS.singleton <$> + satisfy (\c -> c /= _backslash + && c /= _quotedbl) + +zero :: Parser Word8 +zero = word8 _0 + +hexNumParser :: Parser Token +hexNumParser = do + z <- zero + x <- satisfy isxX + hex <- takeWhile1 isHexDigit + f <- unsigned + + pure $ HexNumber (z `BS.cons` x `BS.cons` hex <> f) + +octNumParser :: Parser Token +octNumParser = do + z <- zero + oct <- takeWhile1 isOctDigit + f <- unsigned + + pure $ OctNumber (z `BS.cons` oct <> f) + +numParser :: Parser Token +numParser = do + num <- takeWhile1 isDigit + f <- unsigned + + pure $ Number (num <> f) + +scientificParser :: Parser Token +scientificParser = do + digits <- takeWhile1 isDigit + expon <- exponentParser + f <- option "" (BS.singleton <$> float) + pure $ Scientific $ digits <> expon <> f + +dotFloatParser :: Parser Token +dotFloatParser = do + leadingDigits <- takeWhile isDigit + d <- BS.singleton <$> word8 _period + afterDigits <- takeWhile1 isDigit + expon <- option "" exponentParser + f <- option "" (BS.singleton <$> float) + pure $ DotFloat $ leadingDigits <> d <> afterDigits <> expon <> f + +leadingDotParser :: Parser Token +leadingDotParser = do + leadingDigits <- takeWhile1 isDigit + d <- BS.singleton <$> word8 _period + afterDigits <- takeWhile isDigit + expon <- option "" exponentParser + f <- option "" (BS.singleton <$> float) + pure $ LeadingDot $ leadingDigits <> d <> afterDigits <> expon <> f + +identifierParser :: Parser Token +identifierParser = Identifier <$> takeWhile1 isLetter diff --git a/homework/81248/clex/src/Clex/Types.hs b/homework/81248/clex/src/Clex/Types.hs new file mode 100644 index 0000000..9083514 --- /dev/null +++ b/homework/81248/clex/src/Clex/Types.hs @@ -0,0 +1,377 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Clex.Types + ( Token(..) + , allKeyWords + , allOpers + , recognize + , tokenToBS + , tokensToBS + ) where + +import Data.ByteString (ByteString) +import qualified Data.ByteString as BS + +data Token = + Whitespace ByteString + | Auto + | Break + | Case + | Char + | Const + | Continue + | Default + | Do + | Double + | Else + | Enum + | Extern + | Float + | For + | Goto + | If + | Int + | Long + | Register + | Return + | Short + | Signed + | Sizeof + | Static + | Struct + | Switch + | Typedef + | Union + | Unsigned + | Void + | Volatile + | While + | Ellipsis + | RightShiftAssign + | LeftShiftAssign + | AddAssign + | SubAssign + | MulAssign + | DivAssign + | ModAssign + | AndAssign + | XorAssign + | OrAssign + | Incr + | Decr + | Ptr + | Dot + | And + | Or + | Not + | LtEq + | GtEq + | Eq + | NEq + | Lt + | Gt + | Semicolon + | LeftCurly + | RightCurly + | Comma + | Colon + | Assign + | LeftBracket + | RightBracket + | LeftSquare + | RightSquare + | RightShift + | LeftShift + | BitAnd + | BitOr + | BitNot + | Sub + | Add + | Mul + | Div + | Mod + | Xor + | Ternary + | HexNumber ByteString + | OctNumber ByteString + | Number ByteString + | CharLit ByteString + | Scientific ByteString + | DotFloat ByteString + | LeadingDot ByteString + | StringLit ByteString + | Identifier ByteString + deriving (Show, Eq) + +-- Mapping from ByteStrings to Tokens +recognize :: ByteString -> Token +-- keywords +recognize "auto" = Auto +recognize "break" = Break +recognize "case" = Case +recognize "char" = Char +recognize "const" = Const +recognize "continue" = Continue +recognize "default" = Default +recognize "do" = Do +recognize "double" = Double +recognize "else" = Else +recognize "enum" = Enum +recognize "extern" = Extern +recognize "float" = Float +recognize "for" = For +recognize "goto" = Goto +recognize "if" = If +recognize "int" = Int +recognize "long" = Long +recognize "register" = Register +recognize "return" = Return +recognize "short" = Short +recognize "signed" = Signed +recognize "sizeof" = Sizeof +recognize "static" = Static +recognize "struct" = Struct +recognize "switch" = Switch +recognize "typedef" = Typedef +recognize "union" = Union +recognize "unsigned" = Unsigned +recognize "void" = Void +recognize "volatile" = Volatile +recognize "while" = While + +-- operators +-- +-- two-byte +recognize "..." = Ellipsis +recognize ">>=" = RightShiftAssign +recognize "<<=" = LeftShiftAssign +recognize "+=" = AddAssign +recognize "-=" = SubAssign +recognize "*=" = MulAssign +recognize "/=" = DivAssign +recognize "%=" = ModAssign +recognize "&=" = AndAssign +recognize "^=" = XorAssign +recognize "|=" = OrAssign +recognize "++" = Incr +recognize "--" = Decr +recognize "->" = Ptr +recognize "&&" = And +recognize "||" = Or +recognize "<=" = LtEq +recognize ">=" = GtEq +recognize "==" = Eq +recognize "!=" = NEq +recognize ">>" = RightShift +recognize "<<" = LeftShift + +-- one-byte +recognize "." = Dot +recognize "<" = Lt +recognize ">" = Gt +recognize "&" = BitAnd +recognize "|" = BitOr +recognize "~" = BitNot +recognize "!" = Not +recognize ";" = Semicolon +recognize "{" = LeftCurly +recognize "}" = RightCurly +recognize "," = Comma +recognize ":" = Colon +recognize "=" = Assign +recognize "(" = LeftBracket +recognize ")" = RightBracket +recognize "[" = LeftSquare +recognize "]" = RightSquare +recognize "-" = Sub +recognize "+" = Add +recognize "*" = Mul +recognize "/" = Div +recognize "%" = Mod +recognize "^" = Xor +recognize "?" = Ternary + +-- if all else fails an identifier +recognize x = Identifier x + +allKeyWords :: [ByteString] +allKeyWords = + [ "auto" + , "break" + , "case" + , "char" + , "const" + , "continue" + , "default" + , "do" + , "double" + , "else" + , "enum" + , "extern" + , "float" + , "for" + , "goto" + , "if" + , "int" + , "long" + , "register" + , "return" + , "short" + , "signed" + , "sizeof" + , "static" + , "struct" + , "switch" + , "typedef" + , "union" + , "unsigned" + , "void" + , "volatile" + , "while" + ] + +-- Opers need to be seperate because they have different rules +-- for formation. For example you can have @"()()()"@, but you can't +-- have @"constconstconst"@. +allOpers :: [ByteString] +allOpers = + [ "..." + , ">>=" + , "<<=" + , "+=" + , "-=" + , "*=" + , "/=" + , "%=" + , "&=" + , "^=" + , "|=" + , "++" + , "--" + , "->" + , "&&" + , "||" + , "<=" + , ">=" + , "==" + , "!=" + , ">>" + , "<<" + , "." + , "<" + , ">" + , "&" + , "|" + , "~" + , "!" + , ";" + , "{" + , "}" + , "," + , ":" + , "=" + , "(" + , ")" + , "[" + , "]" + , "-" + , "+" + , "*" + , "/" + , "%" + , "^" + , "?" + ] + +tokensToBS :: [Token] -> ByteString +tokensToBS = BS.concat . map tokenToBS + +tokenToBS :: Token -> ByteString +tokenToBS (Whitespace bs) = bs +tokenToBS Auto = "auto" +tokenToBS Break = "break" +tokenToBS Case = "case" +tokenToBS Char = "char" +tokenToBS Const = "const" +tokenToBS Continue = "continue" +tokenToBS Default = "default" +tokenToBS Do = "do" +tokenToBS Double = "double" +tokenToBS Else = "else" +tokenToBS Enum = "enum" +tokenToBS Extern = "extern" +tokenToBS Float = "float" +tokenToBS For = "for" +tokenToBS Goto = "goto" +tokenToBS If = "if" +tokenToBS Int = "int" +tokenToBS Long = "long" +tokenToBS Register = "register" +tokenToBS Return = "return" +tokenToBS Short = "short" +tokenToBS Signed = "signed" +tokenToBS Sizeof = "sizeof" +tokenToBS Static = "static" +tokenToBS Struct = "struct" +tokenToBS Switch = "switch" +tokenToBS Typedef = "typedef" +tokenToBS Union = "union" +tokenToBS Unsigned = "unsigned" +tokenToBS Void = "void" +tokenToBS Volatile = "volatile" +tokenToBS While = "while" +tokenToBS Ellipsis = "..." +tokenToBS RightShiftAssign = ">>=" +tokenToBS LeftShiftAssign = "<<=" +tokenToBS AddAssign = "+=" +tokenToBS SubAssign = "-=" +tokenToBS MulAssign = "*=" +tokenToBS DivAssign = "/=" +tokenToBS ModAssign = "%=" +tokenToBS AndAssign = "&=" +tokenToBS XorAssign = "^=" +tokenToBS OrAssign = "|=" +tokenToBS Incr = "++" +tokenToBS Decr = "--" +tokenToBS Ptr = "->" +tokenToBS Dot = "." +tokenToBS And = "&&" +tokenToBS Or = "||" +tokenToBS Not = "!" +tokenToBS LtEq = "<=" +tokenToBS GtEq = ">=" +tokenToBS Eq = "==" +tokenToBS NEq = "!=" +tokenToBS Lt = "<" +tokenToBS Gt = ">" +tokenToBS Semicolon = ";" +tokenToBS LeftCurly = "{" +tokenToBS RightCurly = "}" +tokenToBS Comma = "," +tokenToBS Colon = ":" +tokenToBS Assign = "=" +tokenToBS LeftBracket = "(" +tokenToBS RightBracket = ")" +tokenToBS LeftSquare = "[" +tokenToBS RightSquare = "]" +tokenToBS RightShift = ">>" +tokenToBS LeftShift = "<<" +tokenToBS BitAnd = "&" +tokenToBS BitOr = "|" +tokenToBS BitNot = "~" +tokenToBS Sub = "-" +tokenToBS Add = "+" +tokenToBS Mul = "*" +tokenToBS Div = "/" +tokenToBS Mod = "%" +tokenToBS Xor = "^" +tokenToBS Ternary = "?" +tokenToBS (HexNumber bs) = bs +tokenToBS (OctNumber bs) = bs +tokenToBS (Number bs) = bs +tokenToBS (CharLit bs) = bs +tokenToBS (Scientific bs) = bs +tokenToBS (DotFloat bs) = bs +tokenToBS (LeadingDot bs) = bs +tokenToBS (StringLit bs) = bs +tokenToBS (Identifier bs) = bs diff --git a/homework/81248/clex/stack.yaml b/homework/81248/clex/stack.yaml new file mode 100644 index 0000000..5512547 --- /dev/null +++ b/homework/81248/clex/stack.yaml @@ -0,0 +1,7 @@ +resolver: lts-12.13 + +packages: +- . + +extra-deps: +- bytestring-arbitrary-0.1.2 diff --git a/homework/81248/clex/test/Clex/LexSpec.hs b/homework/81248/clex/test/Clex/LexSpec.hs new file mode 100644 index 0000000..c5fec81 --- /dev/null +++ b/homework/81248/clex/test/Clex/LexSpec.hs @@ -0,0 +1,58 @@ +module Clex.LexSpec + ( spec + ) where + +import qualified Data.ByteString as BS +import Data.ByteString (ByteString) +import Data.ByteString.Char8 (pack) + +import Clex.Types +import Clex.Lex (lexC) + +import Test.QuickCheck +import Test.Hspec +import Test.Hspec.QuickCheck +import Clex.TypesSpec () + +spec :: Spec +spec = describe "lexing" $ do + prop "tokensToBS after lexC is id" $ + \x -> let x' = unValidC x in + case lexC x' of + (Left _) -> False + (Right x'') -> x' == tokensToBS x'' + prop "lexing after tokensToBS is id" $ + \x -> let x' = unTokenC x in + case lexC $ tokensToBS x' of + (Left _) -> False + (Right x'') -> x' == x'' + +newtype TokenC = TokenC {unTokenC :: [Token]} + deriving (Eq, Show) + +newtype ValidC = ValidC {unValidC :: ByteString} + deriving (Eq, Show) + +instance Arbitrary TokenC where + arbitrary = TokenC . packWhitespaceTokens . concat <$> + listOf1 (do + x <- arbitrary + whitespace <- whitespaceGen + pure [x, whitespace]) + where + whitespaceGen = Whitespace . pack <$> do listOf1 $ elements ['\t', ' ', '\n'] + +instance Arbitrary ValidC where + arbitrary = ValidC . BS.concat . map tokenToBS <$> listOf1 (frequency + [ (30, arbitrary) + , (60, whitespaceGen) + ]) + where + whitespaceGen = Whitespace . pack <$> do listOf1 $ elements ['\t', ' ', '\n'] + +packWhitespaceTokens :: [Token] -> [Token] +packWhitespaceTokens [] = [] +packWhitespaceTokens [x] = [x] +packWhitespaceTokens (Whitespace x : Whitespace y : xs) + = packWhitespaceTokens $ Whitespace (x <> y) : xs +packWhitespaceTokens (x:xs) = x : packWhitespaceTokens xs diff --git a/homework/81248/clex/test/Clex/TypesSpec.hs b/homework/81248/clex/test/Clex/TypesSpec.hs new file mode 100644 index 0000000..805affc --- /dev/null +++ b/homework/81248/clex/test/Clex/TypesSpec.hs @@ -0,0 +1,177 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# OPTIONS -fno-warn-orphans #-} + +module Clex.TypesSpec + ( spec + ) where + +import Data.ByteString.Char8 (pack) + +import Clex.Types + +import Test.QuickCheck +import Test.Hspec +import Test.Hspec.QuickCheck +import Data.ByteString.Arbitrary + +spec :: Spec +spec = describe "types" + conversionSpec + +conversionSpec :: Spec +conversionSpec = describe "ByteString conversions" $ do + prop "recognize is inverse to tokenToBS" $ + \x -> (not . isLiteral) x ==> x == recognize (tokenToBS x) + prop "tokenToBS is inverse to recognize" $ + \x -> let x' = fromABS x in x' == tokenToBS (recognize x') + +isLiteral :: Token -> Bool +isLiteral (HexNumber _) = True +isLiteral (OctNumber _) = True +isLiteral (Number _) = True +isLiteral (CharLit _) = True +isLiteral (Scientific _) = True +isLiteral (DotFloat _) = True +isLiteral (LeadingDot _) = True +isLiteral (StringLit _) = True +isLiteral _ = False + +instance Arbitrary Token where + arbitrary = frequency + [ (30, nullaryGen) + , (30, identifierGen) + , (10, literalGen) + ] + where + nullaryGen = elements + [ Auto + , Break + , Case + , Char + , Const + , Continue + , Default + , Do + , Double + , Else + , Enum + , Extern + , Float + , For + , Goto + , If + , Int + , Long + , Register + , Return + , Short + , Signed + , Sizeof + , Static + , Struct + , Switch + , Typedef + , Union + , Unsigned + , Void + , Volatile + , While + , Ellipsis + , RightShiftAssign + , LeftShiftAssign + , AddAssign + , SubAssign + , MulAssign + , DivAssign + , ModAssign + , AndAssign + , XorAssign + , OrAssign + , Incr + , Decr + , Ptr + , Dot + , And + , Or + , Not + , LtEq + , GtEq + , Eq + , NEq + , Lt + , Gt + , Semicolon + , LeftCurly + , RightCurly + , Comma + , Colon + , Assign + , LeftBracket + , RightBracket + , LeftSquare + , RightSquare + , RightShift + , LeftShift + , BitAnd + , BitOr + , BitNot + , Sub + , Add + , Mul + , Div + , Mod + , Xor + , Ternary + ] + identifierGen = Identifier <$> (pack <$> do listOf1 $ elements $ '_' : ['a'..'z'] ++ ['A'..'Z']) + `suchThat` (not . reserved) + reserved = (`elem` allKeyWords) + -- don't feel like overdoing this right now + literalGen = oneof + [ elements + [ HexNumber "0xb23abcf4" + , HexNumber "0x123ab5f4ul" + , HexNumber "0X123a4cf4" + , HexNumber "0x1e3ebcf4lull" + + , OctNumber "0234" + , OctNumber "012354ul" + , OctNumber "012744" + , OctNumber "0134lull" + + , Number "234" + , Number "12354ul" + , Number "12744" + , Number "134lull" + ] + , elements + [ CharLit "'c'" + , CharLit "L'c'" + , CharLit "'\n'" + , CharLit "'\t'" + , CharLit "'a'" + ] + , elements + [ DotFloat ".213E+10f" + , DotFloat ".213e-10" + , DotFloat ".213e-10l" + , DotFloat ".213e10l" + , DotFloat ".213l" + , DotFloat ".213" + , DotFloat ".213e9L" + + , LeadingDot "213.2E+10f" + , LeadingDot "213.3e-10" + , LeadingDot "213.e-10l" + , LeadingDot "213.e10l" + , LeadingDot "213.e9L" + , LeadingDot "213.9L" + , LeadingDot "213.L" + ] + , elements + [ StringLit "\"\"" + , StringLit "L\"asdfwqe\"" + , StringLit "\"asd\nfw\tqe\"" + , StringLit "L\"\"" + ] + ] diff --git a/homework/81248/clex/test/Spec.hs b/homework/81248/clex/test/Spec.hs new file mode 100644 index 0000000..a824f8c --- /dev/null +++ b/homework/81248/clex/test/Spec.hs @@ -0,0 +1 @@ +{-# OPTIONS_GHC -F -pgmF hspec-discover #-} diff --git a/homework/81248/stack.yaml b/homework/81248/stack.yaml new file mode 100644 index 0000000..ed8960d --- /dev/null +++ b/homework/81248/stack.yaml @@ -0,0 +1,8 @@ +resolver: lts-12.13 + +packages: +- ./clex +- ./cToHtml + +extra-deps: +- bytestring-arbitrary-0.1.2 |
