24 Commits

Author SHA1 Message Date
Phillip Burch
a9696e2898
fix: retain line breaks in tokens properly (#2341)
* Fix lexer and tokenizer to retain line breaks properly

* Add test for bug

* Check for line breaks not just spaces

* Fix lint

* Fix spacing in test

* clean up code

Co-authored-by: Tony Brix <tony@brix.ninja>
2022-01-06 09:31:58 -06:00
Emmet Jang
5714212afd
fix: table after paragraph without blank line (#2298)
* fix: gfm table requires leading empty line

* test(gfm/table): add some test cases

* revert changes to `gfm.0.29.json`

* test(gfm/table): add `table_following_text` testcase
2021-11-24 18:10:22 -06:00
Ben McCann
4afb228d95
fix: Convert to ESM (#2227)
BREAKING CHANGE:

- Default export removed. Use `import { marked } from 'marked'` or `const { marked } = require('marked')` instead.
- `/lib/marked.js` removed. Use `/marked.min.js` in script tag instead.
- When using marked in a script tag use `marked.parse(...)` instead of `marked(...)`
2021-11-02 09:32:17 -05:00
Trevor Buckner
bc400ac789
fix: Refactor table tokens (#2166)
BREAKING CHANGE:

- `table` tokens `header` property changed to contain an array of objects for each header cell with `text` and `tokens` properties.
- `table` tokens `cells` property changed to `rows` and is an array of rows where each row contains an array of objects for each cell with `text` and `tokens` properties.

v2:

```json
{
  "type": "table",
  "align": [null, null],
  "raw": "| a | b |\n|---|---|\n| 1 | 2 |\n",
  "header": ["a", "b"],
  "cells": [["1", "2"]],
  "tokens": {
    "header": [
      [{ "type": "text", "raw": "a", "text": "a" }],
      [{ "type": "text", "raw": "b", "text": "b" }]
    ],
    "cells": [[
      [{ "type": "text", "raw": "1", "text": "1" }],
      [{ "type": "text", "raw": "2", "text": "2" }]
    ]]
  }
}
```

v3:

```json
{
  "type": "table",
  "align": [null, null],
  "raw": "| a | b |\n|---|---|\n| 1 | 2 |\n",
  "header": [
    {
      "text": "a",
      "tokens": [{ "type": "text", "raw": "a", "text": "a" }]
    },
    {
      "text": "b",
      "tokens": [{ "type": "text", "raw": "b", "text": "b" }]
    }
  ],
  "rows": [
    {
      "text": "1",
      "tokens": [{ "type": "text", "raw": "1", "text": "1" }]
    },
    {
      "text": "2",
      "tokens": [{ "type": "text", "raw": "2", "text": "2" }]
    }
  ]
}
```
2021-08-15 22:02:39 -05:00
Trevor Buckner
eb33d3b3a9
fix: Full Commonmark compliance for Lists (#2112) 2021-08-09 22:41:45 -05:00
Trevor Buckner
288f1cbe2f
Tokenizers lex their own child tokens (#2124)
BREAKING CHANGES:

- Tokenizers will create their own tokens with `this.lexer.inline(text, tokens)`. The `inline` function will queue the token creation until after all block tokens are rendered.
- `nptable` tokenizer is removed and merged with `table` tokenizer.
- Extensions tokenizer `this` object will include the `lexer` as a property. `this.inlineTokens` becomes `this.lexer.inline`.
- Extensions parser `this` object will include the `parser` as a property. `this.parseInline` becomes `this.parser.parseInline`.
- `tag` and `inlineText` tokenizer function signatures have changed.
2021-08-02 14:12:43 -05:00
Tony Brix
46cdfc1ece
fix: fix items between lists (#1936) 2021-02-26 23:51:21 -06:00
Trevor Buckner
f848e77039
fix: Join adjacent inlineText tokens (#1926) 2021-02-07 16:22:47 -06:00
jun-sheaf
fcfe15fba6 Fixed Example 272 2020-06-17 18:28:37 +02:00
jun-sheaf
5920cf9cf8 Updated rules and created tests. 2020-06-17 11:50:00 +02:00
Randolf C
b4b909c00d
Update Lexer-spec.js 2020-06-17 11:33:12 +02:00
Randolf C
242933e9e1
Update Lexer-spec.js 2020-06-17 11:32:45 +02:00
Randolf C
33d30247e9
Update Lexer-spec.js 2020-06-17 11:29:56 +02:00
Randolf C
1dccde3075
Update Lexer-spec.js 2020-06-17 11:28:52 +02:00
Randolf C
c3a964d6c2
Update Lexer-spec.js 2020-06-17 11:28:11 +02:00
Tony Brix
cb42b9003c add list_item type to list item token 2020-05-12 19:57:27 -05:00
Tony Brix
c2d6d2fa31 only strip spaces 2020-04-24 01:00:27 -05:00
Tony Brix
6ad20acfc1 add newline to space test 2020-04-24 00:51:16 -05:00
Tony Brix
60d810f3de test only remove one space 2020-04-24 00:44:02 -05:00
Tony Brix
143ae5d09a fix codespan spaces 2020-04-24 00:39:47 -05:00
Tony Brix
4947ce5078 only send needed variables to tokenizer 2020-04-14 13:29:51 -05:00
Tony Brix
17d81796da add tokenizer 2020-04-08 13:04:29 -05:00
Tony Brix
5da102d024 rename functions 2020-04-06 22:33:40 -05:00
Tony Brix
4d24272617 add tests 2020-04-02 00:24:21 -05:00