clo/sitegen
clo/sitegen
1
Code Issues 44 Pull requests 2 Activity

framework: remove @mdx-js/mdx and other markdown packages #1

New issue
Closed
opened 2025-06-08 16:13:51 -07:00 by clo · 4 comments
clo commented 2025-06-08 16:13:51 -07:00
Owner
Copy link

the two biggest dependencies of the framework are

  • @mdx-js/mdx
  • marko (separate issue, unlikely to take action)
MDX has 111 total dependencies. which is insane because it is not doing anything revolutionary 
@mdx-js/mdx@3.1.0
+-- @types/estree-jsx@1.0.5
+-- @types/estree@1.0.8
+-- @types/hast@3.0.4
| `-- @types/unist@3.0.3
+-- @types/mdx@2.0.13
+-- collapse-white-space@2.1.0
+-- devlop@1.1.0
| `-- dequal@2.0.3
+-- estree-util-is-identifier-name@3.0.0
+-- estree-util-scope@1.0.0
+-- estree-walker@3.0.3
+-- hast-util-to-jsx-runtime@2.3.6
| +-- comma-separated-tokens@2.0.3
| +-- hast-util-whitespace@3.0.0
| +-- mdast-util-mdx-expression@2.0.1
| | `-- mdast-util-to-markdown@2.1.2
| |   +-- longest-streak@3.1.0
| |   +-- mdast-util-phrasing@4.1.0
| |   +-- micromark-util-classify-character@2.0.1
| +-- mdast-util-mdx-jsx@3.2.0
| | +-- ccount@2.0.1
| | +-- parse-entities@4.0.2
| | | +-- @types/unist@2.0.11
| | | +-- character-entities-legacy@3.0.0
| | | +-- character-reference-invalid@2.0.1
| | | +-- is-alphanumerical@2.0.1
| | | | +-- is-alphabetical@2.0.1
| | | +-- is-decimal@2.0.1
| | | `-- is-hexadecimal@2.0.1
| | +-- stringify-entities@4.0.4
| | | +-- character-entities-html4@2.1.0
| +-- mdast-util-mdxjs-esm@2.0.1
| +-- property-information@7.1.0
| +-- space-separated-tokens@2.0.2
| +-- style-to-js@1.1.16
| | `-- style-to-object@1.0.8
| |   `-- inline-style-parser@0.2.4
| +-- unist-util-position@5.0.0
| `-- vfile-message@4.0.2
+-- markdown-extensions@2.0.0
+-- recma-build-jsx@1.0.0
| +-- estree-util-build-jsx@3.0.1
+-- recma-jsx@1.0.0
| +-- acorn-jsx@5.3.2
| | `-- acorn@8.15.0
| +-- estree-util-to-js@2.0.0
| | +-- astring@1.9.0
| +-- recma-parse@1.0.0
| | +-- esast-util-from-js@2.0.1
| | | +-- esast-util-from-estree@2.0.0
| | | | +-- estree-util-visit@2.0.0
+-- recma-stringify@1.0.0
+-- rehype-recma@1.0.0
| `-- hast-util-to-estree@3.1.3
|   +-- estree-util-attach-comments@3.0.0
|   `-- zwitch@2.0.4
+-- remark-mdx@3.1.0
| +-- mdast-util-mdx@3.0.0
| `-- micromark-extension-mdxjs@3.0.0
|   +-- micromark-extension-mdx-expression@3.0.1
|   | +-- micromark-factory-mdx-expression@2.0.3
|   | +-- micromark-factory-space@2.0.1
|   | +-- micromark-util-character@2.1.1
|   | +-- micromark-util-events-to-acorn@2.0.3
|   +-- micromark-extension-mdx-jsx@3.0.2
|   +-- micromark-extension-mdx-md@2.0.0
|   +-- micromark-extension-mdxjs-esm@3.0.0
|   | +-- micromark-core-commonmark@2.0.3
|   | | +-- micromark-factory-destination@2.0.1
|   | | +-- micromark-factory-label@2.0.1
|   | | +-- micromark-factory-title@2.0.1
|   | | +-- micromark-factory-whitespace@2.0.1
|   | | +-- micromark-util-html-tag-name@2.0.1
|   +-- micromark-util-combine-extensions@2.0.1
|   | +-- micromark-util-chunked@2.0.1
+-- remark-parse@11.0.0
| +-- @types/mdast@4.0.4
| +-- mdast-util-from-markdown@2.0.2
| | +-- decode-named-character-reference@1.1.0
| | | `-- character-entities@2.0.2
| | +-- mdast-util-to-string@4.0.0
| | +-- micromark-util-decode-numeric-character-reference@2.0.2
| | +-- micromark-util-decode-string@2.0.1
| | +-- micromark-util-normalize-identifier@2.0.1
| | +-- micromark-util-symbol@2.0.1
| | +-- micromark@4.0.2
| | | +-- @types/debug@4.1.12
| | | | `-- @types/ms@2.1.0
| | | +-- debug@4.4.1
| | | | `-- ms@2.1.3
| | | +-- micromark-util-encode@2.0.1
| | | +-- micromark-util-resolve-all@2.0.1
| | | +-- micromark-util-subtokenize@2.1.0
| +-- micromark-util-types@2.0.2
+-- remark-rehype@11.1.2
| +-- mdast-util-to-hast@13.2.0
| | +-- @ungap/structured-clone@1.3.0
| | +-- micromark-util-sanitize-uri@2.0.1
| | +-- trim-lines@3.0.1
+-- source-map@0.7.4
+-- unified@11.0.5
| +-- bail@2.0.2
| +-- extend@3.0.2
| +-- is-plain-obj@4.1.0
| +-- trough@2.2.0
+-- unist-util-position-from-estree@2.0.0
+-- unist-util-stringify-position@4.0.0
+-- unist-util-visit@5.0.0
| +-- unist-util-is@6.0.0
| `-- unist-util-visit-parents@6.0.1
`-- vfile@6.0.3
for completion, marko has 71 dependencies. not off the hook but this dependency provides a lot of value currently. 
marko@6.0.21
+-- @marko/compiler@5.39.21
| +-- @babel/code-frame@7.27.1
| | +-- @babel/helper-validator-identifier@7.27.1
| | +-- js-tokens@4.0.0
| | `-- picocolors@1.1.1
| +-- @babel/core@7.27.4
| | +-- @ampproject/remapping@2.3.0
| | +-- @babel/helper-compilation-targets@7.27.2
| | | +-- @babel/compat-data@7.27.5
| | | +-- @babel/helper-validator-option@7.27.1
| | | +-- browserslist@4.25.0
| | | | +-- caniuse-lite@1.0.30001721
| | | | +-- electron-to-chromium@1.5.165
| | | | +-- node-releases@2.0.19
| | | | `-- update-browserslist-db@1.1.3
| | | |   +-- escalade@3.2.0
| | | +-- lru-cache@5.1.1
| | | | `-- yallist@3.1.1
| | +-- @babel/helper-module-transforms@7.27.3
| | | +-- @babel/helper-module-imports@7.27.1
| | +-- @babel/helpers@7.27.6
| | +-- @babel/template@7.27.2
| | +-- convert-source-map@2.0.0
| | +-- debug@4.4.1
| | | `-- ms@2.1.3
| | +-- gensync@1.0.0-beta.2
| | +-- json5@2.2.3
| | `-- semver@6.3.1
| +-- @babel/generator@7.27.5
| | +-- @jridgewell/gen-mapping@0.3.8
| | | +-- @jridgewell/set-array@1.2.1
| | +-- @jridgewell/trace-mapping@0.3.25
| | | +-- @jridgewell/resolve-uri@3.1.2
| +-- @babel/parser@7.27.5
| +-- @babel/plugin-syntax-typescript@7.27.1
| | `-- @babel/helper-plugin-utils@7.27.1
| +-- @babel/plugin-transform-modules-commonjs@7.27.1
| +-- @babel/plugin-transform-typescript@7.27.1
| | +-- @babel/helper-annotate-as-pure@7.27.3
| | +-- @babel/helper-create-class-features-plugin@7.27.1
| | | +-- @babel/helper-member-expression-to-functions@7.27.1
| | | +-- @babel/helper-optimise-call-expression@7.27.1
| | | +-- @babel/helper-replace-supers@7.27.1
| | +-- @babel/helper-skip-transparent-expression-wrappers@7.27.1
| +-- @babel/runtime@7.27.6
| +-- @babel/traverse@7.27.4
| | `-- globals@11.12.0
| +-- @babel/types@7.27.6
| | +-- @babel/helper-string-parser@7.27.1
| +-- @luxass/strip-json-comments@1.4.0
| +-- complain@1.6.1
| | `-- error-stack-parser@2.1.4
| |   `-- stackframe@1.3.4
| +-- he@1.2.0
| +-- htmljs-parser@5.5.4
| +-- jsesc@3.1.0
| +-- kleur@4.1.5
| +-- lasso-package-root@1.0.1
| | `-- lasso-caching-fs@1.0.2
| |   `-- raptor-async@1.1.3
| +-- raptor-regexp@1.0.1
| +-- raptor-util@3.2.0
| +-- relative-import-path@1.0.0
| +-- resolve-from@5.0.0
| +-- self-closing-tags@1.0.1
| `-- source-map-support@0.5.21
|   +-- buffer-from@1.1.2
|   `-- source-map@0.6.1
+-- csstype@3.1.3
`-- magic-string@0.30.17
  `-- @jridgewell/sourcemap-codec@1.5.0

there is already an extensible markdown parser in ./src/q+a/simple-markdown.ts. this file probably needs a lot of cleanup, because it is old. it can be moved into the framework. the JSX segments can be emitted verbatim since the file can go through esbuild. this also lets us try a "markodown" version where marko syntax is used instead of JSX. there is a good chance that would be much more preferable than JSX since it is universally less verbose to write.

the two biggest dependencies of the framework are - `@mdx-js/mdx` - `marko` (separate issue, unlikely to take action) <details> <summary>MDX has 111 total dependencies. which is insane because it is not doing anything revolutionary</summary> ``` @mdx-js/mdx@3.1.0 +-- @types/estree-jsx@1.0.5 +-- @types/estree@1.0.8 +-- @types/hast@3.0.4 | `-- @types/unist@3.0.3 +-- @types/mdx@2.0.13 +-- collapse-white-space@2.1.0 +-- devlop@1.1.0 | `-- dequal@2.0.3 +-- estree-util-is-identifier-name@3.0.0 +-- estree-util-scope@1.0.0 +-- estree-walker@3.0.3 +-- hast-util-to-jsx-runtime@2.3.6 | +-- comma-separated-tokens@2.0.3 | +-- hast-util-whitespace@3.0.0 | +-- mdast-util-mdx-expression@2.0.1 | | `-- mdast-util-to-markdown@2.1.2 | | +-- longest-streak@3.1.0 | | +-- mdast-util-phrasing@4.1.0 | | +-- micromark-util-classify-character@2.0.1 | +-- mdast-util-mdx-jsx@3.2.0 | | +-- ccount@2.0.1 | | +-- parse-entities@4.0.2 | | | +-- @types/unist@2.0.11 | | | +-- character-entities-legacy@3.0.0 | | | +-- character-reference-invalid@2.0.1 | | | +-- is-alphanumerical@2.0.1 | | | | +-- is-alphabetical@2.0.1 | | | +-- is-decimal@2.0.1 | | | `-- is-hexadecimal@2.0.1 | | +-- stringify-entities@4.0.4 | | | +-- character-entities-html4@2.1.0 | +-- mdast-util-mdxjs-esm@2.0.1 | +-- property-information@7.1.0 | +-- space-separated-tokens@2.0.2 | +-- style-to-js@1.1.16 | | `-- style-to-object@1.0.8 | | `-- inline-style-parser@0.2.4 | +-- unist-util-position@5.0.0 | `-- vfile-message@4.0.2 +-- markdown-extensions@2.0.0 +-- recma-build-jsx@1.0.0 | +-- estree-util-build-jsx@3.0.1 +-- recma-jsx@1.0.0 | +-- acorn-jsx@5.3.2 | | `-- acorn@8.15.0 | +-- estree-util-to-js@2.0.0 | | +-- astring@1.9.0 | +-- recma-parse@1.0.0 | | +-- esast-util-from-js@2.0.1 | | | +-- esast-util-from-estree@2.0.0 | | | | +-- estree-util-visit@2.0.0 +-- recma-stringify@1.0.0 +-- rehype-recma@1.0.0 | `-- hast-util-to-estree@3.1.3 | +-- estree-util-attach-comments@3.0.0 | `-- zwitch@2.0.4 +-- remark-mdx@3.1.0 | +-- mdast-util-mdx@3.0.0 | `-- micromark-extension-mdxjs@3.0.0 | +-- micromark-extension-mdx-expression@3.0.1 | | +-- micromark-factory-mdx-expression@2.0.3 | | +-- micromark-factory-space@2.0.1 | | +-- micromark-util-character@2.1.1 | | +-- micromark-util-events-to-acorn@2.0.3 | +-- micromark-extension-mdx-jsx@3.0.2 | +-- micromark-extension-mdx-md@2.0.0 | +-- micromark-extension-mdxjs-esm@3.0.0 | | +-- micromark-core-commonmark@2.0.3 | | | +-- micromark-factory-destination@2.0.1 | | | +-- micromark-factory-label@2.0.1 | | | +-- micromark-factory-title@2.0.1 | | | +-- micromark-factory-whitespace@2.0.1 | | | +-- micromark-util-html-tag-name@2.0.1 | +-- micromark-util-combine-extensions@2.0.1 | | +-- micromark-util-chunked@2.0.1 +-- remark-parse@11.0.0 | +-- @types/mdast@4.0.4 | +-- mdast-util-from-markdown@2.0.2 | | +-- decode-named-character-reference@1.1.0 | | | `-- character-entities@2.0.2 | | +-- mdast-util-to-string@4.0.0 | | +-- micromark-util-decode-numeric-character-reference@2.0.2 | | +-- micromark-util-decode-string@2.0.1 | | +-- micromark-util-normalize-identifier@2.0.1 | | +-- micromark-util-symbol@2.0.1 | | +-- micromark@4.0.2 | | | +-- @types/debug@4.1.12 | | | | `-- @types/ms@2.1.0 | | | +-- debug@4.4.1 | | | | `-- ms@2.1.3 | | | +-- micromark-util-encode@2.0.1 | | | +-- micromark-util-resolve-all@2.0.1 | | | +-- micromark-util-subtokenize@2.1.0 | +-- micromark-util-types@2.0.2 +-- remark-rehype@11.1.2 | +-- mdast-util-to-hast@13.2.0 | | +-- @ungap/structured-clone@1.3.0 | | +-- micromark-util-sanitize-uri@2.0.1 | | +-- trim-lines@3.0.1 +-- source-map@0.7.4 +-- unified@11.0.5 | +-- bail@2.0.2 | +-- extend@3.0.2 | +-- is-plain-obj@4.1.0 | +-- trough@2.2.0 +-- unist-util-position-from-estree@2.0.0 +-- unist-util-stringify-position@4.0.0 +-- unist-util-visit@5.0.0 | +-- unist-util-is@6.0.0 | `-- unist-util-visit-parents@6.0.1 `-- vfile@6.0.3 ``` </details> <details> <summary> for completion, marko has 71 dependencies. not off the hook but this dependency provides a lot of value currently. </summary> ``` marko@6.0.21 +-- @marko/compiler@5.39.21 | +-- @babel/code-frame@7.27.1 | | +-- @babel/helper-validator-identifier@7.27.1 | | +-- js-tokens@4.0.0 | | `-- picocolors@1.1.1 | +-- @babel/core@7.27.4 | | +-- @ampproject/remapping@2.3.0 | | +-- @babel/helper-compilation-targets@7.27.2 | | | +-- @babel/compat-data@7.27.5 | | | +-- @babel/helper-validator-option@7.27.1 | | | +-- browserslist@4.25.0 | | | | +-- caniuse-lite@1.0.30001721 | | | | +-- electron-to-chromium@1.5.165 | | | | +-- node-releases@2.0.19 | | | | `-- update-browserslist-db@1.1.3 | | | | +-- escalade@3.2.0 | | | +-- lru-cache@5.1.1 | | | | `-- yallist@3.1.1 | | +-- @babel/helper-module-transforms@7.27.3 | | | +-- @babel/helper-module-imports@7.27.1 | | +-- @babel/helpers@7.27.6 | | +-- @babel/template@7.27.2 | | +-- convert-source-map@2.0.0 | | +-- debug@4.4.1 | | | `-- ms@2.1.3 | | +-- gensync@1.0.0-beta.2 | | +-- json5@2.2.3 | | `-- semver@6.3.1 | +-- @babel/generator@7.27.5 | | +-- @jridgewell/gen-mapping@0.3.8 | | | +-- @jridgewell/set-array@1.2.1 | | +-- @jridgewell/trace-mapping@0.3.25 | | | +-- @jridgewell/resolve-uri@3.1.2 | +-- @babel/parser@7.27.5 | +-- @babel/plugin-syntax-typescript@7.27.1 | | `-- @babel/helper-plugin-utils@7.27.1 | +-- @babel/plugin-transform-modules-commonjs@7.27.1 | +-- @babel/plugin-transform-typescript@7.27.1 | | +-- @babel/helper-annotate-as-pure@7.27.3 | | +-- @babel/helper-create-class-features-plugin@7.27.1 | | | +-- @babel/helper-member-expression-to-functions@7.27.1 | | | +-- @babel/helper-optimise-call-expression@7.27.1 | | | +-- @babel/helper-replace-supers@7.27.1 | | +-- @babel/helper-skip-transparent-expression-wrappers@7.27.1 | +-- @babel/runtime@7.27.6 | +-- @babel/traverse@7.27.4 | | `-- globals@11.12.0 | +-- @babel/types@7.27.6 | | +-- @babel/helper-string-parser@7.27.1 | +-- @luxass/strip-json-comments@1.4.0 | +-- complain@1.6.1 | | `-- error-stack-parser@2.1.4 | | `-- stackframe@1.3.4 | +-- he@1.2.0 | +-- htmljs-parser@5.5.4 | +-- jsesc@3.1.0 | +-- kleur@4.1.5 | +-- lasso-package-root@1.0.1 | | `-- lasso-caching-fs@1.0.2 | | `-- raptor-async@1.1.3 | +-- raptor-regexp@1.0.1 | +-- raptor-util@3.2.0 | +-- relative-import-path@1.0.0 | +-- resolve-from@5.0.0 | +-- self-closing-tags@1.0.1 | `-- source-map-support@0.5.21 | +-- buffer-from@1.1.2 | `-- source-map@0.6.1 +-- csstype@3.1.3 `-- magic-string@0.30.17 `-- @jridgewell/sourcemap-codec@1.5.0 ``` </details> there is already an extensible markdown parser in `./src/q+a/simple-markdown.ts`. this file probably needs a lot of cleanup, because it is old. it can be moved into the framework. the JSX segments can be emitted verbatim since the file can go through esbuild. this also lets us try a "markodown" version where marko syntax is used instead of JSX. there is a good chance that would be much more preferable than JSX since it is universally less verbose to write.
clo commented 2025-07-08 01:11:14 -07:00
Author
Owner
Copy link

new code is here. it'll probably move to engine since it is another way to use the SSR engine instead of a markdown parser

https://git.paperclover.net/clo/sitegen/src/branch/master/framework/lib/markdown.tsx

new code is here. it'll probably move to engine since it is another way to use the SSR engine instead of a markdown parser ~~https://git.paperclover.net/clo/sitegen/src/branch/master/framework/lib/markdown.tsx~~
clo commented 2025-08-11 21:16:14 -07:00
Author
Owner
Copy link

damn markdown parsing is complicated

damn markdown parsing is complicated
clo commented 2025-10-01 17:06:06 -07:00
Author
Owner
Copy link

i consider deferring this and using micromark

i consider deferring this and using micromark
clo changed title from framework: remove @mdx-js/mdx and replace with custom implementation to framework: remove @mdx-js/mdx and replace with simpler/custom implementation 2025-10-07 01:25:56 -07:00
clo changed title from framework: remove @mdx-js/mdx and replace with simpler/custom implementation to replace MDX with Markodown 2026-02-04 04:20:37 -08:00
clo changed title from replace MDX with Markodown to framework: remove @mdx-js/mdx and other markdown packages 2026-02-09 14:46:30 -08:00
clo commented 2026-02-09 14:47:11 -08:00
Author
Owner
Copy link

https://git.paperclover.net/clo/markodown

https://git.paperclover.net/clo/markodown
clo closed this issue 2026-02-18 02:37:25 -08:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
blogs
markodown
progress-ema
fix-log
No results found.
Labels
Clear labels   
bug

   
chore

   
feat

   
lib

   
site

paperclover.net

No labels
bug
chore
feat
lib
site
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
clo/sitegen#1
No description provided.