issue_comments

Those sounds to me like they should be promoted to documented, supported internals.

@eyeseast I started work on that plugin: https://github.com/simonw/datasette-export

Had a thought about the implementation of this: it could make a really neat plugin.

Something like datasette-export which adds a export command using https://docs.datasette.io/en/stable/plugin_hooks.html#register-commands-cli - then you could run:

datasette export my-export-dir mydatabase.db -m metadata.json --template-dir templates/

And the command would then:

• Create a Datasette() instance with those databases/metadata/etc
• Executeawait datasette.client.get("/") to get the homepage HTML
• Parse the HTML using BeautifulSoup to find all a[href], link[href], script[src], img[src] elements that reference a relative path as opposed to one that starts with http://
• Write out the homepage to my-export-dir/index.html
• Recursively fetch and dump all of the other pages and assets that it found too

All of that HTML parsing may be over-complicating things. It could alternatively accept options for which pages you want to export:

datasette export my-export-dir \ mydatabase.db -m metadata.json --template-dir templates/ \ --path / \ --path /mydatabase ...

Or a really wild option: it could allow you to define the paths you want to export using a SQL query:

datasette export my-export-dir \ mydatabase.db -m metadata.json --template-dir templates/ \ --sql " select '/' as path, 'index.html' as filename union all select '/mydatabase/articles/' || id as path, 'article-' || id || '.html' as filename from articles union all select '/mydatabase/tags/' || tag as path, 'tag-' || tag || '.html' as filename from tags " Which would save these files: - index.html as the content of / - article-1.html (and more) as the content of /mydatabase/articles/1 - tag-python.html (and more) as the content of /mydatabase/tags/python

Yeah I think this all hinges on: - #1101

Also this comment about streaming full JSON arrays (not just newline-delimited) using this trick: - https://github.com/simonw/datasette/issues/1356#issuecomment-1017016553

I'm about ready to figure these out, as with so much it's still a little bit blocked on the refactor stuff from: - #1518

Oh that's interesting. I was thinking about this from a slightly different angle recently - pondering what a static site generator built on top of Datasette might look like.

Just a sketch at the moment, but I was imagining a YAML configuration file with a SQL query that returns a list of paths - then a tool that runs that query and uses the equivalent of datasette --get to create a static copy of each of those paths.

I think these two ideas can probably be merged. I'd love to know more about how you are solving this right now!