issue_comments: 974754412

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions issue performed_via_github_app
https://github.com/simonw/sqlite-utils/pull/333#issuecomment-974754412 https://api.github.com/repos/simonw/sqlite-utils/issues/333 974754412 IC_kwDOCGYnMM46GZJs 9599 2021-11-21T04:35:32Z 2021-11-21T04:35:32Z OWNER

Some other recent projects (like trying to get this library to work in JupyterLite) have made me much more cautious about adding new dependencies, especially dependencies like pyarrow which require custom C/Rust extensions.

There are a few ways this could work though:

  • Have this as an optional dependency feature - so it only works if the user installs pyarrow as well
  • Implement this as a separate tool, parquet-to-sqlite - which could itself depend on sqlite-utils
  • Add a concept of "plugins" to sqlite-utils, similar to how those work in Datasette: https://docs.datasette.io/en/stable/plugins.html

My favourite option is parquet-to-sqlite because that can be built without any additional changes to sqlite-utils at all!

I find the concept of plugins for sqlite-utils interesting. I've so far not had quite enough potential use-cases to convince me this is worthwhile (especially since it should be very easy to build out separate tools entirely), but I'm ready to be convinced that a plugin mechanism would be worthwhile.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1039037439