github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
718910318 | MDU6SXNzdWU3MTg5MTAzMTg= | 1015 | Research: could Datasette install its own plugins? | 9599 | open | 0 | 1 | 2020-10-11T19:33:06Z | 2020-10-11T19:35:04Z | OWNER | It would be cool if Datasette could offer a plugin browsing interface where users could install plugins by clicking "Install" on them - similar to how VS Code extensions work. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1015/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
718934942 | MDU6SXNzdWU3MTg5MzQ5NDI= | 1 | Documentation on how to use this with Datasette | 9599 | open | 0 | 1 | 2020-10-11T21:56:27Z | 2020-10-11T22:14:00Z | MEMBER | In particular how to use `datasette-render-images` to see the images. | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
718938889 | MDU6SXNzdWU3MTg5Mzg4ODk= | 5 | Figure out how to display images from <en-media> tags inline in Datasette | 9599 | open | 0 | 6 | 2020-10-11T22:17:03Z | 2020-10-16T20:16:28Z | MEMBER | Relates to #1. Evernote XML looks like this: ```xml <?xml version="1.0"?> <en-note> <div>This note includes two images.</div> <div> <b>The Python logo</b> </div> <div> <en-media hash="61098c2c541de7f0a907c301dd6542da" type="image/svg+xml" width="125"/> </div> <div> <b>The Evernote logo</b> </div> <div> <en-media hash="91bd26175acac0b2ffdb6efac199f8ca" type="image/svg+xml" width="125"/> </div> </en-note> ``` That hash is the md5 we use to store resources. It should be possible to turn these into embedded image tags, especially if done in conjunction with the https://github.com/simonw/datasette-media plugin. | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
721068929 | MDU6SXNzdWU3MjEwNjg5Mjk= | 1020 | Method for datasette.client() to forward on authentication | 9599 | open | 0 | 6 | 2020-10-14T01:47:49Z | 2020-10-19T22:45:01Z | OWNER | I stumbled into this while working on Dogsheep Beta: the requests it re-dispatched through `TableView` did not carry authentication cookies, and since this was against a private instance they were thus denied. https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L223-L231 This made me think that `datasette.client.get()` (which Dogsheep Beta will start using shortly) could benefit from some kind of utility mechanism for passing through the cookies and general authenticated state from the current request. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1020/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
722816436 | MDU6SXNzdWU3MjI4MTY0MzY= | 186 | .extract() shouldn't extract null values | 9599 | open | 0 | 7 | 2020-10-16T02:41:08Z | 2021-08-12T12:32:14Z | OWNER | This almost works, but it creates a rogue `type` record with a value of None. ``` In [1]: import sqlite_utils In [2]: db = sqlite_utils.Database(memory=True) In [5]: db["creatures"].insert_all([ {"id": 1, "name": "Simon", "type": None}, {"id": 2, "name": "Natalie", "type": None}, {"id": 3, "name": "Cleo", "type": "dog"}], pk="id") Out[5]: <Table creatures (id, name, type)> In [7]: db["creatures"].extract("type") Out[7]: <Table creatures (id, name, type_id)> In [8]: list(db["creatures"].rows) Out[8]: [{'id': 1, 'name': 'Simon', 'type_id': None}, {'id': 2, 'name': 'Natalie', 'type_id': None}, {'id': 3, 'name': 'Cleo', 'type_id': 2}] In [9]: db["type"] Out[9]: <Table type (id, type)> In [10]: list(db["type"].rows) Out[10]: [{'id': 1, 'type': None}, {'id': 2, 'type': 'dog'}] ``` | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
723499985 | MDExOlB1bGxSZXF1ZXN0NTA1MDc2NDE4 | 5 | Add fitbit-to-sqlite | 4632208 | open | 0 | 0 | 2020-10-16T20:04:05Z | 2020-10-16T20:04:05Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep.github.io/pulls/5 | 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||||
723982480 | MDExOlB1bGxSZXF1ZXN0NTA1NDUzOTAw | 1030 | Make `package` command deal with a configuration directory argument | 299380 | open | 0 | 1 | 2020-10-18T11:07:02Z | 2020-10-19T08:01:51Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/1030 | Currently if we run `datasette package` on a configuration directory we'll get an exception when we try to hard link to the directory. This PR copies the tree and makes the Dockerfile run inspect on all *.db files. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1030/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
724759588 | MDU6SXNzdWU3MjQ3NTk1ODg= | 29 | Add search highlighting snippets | 9599 | open | 0 | 5 | 2020-10-19T16:00:48Z | 2021-08-26T20:23:11Z | MEMBER | Like on https://til.simonwillison.net/til/search?q=Snippet | 197431109 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
724878151 | MDU6SXNzdWU3MjQ4NzgxNTE= | 1032 | Bring date parsing into Datasette core | 9599 | open | 0 | 8 | 2020-10-19T18:30:45Z | 2020-10-19T19:37:55Z | OWNER | Currently this is mainly handled by a plugin - https://github.com/simonw/datasette-dateutil - but I realise now that this really needs to be core functionality. See also Twitter thread: https://twitter.com/simonw/status/1318234808653213696 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1032/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
727848625 | MDU6SXNzdWU3Mjc4NDg2MjU= | 12 | Some workout columns should be float, not text | 9599 | open | 0 | 4 | 2020-10-23T02:47:02Z | 2022-06-23T04:35:02Z | MEMBER | Columns `duration`, `totalDistance` and `totalEnergyBurned` should be converted to float. https://github.com/dogsheep/healthkit-to-sqlite/blob/71e36e1cf034b96de2a8e6652265d782d3fdf63b/healthkit_to_sqlite/utils.py#L50-L57 | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
728905098 | MDU6SXNzdWU3Mjg5MDUwOTg= | 1048 | Documentation and unit tests for urls.row() urls.row_blob() methods | 9599 | open | 0 | 7 | 2020-10-25T00:13:53Z | 2022-07-10T16:23:57Z | OWNER | https://github.com/simonw/datasette/blob/5db7ae3ce165ded57c7fb1cfbdb3258b1cf06c10/datasette/app.py#L1307-L1313 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1048/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
730210880 | MDU6SXNzdWU3MzAyMTA4ODA= | 1055 | query.html and table.html should share the same table implementation | 9599 | open | 0 | 3268330 | 0 | 2020-10-27T07:58:21Z | 2020-10-27T07:58:29Z | OWNER | In #998 I made a change that affected the table page but didn't affect the query page because I incorrectly assumed they shared rendering logic. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1055/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
732674148 | MDU6SXNzdWU3MzI2NzQxNDg= | 1062 | Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows | 9599 | open | 0 | 3268330 | 5 | 2020-10-29T21:25:02Z | 2022-09-28T14:09:54Z | OWNER | This can drive the upgrade of the `register_output_renderer` hook to be able to handle streaming all rows in a large query. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1062/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
733999615 | MDU6SXNzdWU3MzM5OTk2MTU= | 1079 | Handle long breadcrumbs better with new menu | 9599 | open | 0 | 1 | 2020-11-01T15:57:41Z | 2022-01-13T22:21:29Z | OWNER | On this page when signed in as root: https://latest.datasette.io/fixtures/roadside_attraction_characteristics/1 ![EF921CB1-625F-4D04-A850-490B812A72B3](https://user-images.githubusercontent.com/9599/97807807-db0fbf80-1c17-11eb-9c77-ae5169b12c3d.jpeg) ![A49D8B76-5ACF-4F71-A8B4-21A44F5C8D51](https://user-images.githubusercontent.com/9599/97807809-dea34680-1c17-11eb-9511-a49af56a4bd2.jpeg) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1079/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
734777631 | MDU6SXNzdWU3MzQ3Nzc2MzE= | 1080 | "View all" option for facets, to provide a (paginated) list of ALL of the facet counts plus a link to view them | 9599 | open | 0 | 3268330 | 7 | 2020-11-02T19:55:06Z | 2022-02-04T06:25:18Z | OWNER | Can use `/database/-/...` namespace from #296 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1080/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
735852274 | MDU6SXNzdWU3MzU4NTIyNzQ= | 1082 | DigitalOcean buildpack memory errors for large sqlite db? | 39538958 | open | 0 | 3 | 2020-11-04T06:35:32Z | 2020-11-04T19:35:44Z | NONE | 1. Have a sqlite db stored in Dropbox 2. Previously tried the Digital Ocean build pack minimal approach (e.g. Procfile, requirements.txt, bin/post_compile) 3. bin/post_compile with wget from Dropbox 4. download of large sqlite db is successful 5. log reveals that when building Docker container, Digital Ocean runs out of memory for 5gb+ sqlite db but works fine for 2gb+ sqlite db | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1082/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
736365306 | MDU6SXNzdWU3MzYzNjUzMDY= | 1083 | Advanced CSV export for arbitrary queries | 9599 | open | 0 | 2 | 2020-11-04T19:23:05Z | 2021-06-17T18:12:31Z | OWNER | There's no link to download the CSV file - the table page has that as an advanced export option, but this is missing from the query page. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1083/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
741231849 | MDU6SXNzdWU3NDEyMzE4NDk= | 1087 | Idea: ?_extra=urls for getting back URLs to useful things | 9599 | open | 0 | 0 | 2020-11-12T02:55:41Z | 2021-12-15T18:06:16Z | OWNER | Working on https://github.com/simonw/datasette-search-all/issues/10 made me realize that sometimes it can be difficult to calculate the URL for a database, table or row within Datasette. It would be useful to have an optional extra JSON extension (using `?_extra=` from #262) that can help with this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1087/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
741862364 | MDU6SXNzdWU3NDE4NjIzNjQ= | 1090 | Custom widgets for canned query forms | 9599 | open | 0 | 3 | 2020-11-12T19:21:07Z | 2021-03-27T16:25:25Z | OWNER | This is an idea that was cut from the first version of writable canned queries: > I really want the option to use a `<textarea>` for a specific value. > > Idea: metadata syntax like this: > > ```json > { > "databases": { > "my-database": { > "queries": { > "add_twitter_handle": { > "sql": "insert into twitter_handles (username) values (:username)", > "write": true, > "params": { > "username": { > "widget": "textarea" > } > } > } > } > } > } > } > ``` > > I can ship with some default widgets and provide a plugin hook for registering extra widgets. > > This opens up some really exciting possibilities for things like map widgets that let you draw polygons. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-608125928_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1090/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
743071410 | MDExOlB1bGxSZXF1ZXN0NTIxMDU0NjEy | 13 | SQLite does not have case sensitive columns | 1689944 | open | 0 | 1 | 2020-11-14T20:12:32Z | 2021-08-24T13:28:26Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/13 | This solves a weird issue when there is record with metadata key that is only different in letter cases. See the test for details. | 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
743359646 | MDU6SXNzdWU3NDMzNTk2NDY= | 1096 | TSV should be a default export option | 9599 | open | 0 | 1 | 2020-11-15T22:24:02Z | 2021-06-17T18:12:31Z | OWNER | Refs #1095 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1096/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
743371103 | MDU6SXNzdWU3NDMzNzExMDM= | 1099 | Support linking to compound foreign keys | 9599 | open | 0 | 6 | 2020-11-15T23:23:17Z | 2023-01-25T00:58:26Z | OWNER | Reported as a bug in #1098 because they caused 500 errors - but it would be even better if Datasette could hyperlink to related rows via compound foreign keys. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1099/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
743384829 | MDExOlB1bGxSZXF1ZXN0NTIxMjg3OTk0 | 203 | changes to allow for compound foreign keys | 1049910 | open | 0 | 7 | 2020-11-16T00:30:10Z | 2023-01-25T18:47:18Z | FIRST_TIME_CONTRIBUTOR | simonw/sqlite-utils/pulls/203 | Add support for compound foreign keys, as per issue #117 Not sure if this is the right approach. In particular I'm unsure about: - the new `ForeignKey` class, which replaces the namedtuple in order to ensure that `column` and `other_column` are forced into tuples. The class does the job, but doesn't feel very elegant. - I haven't rewritten `guess_foreign_table` to take account of multiple columns, so it just checks for the first column in the foreign key definition. This isn't ideal. - I haven't added any ability to the CLI to add compound foreign keys, it's only in the python API at the moment. The PR also contains a minor related change that columns and tables are always quoted in foreign key definitions. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/203/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
749283032 | MDU6SXNzdWU3NDkyODMwMzI= | 1101 | register_output_renderer() should support streaming data | 9599 | open | 0 | 3268330 | 13 | 2020-11-24T02:17:09Z | 2023-01-21T22:07:19Z | OWNER | > I'd like to implement this by first extending the `register_output_renderer()` hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1096#issuecomment-732542285_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1101/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
750089847 | MDU6SXNzdWU3NTAwODk4NDc= | 1109 | Deprecate --config in Datasette 1.0 (in favour of --setting) | 9599 | open | 0 | 3268330 | 0 | 2020-11-24T21:43:57Z | 2020-12-17T22:07:49Z | OWNER | I added a deprecation warning to this in #992. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1109/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
751195017 | MDU6SXNzdWU3NTExOTUwMTc= | 1111 | Accessing a database's `.json` is slow for very large SQLite files | 15178711 | open | 0 | 3 | 2020-11-26T00:27:27Z | 2021-01-04T19:57:53Z | CONTRIBUTOR | I have a SQLite DB that's pretty large, 23GB and something like 300 million rows. I expect that most queries I run on it will be slow, which is fine, but there are some things that Datasette does that makes working with the DB very slow. Specifically, when I access the `.json` metadata for a table (which I believe it comes from `datasette/views/database.py`, it takes 43 seconds for the request to come in: ```bash $ time curl localhost:9999/out.json {"database": "out", "size": 24291454976, "tables": [{"name": "PageviewsHour", "columns": ["file", "code", "page", "pageviews"], "primary_keys": [], "count": null, "hidden": false, "fts_table": null, "foreign_keys": {"incoming": [], "outgoing": [{"other_table": "PageviewsHourFiles", "column": "file", "other_column": "file_id"}]}, "private": false}, {"name": "PageviewsHourFiles", "columns": ["file_id", "filename", "sha256", "size", "day", "hour"], "primary_keys": ["file_id"], "count": null, "hidden": false, "fts_table": null, "foreign_keys": {"incoming": [{"other_table": "PageviewsHour", "column": "file_id", "other_column": "file"}], "outgoing": []}, "private": false}, {"name": "sqlite_sequence", "columns": ["name", "seq"], "primary_keys": [], "count": 1, "hidden": false, "fts_table": null, "foreign_keys": {"incoming": [], "outgoing": []}, "private": false}], "hidden_count": 0, "views": [], "queries": [], "private": false, "allow_execute_sql": true, "query_ms": 43340.23213386536} real 0m43.417s user 0m0.006s sys 0m0.016s ``` I suspect this is because a `COUNT(*)` is happening under the hood, which, when I run it through sqlite directly, does take around the same time: ```bash $ time sqlite3 out.db < <(echo "select count(*) from PageviewsHour;") 362794272 real 0m44.523s user 0m2.497s sys 0m6.703s ``` I'm using the `.json` request in the [Observable Datasette Client](https://observablehq.com/@asg017/datasette-client) to 1) verify that a link passed in is a reachable Datasette instance, and 2) a quick way to look at metadata for a db. A few differe… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1111/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
753000405 | MDU6SXNzdWU3NTMwMDA0MDU= | 53 | Command for fetching file contents | 9599 | open | 0 | 1 | 2020-11-29T20:31:04Z | 2020-11-30T00:36:09Z | MEMBER | Something like this: github-to-sqlite files github.db simonw/datasette This would fetch all files from the `main` branch into a `files` table. Additional options could handle things like pulling files from a branch or tag, or just pulling files that match a specific glob or that exist in a specific directory. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/53/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
756875827 | MDU6SXNzdWU3NTY4NzU4Mjc= | 1129 | Fix footer to the bottom of the page | 3243482 | open | 0 | 0 | 2020-12-04T07:28:07Z | 2020-12-04T16:04:29Z | CONTRIBUTOR | Footer doesn't stick to the bottom if the body content isn't long enough to reach the end of viewport. ![before & after](https://user-images.githubusercontent.com/3243482/101134785-f6595a80-361b-11eb-81ce-b8b5cb9c5bc2.png) This can be fixed using flexbox. ```css body { min-height: 100vh; display: flex; flex-direction: column; } .content { flex-grow: 1; } ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1129/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
756876238 | MDExOlB1bGxSZXF1ZXN0NTMyMzQ4OTE5 | 1130 | Fix footer not sticking to bottom in short pages | 3243482 | open | 0 | 4 | 2020-12-04T07:29:01Z | 2021-06-15T13:27:48Z | CONTRIBUTOR | simonw/datasette/pulls/1130 | Fixes https://github.com/simonw/datasette/issues/1129 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1130/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
763361458 | MDU6SXNzdWU3NjMzNjE0NTg= | 1142 | "Stream all rows" is not at all obvious | 9599 | open | 0 | 9 | 2020-12-12T06:24:57Z | 2021-06-17T18:12:31Z | OWNER | Got a question about how to download all rows - the current option isn't at all clear. <img width="668" alt="loans__ppp_loans__9_511_rows_where_where_search_matches__tech__sorted_by_rowid" src="https://user-images.githubusercontent.com/9599/101977057-ac660b00-3bff-11eb-88f4-c93ffd03d3e0.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
764059235 | MDU6SXNzdWU3NjQwNTkyMzU= | 1143 | More flexible CORS support in core, to encourage good security practices | 114388 | open | 0 | 3268330 | 6 | 2020-12-12T17:06:35Z | 2022-02-13T17:41:17Z | NONE | It would be nice if the `--cors` option accepted an origin regex to more securely allow secure local development. As an example, Observable notebooks namespace every user's notebooks by their username and user content is served from username.observableusercontent.com, so you would set `--cors-origin username.observableusercontent.com` to restrict access to a local development Datasette instance to only your own notebooks, rather than exposing the data to any website that makes a request. Thank you for all of your work on Datasette! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1143/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
765637324 | MDU6SXNzdWU3NjU2MzczMjQ= | 1144 | JavaScript to help plugins interact with the fragment part of the URL | 9599 | open | 0 | 1 | 2020-12-13T20:36:06Z | 2020-12-14T14:47:11Z | OWNER | Suggested by Markus Holtermann on Twitter, who is building https://github.com/MarkusH/datasette-chartjs > I've been looking at datasette-vega for how you persist chart settings between form submissions. I've adopted that for datasette-chartjs. Any thoughts on adding a public JS API to #datasette itself, that plugins can rely on? > > I'm talking about functions like onFragmentChange, serialize, unserialize, ... That turn an object into a URL encoded string and put it into the location's hash. And also updating all links/forms automatically. > > Essentially, a plugins could do something like `document.datasette.setConfigValue('prefix', 'foo', 'bar')` and `.getConfigValue('prefix', 'foo')`. And the functions would take care of updating document.location.hash, all (necessary) a.href and form.action https://twitter.com/m_holtermann/status/1338183973311295492 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1144/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
769376447 | MDU6SXNzdWU3NjkzNzY0NDc= | 2 | killed by oomkiller on large location-history | 231498 | open | 0 | 2 | 2020-12-17T00:32:24Z | 2020-12-17T00:48:32Z | NONE | memory seems to grow unbounded and is oom-killed after about 20GB memory usage. this is happening while loading a ~1GB uncompressed location history. | 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
769397742 | MDU6SXNzdWU3NjkzOTc3NDI= | 3 | sqlite-utils error on takeout import | 231498 | open | 0 | 0 | 2020-12-17T01:18:48Z | 2020-12-17T01:19:04Z | NONE | ``` $ google-takeout-to-sqlite my-activity takeout.db /path/to/zip ... sqlite3.OperationalError: no such table: main.my_activity ``` there is no table create in `utils.py`, unlike other importers such as github-to-sqlite additionally, this package and hackernews-to-sqlite have conflicting `sqlite-utils` dep with datasette and dogsheep-beta | 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/3/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
769520939 | MDU6SXNzdWU3Njk1MjA5Mzk= | 1149 | Make it easier to theme Datasette with CSS | 9599 | open | 0 | 3268330 | 3 | 2020-12-17T05:01:26Z | 2021-03-22T21:43:16Z | OWNER | I want to theme https://datasette.io/ so that when you visit https://datasette.io/content (the Datasette UI part of it) the navigation from the parent site is used. I tried dropping in a `base.html` template like this: ```html {% extends "page_base.html" %} {% block base_extra_head %} <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> {% for url in extra_css_urls %} <link rel="stylesheet" href="{{ url.url }}"{% if url.sri %} integrity="{{ url.sri }}" crossorigin="anonymous"{% endif %}> {% endfor %} {% for url in extra_js_urls %} <script src="{{ url.url }}"{% if url.sri %} integrity="{{ url.sri }}" crossorigin="anonymous"{% endif %}></script> {% endfor %} {% block extra_head %}{% endblock %} {% endblock %} {% block extra_body_end %} {% include "_close_open_menus.html" %} {% for body_script in body_scripts %} <script>{{ body_script }}</script> {% endfor %} {% endblock %} ``` But this resulted in pages looking like this: <img width="1067" alt="content__categories__3_rows" src="https://user-images.githubusercontent.com/9599/102446045-c168e280-3fe1-11eb-94d6-e7350798eb96.png"> Note that the cog menu is broken and the filter UI is unstyled. To get these working correctly I would need to copy over a whole lot of Datasette's default CSS - and that means that when Datasette changes in the future those pages could break in subtle ways. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1149/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
770598024 | MDU6SXNzdWU3NzA1OTgwMjQ= | 1152 | Efficiently calculate list of databases/tables a user can view | 9599 | open | 0 | 12 | 2020-12-18T06:13:01Z | 2021-12-27T23:04:31Z | OWNER | > The homepage currently performs a massive flurry of permission checks - one for each, database, table and view: https://github.com/simonw/datasette/blob/0.53/datasette/views/index.py#L21-L75 > > A paginated version of this is a little daunting as the permission checks would have to be carried out in every single table just to calculate the count that will be paginated. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747864831_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1152/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
771511344 | MDExOlB1bGxSZXF1ZXN0NTQzMDE1ODI1 | 31 | Update for Big Sur | 41546558 | open | 0 | 7 | 2020-12-20T04:36:45Z | 2023-08-08T15:52:52Z | CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/31 | Refactored out the SQL for extracting aesthetic scores to use osxphotos -- adds compatbility for Big Sur via osxphotos which has been updated for new table names in Big Sur. Have not yet refactored the SQL for extracting labels which is still compatible with Big Sur. | 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
771608692 | MDU6SXNzdWU3NzE2MDg2OTI= | 14 | UNIQUE constraint failed: workouts.id | 1234956 | open | 0 | 5 | 2020-12-20T15:11:20Z | 2023-07-10T14:46:52Z | NONE | I'm getting an error on my initial attempt to import data: ```console $ healthkit-to-sqlite 20201119\ healthkit\ export.zip healthkit.db Importing from HealthKit [###################################-] 98% 00:00:01 Traceback (most recent call last): File "venv/bin/healthkit-to-sqlite", line 8, in <module> sys.exit(cli()) File "venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "venv/lib/python3.9/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "venv/lib/python3.9/site-packages/healthkit_to_sqlite/cli.py", line 57, in cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) File "venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py", line 34, in convert_xml_to_sqlite workout_to_db(el, db, zipfile) File "venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py", line 57, in workout_to_db pk = db["workouts"].insert(record, alter=True, hash_id="id").last_pk File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 1660, in insert return self.insert_all( File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 1778, in insert_all self.insert_chunk( File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 1588, in insert_chunk result = self.db.execute(query, params) File "venv/lib/python3.9/site-packages/sqlite_utils/db.py", line 213, in execute return self.conn.execute(sql, parameters) sqlite3.IntegrityError: UNIQUE constraint failed: workouts.id ``` | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
774332247 | MDExOlB1bGxSZXF1ZXN0NTQ1MjY0NDM2 | 1159 | Improve the display of facets information | 552629 | open | 0 | 3268330 | 9 | 2020-12-24T11:01:47Z | 2023-07-31T18:57:59Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/1159 | This PR changes the display of facets to hopefully make them more readable. Before | After ---|--- ![image](https://user-images.githubusercontent.com/552629/103084609-b1ec2980-45df-11eb-85bc-68ab8df3e8d9.png) | ![image](https://user-images.githubusercontent.com/552629/103085220-620e6200-45e1-11eb-8189-5dd5d3e2569e.png) | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1159/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
775666296 | MDU6SXNzdWU3NzU2NjYyOTY= | 1160 | "datasette insert" command and plugin hook | 9599 | open | 0 | 23 | 2020-12-29T02:37:03Z | 2021-06-17T18:12:32Z | OWNER | Tools for loading data into Datasette currently mostly exist as separate utilities - `yaml-to-sqlite` and `csvs-to-sqlite` and suchlike. Bringing these into Datasette could have some interesting properties: - A `datasette insert` command could be extended with plugins to handle more formats - Any format that can be inserted on the command-line could also be inserted using a web UI or web API - which would benefit from new format plugin hooks - If Datasette ever grows beyond SQLite (see #670) a built-in import mechanism could work for those other databases as well - without me needing to write `yaml-to-postgresql` and suchlike | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776101101 | MDU6SXNzdWU3NzYxMDExMDE= | 1161 | Update a whole bunch of links to datasette.io instead of datasette.readthedocs.io | 9599 | open | 0 | 1 | 2020-12-29T21:47:31Z | 2020-12-29T21:49:57Z | OWNER | https://ripgrep.datasette.io/-/ripgrep?pattern=%28datasette%7Csqlite-utils%29%5C.readthedocs%5C.io | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1161/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128269 | MDU6SXNzdWU3NzYxMjgyNjk= | 1162 | First working version of "datasette insert data.db file.csv" | 9599 | open | 0 | 0 | 2020-12-29T23:20:11Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128565 | MDU6SXNzdWU3NzYxMjg1NjU= | 1163 | "datasette insert data.db url-to-csv" | 9599 | open | 0 | 1 | 2020-12-29T23:21:21Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 - get filesystem imports working first for #1162, then add import-from-URL. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1163/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776634318 | MDU6SXNzdWU3NzY2MzQzMTg= | 1164 | Mechanism for minifying JavaScript that ships with Datasette | 9599 | open | 0 | 9 | 2020-12-30T20:59:06Z | 2022-01-13T22:21:29Z | OWNER | > If I'm going to minify it I'll need to figure out a build step in Datasette itself so that I can easily work on that minified version. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752748496_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1164/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776635426 | MDU6SXNzdWU3NzY2MzU0MjY= | 1165 | Mechanism for executing JavaScript unit tests | 9599 | open | 0 | 9 | 2020-12-30T21:02:34Z | 2022-01-13T22:21:29Z | OWNER | > I'm going to need to add JavaScript unit tests for this new plugin system. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752757289_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1165/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
777140799 | MDU6SXNzdWU3NzcxNDA3OTk= | 1166 | Adopt Prettier for JavaScript code formatting | 9599 | open | 0 | 10 | 2020-12-31T21:25:27Z | 2022-01-13T22:22:18Z | OWNER | https://prettier.io/ - I'm going to go with 2 spaces. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1166/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
777333388 | MDU6SXNzdWU3NzczMzMzODg= | 1168 | Mechanism for storing metadata in _metadata tables | 9599 | open | 0 | 21 | 2021-01-01T18:47:27Z | 2023-09-28T18:29:05Z | OWNER | _Original title: Perhaps metadata should all live in a `_metadata` in-memory database_ Inspired by #1150 - metadata should be exposed as an API, and for large Datasette instances that API may need to be paginated. So why not expose it through an in-memory database table? One catch to this: plugins. #860 aims to add a plugin hook for metadata. But if the metadata comes from an in-memory table, how do the plugins interact with it? The need to paginate over metadata does make a plugin hook that returns metadata for an individual table seem less wise, since we don't want to have to do 10,000 plugin hook invocations to show a list of all metadata. If those plugins write directly to the in-memory table how can their contributions survive the server restarting? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1168/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778380836 | MDU6SXNzdWU3NzgzODA4MzY= | 4 | Feature Request: Gmail | 203343 | open | 0 | 5 | 2021-01-04T21:31:09Z | 2021-03-04T20:54:44Z | NONE | From takeout, I only exported my Gmail account. Ideally I could parse this into sqlite via this tool. | 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778450486 | MDU6SXNzdWU3Nzg0NTA0ODY= | 1171 | GitHub Actions workflow to build and sign macOS binary executables | 9599 | open | 0 | 8 | 2021-01-04T23:36:59Z | 2021-01-07T19:36:00Z | OWNER | Using PyInstaller, as explored in #93 and https://til.simonwillison.net/python/packaging-pyinstaller The bigger challenge will be the code signing bit. I'll need a Apple Developer account ($99/year) and some extensive CI fiddling. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1171/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778530523 | MDU6SXNzdWU3Nzg1MzA1MjM= | 1172 | /-/static should be excluded from auth and permission checks | 9599 | open | 0 | 0 | 2021-01-05T02:53:41Z | 2021-01-05T02:53:41Z | OWNER | I want to set far future / immutable cache headers on everything served from `/-/static` and `/-/static-plugins` This has security implications since it will be possible to see what plugins are installed by checking for known static URLs. I'm fine with that - performance is more important here. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1172/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
778682317 | MDU6SXNzdWU3Nzg2ODIzMTc= | 1173 | GitHub Actions workflow to build manylinux binary | 9599 | open | 0 | 1 | 2021-01-05T07:41:11Z | 2021-01-05T07:41:43Z | OWNER | Refs #1171 and #93 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1173/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
779088071 | MDU6SXNzdWU3NzkwODgwNzE= | 54 | Archive import appears to be broken on recent exports | 21148 | open | 0 | 5 | 2021-01-05T14:18:01Z | 2023-01-04T11:06:55Z | CONTRIBUTOR | I requested a Twitter export yesterday, and unfortunately they seem to have changed it such that `twitter-to-sqlite import` can't handle it anymore 😢 So far I've ran into two issues. The first was easy to work around, but the second will take more investigation. If I can find the time I'll keep working on it and update this issue accordingly. The issues (so far): ### 1. Data seems to have moved to a `data/` subdirectory Running `twitter-to-sqlite import` on the raw zip file reports a bunch of "not yet implemented" errors, and then exits without actually importing anything: ``` ❯ twitter-to-sqlite import tarchive.db twitter.zip ... data/manifest: not yet implemented data/account-creation-ip: not yet implemented data/account-suspension: not yet implemented ... (dozens of more lines like this, including critical stuff like data/tweets) ... ``` (`tarchive.db` now exists, but is empty) Workaround: unpack the zip file, and run `twitter-to-sqlite import tarchive.db path/to/archive/data` That gets further, but: ### 2. Some schema(s?) have changed At least, the `blocks` schema seems different now: ``` ❯ twitter-to-sqlite import tarchive.db archive/data direct-messages-group: not yet implemented branch-links: not yet implemented periscope-expired-broadcasts: not yet implemented direct-messages: not yet implemented mute: not yet implemented Traceback (most recent call last): File "/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/bin/twitter-to-sqlite", line 8, in <module> sys.exit(cli()) File "/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN… | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
779156520 | MDU6SXNzdWU3NzkxNTY1MjA= | 1175 | Use structlog for logging | 9599 | open | 0 | 4 | 2021-01-05T15:11:36Z | 2022-07-26T12:52:10Z | OWNER | To solve #241 JSON logging. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1175/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
780153562 | MDU6SXNzdWU3ODAxNTM1NjI= | 1177 | Ability to stream all rows as newline-delimited JSON | 9599 | open | 0 | 3268330 | 1 | 2021-01-06T07:10:48Z | 2022-03-21T15:08:52Z | OWNER | > Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas: > > pandas.read_json("https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on", lines=True) _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1177/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
780278550 | MDU6SXNzdWU3ODAyNzg1NTA= | 1179 | Make original path available to render hooks | 9599 | open | 0 | 8 | 2021-01-06T08:31:45Z | 2021-01-25T04:44:33Z | OWNER | https://github.com/simonw/datasette-export-notebook/blob/0.1/datasette_export_notebook/__init__.py ```python async def render_notebook(datasette, request): return Response.html( await datasette.render_template( "export_notebook.html", { "csv_stream_url": datasette.absolute_url( request, path_with_format( request=request, format="csv", extra_qs={"_stream": "on"} ), ), "json_url": datasette.absolute_url( request, path_with_format( request=request, format="json", extra_qs={"_shape": "array"} ), ), "json": json, }, ) ) ``` This results in https://latest-with-plugins.datasette.io/github/issue_comments.Notebook showing `http://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_format=json&_shape=array` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1179/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
780767542 | MDU6SXNzdWU3ODA3Njc1NDI= | 1180 | Lazily evaluated arguments for call_with_supported_arguments | 9599 | open | 0 | 2 | 2021-01-06T18:43:34Z | 2021-01-07T18:56:24Z | OWNER | While building https://github.com/simonw/datasette-export-notebook I thought it would be nice to be able to show a count of exported records on the page "This will stream 10,422 records to your notebook". None of the documented arguments on https://docs.datasette.io/en/0.53/plugin_hooks.html#register-output-renderer-datasette expose the count. The closest is `sql` which could be executed as `select count(*) from ({sql})` but that's a bit inelegant. So, idea: if your defined render function takes a `total` argument, the caller runs a `count(*)` query and passes you that total. To implement this I would need to teach the `call_with_supported_arguments` that some arguments are lazy - they should execute a function (or an async function) but only if they are needed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1180/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
782708469 | MDU6SXNzdWU3ODI3MDg0Njk= | 1183 | Take advantage of sqlite-utils cached table counts, if available | 9599 | open | 0 | 2 | 2021-01-09T23:51:48Z | 2021-01-12T02:42:08Z | OWNER | sqlite-utils 3.2 now has a mechanism for creating a `_counts` table with triggers to maintain counts for individual tables. Datasette could look for this table and use it for counts, dramatically speeding up places that currently suffer from slow counts. Refs #859. https://sqlite-utils.datasette.io/en/stable/python-api.html#cached-table-counts-using-triggers | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
787098345 | MDU6SXNzdWU3ODcwOTgzNDU= | 1191 | Ability for plugins to collaborate when adding extra HTML to blocks in default templates | 9599 | open | 0 | 3268330 | 12 | 2021-01-15T18:18:51Z | 2023-09-18T06:55:52Z | OWNER | Sometimes a plugin may want to add content to an existing default template - for example `datasette-search-all` adds a new search box at the top of `index.html`. I also want `datasette-upload-csvs` to add a CTA on the `database.html` page: https://github.com/simonw/datasette-upload-csvs/issues/18 Currently plugins can do this by providing a new version of the `index.html` template - but if multiple plugins try to do that only one of them will succeed. It would be better if there were known areas of those templates which plugins could add additional content to, such that multiple plugins can use the same spot. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1191/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
787104850 | MDU6SXNzdWU3ODcxMDQ4NTA= | 1192 | Form Plugin for in-depth Datasette Querying | 1024355 | open | 0 | 0 | 2021-01-15T18:24:50Z | 2021-01-15T18:24:50Z | NONE | I envision a sort of easy-to-build form plugin that would be able to map a user's inputs to different fields/columns in a Datasette database. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1192/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
787173276 | MDU6SXNzdWU3ODcxNzMyNzY= | 1193 | Research plugin hook for alternative database backends | 9599 | open | 0 | 1 | 2021-01-15T20:27:50Z | 2021-03-12T01:01:54Z | OWNER | I started exploring what Datasette would like running against PostgreSQL in #670 and @dazzag24 did some work on Parquet described in #657. I had initially thought this was WAY too much additional complexity, but I'm beginning to think that the `Database` class may be small enough that having it abstract away the details of running queries against alternative database backends could be feasible. A bigger issue is SQL generation, but I realized that most of Datasette's SQL generation code exists just in the `TableView` class that runs the table page. If this was abstracted into some kind of SQL builder that could be then customized per-database it might be reasonable to get it working. Very unlikely for this to make it into Datasette 1.0, but maybe this would be the defining feature of Datasette 2.0? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1193/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
789336592 | MDU6SXNzdWU3ODkzMzY1OTI= | 1195 | view_name = "query" for the query page | 9599 | open | 0 | 4 | 2021-01-19T20:21:36Z | 2021-01-25T04:40:08Z | OWNER | It uses `view_name` of `database` at the moment which isn't as useful. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1195/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
791237799 | MDU6SXNzdWU3OTEyMzc3OTk= | 1196 | Access Denied Error in Windows | 2826376 | open | 0 | 2 | 2021-01-21T15:40:40Z | 2021-04-14T19:28:38Z | NONE | I am trying to publish a db to vercel. But while issuing the below command throwing `Access Denied` error which is leading to `RecursionError: maximum recursion depth exceeded while calling a Python object`. I am using PyCharm and Python 3.9. I have reinstalled both and launched PyCharm as Admin in Windows 10. But still the issue persists. Issued command `datasette publish vercel jmeter.db --project jmeter --install datasette-vega` PS: localhost is working fine. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1196/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
792652391 | MDU6SXNzdWU3OTI2NTIzOTE= | 1199 | Experiment with PRAGMA mmap_size=N | 9599 | open | 0 | 2 | 2021-01-23T21:24:09Z | 2021-07-17T17:39:17Z | OWNER | https://sqlite.org/mmap.html - SQLite supports memory-mapped I/O but it's disabled by default. The `PRAGMA mmap_size=N` option can be used to enable it. It would be very interesting to understand the impact this could have on Datasette performance for various different shapes of data. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
792890765 | MDU6SXNzdWU3OTI4OTA3NjU= | 1200 | ?_size=10 option for the arbitrary query page would be useful | 9599 | open | 0 | 2 | 2021-01-24T20:55:35Z | 2021-02-11T03:13:59Z | OWNER | https://latest.datasette.io/fixtures?sql=select+*+from+compound_three_primary_keys&_size=10 - `_size=10` does not do anything at the moment. It would be useful if it did. Would also be good if it persisted in a hidden form field. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1200/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
793002853 | MDExOlB1bGxSZXF1ZXN0NTYwNzYwMTQ1 | 1204 | WIP: Plugin includes | 9599 | open | 0 | 3 | 2021-01-25T03:59:06Z | 2021-12-17T07:10:49Z | OWNER | simonw/datasette/pulls/1204 | Refs #1191 Next steps: - [ ] Get comfortable that this pattern is the right way to go - [ ] Implement it for all of the other pages, not just the table page - [ ] Add a new set of plugin tests that exercise ALL of these new hook locations - [ ] Document, then ship | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1204/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | ||||||
793907673 | MDExOlB1bGxSZXF1ZXN0NTYxNTEyNTAz | 15 | added try / except to write_records | 9857779 | open | 0 | 0 | 2021-01-26T03:56:21Z | 2021-01-26T03:56:21Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/15 | to keep the data write from failing if it came across an error during processing. In particular when trying to convert my HealthKit zip file (and that of my wife's) it would consistently error out with the following: ``` db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: too many SQL variables --------------------------------------------------------------------------------------------------------------------------------------------------------------------- db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: too many SQL variables --------------------------------------------------------------------------------------------------------------------------------------------------------------------- db.py 1709 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: table rBodyMass has no column named metadata_HKWasUserEntered --------------------------------------------------------------------------------------------------------------------------------------------------------------------- healthkit-to-sqlite 8 <module> sys.exit(cli()) core.py 829 __call__ return self.main(*args, **kwargs) core.py 782 main rv = self.invoke(ctx) core.py 1066 invoke return ctx.invoke(self.callback, **ctx.params) core.py 610 invoke return callback(*args, **kwargs) cli.py 57 cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf) utils.py 42 convert_xml_to_sqlite write_records(records, db) utils.py 143 write_records db[table].insert_all( db.py 1899 insert_all self.insert_chunk( db.py 1720 insert_chunk self.insert_chunk( db.py 1720 insert_chunk self.insert_chunk( db.py 1714 insert_chunk result = self.db.execute(query, params) db.py 226 execute return self.co… | 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
795367402 | MDU6SXNzdWU3OTUzNjc0MDI= | 1209 | v0.54 500 error from sql query in custom template; code worked in v0.53; found a workaround | 11788561 | open | 0 | 1 | 2021-01-27T19:08:13Z | 2021-01-28T23:00:27Z | NONE | v0.54 500 error in sql query template; code worked in v0.53; found a workaround **schema:** CREATE TABLE "talks" ("talk" TEXT,"series" INTEGER, "talkdate" TEXT) CREATE TABLE "series" ("id" INTEGER PRIMARY KEY, "series" TEXT, talks_list TEXT default '', website TEXT default ''); **Live example of correctly rendered template in v.053:** https://cosmotalks-cy6xkkbezq-uw.a.run.app/cosmotalks/talks/1 **Description of problem:** I needed 'sql select' code in a custom row-mydatabase-mytable.html template to lookup the series name for a foreign key integer value in the talks table. So `metadata.json` specifies the `datasette-template-sql` plugin. The code below worked perfectly in v0.53 (just the relevant sql statement part is shown; full code is [here](https://github.com/jrdmb/cosmotalks-datasette/blob/main/templates/row-cosmotalks-talks.html)): ``` {# custom addition #} {% for row in display_rows %} ... {% set sname = sql("select series from series where id = ?", [row.series]) %} <strong>Series name: {{ sname[0].series }} ... {% endfor %} {# End of custom addition #} ``` **In v0.54, that code resulted in a 500 error with a 'no such table series' message.** A second query in that template also did not work but the above is fully illustrative of the problem. All templates were up-to-date along with datasette v0.54. **Workaround:** After fiddling around with trying different things, what worked was the syntax from [Querying a different database from the datasette-template-sql github repo](https://github.com/simonw/datasette-template-sql#querying-a-different-database) to add the database name to the sql statement: `{% set sname = sql("select series from series where id = ?", [row.series], database="cosmotalks") %}` Though this was found to work, it should not be necessary to add `database="cosmotalks"` since per the `datasette-template-sql` README, it's only needed when querying a different database, but here it's a table within the same databa… | 107914493 | issue | |||||||||
797097140 | MDU6SXNzdWU3OTcwOTcxNDA= | 60 | Use Data from SQLite in other commands | 22578954 | open | 0 | 3 | 2021-01-29T18:35:52Z | 2021-02-12T18:29:43Z | CONTRIBUTOR | As a total beginner here how could you access data from the sqlite table to run other commands. What I am thinking is I want to get all the repos in an organization then using the repo list pull all the commit messages for each repo. I love this project by the way! | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
797728929 | MDU6SXNzdWU3OTc3Mjg5Mjk= | 8 | QUESTION: extract full text | 417363 | open | 0 | 0 | 2021-01-31T14:50:10Z | 2021-01-31T14:50:10Z | NONE | This may be solved or a feature already, but I couldn't figure it out, is it possible to extract and store also full text from the saved pages? The same way that Pocket parses the text, it'd be amazing to be able to store (and thus make searchable later) the text. Thank you very much for the project, it's such an amazing idea! | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
797784080 | MDU6SXNzdWU3OTc3ODQwODA= | 62 | Stargazers and workflows commands always require an auth file when using GITHUB_TOKEN | 631242 | open | 0 | 0 | 2021-01-31T18:56:05Z | 2021-01-31T18:56:05Z | CONTRIBUTOR | Requested fix in https://github.com/dogsheep/github-to-sqlite/pull/59 The stargazers and workflows commands always require an auth file, even when using a `GITHUB_TOKEN`. Other commands don't require the auth file. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/62/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
799663959 | MDU6SXNzdWU3OTk2NjM5NTk= | 1213 | gzip support for HTML (and JSON) responses | 9599 | open | 0 | 3 | 2021-02-02T20:36:28Z | 2021-02-02T20:41:55Z | OWNER | This page https://datasette-tiles-demo.datasette.io/San_Francisco/tiles is 2MB because of all of the base64 images. Gzipped it's 1.5MB. Since Datasette is usually deployed without a frontend gzipping proxy, Datasette itself needs to solve for this. Gzipping everything won't work because some endpoints - the all-rows CSV endpoint and the download-database endpoint - are streaming and hence can't be buffered-and-gzipped. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1213/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
801780625 | MDU6SXNzdWU4MDE3ODA2MjU= | 9 | SSL Error | 12669260 | open | 0 | 2 | 2021-02-05T02:12:56Z | 2021-02-07T18:45:04Z | NONE | Here's the error I get when running `pip install pocket-to-sqlite`: ``` Could not fetch URL https://pypi.python.org/simple/pocket-to-sqlite/: There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:661) - skipping Could not find a version that satisfies the requirement pocket-to-sqlite (from versions: ) No matching distribution found for pocket-to-sqlite ``` Does this require python 3? | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
802513359 | MDU6SXNzdWU4MDI1MTMzNTk= | 1217 | Possible to deploy as a python app (for Rstudio connect server)? | 6165713 | open | 0 | 4 | 2021-02-05T22:21:24Z | 2022-11-04T11:37:52Z | NONE | Is it possible to deploy a `datasette` application as a python web app? In my enterprise, I have option to deploy python apps via [Rstudio Connect](https://github.com/rstudio/rsconnect-python), and I would like to publish a `datasette` dashboard for sharing. I welcome any pointers to converting `datasette serve` into a python app that can be run as something like `python datasette.py --my_data.db` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1217/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
803333769 | MDU6SXNzdWU4MDMzMzM3Njk= | 32 | KeyError: 'Contents' on running upload | 11855322 | open | 0 | 3 | 2021-02-08T08:36:37Z | 2021-07-22T06:40:25Z | NONE | Following the readme, on big sur, and having entered my auth creds via `dogsheep-photos s3-auth`: ``` (venv) (base) Robins-MacBook:datasette robin$ dogsheep-photos upload photos.db ~/Pictures/Photos\ /Users/robin/Pictures/Library.photoslibrary --dry-run Fetching existing keys from S3... Traceback (most recent call last): File "/Users/robin/datasette/venv/bin/dogsheep-photos", line 8, in <module> sys.exit(cli()) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/dogsheep_photos/cli.py", line 96, in upload key.split(".")[0] for key in get_all_keys(client, creds["photos_s3_bucket"]) File "/Users/robin/datasette/venv/lib/python3.8/site-packages/dogsheep_photos/utils.py", line 46, in get_all_keys for row in page["Contents"]: KeyError: 'Contents' ``` Possibly since the bucket is in `EU (London) eu-west-2` and this into is not requested? | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
803338729 | MDU6SXNzdWU4MDMzMzg3Mjk= | 33 | photo-to-sqlite: command not found | 11855322 | open | 0 | 4 | 2021-02-08T08:42:57Z | 2021-02-12T15:00:44Z | NONE | Having installed in a venv I get: ``` (venv) (base) Robins-MacBook:datasette robin$ photo-to-sqlite apple-photos photos.db -bash: photo-to-sqlite: command not found ``` | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/33/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
803356942 | MDU6SXNzdWU4MDMzNTY5NDI= | 1218 | /usr/local/opt/python3/bin/python3.6: bad interpreter: No such file or directory | 11855322 | open | 0 | 1 | 2021-02-08T09:07:00Z | 2021-02-23T12:12:17Z | NONE | Error as above, however I do have python3.8 and the readme indicates this is supported. ``` (venv) (base) Robins-MacBook:datasette robin$ ls /usr/local/opt/python3/bin/ .. pip3 python3 python3.8 ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1218/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
803929694 | MDU6SXNzdWU4MDM5Mjk2OTQ= | 1219 | Try profiling Datasette using scalene | 9599 | open | 0 | 2 | 2021-02-08T20:37:06Z | 2021-02-08T22:13:00Z | OWNER | https://github.com/emeryberger/scalene looks like an interesting profiling tool. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1219/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
808771690 | MDU6SXNzdWU4MDg3NzE2OTA= | 1225 | More flexible formatting of records with CSS grid | 649467 | open | 0 | 0 | 2021-02-15T19:28:17Z | 2021-02-15T19:28:35Z | NONE | In several applications I've been experimenting with alternate formatting of datasette query results. Lately I've found that CSS grids work very well and seem quite general for formatting rows. In CSS I use grid templates to define the layout of each record and the regions for each field, hiding the fields I don't want. It's pretty flexible and looks good. It's also a great basis for highly responsive layout. I initially thought I'd only use this feature for record detail views, but now I use it for index views as well. However, there are some limitations: * With the existing table templates, it seems that you can change the `display` property on the enclosing `table`, `tbody`, and `tr` to make them be grid-like, but that seems hacky (convert `table` and `tbody` to be `display: block` and `tr` to be `display: grid`). * More significantly, it's very nice to have the column name available when rendering each record to display headers/field labels. The existing templates don't do that, so a custom `_table` template is necessary. * I don't know if any plugins are sensitive to whether data is rendered as a table or not since I'm not completely clear how plugins get their data. * Regardless, you need custom CSS to take full advantage of grids. I don't have a proposal on how to integrate them more deeply. It would be helpful to at least have an official example or test that used a grid layout for records to make sure nothing in datasette breaks with it. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1225/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
811054000 | MDU6SXNzdWU4MTEwNTQwMDA= | 1230 | Vega charts are plotted only for rows on the visible page, cluster maps only for rows in the remaining pages | 7107523 | open | 0 | 1 | 2021-02-18T12:27:02Z | 2021-02-18T15:22:15Z | NONE | I filtered a data set on some criteria and obtain 265 results, split over three pages (100, 100, 65), and reazlized that Vega plots are only applied to the results displayed on the current page, instead of the whole filtered data, _e.g._, 100 on page 1, 100 on page 2, 65 on page 3. Is there a way to force the graphs to consider all results instead of just the page, considering that pages rarely represent sensible information? Likewise, while the cluster map does show all results on the first page, if you go to next pages, it will show all remaining results except the previous page(s), _e.g._, 265 on page 1, 165 on page 2, 65 on page 3. In both cases, I don't see many situations where one would like to represent the data this way, and it might even lead to interpretation errors when viewing the data. Am I missing some cases where this would be best? Perhaps a clickable option to subset visual representations according visible pages _vs._ display all search results would do? [Edit] Oh, I just saw the "Load all" button under the cluster map as well as the [setting to alter the max number or results](https://docs.datasette.io/en/stable/settings.html#max-returned-rows). So I guess this issue only is about the Vega charts. | 107914493 | issue | |||||||||
811458446 | MDU6SXNzdWU4MTE0NTg0NDY= | 1233 | "datasette publish cloudrun" cannot publish files with spaces in their name | 9599 | open | 0 | 1 | 2021-02-18T21:08:31Z | 2021-02-18T21:10:08Z | OWNER | Got this error: ``` Step 6/9 : RUN datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json ---> Running in db9da0068592 Usage: datasette inspect [OPTIONS] [FILES]... Try 'datasette inspect --help' for help. Error: Invalid value for '[FILES]...': Path 'extra' does not exist. The command '/bin/sh -c datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json' returned a non-zero code: 2 ERROR ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 2 ``` While working on the demo for #1232, using this deploy command: ``` GITHUB_SHA=crossdb datasette publish cloudrun fixtures.db 'extra database.db' \ -m fixtures.json \ --plugins-dir=plugins \ --branch=$GITHUB_SHA \ --version-note=$GITHUB_SHA \ --extra-options="--setting template_debug 1 --crossdb" \ --install=pysqlite3-binary \ --service=datasette-latest-crossdb ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1233/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
811505638 | MDU6SXNzdWU4MTE1MDU2Mzg= | 1234 | Runtime support for ATTACHing multiple databases | 9599 | open | 0 | 1 | 2021-02-18T22:06:47Z | 2021-02-22T21:06:28Z | OWNER | > The implementation in #1232 is ready to land. It's the simplest-thing-that-could-possibly-work: you can run `datasette one.db two.db three.db --crossdb` and then use the `/_memory` page to run joins across tables from multiple databases. > > It only works on the first 10 databases that were passed to the command-line. This means that if you have a Datasette instance with hundreds of attached databases (see [Datasette Library](https://github.com/simonw/datasette/issues/417)) this won't be particularly useful for you. > > So... a better, future version of this feature would be one that lets you join across databases on command - maybe by hitting `/_memory?attach=db1&attach=db2` to get a special connection. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/283#issuecomment-781665560_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1234/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
812704869 | MDU6SXNzdWU4MTI3MDQ4Njk= | 1237 | ?_pretty=1 option for pretty-printing JSON output | 9599 | open | 0 | 3268330 | 1 | 2021-02-20T20:54:40Z | 2021-11-16T18:28:33Z | OWNER | Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
813880401 | MDExOlB1bGxSZXF1ZXN0NTc3OTUzNzI3 | 5 | WIP: Add Gmail takeout mbox import | 306240 | open | 0 | 25 | 2021-02-22T21:30:40Z | 2021-07-28T07:18:56Z | FIRST_TIME_CONTRIBUTOR | dogsheep/google-takeout-to-sqlite/pulls/5 | WIP This PR adds the ability to import emails from a Gmail mbox export from Google Takeout. This is my first PR to a datasette/dogsheep repo. I've tested this on my personal Google Takeout mbox with ~520,000 emails going back to 2004. This took around ~20 minutes to process. To provide some feedback on the progress of the import I added the "rich" python module. I'm happy to remove that if adding a dependency is discouraged. However, I think it makes a nice addition to give feedback on the progress of a long import. Do we want to log emails that have errors when trying to import them? Dealing with encodings with emails is a bit tricky. I'm very open to feedback on how to deal with those better. As well as any other feedback for improvements. | 206649770 | pull | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
814595021 | MDU6SXNzdWU4MTQ1OTUwMjE= | 1241 | Share button for copying current URL | 7107523 | open | 0 | 6 | 2021-02-23T15:55:40Z | 2023-08-24T20:09:52Z | NONE | I use datasette in an `iframe` inside another HTML file that contains other ways to represent my data (mostly leaflets maps built with R on summarized data), and the datasette `iframe` is a tab in that page. This particular use prevents users to access the full URLs of their datasette views and queries, which is a shame because the way datasette handles URLs to make every view or query easy to share is awesome. I know how to get the URL from the context menu of my browser, but I don't think many visitors would do it or even notice that datasette uses permalinks for pretty much every action they do. Would it be possible to add a "Share link" button to the interface, either in datasette itself or in a plugin? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1241/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
816526538 | MDU6SXNzdWU4MTY1MjY1Mzg= | 239 | sqlite-utils extract could handle nested objects | 9599 | open | 0 | 16 | 2021-02-25T15:10:28Z | 2022-09-03T23:46:02Z | OWNER | Imagine a table (imported from a nested JSON file) where one of the columns contains values that look like this: {"email": "anonymous@noreply.airtable.com", "id": "usrROSHARE0000000", "name": "Anonymous"} The `sqlite-utils extract` command already uses single text values in a column to populate a new table. It would not be much of a stretch for it to be able to use JSON instead, including specifying which of those values should be used as the primary key in the new table. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/239/reactions", "total_count": 6, "+1": 5, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
816601354 | MDExOlB1bGxSZXF1ZXN0NTgwMjM1NDI3 | 241 | Extract expand - work in progress | 9599 | open | 0 | 0 | 2021-02-25T16:36:38Z | 2021-02-25T16:36:38Z | OWNER | simonw/sqlite-utils/pulls/241 | Refs #239. Still needs documentation and CLI implementation. | 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/241/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | ||||||
817544251 | MDU6SXNzdWU4MTc1NDQyNTE= | 1245 | Sticky table column headers would be useful, especially on the query page | 9599 | open | 0 | 1 | 2021-02-26T17:42:51Z | 2021-04-02T20:53:35Z | OWNER | Suggestion from office hours. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1245/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
817989436 | MDU6SXNzdWU4MTc5ODk0MzY= | 242 | Async support | 25778 | open | 0 | 13 | 2021-02-27T18:29:38Z | 2021-10-28T14:37:56Z | CONTRIBUTOR | Following our conversation last week, want to note this here before I forget. I've had a couple situations where I'd like to do a bunch of updates in an async event loop, but I run into SQLite's issues with concurrent writes. This feels like something sqlite-utils could help with. PeeWee ORM has a [SQLite write queue](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#sqliteq) that might be a good model. It's using threads or gevent, but I _think_ that approach would translate well enough to asyncio. Happy to help with this, too. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/242/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
818684978 | MDU6SXNzdWU4MTg2ODQ5Nzg= | 243 | How can i use this utils to deal with fts on column meta of tables ? | 27874014 | open | 0 | 0 | 2021-03-01T09:45:05Z | 2021-03-01T09:45:05Z | NONE | Thank you to release this bravo project. When i use this project on multi table db, I want to implement convenient search on column name from different tables. I want to develop a meta table to save the meta data of different columns of different tables and search on this meta table to get rows from the data table (which the meta table describes) does this project provide some simple function on it ? You can think a have a knowledge graph about the table in the db, and i save this knowledge graph into the db with fts enabled. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/243/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
821841046 | MDU6SXNzdWU4MjE4NDEwNDY= | 6 | Upgrade to latest sqlite-utils | 9599 | open | 0 | 1 | 2021-03-04T07:21:54Z | 2021-03-04T07:22:51Z | MEMBER | This is pinned to v1 at the moment. | 206649770 | issue | { "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
824750134 | MDU6SXNzdWU4MjQ3NTAxMzQ= | 1251 | facet option not appearing when table is big | 15836677 | open | 0 | 0 | 2021-03-08T16:54:04Z | 2021-03-08T16:54:16Z | NONE | I have a big table with more than 500.000 rows. Trying to facet by one of my columns, the options are not available as for the other smaller tables. I have tried to set it in URL as: `&_facet=city_id` to no avail. is there any limit? how can I force the option "facet" to appear for big tables? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
826064552 | MDU6SXNzdWU4MjYwNjQ1NTI= | 1253 | Capture "Ctrl + Enter" or "⌘ + Enter" to send SQL query? | 9308268 | open | 0 | 1 | 2021-03-09T15:00:50Z | 2021-10-30T16:00:42Z | NONE | It appears as though "Shift + Enter" triggers the form submit action to submit SQL, but could that action be bound to the "Ctrl + Enter" or "⌘ + Enter" action? I feel like that pattern already exists in a number of similar tools and could improve usability of the editor. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1253/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
826700095 | MDU6SXNzdWU4MjY3MDAwOTU= | 1255 | Facets timing out but work when filtering | 1219001 | open | 0 | 2 | 2021-03-09T22:01:39Z | 2021-04-02T20:50:08Z | NONE | System info: Windows 10 Datasette 0.55 installed via pip Python 3.8.5 in a conda environment I'm getting the message `These facets timed out` on any faceting operation. However, when I apply a filter, the facets appear in the filtered view. The error returns when the filter is removed. My data only has 38,450 rows. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1255/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
828858421 | MDU6SXNzdWU4Mjg4NTg0MjE= | 1258 | Allow canned query params to specify default values | 1385831 | open | 0 | 5 | 2021-03-11T07:19:02Z | 2023-02-20T23:39:58Z | NONE | If I call a canned query that includes named parameters, without passing any parameters, datasette runs the query anyway, resulting in an HTTP status code 400, and a visible error in the browser, with only a link back to home. This means that one of the default links on https://site/database/ will lead to a broken page with no apparent way out. ![image](https://user-images.githubusercontent.com/1385831/110748683-13e72300-820e-11eb-855c-32e03dfef5bf.png) Is there any way to skip performing the query when parameters aren't supplied, but otherwise render the usual canned query page? Alternatively, can I supply default values for my parameters, either when defining my canned queries or when linking to the canned query page from the default database template. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1258/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
830283447 | MDU6SXNzdWU4MzAyODM0NDc= | 34 | bucket name | 6213 | open | 0 | 0 | 2021-03-12T16:40:57Z | 2021-03-12T16:40:57Z | NONE | I followed the instructions to setup credentials but I am getting a invalid bucket name. Can you put a sample auth.json file in the base that shows the correct format for this? Thanks | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/34/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
830567275 | MDU6SXNzdWU4MzA1NjcyNzU= | 1259 | Research using CTEs for faster facet counts | 9599 | open | 0 | 5 | 2021-03-12T22:19:49Z | 2021-03-21T22:55:31Z | OWNER | https://www.sqlite.org/changes.html#version_3_35_0 > Add support for the [MATERIALIZED](https://www.sqlite.org/lang_with.html#mathint) and [NOT MATERIALIZED](https://www.sqlite.org/lang_with.html#mathint) hints when specifying [common table expressions](https://www.sqlite.org/lang_with.html). The default behavior was formerly NOT MATERIALIZED, but is now changed to MATERIALIZED for CTEs that are used more than once. If a CTE creates a table that is used multiple time in that query, SQLite will now default to creating a materialized table for the duration of that query. This could be a big performance boost when applying faceting multiple times against the same query. Consider this example query: ```sql WITH data as ( select * from [global-power-plants] ), country_long as (select 'country_long' as col, country_long as value, count(*) as c from data group by country_long order by c desc limit 10 ), primary_fuel as ( select 'primary_fuel' as col, primary_fuel as value, count(*) as c from data group by primary_fuel order by c desc limit 10 ) select * from primary_fuel union select * from country_long order by col, c desc ``` https://global-power-plants.datasettes.com/global-power-plants?sql=WITH+data+as+%28%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++%5Bglobal-power-plants%5D%0D%0A%29%2C%0D%0Acountry_long+as+%28select+%0D%0A++%27country_long%27+as+col%2C+country_long+as+value%2C+count%28*%29+as+c+from+data+group+by+country_long%0D%0A++order+by+c+desc+limit+10%0D%0A%29%2C%0D%0Aprimary_fuel+as+%28%0D%0Aselect%0D%0A++%27primary_fuel%27+as+col%2C+primary_fuel+as+value%2C+count%28*%29+as+c+from+data+group+by+primary_fuel%0D%0A++order+by+c+desc+limit+10%0D%0A%29%0D%0Aselect+*+from+primary_fuel+union+select+*+from+country_long+order+by+col%2C+c+desc Outputs: col | value | c -- | -- | -- country_long | United States of America | 8688 country_long | China | 4235 country_long | United Kingdom | 2603 country_long | Brazil | 2360 country_long | France | 2155 country… | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
830901133 | MDExOlB1bGxSZXF1ZXN0NTkyMzY0MjU1 | 16 | Add a fallback ID, print if no ID found | 1234956 | open | 0 | 0 | 2021-03-13T13:38:29Z | 2021-03-13T14:44:04Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/16 | Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/14 | 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
834602299 | MDU6SXNzdWU4MzQ2MDIyOTk= | 1262 | Plugin hook that could support 'order by random()' for table view | 19328961 | open | 0 | 3 | 2021-03-18T10:02:01Z | 2021-03-18T17:55:01Z | NONE | I am frequently using Datasette to quickly get a visual impression for a table without reviewing it in its entirety. Because I have some groups of similar records, the default sorting options mean that each page is very similar and not representative of the full dataset. The current interface allows sorting by columns, but random sorting is only available via custom SQL. Maybe this could be a button or link. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1262/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
836063389 | MDU6SXNzdWU4MzYwNjMzODk= | 17 | Datetime columns are not properly formatted to be recognizes as datetime | 1234956 | open | 0 | 0 | 2021-03-19T14:33:04Z | 2021-03-19T14:33:04Z | NONE | Currently, the datetimes are formatted in a way that is not recognized by datasette-vega for plotting with a `Date/time` type for the axis. For example, if you have datasette running locally with `datasette-vega` installed and have a database that includes resting heart rate: ``` http://localhost:8001/healthkit/rRestingHeartRate#g.mark=line&g.x_column=startDate&g.x_type=temporal&g.y_column=value&g.y_type=quantitative ``` The plot is blank unless you choose `Label` as the type for the date data. The `startDate` (and `creationDate` and `endDate`) columns appear like: `2019-11-14 18:22:18 -0700` If instead the format for this column is changed slightly: `2019-11-14T18:22:18-07:00` they are recognized as proper dates and the charting works as expected. I have a PR that addresses this issue, will submit shortly. | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
836064851 | MDExOlB1bGxSZXF1ZXN0NTk2NjI3Nzgw | 18 | Add datetime parsing | 1234956 | open | 0 | 0 | 2021-03-19T14:34:22Z | 2021-03-19T14:34:22Z | FIRST_TIME_CONTRIBUTOR | dogsheep/healthkit-to-sqlite/pulls/18 | Parses the datetime columns so they are subsequently properly recognized as datetime. Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/17 | 197882382 | pull | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
836829560 | MDU6SXNzdWU4MzY4Mjk1NjA= | 248 | support for Apache Arrow / parquet files I/O | 649467 | open | 0 | 1 | 2021-03-20T14:59:30Z | 2021-10-28T23:46:48Z | NONE | I just started looking at Apache Arrow using pyarrow for import and export of tabular datasets, and it looks quite compelling. It might be worth looking at for sqlite-utils and/or datasette. As a test, I took a random jsonl data dump of a dataset I have with floats, strings, and ints and converted it to arrow's parquet format using the naive `pyarrow.parquet.write_file()` command, which has automatic type inferrence. It compressed down to 7% of the original size. Conversion of a 26MB JSON file and serializing it to parquet was eyeblink instantaneous. Parquet files are portable and can be directly imported into pandas and other analytics software. The only hangup is the automatic type inference of the naive reader. It's great for general laziness and for parsing JSON columns (it correctly interpreted a table of mine with a JSON array). However, I did get an exception for a string column where most entries looked integer-like but had a couple values that weren't -- the reader tried to coerce all of them for some reason, even though the JSON type is string. Since the writer optionally takes a schema, it shouldn't be too hard to grab the sqlite header types. With some additional hinting, you might get datetime columns and JSON, which are native Arrow types. Somewhat tangentially, someone even wrote an sqlite vfs extension for Parquet: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/248/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |