github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
470691999 | MDU6SXNzdWU0NzA2OTE5OTk= | 43 | .add_column() doesn't match indentation of initial creation | 9599 | closed | 0 | 3 | 2019-07-20T16:33:10Z | 2019-07-23T13:09:11Z | 2019-07-23T13:09:05Z | OWNER | I spotted a table which was created once and then had columns added to it and the formatted SQL looks like this: ```sql CREATE TABLE [records] ( [type] TEXT, [sourceName] TEXT, [sourceVersion] TEXT, [unit] TEXT, [creationDate] TEXT, [startDate] TEXT, [endDate] TEXT, [value] TEXT, [metadata_Health Mate App Version] TEXT, [metadata_Withings User Identifier] TEXT, [metadata_Modified Date] TEXT, [metadata_Withings Link] TEXT, [metadata_HKWasUserEntered] TEXT , [device] TEXT, [metadata_HKMetadataKeyHeartRateMotionContext] TEXT, [metadata_HKDeviceManufacturerName] TEXT, [metadata_HKMetadataKeySyncVersion] TEXT, [metadata_HKMetadataKeySyncIdentifier] TEXT, [metadata_HKSwimmingStrokeStyle] TEXT, [metadata_HKVO2MaxTestType] TEXT, [metadata_HKTimeZone] TEXT, [metadata_Average HR] TEXT, [metadata_Recharge] TEXT, [metadata_Lights] TEXT, [metadata_Asleep] TEXT, [metadata_Rating] TEXT, [metadata_Energy Threshold] TEXT, [metadata_Deep Sleep] TEXT, [metadata_Nap] TEXT, [metadata_Edit Slots] TEXT, [metadata_Tags] TEXT, [metadata_Daytime HR] TEXT) ``` It would be nice if the columns that were added later matched the indentation of the initial columns. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/43/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
471628483 | MDU6SXNzdWU0NzE2Mjg0ODM= | 44 | Utilities for building lookup tables | 9599 | closed | 0 | 2 | 2019-07-23T10:59:58Z | 2019-07-23T13:07:01Z | 2019-07-23T13:07:01Z | OWNER | While building https://github.com/dogsheep/healthkit-to-sqlite I found a need for a neat mechanism for easily building lookup tables - tables where each unique value in a column is replaced by a foreign key to a separate table. csvs-to-sqlite currently creates those with its "extract" mechanism - but that's written as custom code against Pandas. I'd like to eventually replace Pandas with sqlite-utils there. See also #42 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/44/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
351845423 | MDU6SXNzdWUzNTE4NDU0MjM= | 3 | Experiment with contentless FTS tables | 9599 | closed | 0 | 1 | 2018-08-18T19:31:01Z | 2019-07-22T20:58:55Z | 2019-07-22T20:58:55Z | OWNER | Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
463544206 | MDU6SXNzdWU0NjM1NDQyMDY= | 537 | Populate "endpoint" key in ASGI scope | 9599 | open | 0 | 12 | 2019-07-03T04:54:47Z | 2019-07-22T06:03:18Z | OWNER | This is a trick used by Starlette so that other layers of ASGI middleware can see which route was selected. They added it here: https://github.com/encode/starlette/commit/34d0097feb6f057bd050d5057df5a2f96b97384e If Datasette supports it as well we can benefit from it if we integrate this sentry_asgi middleware (probably as a `datasette-sentry` plugin): https://github.com/encode/sentry-asgi/blob/c6a42d44d31f85885b79e4ee898683ecf8104971/sentry_asgi/middleware.py#L34-L35 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/537/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
470691622 | MDU6SXNzdWU0NzA2OTE2MjI= | 5 | Add progress bar | 9599 | closed | 0 | 2 | 2019-07-20T16:29:07Z | 2019-07-22T03:30:13Z | 2019-07-22T02:49:22Z | MEMBER | Showing a progress bar would be nice, using Click. The easiest way to do this would probably be be to hook it up to the length of the compressed content, and update it as this code pushes more XML bytes through the parser: https://github.com/dogsheep/healthkit-to-sqlite/blob/d64299765064501f4efdd9a0b21dbdba9ec4287f/healthkit_to_sqlite/utils.py#L6-L10 | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
470856782 | MDU6SXNzdWU0NzA4NTY3ODI= | 6 | Break up records into different tables for each type | 9599 | closed | 0 | 1 | 2019-07-22T01:54:59Z | 2019-07-22T03:28:55Z | 2019-07-22T03:28:50Z | MEMBER | I don't think there's much benefit to having all of the different record types stored in the same enormous table. Here's what I get when I use `_facet=type`: <img width="358" alt="hello2__records__2_672_233_rows" src="https://user-images.githubusercontent.com/9599/61601118-e2f54d00-abe8-11e9-8bf6-3df2ef969112.png"> I'm going to try splitting these up into separate tables - so `HKQuantityTypeIdentifierBodyMassIndex` becomes a table called `rBodyMassIndex` - and see if that's nicer to work with. | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
470637152 | MDU6SXNzdWU0NzA2MzcxNTI= | 2 | Import workouts | 9599 | closed | 0 | 1 | 2019-07-20T05:20:21Z | 2019-07-20T06:21:41Z | 2019-07-20T06:21:41Z | MEMBER | From #1 | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
470640505 | MDU6SXNzdWU0NzA2NDA1MDU= | 4 | Import Records | 9599 | closed | 0 | 1 | 2019-07-20T06:11:20Z | 2019-07-20T06:21:41Z | 2019-07-20T06:21:41Z | MEMBER | From #1: ```python 'Record': {'attr_counts': {'creationDate': 2672233, 'device': 2665111, 'endDate': 2672233, 'sourceName': 2672233, 'sourceVersion': 2671779, 'startDate': 2672233, 'type': 2672233, 'unit': 2650012, 'value': 2672232}, 'child_counts': {'HeartRateVariabilityMetadataList': 2318, 'MetadataEntry': 287974}, 'count': 2672233, 'parent_counts': {'Correlation': 2, 'HealthData': 2672231}}, ``` | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
470637206 | MDU6SXNzdWU0NzA2MzcyMDY= | 3 | Import ActivitySummary | 9599 | closed | 0 | 0 | 2019-07-20T05:21:00Z | 2019-07-20T05:58:07Z | 2019-07-20T05:58:07Z | MEMBER | From #1 ```python 'ActivitySummary': {'attr_counts': {'activeEnergyBurned': 980, 'activeEnergyBurnedGoal': 980, 'activeEnergyBurnedUnit': 980, 'appleExerciseTime': 980, 'appleExerciseTimeGoal': 980, 'appleStandHours': 980, 'appleStandHoursGoal': 980, 'dateComponents': 980}, 'child_counts': {}, 'count': 980, 'parent_counts': {'HealthData': 980}}, ``` | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
470637068 | MDU6SXNzdWU0NzA2MzcwNjg= | 1 | Use XML Analyser to figure out the structure of the export XML | 9599 | closed | 0 | 1 | 2019-07-20T05:19:02Z | 2019-07-20T05:20:09Z | 2019-07-20T05:20:09Z | MEMBER | https://github.com/simonw/xml_analyser | 197882382 | issue | { "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
470131537 | MDU6SXNzdWU0NzAxMzE1Mzc= | 41 | sqlite-utils insert --tsv option | 9599 | closed | 0 | 0 | 2019-07-19T04:27:21Z | 2019-07-19T04:50:47Z | 2019-07-19T04:50:47Z | OWNER | Right now we only support ingesting CSV, but sometimes interesting data is released as TSV. https://www.washingtonpost.com/national/2019/07/18/how-download-use-dea-pain-pills-database/ for example. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/41/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
467864071 | MDU6SXNzdWU0Njc4NjQwNzE= | 39 | table.get(...) method | 9599 | closed | 0 | 0 | 2019-07-14T17:20:51Z | 2019-07-15T04:28:53Z | 2019-07-15T04:28:53Z | OWNER | Utility method for fetching a record by its primary key. Accepts a single value (for primary key / rowid tables) or a list/tuple of values (for compound primary keys, refs #36). Raises a `NotFoundError` if the record cannot be found. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/39/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
462817589 | MDU6SXNzdWU0NjI4MTc1ODk= | 36 | Support compound primary keys | 9599 | closed | 0 | 0 | 2019-07-01T17:00:07Z | 2019-07-15T04:28:52Z | 2019-07-15T04:28:52Z | OWNER | This should work: ```python table = db["dog_breeds"].insert({ "dog_id": 1, "breed_id": 2 }, pk=("dog_id", "breed_id")) ``` Needed for m2m work in #23 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/36/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
455996809 | MDU6SXNzdWU0NTU5OTY4MDk= | 28 | Rearrange the docs by area, not CLI vs Python | 9599 | closed | 0 | 1 | 2019-06-13T23:33:35Z | 2019-07-15T02:37:20Z | 2019-07-15T02:37:20Z | OWNER | The docs for eg inserting data should live on the same page, rather than being split across the API and CLI pages. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
467218270 | MDU6SXNzdWU0NjcyMTgyNzA= | 558 | Support unicode in url | 380586 | closed | 0 | 4 | 2019-07-12T04:43:24Z | 2019-07-15T01:29:30Z | 2019-07-14T02:49:33Z | NONE | Hi, I defined some custom queries in my `metadata.json`. There are Chinese characters in the names of the queries. So the urls are like `http://127.0.0.1:8001/mydb/测试查询`. When opening such urls, datasette will throw an exception. ``` Traceback (most recent call last): File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/asgi.py", line 100, in __call__ return await view(new_scope, receive, send) File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/asgi.py", line 172, in view request, **scope["url_route"]["kwargs"] File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/views/base.py", line 267, in get request, database, hash, correct_hash_provided, **kwargs File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/views/base.py", line 471, in view_get for key in self.ds.renderers.keys() File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/views/base.py", line 471, in <dictcomp> for key in self.ds.renderers.keys() File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/__init__.py", line 655, in path_with_format path = request.path File "/home/zhe/miniconda3/lib/python3.7/site-packages/datasette/utils/asgi.py", line 49, in path self.scope.get("raw_path", self.scope["path"].encode("latin-1")) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 9-11: ordinal not in range(256) ``` This used to work when datasette was based on sanic. Btw, thanks for the great work! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/558/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
465731062 | MDU6SXNzdWU0NjU3MzEwNjI= | 555 | Static mounts with relative paths not working | 3243482 | closed | 0 | 0 | 2019-07-09T11:38:35Z | 2019-07-11T16:13:22Z | 2019-07-11T16:13:22Z | CONTRIBUTOR | Datasette fails to serve files from static mounts that are created using relative paths `datasette --static mystatic:rel/path/to/static/dir`. I've explained the problem and the solution in the pull request: https://github.com/simonw/datasette/pull/554 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/555/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
465003070 | MDU6SXNzdWU0NjUwMDMwNzA= | 551 | Ship many-to-many faceting support (and facet-by-delimiter) | 9599 | open | 0 | 2 | 2019-07-07T23:11:45Z | 2019-07-08T15:45:23Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/551/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
456569067 | MDU6SXNzdWU0NTY1NjkwNjc= | 510 | Ability to facet by delimiter (e.g. comma separated fields) | 9599 | open | 0 | 9599 | 1 | 2019-06-15T19:34:41Z | 2019-07-08T15:44:51Z | OWNER | E.g. if a field contains "Tags,With,Commas" be able to facet them in the same way as `_facet_array=` lets you facet `["Tags", "With", "Commas"]` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/510/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
462117311 | MDU6SXNzdWU0NjIxMTczMTE= | 531 | /database/-/inspect | 9599 | open | 0 | 1 | 2019-06-28T16:33:41Z | 2019-07-08T15:43:57Z | OWNER | Build `/database/-/inspect` which shows tables, columns, column types and foreign keys It won't show table counts. Or maybe it will include them optionally but only for `-i` databases, in a special area of the JSON reserved for immutable-only inspect details. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/465#issuecomment-506797086_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/531/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
465327844 | MDU6SXNzdWU0NjUzMjc4NDQ= | 553 | Potential improvements to facet-by-date | 9599 | open | 0 | 3 | 2019-07-08T15:37:53Z | 2019-07-08T15:41:55Z | OWNER | In addition to #483 Tobias had some useful suggestions on Twitter: https://twitter.com/rixxtr/status/1148253926476701696 > I think for date facets, it might be more meaningful to order them by date, rather than by size? Or offer both? I'm *definitely* often interested in size-over-time, so https://data.rixx.de/django_tickets/tickets?_facet_date=created#facet-created … isn't all that helpful! Screenshot of that link: <img width="1092" alt="django_tickets__tickets__29_846_rows" src="https://user-images.githubusercontent.com/9599/60823090-9f680100-a15b-11e9-84e9-52b9d666e90f.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/553/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
465001185 | MDU6SXNzdWU0NjUwMDExODU= | 549 | Send pull request to the repo that the _table.html template will break | 9599 | closed | 0 | 4471010 | 1 | 2019-07-07T22:45:17Z | 2019-07-08T03:36:46Z | 2019-07-08T03:36:45Z | OWNER | Bump this to 0.29 https://github.com/simonw/salaries-datasette/blob/master/requirements/base.txt And rename https://github.com/simonw/salaries-datasette/blob/master/templates/_rows_and_columns.html to _table.html | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/549/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
464990184 | MDU6SXNzdWU0NjQ5OTAxODQ= | 547 | Release notes for 0.29 | 9599 | closed | 0 | 4471010 | 2 | 2019-07-07T20:30:28Z | 2019-07-08T03:31:59Z | 2019-07-08T03:31:59Z | OWNER | There's a lot of stuff... https://github.com/simonw/datasette/compare/0.28...master | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/547/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
445868234 | MDU6SXNzdWU0NDU4NjgyMzQ= | 478 | Make it so Docker build doesn't delay PyPI release | 9599 | closed | 0 | 4471010 | 3 | 2019-05-19T21:52:10Z | 2019-07-08T03:30:41Z | 2019-07-07T20:03:20Z | OWNER | Datasette automated releases currently include building a Docker image that has a full custom-compiled version of SQLite and SpatiaLite. This takes ages! I still want to publish this Docker image (to https://hub.docker.com/r/datasetteproject/datasette/tags ) but I'd like it if this wasn't a blocker on pushing the new package to PyPI. Ideally PyPI publish would happen first. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/478/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
464868844 | MDU6SXNzdWU0NjQ4Njg4NDQ= | 543 | datasette publish option for setting plugin configuration secrets | 9599 | closed | 0 | 4471010 | 3 | 2019-07-06T16:21:23Z | 2019-07-08T02:06:34Z | 2019-07-08T02:06:34Z | OWNER | Follow-on from #538 - the `datasette publish` command needs a way of passing secrets which will be made available to plugin configuration but will not be exposed in `/-/metadata.json`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/543/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
464994105 | MDU6SXNzdWU0NjQ5OTQxMDU= | 548 | Add datasette-cors and datasette-auth-github plugins to Ecosystem page | 9599 | closed | 0 | 4471010 | 0 | 2019-07-07T21:14:14Z | 2019-07-08T02:02:36Z | 2019-07-08T02:02:36Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/548/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
465019882 | MDU6SXNzdWU0NjUwMTk4ODI= | 552 | Add --plugin-secret support to "datasette package" | 9599 | open | 0 | 1 | 2019-07-08T01:46:47Z | 2019-07-08T01:47:30Z | OWNER | Split out from #544. I think I should combine this with #347 (renaming `datasette package` to `datasette publish docker`). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/552/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
465002978 | MDU6SXNzdWU0NjUwMDI5Nzg= | 550 | Pull m2m faceting out of master so we can ship a release without it | 9599 | closed | 0 | 4471010 | 1 | 2019-07-07T23:10:48Z | 2019-07-07T23:21:22Z | 2019-07-07T23:21:22Z | OWNER | After spending some time with #495 I believe I need to make some pretty major changes to how m2m faceting works. I don't want it to block the release of ASGI Datasette so I'm going to revert it back out of master for the moment and merge it back in after the release has gone out. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/550/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
446433735 | MDU6SXNzdWU0NDY0MzM3MzU= | 482 | Example of a custom facet plugin is incorrect | 9599 | closed | 0 | 4471010 | 0 | 2019-05-21T06:12:47Z | 2019-07-07T23:19:10Z | 2019-07-07T23:19:10Z | OWNER | The function signatures are wrong on https://datasette.readthedocs.io/en/0.28/plugins.html#register-facet-classes The new signatures are: `async def suggest(self)` and `async def facet_results(self)` - the `sql` and `params` are now passed to the class constructor. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/482/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
453829910 | MDU6SXNzdWU0NTM4Mjk5MTA= | 505 | Add white-space: pre-wrap to SQL create statement | 9599 | closed | 0 | 9599 | 4471010 | 0 | 2019-06-08T19:59:56Z | 2019-07-07T20:26:55Z | 2019-07-07T20:26:55Z | OWNER | Right now a super-long CREATE TABLE statement causes the table page to be even wider than the table itself: <img width="1067" alt="many-photos-tables__RKMaster__1_row_where_where_modelId___18479_and_New_Issue_·_simonw_datasette_and_many-photos-tables__RKPlace__1_274_rows" src="https://user-images.githubusercontent.com/9599/59151726-27d47400-89ed-11e9-9ff7-76c129127329.png"> Adding `white-space: pre-wrap` to that `<pre>` element is an easy fix: <img width="1070" alt="many-photos-tables__RKMaster__28_260_rows_and_New_Issue_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/59151732-476b9c80-89ed-11e9-8c1b-c9af98bad5b4.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/505/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||
464905894 | MDU6SXNzdWU0NjQ5MDU4OTQ= | 545 | Fix header on 404 page | 9599 | closed | 0 | 4471010 | 1 | 2019-07-07T01:47:40Z | 2019-07-07T20:26:55Z | 2019-07-07T20:26:55Z | OWNER | <img width="707" alt="Error_404" src="https://user-images.githubusercontent.com/9599/60762932-83127a00-a01e-11e9-853a-6ea6aa5e8f6d.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/545/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
464449570 | MDU6SXNzdWU0NjQ0NDk1NzA= | 540 | Add a universal navigation bar which can be modified by plugins | 9599 | closed | 0 | 8 | 2019-07-05T03:50:33Z | 2019-07-06T23:13:29Z | 2019-07-06T23:11:35Z | OWNER | Needed by https://github.com/simonw/datasette-auth-github/issues/5 We already have a navigation breadcrumbs header on some pages, I can extend that to be present on every page and make it easy to modify with custom templates. <img width="768" alt="fixtures__facetable" src="https://user-images.githubusercontent.com/9599/60697075-5aaf4200-9e9d-11e9-9009-8f794971f5e5.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/540/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
464779810 | MDU6SXNzdWU0NjQ3Nzk4MTA= | 541 | Plugin hook for adding extra template context variables | 9599 | closed | 0 | 2 | 2019-07-05T21:37:05Z | 2019-07-06T00:05:59Z | 2019-07-06T00:05:59Z | OWNER | It turns out I need this for https://github.com/simonw/datasette-auth-github/issues/5 It can be modelled on the `extra_body_script` hook: https://datasette.readthedocs.io/en/stable/plugins.html#extra-body-script-template-database-table-view-name-datasette | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/541/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
463915863 | MDU6SXNzdWU0NjM5MTU4NjM= | 538 | Mechanism for secrets in plugin configuration | 9599 | closed | 0 | 3 | 2019-07-03T19:23:34Z | 2019-07-04T05:47:54Z | 2019-07-04T05:47:54Z | OWNER | See https://github.com/simonw/datasette-auth-github/issues/1 We need a mechanism where by plugins can tap into "secret" config options without exposing them in the visible metadata.json (where plugin configs currently live, see https://datasette.readthedocs.io/en/stable/plugins.html#plugin-configuration ) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/538/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
459598080 | MDU6SXNzdWU0NTk1OTgwODA= | 520 | asgi_wrapper plugin hook | 9599 | closed | 0 | 9599 | 3 | 2019-06-23T17:16:45Z | 2019-07-03T04:40:34Z | 2019-07-03T04:06:28Z | OWNER | After #272 we can finally add this hook. It will allow plugins to wrap their own ASGI middleware around Datasette. Potential use-cases include: * adding authentication * custom CORS headers (see #454) * maybe gzip support? * possibly defining entirely new routes, though that may be better handled by a separate hook | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/520/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
462928038 | MDU6SXNzdWU0NjI5MjgwMzg= | 532 | Switch setup.py to using ~= for dependencies | 9599 | closed | 0 | 0 | 2019-07-01T21:53:48Z | 2019-07-03T04:32:58Z | 2019-07-03T04:32:58Z | OWNER | `~=` means "compatible release" https://www.python.org/dev/peps/pep-0440/#compatible-release See also https://stackoverflow.com/questions/39590187/in-requirements-txt-what-does-tilde-equals-mean | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/532/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
459621683 | MDU6SXNzdWU0NTk2MjE2ODM= | 521 | Easier way of creating custom row templates | 9599 | closed | 0 | 6 | 2019-06-23T21:49:27Z | 2019-07-03T03:23:56Z | 2019-07-03T03:23:56Z | OWNER | I was messing around with a custom `_rows_and_columns.html` template and ended up with this: ```html {% for row in display_rows %} <div> <hr> {% for cell in row %} {% if cell.column == "First_Name" %} <h2 class="scientist">{{ cell.value }} {% elif cell.column == "Last_Name" %} {{ cell.value }}</h1> {% elif cell.column == "Short_Description" %} <p><strong>{{ cell.column }}: </strong>{{ cell.value }}</p> <p> {% else %} <strong>{{ cell.column }}: </strong>{{ cell.value }} {% endif %} {% endfor %} </div> {% endfor %} ``` This is nasty. I'd like to be able to do something like this instead: ``` {% for row in display_rows %} <h2 class="scientist">{{ row["First_Name"] }} {{ row["Last_Name"] }}</h2> ... ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/521/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
462423839 | MDU6SXNzdWU0NjI0MjM4Mzk= | 33 | index_foreign_keys / index-foreign-keys utilities | 9599 | closed | 0 | 2 | 2019-06-30T16:42:03Z | 2019-06-30T23:54:11Z | 2019-06-30T23:50:55Z | OWNER | Sometimes it's good to have indices on all columns that are foreign keys, to allow for efficient reverse lookups. This would be a useful utility: $ sqlite-utils index-foreign-keys database.db | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/33/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
461237618 | MDU6SXNzdWU0NjEyMzc2MTg= | 31 | Mechanism for adding multiple foreign key constraints at once | 9599 | closed | 0 | 0 | 2019-06-27T00:04:30Z | 2019-06-29T06:27:40Z | 2019-06-29T06:27:40Z | OWNER | Needed by [db-to-sqlite](https://github.com/simonw/db-to-sqlite). It currently works by collecting all of the foreign key relationships it can find and then applying them at the end of the process. The problem is, the `add_foreign_key()` method looks like this: https://github.com/simonw/sqlite-utils/blob/86bd2bba689e25f09551d611ccfbee1e069e5b66/sqlite_utils/db.py#L498-L516 That means it's doing a full `VACUUM` for every single relationship it sets up - and if you have hundreds of foreign key relationships in your database this can take hours. I think the right solution is to have a `.add_foreign_keys(list_of_args)` method which does the bulk operation and then a single `VACUUM`. `.add_foreign_key(...)` can then call the bulk action with a single list item. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/31/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327395270 | MDU6SXNzdWUzMjczOTUyNzA= | 296 | Per-database and per-table /-/ URL namespace | 9599 | open | 0 | 3 | 2018-05-29T16:23:13Z | 2019-06-28T16:46:34Z | OWNER | Initially this will be for subsets of `/-/inspect` and `/-/metadata` but it will also give us a URL namespace for future features like `/-/facet` (expanded list of a specific facet, linked to from `...`) and `/-/graph` To start: * `/dbname/-/inspect` * `/dbname/-/metadata` * `/dbname/tablename/-/inspect` * `/dbname/tablename/-/metadata` This means we will no longer allow databases or tables to have the name `"-"` - I think that's OK We will continue to support rows with a primary key of `"-"` at the following URL: * `/dbname/tablename/-` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/296/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
327365110 | MDU6SXNzdWUzMjczNjUxMTA= | 294 | inspect should record column types | 9599 | open | 0 | 7 | 2018-05-29T15:10:41Z | 2019-06-28T16:45:28Z | OWNER | For each table we want to know the columns, their order and what type they are. I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a `geometry` type for columns that are detected as SpatiaLite geometries. Possible JSON design: "columns": [{ "name": "title", "type": "text" }, ...] Refs #276 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
443038584 | MDU6SXNzdWU0NDMwMzg1ODQ= | 465 | Decide what to do about /-/inspect | 9599 | closed | 0 | 4 | 2019-05-11T21:39:46Z | 2019-06-28T16:34:33Z | 2019-06-28T16:34:33Z | OWNER | It's not clear to me what this endpoint should do now as a result of #419 - it's still useful to be able to introspect databases for tools like datasette-registry, but since we aren't pre-calculating introspection data any more I need to rethink the approach. For one thing, this endpoint may need to be paginated. Or maybe it should be split up into separate endpoints for each connected database? Those should probably be paginated too seeing as fivethirtyeight has 400+ tables. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/465/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
459622390 | MDU6SXNzdWU0NTk2MjIzOTA= | 522 | Handle case-insensitive headers in a nicer way | 9599 | open | 0 | 1 | 2019-06-23T21:56:34Z | 2019-06-26T18:48:53Z | OWNER | Spun out from https://github.com/simonw/datasette/pull/518#discussion_r296486289 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/522/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
460095928 | MDU6SXNzdWU0NjAwOTU5Mjg= | 528 | Establish a pattern for Datasette plugins built on top of Pandas | 9599 | open | 0 | 0 | 2019-06-24T21:05:52Z | 2019-06-24T21:05:52Z | OWNER | The Pandas ecosystem is huge, varied and full of tools that are really good at doing interesting analysis on top of tabular data. Pandas should not be a dependency of Datasette core, but I think there is a lot of potential in having plugins which use Pandas to apply interesting analysis to data sucked out of Datasette's SQLite tables. One example ([thanks, Tony](https://twitter.com/psychemedia/status/1143259809715752962)): https://github.com/ResidentMario/missingno could form the basis of a fantastic plugin for getting a high-level overview of how complete each column in a table is. Some thought is needed here about what shape these kind of plugins might take, and what plugin hooks they would use. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/528/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
459714943 | MDU6SXNzdWU0NTk3MTQ5NDM= | 525 | Add section on sqite-utils enable-fts to the search documentation | 9599 | closed | 0 | 9599 | 2 | 2019-06-24T06:39:16Z | 2019-06-24T16:36:35Z | 2019-06-24T16:29:43Z | OWNER | https://datasette.readthedocs.io/en/stable/full_text_search.html already has a section about csvs-to-sqlite, sqlite-utils is even more relevant. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/525/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
459936585 | MDU6SXNzdWU0NTk5MzY1ODU= | 527 | Unable to use rank when fts-table generated with csvs-to-sqlite | 2181410 | closed | 0 | 3 | 2019-06-24T14:49:48Z | 2019-06-24T15:21:18Z | 2019-06-24T15:09:10Z | NONE | Hi Simon. If i generate a fts-table with the csvs-to-sqlite f-option, I'm unable to use (in datasette's GUI) the internal ranking of the table for sorting or viewing, but if I generate the fts-table with the enable-fts argument from sqlite-utils, everyrthing works ok. Eg.: datasette, version 0.28 sqlite-utils, version 1.2.1 csvs-to-sqlite, version 0.9 No column named rank with these commands: $ csvs-to-sqlite minutes.csv minutes.db -f text_data $ datasette -i minutes.db select rank, * from minutes_fts where minutes_fts match 'dog' Everything ok with these commands: $ csvs-to-sqlite minutes.csv minutes.db $ sqlite-utils enable-fts minutes.db text_data $ datasette -i minutes.db select rank, * from minutes_fts where minutes_fts match 'dog' Am I doing something wrong? Thank you for a great application! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/527/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
272391665 | MDU6SXNzdWUyNzIzOTE2NjU= | 48 | Switch to ujson | 9599 | closed | 0 | 4 | 2017-11-08T23:50:29Z | 2019-06-24T06:57:54Z | 2019-06-24T06:57:43Z | OWNER | ujson is already a dependency of Sanic, and should be quite a bit faster. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
317714268 | MDU6SXNzdWUzMTc3MTQyNjg= | 238 | External metadata.json | 9599 | closed | 0 | 3 | 2018-04-25T17:02:30Z | 2019-06-24T06:52:55Z | 2019-06-24T06:52:45Z | OWNER | A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json` Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340730961 | MDU6SXNzdWUzNDA3MzA5NjE= | 340 | Embrace black | 9599 | closed | 0 | 1 | 2018-07-12T17:32:29Z | 2019-06-24T06:50:27Z | 2019-06-24T06:50:26Z | OWNER | Run [black](https://github.com/ambv/black) against everything. Then set up CI to fail if code doesn't conform to black's style. Here's how Starlette does this: * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/.travis.yml#L14 * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/scripts/lint - essentially runs `black starlette tests --check` And here's an example of a test run that failed: https://travis-ci.org/encode/starlette/jobs/403172478 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/340/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
276455748 | MDU6SXNzdWUyNzY0NTU3NDg= | 146 | datasette publish gcloud | 9599 | closed | 0 | 2 | 2017-11-23T18:55:03Z | 2019-06-24T06:48:20Z | 2019-06-24T06:48:20Z | OWNER | See also #103 It looks like you can start a Google Cloud VM with a "docker container" option - and the Google Cloud Registry is easy to push containers to. So it would be feasible to have `datasette publish gcloud ...` automatically build a container, push it to GCR, then start a new VM instance with it: https://cloud.google.com/container-registry/docs/pushing-and-pulling | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/146/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
291639118 | MDU6SXNzdWUyOTE2MzkxMTg= | 183 | Custom Queries - escaping strings | 82988 | closed | 0 | 2 | 2018-01-25T16:49:13Z | 2019-06-24T06:45:07Z | 2019-06-24T06:45:07Z | CONTRIBUTOR | If a SQLite table column name contains spaces, they are usually referred to in double quotes: `SELECT * FROM mytable WHERE "gappy column name"="my value";` In the JSON metadata file, this is passed by escaping the double quotes: `"queries": {"my query": "SELECT * FROM mytable WHERE \"gappy column name\"=\"my value\";"}` When specifying a custom query in `metadata.json` using double quotes, these are then rendered in the *datasette* query box using single quotes: `SELECT * FROM mytable WHERE 'gappy column name'='my value';` which does not work. Alternatively, a valid custom query can be passed using backticks (\`) to quote the column name and single (unescaped) quotes for the matched value: ``"queries": {"my query": "SELECT * FROM mytable WHERE `gappy column name`='my value';"}`` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/183/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
275125805 | MDU6SXNzdWUyNzUxMjU4MDU= | 124 | Option to open readonly but not immutable | 9599 | closed | 0 | 5 | 2017-11-19T02:11:03Z | 2019-06-24T06:43:46Z | 2019-06-24T06:43:46Z | OWNER | Immutable assumes no other process can modify the file. An option to open reqdonly instead would enable other processes to update the file in place. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/124/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
274315193 | MDU6SXNzdWUyNzQzMTUxOTM= | 106 | Document how pagination works | 9599 | closed | 0 | 1 | 2017-11-15T21:44:32Z | 2019-06-24T06:42:33Z | 2019-06-24T06:42:33Z | OWNER | I made a start at that in this comment: https://news.ycombinator.com/item?id=15691926 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/106/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323716411 | MDU6SXNzdWUzMjM3MTY0MTE= | 267 | Documentation for URL hashing, redirects and cache policy | 9599 | closed | 0 | 3 | 2018-05-16T17:29:01Z | 2019-06-24T06:41:02Z | 2019-06-24T06:41:02Z | OWNER | See my comments on #258 for a starting point | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/267/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
329147284 | MDU6SXNzdWUzMjkxNDcyODQ= | 305 | Add contributor guidelines to docs | 9599 | closed | 0 | 2 | 2018-06-04T17:25:30Z | 2019-06-24T06:40:19Z | 2019-06-24T06:40:19Z | OWNER | https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/305/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
324188953 | MDU6SXNzdWUzMjQxODg5NTM= | 272 | Port Datasette to ASGI | 9599 | closed | 0 | 9599 | 3268330 | 42 | 2018-05-17T21:16:32Z | 2019-06-24T04:54:15Z | 2019-06-24T03:33:06Z | OWNER | Datasette doesn't take much advantage of Sanic, and I'm increasingly having to work around parts of it because of idiosyncrasies that are specific to Datasette - caring about the exact order of querystring arguments for example. Since Datasette is GET-only our needs from a web framework are actually pretty slim. This becomes more important as I expand the plugins #14 framework. Am I sure I want the plugin ecosystem to depend on a Sanic if I might move away from it in the future? If Datasette wasn't all about async/await I would use WSGI, but today it makes more sense to use ASGI. I'd like to be confident that switching to ASGI would still give me the excellent performance that Sanic provides. https://github.com/django/asgiref/blob/master/specs/asgi.rst | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/272/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||
459627549 | MDU6SXNzdWU0NTk2Mjc1NDk= | 523 | Show total/unfiltered row count when filtering | 2657547 | closed | 0 | 2 | 2019-06-23T22:56:48Z | 2019-06-24T01:38:14Z | 2019-06-24T01:38:14Z | CONTRIBUTOR | When I'm seeing a filtered view of a table, I'd like to be able to see something like '2 rows where status != "closed" (of 1000 total)' to have a context for the data I'm seeing – e.g. currently my database is being filled by an importer, so this information would be super helpful. Since this information would be a performance hit, maybe something like '12 rows where status != "closed" (of ??? total)' with lazy-loading on-click(?) could be applied (Or via a "How many total?" tooltip, or …) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/523/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
459469278 | MDU6SXNzdWU0NTk0NjkyNzg= | 515 | Try shrinking official image with docker-slim | 9599 | open | 0 | 0 | 2019-06-22T12:25:37Z | 2019-06-22T12:25:37Z | OWNER | This looks really promising: https://github.com/docker-slim/docker-slim If it can shave substantial size from our official container reliably we could add it to the automated build process. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/515/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
457201907 | MDU6SXNzdWU0NTcyMDE5MDc= | 513 | Is it possible to publish to Heroku despite slug size being too large? | 7936571 | closed | 0 | 2 | 2019-06-18T00:12:02Z | 2019-06-21T22:35:54Z | 2019-06-21T22:35:54Z | NONE | I'm trying to push more than 1.5GB worth of SQLite databases -- 535MB compressed -- to Heroku but I get this error when I run the `datasette publish heroku` command. Compiled slug size: 535.5M is too large (max is 500M). Can I publish the databases and make datasette work on Heroku despite the large slug size? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/513/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
458941203 | MDU6SXNzdWU0NTg5NDEyMDM= | 29 | Prevent accidental add-foreign-key with invalid column | 9599 | closed | 0 | 0 | 2019-06-20T23:57:24Z | 2019-06-20T23:58:26Z | 2019-06-20T23:58:26Z | OWNER | You can corrupt your database by running: $ sqlite-utils add-foreign-key my.db table non_existent_column other_table other_column | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/29/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
309033998 | MDU6SXNzdWUzMDkwMzM5OTg= | 187 | Windows installation error | 11855322 | closed | 0 | 7 | 2018-03-27T16:04:37Z | 2019-06-15T21:44:23Z | 2019-06-15T21:44:23Z | NONE | On attempting install on a Win 7 PC with py 3.6.2 (Anaconda dist) I get the error: ``` Collecting uvloop>=0.5.3 (from Sanic==0.7.0->datasette) Downloading uvloop-0.9.1.tar.gz (1.8MB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 1.8MB 12.8MB/s Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\RCole\AppData\Local\Temp\pip-build-juakfqt8\uvloop\setup.py ", line 10, in <module> raise RuntimeError('uvloop does not support Windows at the moment') RuntimeError: uvloop does not support Windows at the moment ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/187/reactions", "total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
449818897 | MDU6SXNzdWU0NDk4MTg4OTc= | 24 | Additional Column Constraints? | 98555 | closed | 0 | 6 | 2019-05-29T13:47:03Z | 2019-06-13T06:47:17Z | 2019-06-13T06:30:26Z | NONE | I'm looking to import data from XML with a pre-defined schema that maps fairly closely to a relational database. In particular, it has explicit annotations for when fields are required, optional, or when a default value should be inferred. Would there be value in adding the ability to define `NOT NULL` and `DEFAULT` column constraints to sqlite-utils? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/24/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
449848803 | MDU6SXNzdWU0NDk4NDg4MDM= | 25 | Allow .insert(..., foreign_keys=()) to auto-detect table and primary key | 9599 | closed | 0 | 4 | 2019-05-29T14:39:22Z | 2019-06-13T05:32:32Z | 2019-06-13T05:32:32Z | OWNER | The `foreign_keys=` argument currently takes a list of triples: ```python db["usages"].insert_all( usages_to_insert, foreign_keys=( ("line_id", "lines", "id"), ("definition_id", "definitions", "id"), ), ) ``` As of #16 we have a mechanism for detecting the primary key column (the third item in this triple) - we should use that here too, so foreign keys can be optionally defined as a list of pairs. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
453846217 | MDU6SXNzdWU0NTM4NDYyMTc= | 506 | Option to display binary data | 9599 | closed | 0 | 10 | 2019-06-08T23:44:12Z | 2019-06-11T15:48:27Z | 2019-06-09T16:07:39Z | OWNER | In #442 we suppressed rendering of binary data: <img width="358" alt="many-photos-tables__RKAlbumVersion_albumId_RidIndex__36_rows" src="https://user-images.githubusercontent.com/9599/59153367-865d1a80-8a0c-11e9-9384-27dcba0b61b6.png"> It turns out there is one use-case where displaying binary data is useful: when you're poking around looking at random SQLite databases you find in `~/Library` trying to figure out what they are for. So, a mechanism for opting in to ugly display of binary data again would be useful. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/506/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
451585764 | MDU6SXNzdWU0NTE1ODU3NjQ= | 499 | Accessibility for non-techie newsies? | 7936571 | open | 0 | 3 | 2019-06-03T16:49:37Z | 2019-06-05T21:22:55Z | NONE | Hi again, I'm having fun uploading datasets to Heroku via datasette. I'd like to set up datasette so that it's easy for other newsroom workers, who don't use Linux and aren't programmers, to upload datasets. Does datsette provide this out-of-the-box, or as a plugin? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/499/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
450862577 | MDU6SXNzdWU0NTA4NjI1Nzc= | 496 | Additional options to gcloud build command in cloudrun - timeout | 1740337 | closed | 0 | 1 | 2019-05-31T15:43:55Z | 2019-05-31T23:05:05Z | 2019-05-31T23:05:05Z | NONE | I am trying to deploy a 3.1 GB dataset to cloudrun with datasette. Currrently the docker build times out. Would be nice to have a timeout flag or additional gcloud commands that could be specified. Here is the line https://github.com/simonw/datasette/blob/f825e2012109247fa246e2b938f8174069e574f1/datasette/publish/cloudrun.py#L78 I would be happy to submit a PR to allow for a timeout option. What are your ideas of allowing the user additional build publishing flag options? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/496/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
322477187 | MDU6SXNzdWUzMjI0NzcxODc= | 255 | Facets | 9599 | closed | 0 | 16 | 2018-05-12T03:00:07Z | 2019-05-29T21:39:12Z | 2018-05-16T15:32:12Z | OWNER | Ability to display facets and facet counts on the table view. Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `"facets": [...]` property in `metadata.json` - [x] Implement `?_facet=` - [x] Implement `metadata.json` `facets` key - [x] Design for how facets should be presented - [x] Facets should be able to toggle off as well as on - [x] Expand labels for facets that are foreign keys - [x] Suggest potential facets (if we can do so within a tight time limit) - [x] Documentation | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/255/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
334148669 | MDU6SXNzdWUzMzQxNDg2Njk= | 318 | Facets with value of 0 displayed incorrectly | 9599 | closed | 0 | 3439337 | 1 | 2018-06-20T16:06:46Z | 2019-05-29T21:39:12Z | 2018-06-21T04:30:45Z | OWNER | https://registry.datasette.io/registry-7d4f81f/tables?_facet=is_hidden#facet-is_hidden ![2018-06-20 at 9 05 am](https://user-images.githubusercontent.com/9599/41670448-2c06e642-7469-11e8-86be-4664269582b1.png) Displays correctly if you select it: https://registry.datasette.io/registry-7d4f81f/tables?_facet=is_hidden&is_hidden=0 ![2018-06-20 at 9 06 am](https://user-images.githubusercontent.com/9599/41670471-3e61e486-7469-11e8-8710-5da90ef65787.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/318/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
346028655 | MDU6SXNzdWUzNDYwMjg2NTU= | 356 | Ability to display facet counts for many-to-many relationships | 9599 | closed | 0 | 4 | 2018-07-31T04:14:26Z | 2019-05-29T21:39:12Z | 2019-05-25T16:30:09Z | OWNER | Parent: #354 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
349827640 | MDU6SXNzdWUzNDk4Mjc2NDA= | 359 | Faceted browse against a JSON list of tags | 9599 | closed | 0 | 6 | 2018-08-12T17:01:14Z | 2019-05-29T21:39:12Z | 2019-05-03T00:21:44Z | OWNER | If a table has a `["foo", "bar", "baz"]` JSON column allow that to be faceted against. - [x] Support `?column__arraycontains=x` filter queries - [x] Support `?_facet_array=column` faceting | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
431800286 | MDU6SXNzdWU0MzE4MDAyODY= | 427 | New design for facet abstraction, including querystring and metadata.json | 9599 | closed | 0 | 10 | 2019-04-11T02:24:15Z | 2019-05-29T21:39:12Z | 2019-05-03T00:11:29Z | OWNER | I need a better design for query strings for facets (and for how facets are enabled in `metadata.json`). Think of all of the potential kinds of facets: * `?_facet_array=tags` where tags is a JSON array of values * `_facet_date=datetimecol` - faceted by date part of a datetime * `_facet_bins=numeric_column` - can I do some kind of fancy binning here? Might need to take an argument * `?_facet_bins=numeric_column:5` - could be a way to take an argument. We’ll ignore columns with a : in their name. * `?_facet_json=jsoncol:jsonpath` - could use a JSON path to extract out something to facet on? * `?_facet_percentile=numericcolumn` - could this work? * `?_facet_function=column:sqlfunctionname` - maybe this could be interesting? Would allow for e.g. facet by soundex * `?_facet_prefix=column:prefix` - facet by terms but only if they start with a specific prefix * `?_facet_substring=column:3,6` - facet by a substr(column, 3, 6) Maybe bundling JSON in querystrings is a way to do options? `?_facet_distance={"latitude_column":"x",...}` Could detect values starting with `{` - and if for some weird reason you have a column starting with that character you can pass this instead: `?_facet_percentile={"column": "{value}"}` This could even be the mechanism that allows us to extend regular facets to support additional options like adding a sum or max to each one. Problem: it’s not obvious what the name associated with these facets should be. What if one column is faceted multiple times using multiple facet variants? Maybe just number them? name1=… name2=… etc? Other option is to use Solr style querystring syntax for notation. Solr does this: `?f.price.facet.range.gap=100&f.age.facet.range.gap=10` So how about this: `?_facet_range=age&_facet_range.span=5` Related: #359 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/427/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
445862501 | MDU6SXNzdWU0NDU4NjI1MDE= | 477 | Documentation for ArrayFacet (facet by JSON array) | 9599 | closed | 0 | 4305096 | 0 | 2019-05-19T20:47:27Z | 2019-05-29T21:39:12Z | 2019-05-19T21:19:43Z | OWNER | This is missing from https://datasette.readthedocs.io/en/0.27.1/facets.html right now | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/477/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
446429421 | MDU6SXNzdWU0NDY0Mjk0MjE= | 481 | Facet by date | 9599 | closed | 0 | 1 | 2019-05-21T05:55:54Z | 2019-05-29T21:39:12Z | 2019-05-21T06:09:49Z | OWNER | Ability to facet on datetime fields by their date. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/481/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
447408527 | MDU6SXNzdWU0NDc0MDg1Mjc= | 483 | Option to facet by date using month or year | 9599 | open | 0 | 5 | 2019-05-23T01:25:29Z | 2019-05-29T21:38:27Z | OWNER | Facet by date (from #481) can take datetimes and facet them by the day component. https://latest.datasette.io/fixtures/facetable?_facet_date=created I'd like to also be able to facet by month or year. I'm not sure what the best way to achieve this is. Could be two more Facet classes (YearFacet and MonthFacet) but I think it might be nicer if the existing DateFacet could take an optional argument that changed its behaviour. But... if I do that, do I expose it in the UI somewhere or is it only available to URL-hackers? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/483/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
413867537 | MDU6SXNzdWU0MTM4Njc1Mzc= | 16 | add_column() should support REFERENCES {other_table}({other_column}) | 9599 | closed | 0 | 4 | 2019-02-24T21:00:45Z | 2019-05-29T05:17:59Z | 2019-05-29T04:56:18Z | OWNER | Related to #2 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
448391492 | MDU6SXNzdWU0NDgzOTE0OTI= | 21 | Option to ignore inserts if primary key exists already | 9599 | closed | 0 | 3 | 2019-05-25T00:17:12Z | 2019-05-29T05:09:01Z | 2019-05-29T04:18:26Z | OWNER | > I've just noticed that SQLite lets you IGNORE inserts that collide with a pre-existing key. This can be quite handy if you have a dataset that keeps changing in part, and you don't want to upsert and replace pre-existing PK rows but you do want to ignore collisions to existing PK rows. > > Do `sqlite_utils` support such (cavalier!) behaviour? _Originally posted by @psychemedia in https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924_ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/21/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
449445715 | MDU6SXNzdWU0NDk0NDU3MTU= | 491 | Figure out how to use Firebase with cloudrun to enable vanity URLs and CDN caching | 9599 | open | 0 | 0 | 2019-05-28T19:48:06Z | 2019-05-28T19:48:35Z | OWNER | It looks like Firebase can solve a couple of problems with the existing `datasette publish cloudrun` hosting mechanism: * The URLs it produces aren't pretty enough. Firebase offers more control over vanity URLs. * CDN caching (as seen in `datasette publish now`) is great for improving performance and saving money on Cloud Run execution time. https://firebase.google.com/docs/hosting/cloud-run looks like it can help with both of these. Lots of interesting questions: * Should this be a new `datasette publish firebase` command or should it instead be implemented as additional custom options to `datasette publish cloudrun`? * How much harder does it become to do account setup? * How much will this option cost users? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/491/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
448977444 | MDU6SXNzdWU0NDg5Nzc0NDQ= | 489 | Pagination breaks when combined with expanded foreign keys | 9599 | closed | 0 | 1 | 2019-05-27T19:56:56Z | 2019-05-28T02:48:57Z | 2019-05-28T02:23:27Z | OWNER | Consider https://edb3662.datasette.io/fixtures/roadside_attraction_characteristics?_sort=attraction_id&_size=2 The "Next page" link goes here, which returns 0 rows: https://edb3662.datasette.io/fixtures/roadside_attraction_characteristics?_size=2&_next=%257B%2527value%2527%253A%2B2%252C%2B%2527label%2527%253A%2B%2527Winchester%2BMystery%2BHouse%2527%257D%2C2&_sort=attraction_id That's because if you double-url-decode that `_next` link you get this: `_next={'value': 2, 'label': 'Winchester Mystery House'},2` It should be `_next=2,2` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/489/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
448978907 | MDU6SXNzdWU0NDg5Nzg5MDc= | 490 | Rename InterruptedError exception class | 9599 | closed | 0 | 1 | 2019-05-27T20:04:25Z | 2019-05-28T00:16:45Z | 2019-05-28T00:16:45Z | OWNER | https://github.com/simonw/datasette/blob/edb36629e7356f70f42b9d37fea5dfe9cc3c364a/datasette/utils.py#L49-L50 Python has a built-in exception called this, so we should call ours something else. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/490/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
267769431 | MDU6SXNzdWUyNjc3Njk0MzE= | 22 | Refactor to use class based views | 9599 | closed | 0 | 2857392 | 0 | 2017-10-23T18:26:22Z | 2019-05-27T20:05:56Z | 2017-10-24T02:25:53Z | OWNER | http://sanic.readthedocs.io/en/latest/sanic/class_based_views.html | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
448664792 | MDU6SXNzdWU0NDg2NjQ3OTI= | 487 | Refactor database methods off Datasette class | 9599 | closed | 0 | 1 | 2019-05-27T04:52:41Z | 2019-05-27T20:05:34Z | 2019-05-27T05:08:01Z | OWNER | Methods like this one: https://github.com/simonw/datasette/blob/182a3017c24e3fa3af60e4ac0c91c7e48f8736fd/datasette/app.py#L497-L503 Should live on the `ConnectedDatabase` class instead. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/487/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
448668204 | MDU6SXNzdWU0NDg2NjgyMDQ= | 488 | Move detect_primary_keys to Database class method | 9599 | closed | 0 | 0 | 2019-05-27T05:11:51Z | 2019-05-27T20:05:34Z | 2019-05-27T18:29:02Z | OWNER | e.g. https://github.com/simonw/datasette/blob/026c84db30bd0a75ecde146a80a5d142078dc299/datasette/views/table.py#L73-L75 Should be ``` pks = await db.primary_keys(table) ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/488/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
448395665 | MDU6SXNzdWU0NDgzOTU2NjU= | 22 | Release notes for 1.0 | 9599 | closed | 0 | 4348046 | 2 | 2019-05-25T00:58:03Z | 2019-05-25T01:18:27Z | 2019-05-25T01:06:52Z | OWNER | https://github.com/simonw/sqlite-utils/compare/0.14...251e473 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
432727685 | MDU6SXNzdWU0MzI3Mjc2ODU= | 20 | JSON column values get extraneously quoted | 649467 | closed | 0 | 4348046 | 1 | 2019-04-12T20:15:30Z | 2019-05-25T00:57:19Z | 2019-05-25T00:57:19Z | NONE | If the input to `sqlite-utils insert` includes a column that is a JSON array or object, `sqlite-utils query` will introduce an extra level of quoting on output: ``` # echo '[{"key": ["one", "two", "three"]}]' | sqlite-utils insert t.db t - # sqlite-utils t.db 'select * from t' [{"key": "[\"one\", \"two\", \"three\"]"}] # sqlite3 t.db 'select * from t' ["one", "two", "three"] ``` This might require an imperfect solution, since sqlite3 doesn't have a JSON type. Perhaps fields that start with `["` or `{"` and end with `"]` or `"}` could be detected, with a flag to turn off that behavior for weird text fields (or vice versa). | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
432217625 | MDU6SXNzdWU0MzIyMTc2MjU= | 19 | Incorrect help text for enable-fts command | 9599 | closed | 0 | 4348046 | 0 | 2019-04-11T19:46:44Z | 2019-05-25T00:44:31Z | 2019-05-25T00:44:31Z | OWNER | I clearly copied-and-pasted this from the `tables` command without updating it: https://github.com/simonw/sqlite-utils/blob/0b1af42ead3b3902347951180b3364ce1942da6e/sqlite_utils/cli.py#L216-L222 | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
413871266 | MDU6SXNzdWU0MTM4NzEyNjY= | 18 | .insert/.upsert/.insert_all/.upsert_all should add missing columns | 9599 | closed | 0 | 4348046 | 2 | 2019-02-24T21:36:11Z | 2019-05-25T00:42:11Z | 2019-05-25T00:42:11Z | OWNER | This is a larger change, but it would be incredibly useful: if you attempt to insert or update a document with a field that does not currently exist in the underlying table, sqlite-utils should add the appropriate column for you. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
448189298 | MDU6SXNzdWU0NDgxODkyOTg= | 486 | Ability to add extra routes and related templates | 2181410 | closed | 0 | 2 | 2019-05-24T14:04:25Z | 2019-05-24T14:43:28Z | 2019-05-24T14:43:09Z | NONE | Hi Simon Thank for an excellent job! Datasette is such an obviously good idea (once you have that idea!) and so well done. The only thing that I miss, is the ability to add extras routes (with associated jinja2-templates). For most of the datasets, that I would like to publish, I would also like at least a page, that describes the data (semantics, provenance, biases...) and a page explaining our cookie- and privacy-policies (which would allows us to use something like Goggle Analytics). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/486/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
447451492 | MDU6SXNzdWU0NDc0NTE0OTI= | 484 | Mechanism for displaying summary of m2m relationships in rows on table view | 9599 | open | 0 | 1 | 2019-05-23T05:02:41Z | 2019-05-23T06:34:05Z | OWNER | Part of #354 (m2m support) It would be fantastic if rows that are part of a m2m relationship could display it in an additional column in the table view. It might look something like this: https://russian-ira-facebook-ads.datasettes.com/russian-ads-919cbfd/display_ads?_search=black+lives+matter <img width="1475" alt="russian-ads__display_ads__50_rows_where_where_search_matches__black_lives_matter_" src="https://user-images.githubusercontent.com/9599/58226788-12690580-7cdd-11e9-9925-73153b27a413.png"> That example [was achieved](https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/daf51a8c50a78e8bc7971c211005fd85e66ccf64/russian-ads-metadata.yaml#L72-L77) using a custom SQL query and [datasette-json-html](https://github.com/simonw/datasette-json-html) - but I'd like this to be a built-in feature instead. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/484/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
346027040 | MDU6SXNzdWUzNDYwMjcwNDA= | 355 | Table view should support filtering via many-to-many relationships | 9599 | open | 0 | 10 | 2018-07-31T04:04:16Z | 2019-05-23T06:04:03Z | OWNER | Parent: #354 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
322787470 | MDU6SXNzdWUzMjI3ODc0NzA= | 259 | inspect() should detect many-to-many relationships | 9599 | closed | 0 | 6 | 2018-05-14T12:03:58Z | 2019-05-23T03:55:37Z | 2019-05-23T03:55:37Z | OWNER | Relates to #255 - in particular supporting facets across M2M relationships. It should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables. When rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side. Since SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
292011379 | MDU6SXNzdWUyOTIwMTEzNzk= | 184 | 500 from missing table name | 222245 | closed | 0 | 4 | 2018-01-26T19:46:45Z | 2019-05-21T16:17:29Z | 2018-04-13T18:18:59Z | NONE | https://github.com/simonw/datasette/blob/56623e48da5412b25fb39cc26b9c743b684dd968/datasette/app.py#L517-L519 throws an error if it gets an empty list back. Simplest solution is to write a helper func that just says ```python result = list(await self.execute(name, sql, params) if result: return result[0][0] ``` and use it anywhere `[0][0]` is now. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/184/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
444749373 | MDU6SXNzdWU0NDQ3NDkzNzM= | 469 | publish commands should use new -i option | 9599 | closed | 0 | 1 | 2019-05-16T04:31:40Z | 2019-05-19T22:53:41Z | 2019-05-19T22:53:41Z | OWNER | I can make this change only after releasing 0.28 - if I make the change earlier than that `publish heroku` etc will break because they will install the latest release of Datasette which will not understand the `-i` option. This is a one-line fix: replace this: https://github.com/simonw/datasette/blob/2ad9d15cd6901654e6801e2faa29e6fc08bae5fa/datasette/utils.py#L489 With this: (need to do it for other publishers too though) ``` quoted_files = " ".join( ["-i {}".format(shlex.quote(file_name)) for file_name in file_names] ) ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/469/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
440313209 | MDU6SXNzdWU0NDAzMTMyMDk= | 451 | Update README | 9599 | closed | 0 | 4305096 | 1 | 2019-05-04T11:26:07Z | 2019-05-19T22:23:43Z | 2019-05-19T22:23:43Z | OWNER | The README is quite out of date now. It includes out-dated copies of help files, promotes the old Zeit Now integration and duplicates a lot of material from the docs. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/451/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
443034003 | MDU6SXNzdWU0NDMwMzQwMDM= | 463 | Write release notes for 0.28 | 9599 | closed | 0 | 4305096 | 1 | 2019-05-11T20:36:56Z | 2019-05-19T21:24:44Z | 2019-05-19T21:24:20Z | OWNER | So much new stuff! https://github.com/simonw/datasette/compare/0.27...master | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/463/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
445855789 | MDU6SXNzdWU0NDU4NTU3ODk= | 474 | Do not allow downloads of mutable databases | 9599 | closed | 0 | 4305096 | 1 | 2019-05-19T19:35:32Z | 2019-05-19T20:41:17Z | 2019-05-19T20:41:16Z | OWNER | If the file changes during download it will probably result in a corrupt download. Safer not to allow downloads at all of mutable databases. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/474/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
445855910 | MDU6SXNzdWU0NDU4NTU5MTA= | 475 | Documentation for about and about_url metadata | 9599 | closed | 0 | 4305096 | 0 | 2019-05-19T19:36:59Z | 2019-05-19T20:13:36Z | 2019-05-19T20:13:36Z | OWNER | Added in https://github.com/simonw/datasette/commit/bf6b0f918de4aeee7c1036ac975ce2fb23237da7 without docs. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/475/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
445858491 | MDU6SXNzdWU0NDU4NTg0OTE= | 476 | Remove "datasette skeleton" | 9599 | closed | 0 | 4305096 | 0 | 2019-05-19T20:04:11Z | 2019-05-19T20:06:06Z | 2019-05-19T20:06:06Z | OWNER | It doesn't work any more, and it's not a particularly useful feature - I've hardly used it since I added it. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/476/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
421971339 | MDU6SXNzdWU0MjE5NzEzMzk= | 420 | Fix all the places that currently use .inspect() data | 9599 | closed | 0 | 4305096 | 13 | 2019-03-17T20:54:37Z | 2019-05-19T19:58:31Z | 2019-05-02T01:13:46Z | OWNER | See #419: if Datasette is going to work against mutable SQLite files it can no longer assume that the `.inspect()` method will have cached the correct schema for all tables in all attached databases. So everywhere in the code at the moment that relies on `.inspect()` data needs to be modified to use live introspection of the schema instead. From [a comment later on](https://github.com/simonw/datasette/issues/420#issuecomment-474398127): here are the uses I need to fix as a checklist: - [x] `table_exists()` - [x] `info["file"]` in `.execute()` - [x] `resolve_db_name()` - [x] `.database_url(database)` - [x] `DatabaseDownload` file path - [x] `sortable_columns_for_table()` uses it to find the columns in a table - [x] `expandable_columns()` uses it to find foreign keys - [x] `expand_foreign_keys()` uses it to find foreign keys - [x] `display_columns_and_rows()` uses it to find primary keys and foreign keys... but also has access to a cursor.description which it uses to list the columns - [x] `TableView.data` uses it to lookup columns and primary keys and the table_rows_count (used if the thing isn't a view) and probably a few more things, this method is huge! - [x] `RowView.data` uses it for primary keys - [x] `foreign_key_tables()` uses it for foreign keys - [x] `DatabaseView` list of tables - [x] `IndexView` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/420/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
444997937 | MDU6SXNzdWU0NDQ5OTc5Mzc= | 470 | /-/databases showing currently attached database details | 9599 | closed | 0 | 4305096 | 1 | 2019-05-16T14:45:18Z | 2019-05-19T19:28:44Z | 2019-05-16T14:50:26Z | OWNER | Split from #419. Mainly useful to see what is connected as mutable v.s. immutable. Also helps fill the gap left by `/-/inspect` until #465 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
445230077 | MDU6SXNzdWU0NDUyMzAwNzc= | 472 | Rename "publish now" to "publish nowv1" | 9599 | closed | 0 | 4305096 | 1 | 2019-05-17T01:58:52Z | 2019-05-19T18:07:39Z | 2019-05-19T18:07:39Z | OWNER | This will help clarify that you need a nowv1 account use it. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/472/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
421985685 | MDU6SXNzdWU0MjE5ODU2ODU= | 421 | Documentation for ?_hash=1 and Datasette's hashed URL caching | 9599 | closed | 0 | 4305096 | 2 | 2019-03-17T23:08:36Z | 2019-05-19T05:32:37Z | 2019-05-19T05:31:27Z | OWNER | Follow on from #418 - the Datasette documentation needs an entire section (probably a new page) describing exactly how the hash-in-URL caching mechanism works. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/421/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
421551434 | MDU6SXNzdWU0MjE1NTE0MzQ= | 419 | Default to opening files in mutable mode, special option for immutable files | 9599 | closed | 0 | 4305096 | 10 | 2019-03-15T14:39:27Z | 2019-05-16T15:14:32Z | 2019-05-16T15:14:31Z | OWNER | One of the original ideas behind Datasette was that serving immutable data makes everything way easier. Two examples: You don't have to worry about SQLite concurrency and you can bundle the database inside a Docker container and deploy it to immutable hosting. See [The interesting ideas in Datasette](https://simonwillison.net/2018/Oct/4/datasette-ideas/) for more on this. I'm beginning to see a much stronger case for being able to serve mutable data as well. SQLite is actually perfectly capable of handling reads against a database that is also being written to, even if the writes are coming from another process. https://www.sqlite.org/wal.htm There are all kinds of interesting use-cases which Datasette is currently unsuitable for due to its insistence on immutable databases. Some examples: * Continually run Datasette against a SQLite database updated by another process, e.g. Firefox bookmarks * Projects where a cron runs every X minutes and writes new entries gathered from other sources to SQLite * Tail a log file, write those log updates to a SQLite file, view recent log entries in Datasette This is also relevant to #417, Datasette Library. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/419/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |