issues
3,044 rows sorted by milestone
This data as json, CSV (advanced)
repo 16
- datasette 1,898
- sqlite-utils 604
- github-to-sqlite 79
- twitter-to-sqlite 73
- dogsheep-photos 40
- dogsheep-beta 38
- healthkit-to-sqlite 24
- evernote-to-sqlite 16
- swarm-to-sqlite 15
- apple-notes-to-sqlite 14
- google-takeout-to-sqlite 13
- pocket-to-sqlite 12
- dogsheep.github.io 8
- hacker-news-to-sqlite 6
- inaturalist-to-sqlite 2
- genome-to-sqlite 2
id | node_id | number | title | user | state | locked | assignee | milestone ▼ | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
326783670 | MDU6SXNzdWUzMjY3ODM2NzA= | 291 | Avoid plugins accidentally loading dependencies twice | simonw 9599 | closed | 0 | 3 | 2018-05-27T03:15:21Z | 2020-09-30T20:36:12Z | 2018-05-28T20:42:02Z | OWNER | Plugins that include JavaScript files risk loading the same code twice. In particular: I want to build a second plugin that uses the Leaflet mapping library (the first was datasette-cluster-map). But I don't want the two plugins to load duplicate copies of Leaflet. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/291/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326800219 | MDU6SXNzdWUzMjY4MDAyMTk= | 292 | Mechanism for customizing the SQL used to select specific columns in the table view | simonw 9599 | closed | 0 | 15 | 2018-05-27T09:05:52Z | 2021-05-27T04:25:01Z | 2021-05-27T04:25:01Z | OWNER | Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow. We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice. I think there are two features here:
Both features should be available via both querystring arguments and in The querystring argument for custom SQL should only work if Refs #276 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/292/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326987229 | MDExOlB1bGxSZXF1ZXN0MTkwOTAxNDI5 | 293 | Support for external database connectors | jsancho-gpl 11912854 | closed | 0 | 1 | 2018-05-28T11:02:45Z | 2018-09-11T14:32:45Z | 2018-09-11T14:32:45Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/293 | I think it would be nice that Datasette could work with other file formats that aren't SQLite, like files with PyTables format. I've tried to accomplish that using external connectors published with entry points. These external connectors must have a structure similar to the structure PyTables Datasette connector has. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/293/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
327365110 | MDU6SXNzdWUzMjczNjUxMTA= | 294 | inspect should record column types | simonw 9599 | open | 0 | 7 | 2018-05-29T15:10:41Z | 2019-06-28T16:45:28Z | OWNER | For each table we want to know the columns, their order and what type they are. I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a Possible JSON design:
Refs #276 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
327383759 | MDU6SXNzdWUzMjczODM3NTk= | 295 | Extract unit tests for inspect out to test_inspect.py | simonw 9599 | closed | 0 | 2 | 2018-05-29T15:55:04Z | 2019-05-11T21:40:32Z | 2019-05-11T21:40:32Z | OWNER | Right now they are bundled up as API unit tests for a relatively unimportant endpoint. They should be their own thing. Blocks #294 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/295/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327395270 | MDU6SXNzdWUzMjczOTUyNzA= | 296 | Per-database and per-table /-/ URL namespace | simonw 9599 | open | 0 | 3 | 2018-05-29T16:23:13Z | 2019-06-28T16:46:34Z | OWNER | Initially this will be for subsets of To start:
This means we will no longer allow databases or tables to have the name We will continue to support rows with a primary key of
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/296/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
327420945 | MDU6SXNzdWUzMjc0MjA5NDU= | 297 | datasette publish Dockerfile should use python:3.6-slim-stretch | simonw 9599 | closed | 0 | 1 | 2018-05-29T17:40:08Z | 2018-05-31T14:44:37Z | 2018-05-31T14:44:37Z | OWNER | Right now the Dockerfile generated by This appears to result in a SQLite version of
Meanwhile, https://fivethirtyeight.datasettes.com/-/versions is deployed with this Dockerfile https://github.com/simonw/fivethirtyeight-datasette/blob/0849901cae06e957fe04892cd4033bdcd1fcf966/Dockerfile which uses
So not only do we get a more recent SQLite (including https://www.sqlite.org/rowvalue.html added in 3.15) but we also get Refs #191 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/297/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327459829 | MDU6SXNzdWUzMjc0NTk4Mjk= | 298 | URLify URLs in results from custom SQL statements / views | simonw 9599 | closed | 0 | 2 | 2018-05-29T19:41:07Z | 2018-07-24T04:53:20Z | 2018-07-24T03:56:50Z | OWNER | Consider this custom query:
It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators This currently does not happen because the table view render logic takes a different path through |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/298/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327461381 | MDU6SXNzdWUzMjc0NjEzODE= | 299 | Documentation covering ALL datasette URLs | simonw 9599 | closed | 0 | 1 | 2018-05-29T19:46:15Z | 2018-07-28T04:24:05Z | 2018-07-28T04:22:30Z | OWNER | Relates to #296. We need a single page of the docs listing all of the URL patterns Datasette responds to, also detailing which templates are used to render them and linking to examples of the JSON they output when called with |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327541975 | MDU6SXNzdWUzMjc1NDE5NzU= | 300 | Hide sort select box on larger screens | simonw 9599 | closed | 0 | 0 | 2018-05-30T01:34:59Z | 2018-05-31T14:43:13Z | 2018-05-31T14:43:13Z | OWNER | I'm larger screens you can sort by clicking column headers, so no need to show the select box (which was added for the small screen layout that doesn't show headers) |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/300/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
328155946 | MDU6SXNzdWUzMjgxNTU5NDY= | 301 | --spatialite option for "datasette publish heroku" | simonw 9599 | open | 0 | 1 | 2018-05-31T14:13:09Z | 2022-01-20T21:28:50Z | OWNER | Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/301/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
328172521 | MDU6SXNzdWUzMjgxNzI1MjE= | 303 | Support table names ending with .json or .csv | simonw 9599 | closed | 0 | 4 | 2018-05-31T14:53:23Z | 2018-06-15T06:55:50Z | 2018-06-15T06:55:50Z | OWNER | This is needed for #266 - if a table name ends with We should be smarter about this. This does mean we will have some URLs that look like this:
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/303/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
328229224 | MDU6SXNzdWUzMjgyMjkyMjQ= | 304 | Ability to configure SQLite cache_size | simonw 9599 | closed | 0 | 3 | 2018-05-31T17:28:07Z | 2018-06-04T16:13:32Z | 2018-06-04T16:03:19Z | OWNER | See https://www.sqlite.org/pragma.html#pragma_cache_size Let's call the config setting Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
329147284 | MDU6SXNzdWUzMjkxNDcyODQ= | 305 | Add contributor guidelines to docs | simonw 9599 | closed | 0 | 2 | 2018-06-04T17:25:30Z | 2019-06-24T06:40:19Z | 2019-06-24T06:40:19Z | OWNER | https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/305/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
329661905 | MDU6SXNzdWUzMjk2NjE5MDU= | 306 | Custom URL routing with independent tests | simonw 9599 | closed | 0 | 5 | 2018-06-05T23:40:08Z | 2018-06-07T15:29:28Z | 2018-06-07T15:29:28Z | OWNER | The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching. If we go custom, tests should live in |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/306/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
330323860 | MDExOlB1bGxSZXF1ZXN0MTkzMzYxMzQx | 307 | Initial sketch of custom URL routing, refs #306 | simonw 9599 | closed | 0 | 1 | 2018-06-07T15:26:48Z | 2018-06-07T15:29:54Z | 2018-06-07T15:29:41Z | OWNER | simonw/datasette/pulls/307 | See #306 for background on this. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/307/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
330826972 | MDU6SXNzdWUzMzA4MjY5NzI= | 308 | Support extra Heroku apps:create options - region, space, team | annapowellsmith 78156 | open | 0 | 2 | 2018-06-08T23:08:33Z | 2018-09-21T14:09:28Z | NONE | It would be useful to document how to pass Heroku CLI options on |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/308/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
332830309 | MDU6SXNzdWUzMzI4MzAzMDk= | 310 | datasette publish now is broken in master | simonw 9599 | closed | 0 | 0 | 2018-06-15T16:01:14Z | 2018-06-16T16:29:50Z | 2018-06-16T16:29:50Z | OWNER | ```
Turns out the |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/310/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
332998752 | MDExOlB1bGxSZXF1ZXN0MTk1MzM5MTEx | 311 | ?_labels=1 to expand foreign keys (in csv and json), refs #233 | simonw 9599 | closed | 0 | 2 | 2018-06-16T16:31:12Z | 2018-06-16T22:20:31Z | 2018-06-16T22:20:31Z | OWNER | simonw/datasette/pulls/311 | Output looks something like this:
|
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/311/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
333000163 | MDU6SXNzdWUzMzMwMDAxNjM= | 312 | HTML, CSV and JSON views should support ?_col=&_col= | simonw 9599 | closed | 0 | 1 | 2018-06-16T16:53:35Z | 2021-06-17T18:14:24Z | 2018-06-16T17:00:12Z | OWNER | To support whitelisting columns to display. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
333086005 | MDU6SXNzdWUzMzMwODYwMDU= | 313 | Deploy demo of Datasette on every commit that passes tests | simonw 9599 | closed | 0 | 6 | 2018-06-17T19:19:12Z | 2018-06-17T21:52:58Z | 2018-06-17T21:52:58Z | OWNER | We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/313/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
333120982 | MDExOlB1bGxSZXF1ZXN0MTk1NDEzMjQx | 315 | Streaming mode for downloading all rows as a CSV | simonw 9599 | closed | 0 | 0 | 2018-06-18T03:06:59Z | 2018-06-18T03:29:13Z | 2018-06-18T03:21:02Z | OWNER | simonw/datasette/pulls/315 | Refs #266 |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
333238932 | MDU6SXNzdWUzMzMyMzg5MzI= | 316 | datasette inspect takes a very long time on large dbs | gavinband 132230 | closed | 0 | 5 | 2018-06-18T11:56:27Z | 2019-05-11T18:26:25Z | 2019-05-11T18:26:25Z | NONE | Hi, I want to expose data in a very large sqlite database (~600Gb) to the web. I have used datasette with success on smaller test databases with the same schema - it works very well (thanks!). However, using the full db, both (I noticed that the output of Any help on a workaround would be appreciated. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/316/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
334169932 | MDU6SXNzdWUzMzQxNjk5MzI= | 320 | Need unit tests covering the different states for the advanced export box | simonw 9599 | closed | 0 | 1 | 2018-06-20T17:03:40Z | 2018-07-24T04:53:20Z | 2018-07-24T03:38:40Z | OWNER | There are quite a few variants of this box: Test coverage should exercise all of them, since the logic is a little unclear. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/320/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
334592281 | MDExOlB1bGxSZXF1ZXN0MTk2NTI2ODYx | 322 | Feature/in operator | 4e1e0603 2691848 | closed | 0 | 0 | 2018-06-21T17:41:51Z | 2018-06-21T17:45:25Z | 2018-06-21T17:45:25Z | NONE | simonw/datasette/pulls/322 | datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/322/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
334698969 | MDU6SXNzdWUzMzQ2OTg5Njk= | 323 | Speed up Travis CI builds | simonw 9599 | closed | 0 | 1 | 2018-06-21T23:55:27Z | 2018-07-10T15:03:37Z | 2018-07-10T15:03:36Z | OWNER | They've got a bit slow. Part of this is the Zeit Now deploy, but the build-and-test cycle is taking at least a couple of minutes. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/323/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
334731076 | MDExOlB1bGxSZXF1ZXN0MTk2NjI4MzA0 | 324 | Speed up Travis by reusing pip wheel cache across builds | simonw 9599 | closed | 0 | 0 | 2018-06-22T03:20:08Z | 2018-06-24T01:03:47Z | 2018-06-24T01:03:47Z | OWNER | simonw/datasette/pulls/324 | From https://atchai.com/blog/faster-ci/ - refs #323 |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/324/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
335064777 | MDU6SXNzdWUzMzUwNjQ3Nzc= | 325 | Error on row page if table has slashes in the name and ends in .csv | simonw 9599 | closed | 0 | 1 | 2018-06-23T03:43:42Z | 2018-07-09T17:28:27Z | 2018-07-08T05:21:59Z | OWNER | https://v0-23-1.datasette.io/fixtures-e14e080/table%252Fwith%252Fslashes.csv/3
From clicking the row link on https://v0-23-1.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/325/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
335141434 | MDU6SXNzdWUzMzUxNDE0MzQ= | 326 | CSV should respect --cors and return cors headers | simonw 9599 | closed | 0 | 1 | 2018-06-24T00:44:07Z | 2021-06-17T18:14:24Z | 2018-06-24T00:59:45Z | OWNER | Otherwise tools like Vega can't load data via CSV. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/326/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
335200136 | MDU6SXNzdWUzMzUyMDAxMzY= | 327 | Explore if SquashFS can be used to shrink size of packaged Docker containers | simonw 9599 | open | 0 | 4 | 2018-06-24T18:15:16Z | 2022-02-17T23:37:24Z | OWNER | Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed https://en.wikipedia.org/wiki/SquashFS is "a compressed read-only file system for Linux" - which means it could be a really nice fit for Datasette and its read-only SQLite databases. It would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/327/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
336464733 | MDU6SXNzdWUzMzY0NjQ3MzM= | 328 | Installation instructions, including how to use the docker image | simonw 9599 | closed | 0 | 4 | 2018-06-28T03:59:33Z | 2023-09-05T14:10:39Z | 2018-06-28T04:02:10Z | OWNER | datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/328/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
336465018 | MDU6SXNzdWUzMzY0NjUwMTg= | 329 | Travis should push tagged images to Docker Hub for each release | simonw 9599 | closed | 0 | 7 | 2018-06-28T04:01:31Z | 2018-11-05T06:54:10Z | 2018-11-05T06:53:28Z | OWNER | datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/329/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
336924199 | MDU6SXNzdWUzMzY5MjQxOTk= | 330 | Limit text display in cells containing large amounts of text | psychemedia 82988 | closed | 0 | 4 | 2018-06-29T09:15:22Z | 2018-07-24T04:53:20Z | 2018-07-10T16:20:48Z | CONTRIBUTOR | The default preview of a database shows all columns (is the row count limited?) which is fine in many cases but can take a long time to load / offer a large overhead if the table is a SpatiaLite table containing geometry columns that include large shapefiles. Would it make sense to have a setting that can limit the amount of text displayed in any given cell in the table preview, or (less useful?) suppress (with notification) the display of overlong columns unless enabled by the user? An issue then arises if a user does want to see all the text in a cell: 1) for a particular cell; 2) for every cell in the table; 3) for all cells in a particular column or columns (I haven't checked but what if a column contains e.g. raw image data? Does this display as raw data? Or can this be rendered in a context aware way as an image preview? I guess a custom template would be one way to do that?) |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/330/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
336936010 | MDU6SXNzdWUzMzY5MzYwMTA= | 331 | Datasette throws error when loading spatialite db without extension loaded | psychemedia 82988 | closed | 0 | 2 | 2018-06-29T09:51:14Z | 2022-01-20T21:29:40Z | 2018-07-10T15:13:36Z | CONTRIBUTOR | When starting datasette on a SpatialLite database without loading the SpatiaLite extension (using eg
It would be nice to trap this and return a message saying something like: ``` It looks like you're trying to load a SpatiaLite database? Make sure you load in the SpatiaLite extension when starting datasette. Read more: https://datasette.readthedocs.io/en/latest/spatialite.html ``` |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/331/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
337141108 | MDU6SXNzdWUzMzcxNDExMDg= | 332 | Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1 | simonw 9599 | closed | 0 | 12 | 2018-06-29T21:21:27Z | 2018-07-24T04:53:20Z | 2018-07-24T03:08:30Z | OWNER | It turns out if you load this CSV using
I deployed this using:
Datasette outputs that as Demo * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug - HTML view works * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug.json?_shape=array - this outputs the following:
But... in Firefox that gets rendered like this: And if you click the "Show charting options" button you get this error in the console:
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
338768551 | MDU6SXNzdWUzMzg3Njg1NTE= | 333 | Datasette on Zeit Now returns http URLs for facet and next links | simonw 9599 | closed | 0 | 4 | 2018-07-06T00:40:49Z | 2018-07-24T04:53:20Z | 2018-07-24T01:51:53Z | OWNER | e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0
Note that suggested facets doesn't include the full URL at all, which is a consistency bug. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
339095976 | MDU6SXNzdWUzMzkwOTU5NzY= | 334 | extra_options not passed to heroku publisher | kamicut 719357 | closed | 0 | 2 | 2018-07-06T23:26:12Z | 2018-07-24T04:53:21Z | 2018-07-10T01:46:04Z | NONE | I might be wrong but I was not able to publish to Any help appreciated! |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/334/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
339505204 | MDU6SXNzdWUzMzk1MDUyMDQ= | 335 | Package datasette for installation using homebrew | simonw 9599 | closed | 0 | 12 | 2018-07-09T15:45:03Z | 2020-08-11T16:54:06Z | 2020-08-11T16:54:06Z | OWNER | https://docs.brew.sh/Python-for-Formula-Authors describes how.
It recommends using https://github.com/tdsmith/homebrew-pypi-poet |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340039409 | MDU6SXNzdWUzNDAwMzk0MDk= | 336 | Ensure --help examples in docs are always up to date | simonw 9599 | closed | 0 | 3 | 2018-07-10T23:20:01Z | 2018-07-24T16:01:29Z | 2018-07-24T16:01:29Z | OWNER | Ideally I would automatically generate the --help output shown in our docs, but I don't think I can get that working with readthedocs. Instead, I'm going to add a unit test that checks that those extracts in the documentation match the current output of the --help command. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/336/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340065374 | MDU6SXNzdWUzNDAwNjUzNzQ= | 337 | Documentation for datasette publish and datasette package | simonw 9599 | closed | 0 | 1 | 2018-07-11T02:04:06Z | 2018-07-11T02:07:32Z | 2018-07-11T02:05:56Z | OWNER | datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
340282796 | MDU6SXNzdWUzNDAyODI3OTY= | 338 | Only load vegaEmbed if charting tools are enabled | simonw 9599 | closed | 0 | 1 | 2018-07-11T15:02:14Z | 2018-07-11T15:21:47Z | 2018-07-11T15:21:47Z | OWNER | vegaEmbed is a LOT of code (it bundles d3) Inspired by this tweet: https://twitter.com/thelarkinn/status/1017053567641948162 - it would be great if we loaded that code on demand the first time the "Show chart options" button was clicked, or when the page loads with #g. options in the URL. Even better: avoid the overhead if loading React unless the chart options need to be displayed. This would be a pretty major refactoring though. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340396247 | MDU6SXNzdWUzNDAzOTYyNDc= | 339 | Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way | bsilverm 12617395 | closed | 0 | 4 | 2018-07-11T20:38:06Z | 2022-03-21T22:22:40Z | 2022-03-21T22:22:34Z | NONE | Is it possible to configure the sql_time_limit_ms beyond 60 seconds? It seems queries are still timing out at 60 seconds when sql_time_limit_ms is set to 180000. We have a very large data set and often encounter timeouts when testing new queries from the datasette UI. We are optimizing our database as much as we can, but still may require more than 60 seconds for complex queries. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/339/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340730961 | MDU6SXNzdWUzNDA3MzA5NjE= | 340 | Embrace black | simonw 9599 | closed | 0 | 1 | 2018-07-12T17:32:29Z | 2019-06-24T06:50:27Z | 2019-06-24T06:50:26Z | OWNER | Run black against everything. Then set up CI to fail if code doesn't conform to black's style. Here's how Starlette does this:
And here's an example of a test run that failed: https://travis-ci.org/encode/starlette/jobs/403172478 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/340/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340733753 | MDExOlB1bGxSZXF1ZXN0MjAxMDc1NTMy | 341 | Bump aiohttp to fix compatibility with Python 3.7 | simonw 9599 | closed | 0 | 0 | 2018-07-12T17:41:24Z | 2018-07-12T18:07:38Z | 2018-07-12T18:07:38Z | OWNER | simonw/datasette/pulls/341 | Tests failed here: https://travis-ci.org/simonw/datasette/jobs/403223333 |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/341/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
341123355 | MDU6SXNzdWUzNDExMjMzNTU= | 342 | Requesting support for query description | bsilverm 12617395 | closed | 0 | 4 | 2018-07-13T18:50:16Z | 2018-07-24T04:53:21Z | 2018-07-16T02:33:54Z | NONE | It would be great if the metadata file allowed you to enter a description for the query. We have a lot of pre-defined queries that can only be so descriptive by their name. It would be nice if an optional description could be included underneath the name within the UI, or on hover where it currently shows the SQL. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/342/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
341228846 | MDU6SXNzdWUzNDEyMjg4NDY= | 343 | Render boolean fields better by default | russss 45057 | open | 0 | 1 | 2018-07-14T11:10:29Z | 2018-07-14T14:17:14Z | CONTRIBUTOR | These show up as 0 or 1 because sqlite. I think Yes/No would be fine in most cases? |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/343/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
341229113 | MDU6SXNzdWUzNDEyMjkxMTM= | 344 | datasette publish heroku fails without name provided | russss 45057 | closed | 0 | 1 | 2018-07-14T11:15:56Z | 2018-07-14T13:00:48Z | 2018-07-14T13:00:48Z | CONTRIBUTOR | It fails with the following JSON traceback if the
```
Traceback (most recent call last):
File "/usr/local/bin/datasette", line 11, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/datasette/cli.py", line 265, in publish
app_name = json.loads(create_output)["name"]
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/344/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
341235633 | MDExOlB1bGxSZXF1ZXN0MjAxNDUxMzMy | 345 | Allow app names for `datasette publish heroku` | russss 45057 | closed | 0 | 1 | 2018-07-14T13:12:34Z | 2018-07-14T14:09:54Z | 2018-07-14T14:04:44Z | CONTRIBUTOR | simonw/datasette/pulls/345 | Lets you supply the |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/345/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
343728754 | MDU6SXNzdWUzNDM3Mjg3NTQ= | 346 | Logo design for DATASETTE | ggabogarcia 35750428 | closed | 0 | 0 | 2018-07-23T17:40:17Z | 2018-08-02T02:31:59Z | 2018-08-02T02:31:59Z | NONE | Hello :) , I'm a graphic designer, I'm interested in collaborating with open source projects, besides this helps me expand my portfolio. I would like to design a logo for your project. I will be happy to collaborate with you :). |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/346/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
344654623 | MDU6SXNzdWUzNDQ2NTQ2MjM= | 347 | Rename "datasette package" to "datasette publish docker" | simonw 9599 | open | 0 | 0 | 2018-07-26T00:42:46Z | 2018-07-26T00:42:46Z | OWNER | datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/347/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
344656114 | MDU6SXNzdWUzNDQ2NTYxMTQ= | 348 | Unit tests for "datasette publish" | simonw 9599 | closed | 0 | 1 | 2018-07-26T00:52:23Z | 2018-07-26T05:46:10Z | 2018-07-26T05:46:10Z | OWNER | The datasette publish family of commands all work by shelling out to heroku/now/docker
So in tests I should be able to mock those calls:
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/348/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
344695978 | MDExOlB1bGxSZXF1ZXN0MjA0MDI5MTQy | 349 | publish_subcommand hook + default plugins mechanism, used for publish heroku/now | simonw 9599 | closed | 0 | 1 | 2018-07-26T05:03:22Z | 2018-07-26T05:28:54Z | 2018-07-26T05:16:00Z | OWNER | simonw/datasette/pulls/349 | This change introduces a new plugin hook, publish_subcommand, which can be used to implement new subcommands for the "datasette publish" command family. I've used this new hook to refactor out the "publish now" and "publish heroku" implementations into separate modules. I've also added unit tests for these two publishers, mocking the subprocess.call and subprocess.check_output functions. As part of this, I introduced a mechanism for loading default plugins. These are defined in the new "default_plugins" list inside datasette/app.py Closes #217 (Plugin support for "datasette publish") Closes #348 (Unit tests for "datasette publish") Refs #14, #59, #102, #103, #146, #236, #347 |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
344701755 | MDU6SXNzdWUzNDQ3MDE3NTU= | 350 | Don't list default plugins on /-/plugins | simonw 9599 | closed | 0 | 2 | 2018-07-26T05:38:00Z | 2018-08-28T17:13:50Z | 2018-08-28T16:48:19Z | OWNER | https://dbbe707.datasette.io/-/plugins is showing "datasette.publish.now" and "datasette.publish.heroku" |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
345469355 | MDU6SXNzdWUzNDU0NjkzNTU= | 351 | Automatically create a GitHub release linking to release notes for every tagged release | simonw 9599 | closed | 0 | 1 | 2018-07-28T18:31:12Z | 2020-05-28T18:56:16Z | 2020-05-28T18:56:15Z | OWNER | Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
345821500 | MDU6SXNzdWUzNDU4MjE1MDA= | 352 | render_cell(value) plugin hook | simonw 9599 | closed | 0 | 4 | 2018-07-30T15:56:20Z | 2020-02-10T16:18:58Z | 2018-08-05T00:14:57Z | OWNER | To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/352/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
345821778 | MDExOlB1bGxSZXF1ZXN0MjA0ODUxNTEx | 353 | render_cell(value) plugin hook | simonw 9599 | closed | 0 | 0 | 2018-07-30T15:57:08Z | 2018-08-05T00:14:57Z | 2018-08-05T00:14:57Z | OWNER | simonw/datasette/pulls/353 | Closes #352. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
346026869 | MDU6SXNzdWUzNDYwMjY4Njk= | 354 | Handle many-to-many relationships | simonw 9599 | open | 0 | 0 | 2018-07-31T04:03:13Z | 2020-11-24T19:51:18Z | OWNER | This is a master tracking ticket for various many-2-many features. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
346027040 | MDU6SXNzdWUzNDYwMjcwNDA= | 355 | Table view should support filtering via many-to-many relationships | simonw 9599 | open | 0 | 10 | 2018-07-31T04:04:16Z | 2019-05-23T06:04:03Z | OWNER | Parent: #354 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
346028655 | MDU6SXNzdWUzNDYwMjg2NTU= | 356 | Ability to display facet counts for many-to-many relationships | simonw 9599 | closed | 0 | 4 | 2018-07-31T04:14:26Z | 2019-05-29T21:39:12Z | 2019-05-25T16:30:09Z | OWNER | Parent: #354 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
347058326 | MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2 | 1 | Make .indexes compatible with older SQLite versions | simonw 9599 | closed | 0 | 0 | 2018-08-02T15:17:05Z | 2018-08-02T15:17:30Z | 2018-08-02T15:17:30Z | OWNER | simonw/sqlite-utils/pulls/1 | Older SQLite versions return a different set of columns from the PRAGMA we are using. |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
348043884 | MDU6SXNzdWUzNDgwNDM4ODQ= | 357 | Plugin hook for loading metadata.json | simonw 9599 | open | 0 | 6 | 2018-08-06T19:00:01Z | 2020-06-21T22:19:58Z | OWNER | For https://github.com/simonw/russian-ira-facebook-ads-datasette/tree/af6d956995e14afd585c35a6a06bb01da32043ba I wrote a script to convert YAML to JSON because YAML is a better format for embedding multi-line HTML descriptions and canned SQL statements. Example yaml metadata file: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/af6d956995e14afd585c35a6a06bb01da32043ba/russian-ads-metadata.yaml It would be useful if Datasette could be fed a YAML file directly:
Question is... should this be a native feature (hence adding a YAML dependency) or should it be handled by a |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
348534997 | MDExOlB1bGxSZXF1ZXN0MjA2ODYzODAz | 358 | Bump versions of pytest, pluggy and beautifulsoup4 | simonw 9599 | closed | 0 | 0 | 2018-08-08T00:44:38Z | 2018-08-08T01:11:13Z | 2018-08-08T01:11:13Z | OWNER | simonw/datasette/pulls/358 | datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/358/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
349827640 | MDU6SXNzdWUzNDk4Mjc2NDA= | 359 | Faceted browse against a JSON list of tags | simonw 9599 | closed | 0 | 6 | 2018-08-12T17:01:14Z | 2019-05-29T21:39:12Z | 2019-05-03T00:21:44Z | OWNER | If a table has a
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
349850687 | MDU6SXNzdWUzNDk4NTA2ODc= | 2 | Mechanism for adding foreign keys to an existing table | simonw 9599 | closed | 0 | 1 | 2018-08-12T22:50:56Z | 2019-02-24T21:34:41Z | 2019-02-24T21:34:41Z | OWNER | SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running Here's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125 And here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page) |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
351017129 | MDU6SXNzdWUzNTEwMTcxMjk= | 360 | Use pysqlite3 if available | simonw 9599 | closed | 0 | 3 | 2018-08-16T00:50:45Z | 2018-08-16T01:50:42Z | 2018-08-16T00:58:58Z | OWNER | pysqlite3 is a way to provide access to a more recent version of SQLite than the standard library |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/360/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
351017365 | MDExOlB1bGxSZXF1ZXN0MjA4NzE5MDQz | 361 | Import pysqlite3 if available, closes #360 | simonw 9599 | closed | 0 | 0 | 2018-08-16T00:52:21Z | 2018-08-16T00:58:57Z | 2018-08-16T00:58:57Z | OWNER | simonw/datasette/pulls/361 | datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/361/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
351845423 | MDU6SXNzdWUzNTE4NDU0MjM= | 3 | Experiment with contentless FTS tables | simonw 9599 | closed | 0 | 1 | 2018-08-18T19:31:01Z | 2019-07-22T20:58:55Z | 2019-07-22T20:58:55Z | OWNER | Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/ |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
352768017 | MDU6SXNzdWUzNTI3NjgwMTc= | 362 | Add option to include/exclude columns in search filters | annapowellsmith 78156 | open | 0 | 1 | 2018-08-22T01:32:08Z | 2020-11-03T19:01:59Z | NONE | I have a dataset with many columns, of which only some are likely to be of interest for searching. It would be great for usability if the search filters in the UI could be configured to include/exclude columns. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/362/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
355299310 | MDExOlB1bGxSZXF1ZXN0MjExODYwNzA2 | 363 | Search all apps during heroku publish | kevboh 436032 | open | 0 | 1 | 2018-08-29T19:25:10Z | 2018-08-31T14:39:45Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/363 | Adds the |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/363/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
359075028 | MDExOlB1bGxSZXF1ZXN0MjE0NjUzNjQx | 364 | Support for other types of databases using external connectors | jsancho-gpl 11912854 | open | 0 | 0 | 2018-09-11T14:31:47Z | 2018-09-11T14:31:47Z | FIRST_TIME_CONTRIBUTOR | simonw/datasette/pulls/364 | This PR is related to #293, but now all commits have been merged. The purpose is to support other file formats that aren't SQLite, like files with PyTables format. I've tried to accomplish that using external connectors published with entry points. The modifications in the original datasette code are minimal and many are in a separated file. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/364/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
361764460 | MDExOlB1bGxSZXF1ZXN0MjE2NjUxMzE3 | 365 | fix small doc typo | jaywgraves 418191 | closed | 0 | 2 | 2018-09-19T14:02:02Z | 2019-12-19T02:30:33Z | 2018-09-19T17:15:43Z | CONTRIBUTOR | simonw/datasette/pulls/365 | datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/365/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
369716228 | MDU6SXNzdWUzNjk3MTYyMjg= | 366 | Default built image size over Zeit Now 100MiB limit | gfrmin 416374 | closed | 0 | 2 | 2018-10-12T21:27:17Z | 2018-11-05T06:23:32Z | 2018-11-05T06:23:32Z | CONTRIBUTOR | Using |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/366/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
374675798 | MDExOlB1bGxSZXF1ZXN0MjI2MzE0ODYy | 367 | Mark codemirror files as vendored | jaap3 48517 | closed | 0 | 2 | 2018-10-27T18:41:25Z | 2019-05-03T21:12:09Z | 2019-05-03T21:11:20Z | CONTRIBUTOR | simonw/datasette/pulls/367 | GitHub lists datasette as a Javascript project, primarily because of the vendored codemirror files. This is somewhat confusing when you're looking for datasette, knowing it's written in Python. Luckily it's possible exclude certain files from GitHub's code statistics: https://github.com/github/linguist#using-gitattributes |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/367/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
374676773 | MDExOlB1bGxSZXF1ZXN0MjI2MzE1NTEz | 368 | Update installation instructions | jaap3 48517 | closed | 0 | 0 | 2018-10-27T18:52:31Z | 2019-05-03T18:18:43Z | 2019-05-03T18:18:42Z | CONTRIBUTOR | simonw/datasette/pulls/368 | I was writing this as a response to your tweet, but decided I might just make it a pull request. I feel like it might be confusing to those unfamiliar with Python's By also using |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/368/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
377155320 | MDU6SXNzdWUzNzcxNTUzMjA= | 370 | Integration with JupyterLab | psychemedia 82988 | open | 0 | 4 | 2018-11-04T13:57:13Z | 2022-09-29T08:17:47Z | CONTRIBUTOR | I just watched a demo video for the JupyterLab Chart Editor which wraps the plotly chart editor app in a JupyterLab panel and lets you open a plotly chart JSON file in that editor. Essentially, it pops an HTML app into a panel in JupyterLab, and I think registers the app as a file viewer for a particular file type. (I'm not completely taken by it, tbh, because it means you can do irreproducible things to the chart definition file, but that's another issue). JupyterLab extensions can also open files from a dialogue as the iframe/html previewer shows: https://github.com/timkpaine/jupyterlab_iframe. This made me wonder about what For example, by right-clicking on a CSV file (for which there is already a CSV table view) in the file browser, offer a View / Run as datasette file viewer option that will:
(? Create a new SQLite db for each CSV file and launch each datasette view on a new port? Or have a JupyterLab (session?) SQLite db that stores all As a freebie, the Related: |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/370/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
377156339 | MDU6SXNzdWUzNzcxNTYzMzk= | 371 | datasette publish digitalocean plugin | psychemedia 82988 | closed | 0 | 3 | 2018-11-04T14:07:41Z | 2021-01-04T20:14:28Z | 2021-01-04T20:14:28Z | CONTRIBUTOR | Provide support for launching Example: Deploy Docker containers into Digital Ocean. Digital Ocean also has a preconfigured VM running Docker that can be launched from the command line via the Digital Ocean API: Docker One-Click Application. Related: - Launching containers in Digital Ocean servers running docker: How To Provision and Manage Remote Docker Hosts with Docker Machine on Ubuntu 16.04 - How To Use Doctl, the Official DigitalOcean Command-Line Client |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/371/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
377166793 | MDU6SXNzdWUzNzcxNjY3OTM= | 372 | Docker build tools | psychemedia 82988 | open | 0 | 0 | 2018-11-04T16:02:35Z | 2018-11-04T16:02:35Z | CONTRIBUTOR | In terms of small pieces lightly joined, I note that there are several tools starting to appear for building generating Dockerfiles and building Docker containers from simpler components such as If plugin/extensions builders want to include additional packages, then things like incremental builds of composable builds that add additional items into a base Examples of Dockerfile generators / container builders: Discussions / threads (via Binderhub gitter) on:
- why Relates to things like: |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/372/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0 } |
||||||||
377518499 | MDU6SXNzdWUzNzc1MTg0OTk= | 374 | Get Datasette working with Zeit Now v2's 100MB image size limit | simonw 9599 | closed | 0 | 5 | 2018-11-05T18:08:29Z | 2018-12-19T01:35:59Z | 2018-12-19T01:35:59Z | OWNER | Follow-on from #366 Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances. Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/374/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
382471625 | MDExOlB1bGxSZXF1ZXN0MjMyMTcyMTA2 | 389 | Bump dependency versions | simonw 9599 | closed | 0 | 2 | 2018-11-20T02:23:12Z | 2019-11-13T19:13:41Z | 2019-11-13T19:13:41Z | OWNER | simonw/datasette/pulls/389 | datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/389/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
386459810 | MDExOlB1bGxSZXF1ZXN0MjM1MTk0Mjg2 | 390 | tiny typo in customization docs | jaywgraves 418191 | closed | 0 | 1 | 2018-12-01T13:44:42Z | 2019-12-19T02:30:35Z | 2018-12-16T21:32:56Z | CONTRIBUTOR | simonw/datasette/pulls/390 | was looking to add some custom templates to my use of datasette and saw this small typo. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/390/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
392610803 | MDU6SXNzdWUzOTI2MTA4MDM= | 391 | Google Trends example doesn’t work | styfle 229881 | closed | 0 | 1 | 2018-12-19T13:51:38Z | 2019-01-02T19:45:13Z | 2019-01-02T19:45:12Z | NONE | https://google-trends.datasettes.com/ I see a cloud flare error. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/391/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
394751072 | MDExOlB1bGxSZXF1ZXN0MjQxNDE4NDQz | 392 | Fix some regex DeprecationWarnings | simonw 9599 | closed | 0 | 0 | 2018-12-29T02:10:28Z | 2018-12-29T02:22:28Z | 2018-12-29T02:22:28Z | OWNER | simonw/datasette/pulls/392 | datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/392/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
395236066 | MDU6SXNzdWUzOTUyMzYwNjY= | 393 | CSV export in "Advanced export" pane doesn't respect query | ltrgoddard 1727065 | closed | 0 | 6 | 2019-01-02T12:39:41Z | 2021-06-17T18:14:24Z | 2019-01-03T02:44:10Z | NONE | It looks like there's an inconsistency when exporting to CSV via the the web interface. Say I'm looking at songs released in 1989 in the It may be that this is intended behaviour related to the streaming CSV stuff discussed here, but if that's the case then I think it should be a little clearer. |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/393/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
396215043 | MDU6SXNzdWUzOTYyMTUwNDM= | 395 | Find a cleaner pattern for fixtures with arguments | simonw 9599 | closed | 0 | 1 | 2019-01-06T00:31:22Z | 2020-06-07T21:23:22Z | 2020-06-07T21:23:22Z | OWNER | A lot of Datasette tests look like this: The loop here isn't actually expected to loop - it's there because the This pattern works, but it is a little confusing. It would be nice to replace it with something less strange looking. The answer may be to switch to the "factories as fixtures" pattern described here: https://docs.pytest.org/en/latest/fixture.html#factories-as-fixtures In particular some variant of this example: ``` @pytest.fixture def make_customer_record():
def test_customer_records(make_customer_record): customer_1 = make_customer_record("Lisa") customer_2 = make_customer_record("Mike") customer_3 = make_customer_record("Meredith") ``` |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/395/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
397098882 | MDU6SXNzdWUzOTcwOTg4ODI= | 396 | Add pragma compile_options output to /-/versions | simonw 9599 | closed | 0 | 1 | 2019-01-08T21:43:54Z | 2019-01-11T00:55:22Z | 2019-01-11T00:44:56Z | OWNER |
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/396/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
397129564 | MDU6SXNzdWUzOTcxMjk1NjQ= | 397 | Update official datasetteproject/datasette Docker container to SQLite 3.26.0 | claes 43564 | closed | 0 | 3 | 2019-01-08T22:51:50Z | 2019-01-11T01:25:33Z | 2019-01-11T00:56:18Z | NONE | I try to start datasette on a database that contains the below view It fails in a way that makes me think it does not support the window functions SQL syntax.
``` |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/397/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
398089089 | MDU6SXNzdWUzOTgwODkwODk= | 399 | /-/versions for official Docker image returns wrong Datasette version | simonw 9599 | closed | 0 | 2 | 2019-01-11T01:19:58Z | 2019-01-13T23:31:59Z | 2019-01-13T23:10:45Z | OWNER |
|
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/399/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
398559195 | MDU6SXNzdWUzOTg1NTkxOTU= | 400 | datasette publish cloudrun plugin | rprimet 10352819 | closed | 0 | 1 | 2019-01-12T14:35:11Z | 2019-05-03T16:57:35Z | 2019-05-03T16:57:35Z | CONTRIBUTOR | Google announced that they may launch a simple service for running Docker containers (previously serverless containers, now called "cloud run" -- link to alpha here). If/when this happens, it might be a good fit for publishing datasettes? (at least using the current version, manually publishing a datasette seems relatively painless). |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/400/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
400229984 | MDU6SXNzdWU0MDAyMjk5ODQ= | 401 | How to pass configuration to plugins? | dazzag24 1055831 | closed | 0 | 3 | 2019-01-17T11:20:41Z | 2019-01-18T11:48:13Z | 2019-01-18T06:49:07Z | NONE | Hi, Firstly, thanks for your work on datasette, it is a hugely useful tool! I've been working on a fork [https://github.com/dazzag24/datasette-cluster-map] of datasette-cluster-map to allow the tileserver to be easily switched. Primarily because the tiles being served in the current version use localised text for labels and I'd like to have English used for these names instead. It uses http://leaflet-extras.github.io/leaflet-providers/preview/ to allow you to simply set the tile provider using a call like so:
Can you please point me to an example or how to pass configuration from the metadata.json down into a plugin. Once I've over come this issue I was wondering if you would be interested in taking this change into your version? Many thanks Darren |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/401/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
400340905 | MDU6SXNzdWU0MDAzNDA5MDU= | 402 | Use SQLITE_DBCONFIG_DEFENSIVE plus other recommendations from SQLite security docs | simonw 9599 | open | 0 | 3 | 2019-01-17T15:52:28Z | 2019-01-17T16:15:21Z | OWNER |
https://twitter.com/ignoredambience/status/1085926961413869568 |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/402/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
400511206 | MDU6SXNzdWU0MDA1MTEyMDY= | 403 | How does persistence work? | ccorcos 1794527 | closed | 0 | 2 | 2019-01-17T23:41:57Z | 2019-01-19T05:47:55Z | 2019-01-18T06:51:14Z | NONE | I was under the impression that now.sh is for stateless microservices. So where are these SQLite databases stored and when do they get created and destroyed? |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/403/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
403028630 | MDExOlB1bGxSZXF1ZXN0MjQ3NTc2OTQy | 4 | Fts5 | simonw 9599 | closed | 0 | 0 | 2019-01-25T06:54:05Z | 2019-01-25T06:54:33Z | 2019-01-25T06:54:33Z | OWNER | simonw/sqlite-utils/pulls/4 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
403396009 | MDExOlB1bGxSZXF1ZXN0MjQ3ODYxNDE5 | 5 | Run Travis tests against Python 3.8-dev | simonw 9599 | closed | 0 | 0 | 2019-01-26T02:30:55Z | 2019-01-26T02:37:54Z | 2019-01-26T02:37:54Z | OWNER | simonw/sqlite-utils/pulls/5 | sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
403499298 | MDExOlB1bGxSZXF1ZXN0MjQ3OTIzMzQ3 | 404 | Experiment: run Jinja in async mode | simonw 9599 | closed | 0 | 3 | 2019-01-27T00:28:44Z | 2019-11-12T05:02:18Z | 2019-11-12T05:02:13Z | OWNER | simonw/datasette/pulls/404 | See http://jinja.pocoo.org/docs/2.10/api/#async-support Tests all pass. Have not checked performance difference yet. Creating pull request to run tests in Travis. This is not ready to merge - I'm not yet sure if this is a good idea. |
datasette 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/404/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
403617881 | MDU6SXNzdWU0MDM2MTc4ODE= | 405 | .json?_nl=on option for exporting newline-delimited JSON | simonw 9599 | closed | 0 | 2 | 2019-01-28T01:10:45Z | 2019-01-28T01:49:00Z | 2019-01-28T01:48:37Z | OWNER | The neat thing about newline-delimited JSON is that you don't have to read an entire array (of potentially thousands of objects) into memory in order to parse it - you can parse things a line at a time instead. It will look like this:
I added this as part of the It can be offered alongside |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/405/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
403624090 | MDU6SXNzdWU0MDM2MjQwOTA= | 6 | "sqlite-utils insert" should support newline-delimited JSON | simonw 9599 | closed | 0 | 1 | 2019-01-28T02:00:02Z | 2019-01-28T02:17:45Z | 2019-01-28T02:17:45Z | OWNER | We can already export newline delimited JSON. We should learn to import it as well. The neat thing about importing it is that you can import GBs of data without having to read the whole lot into memory in order to decode the wrapping JSON array. Datasette can export it now: https://github.com/simonw/datasette/issues/405 Demo: https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on It should be possible to do this:
|
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
403625674 | MDU6SXNzdWU0MDM2MjU2NzQ= | 7 | .insert_all() should accept a generator and process it efficiently | simonw 9599 | closed | 0 | 3 | 2019-01-28T02:11:58Z | 2019-01-28T06:26:53Z | 2019-01-28T06:26:53Z | OWNER | Right now you have to load every record into memory before passing the list to If you want to process millions of rows, this is inefficient. Python has generators - we should use them! The only catch here is that part of the magic of If a record outside of those first 1,000 has a rogue column, we can crash with an error. This will free us up to make the |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
403922644 | MDU6SXNzdWU0MDM5MjI2NDQ= | 8 | Problems handling column names containing spaces or - | psychemedia 82988 | closed | 0 | 3 | 2019-01-28T17:23:28Z | 2019-04-14T15:29:33Z | 2019-02-23T21:09:03Z | NONE | Irrrespective of whether using column names containing a space or - character is good practice, SQLite does allow it, but ```python from sqlite_utils import Database dbname = 'test.db' DB = Database(sqlite3.connect(dbname)) import pandas as pd df = pd.DataFrame({'col1':range(3), 'col2':range(3)}) Convert pandas dataframe to appropriate list/dict formatDB['test1'].insert_all( df.to_dict(orient='records') ) Works fine``` However:
throws: ```OperationalError Traceback (most recent call last) <ipython-input-27-070b758f4f92> in <module>() 1 import pandas as pd 2 df = pd.DataFrame({'col 1':range(3), 'col2':range(3)}) ----> 3 DB['test1'].insert_all(df.to_dict(orient='records')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near "1": syntax error ``` and:
results in: ```OperationalError Traceback (most recent call last) <ipython-input-28-654523549d20> in <module>() 1 import pandas as pd 2 df = pd.DataFrame({'col-1':range(3), 'col2':range(3)}) ----> 3 DB['test1'].insert_all(df.to_dict(orient='records')) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, upsert, batch_size, column_order) 327 jsonify_if_needed(record.get(key, None)) for key in all_columns 328 ) --> 329 result = self.db.conn.execute(sql, values) 330 self.db.conn.commit() 331 self.last_id = result.lastrowid OperationalError: near "-": syntax error ``` |
sqlite-utils 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
405801771 | MDExOlB1bGxSZXF1ZXN0MjQ5NjgwOTQ0 | 9 | :pencil: Updates my_database.py to my_database.db | jefftriplett 50527 | closed | 0 | 0 | 2019-02-01T17:35:43Z | 2019-02-24T03:55:04Z | 2019-02-24T03:55:04Z | CONTRIBUTOR | simonw/sqlite-utils/pulls/9 | I noticed that both |
sqlite-utils 140912432 | pull | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/9/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
406055201 | MDU6SXNzdWU0MDYwNTUyMDE= | 406 | Support nullable foreign keys in _labels mode | simonw 9599 | closed | 0 | simonw 9599 | 2 | 2019-02-03T05:34:20Z | 2019-11-02T22:39:28Z | 2019-11-02T22:30:27Z | OWNER | Currently if there's a null in a foreign key we get "None" displayed in the inflated view: |
datasette 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/406/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issues] ( [id] INTEGER PRIMARY KEY, [node_id] TEXT, [number] INTEGER, [title] TEXT, [user] INTEGER REFERENCES [users]([id]), [state] TEXT, [locked] INTEGER, [assignee] INTEGER REFERENCES [users]([id]), [milestone] INTEGER REFERENCES [milestones]([id]), [comments] INTEGER, [created_at] TEXT, [updated_at] TEXT, [closed_at] TEXT, [author_association] TEXT, [pull_request] TEXT, [body] TEXT, [repo] INTEGER REFERENCES [repos]([id]), [type] TEXT , [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT); CREATE INDEX [idx_issues_repo] ON [issues] ([repo]); CREATE INDEX [idx_issues_milestone] ON [issues] ([milestone]); CREATE INDEX [idx_issues_assignee] ON [issues] ([assignee]); CREATE INDEX [idx_issues_user] ON [issues] ([user]);