github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
398089089 | MDU6SXNzdWUzOTgwODkwODk= | 399 | /-/versions for official Docker image returns wrong Datasette version | 9599 | closed | 0 | 2 | 2019-01-11T01:19:58Z | 2019-01-13T23:31:59Z | 2019-01-13T23:10:45Z | OWNER | ``` docker run -p 8001:8001 datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 ``` http://0.0.0.0:8001/-/versions returns this: ``` { "datasette": { "version": "0+unknown" }, ... ``` This is because the Docker image is built by copying in the Datasette source code, which confuses versioneer. Maybe the Docker image should install the code using a wheel or similar? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/399/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
398011658 | MDU6SXNzdWUzOTgwMTE2NTg= | 398 | Ensure downloading a 100+MB SQLite database file works | 9599 | closed | 0 | 3268330 | 3 | 2019-01-10T20:57:52Z | 2020-12-05T19:36:27Z | 2020-12-05T19:36:27Z | OWNER | I've seen attempted downloads of large files fail after about ten seconds. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/398/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
397098882 | MDU6SXNzdWUzOTcwOTg4ODI= | 396 | Add pragma compile_options output to /-/versions | 9599 | closed | 0 | 1 | 2019-01-08T21:43:54Z | 2019-01-11T00:55:22Z | 2019-01-11T00:44:56Z | OWNER | ``` sqlite> pragma compile_options ...> ; BUG_COMPATIBLE_20160819 COMPILER=clang-9.0.0 DEFAULT_CACHE_SIZE=2000 DEFAULT_CKPTFULLFSYNC DEFAULT_JOURNAL_SIZE_LIMIT=32768 DEFAULT_PAGE_SIZE=4096 DEFAULT_SYNCHRONOUS=2 DEFAULT_WAL_SYNCHRONOUS=1 ENABLE_API_ARMOR ENABLE_COLUMN_METADATA ENABLE_DBSTAT_VTAB ENABLE_FTS3 ENABLE_FTS3_PARENTHESIS ENABLE_FTS3_TOKENIZER ENABLE_FTS4 ENABLE_FTS5 ENABLE_JSON1 ENABLE_LOCKING_STYLE=1 ENABLE_PREUPDATE_HOOK ENABLE_RTREE ENABLE_SESSION ENABLE_SNAPSHOT ENABLE_SQLLOG ENABLE_UNKNOWN_SQL_FUNCTION ENABLE_UPDATE_DELETE_LIMIT HAVE_ISNAN MAX_LENGTH=2147483645 MAX_MMAP_SIZE=1073741824 MAX_VARIABLE_NUMBER=500000 OMIT_AUTORESET OMIT_LOAD_EXTENSION STMTJRNL_SPILL=131072 THREADSAFE=2 USE_URI sqlite> ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/396/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
396215043 | MDU6SXNzdWUzOTYyMTUwNDM= | 395 | Find a cleaner pattern for fixtures with arguments | 9599 | closed | 0 | 1 | 2019-01-06T00:31:22Z | 2020-06-07T21:23:22Z | 2020-06-07T21:23:22Z | OWNER | A lot of Datasette tests look like this: https://github.com/simonw/datasette/blob/b65d97792a53f78cb14b226231063209d22c4602/tests/test_api.py#L438-L444 The loop here isn't actually expected to loop - it's there because the `make_app_client` function yields a value and then cleans it up afterwards. This pattern works, but it is a little confusing. It would be nice to replace it with something less strange looking. The answer may be to switch to the "factories as fixtures" pattern described here: https://docs.pytest.org/en/latest/fixture.html#factories-as-fixtures In particular some variant of this example: ``` @pytest.fixture def make_customer_record(): created_records = [] def _make_customer_record(name): record = models.Customer(name=name, orders=[]) created_records.append(record) return record yield _make_customer_record for record in created_records: record.destroy() def test_customer_records(make_customer_record): customer_1 = make_customer_record("Lisa") customer_2 = make_customer_record("Mike") customer_3 = make_customer_record("Meredith") ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/395/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
396212021 | MDU6SXNzdWUzOTYyMTIwMjE= | 394 | base_url configuration setting | 9599 | closed | 0 | 5234079 | 27 | 2019-01-05T23:48:48Z | 2020-06-11T09:15:20Z | 2020-03-25T00:18:45Z | OWNER | I've identified a couple of use-cases for running Datasette in a way that over-rides the default way that internal URLs are generated. 1. Running behind a reverse proxy. I tried running Datasette behind a proxy and found that some of the generated internal links incorrectly referenced `http://127.0.0.1:8001/fixtures/...` - when they should have been referencing `http://my-host.my-domain.com/fixtures/...` - this is a problem both for links within the HTML interface but also for the `toggle_url` keys returned in the JSON as part of the facets datastructure. 2. I would like it to be possible to host a Datasette instance at e.g. `https://www.mynewspaper.com/interactives/2018/election-results/` - either through careful HTTP proxying or, once Datasette has been ported to ASGI, by mounting a Datasette ASGI instance deep within an existing set of URL routes. I'm going to add a `url_prefix` configuration option. This will default to `""`, which means Datasette will behave as it does at the moment - it will use `/` for most URL prefixes in the HTML version, and an absolute URL derived from the incoming `Host` header for URLs that are returned as part of the JSON output. If `url_prefix` is set to another value (either a full URL or a path) then this path will be appended to all generated URLs. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/394/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
377518499 | MDU6SXNzdWUzNzc1MTg0OTk= | 374 | Get Datasette working with Zeit Now v2's 100MB image size limit | 9599 | closed | 0 | 5 | 2018-11-05T18:08:29Z | 2018-12-19T01:35:59Z | 2018-12-19T01:35:59Z | OWNER | Follow-on from #366 Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances. Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/374/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
351017129 | MDU6SXNzdWUzNTEwMTcxMjk= | 360 | Use pysqlite3 if available | 9599 | closed | 0 | 3 | 2018-08-16T00:50:45Z | 2018-08-16T01:50:42Z | 2018-08-16T00:58:58Z | OWNER | [pysqlite3](https://github.com/coleifer/pysqlite3) is a way to provide access to a more recent version of SQLite than the standard library `sqlite3` module (which tends to use the version provided with the operating system - which on e.g. the Travis CI Ubuntu build environment can be as old as 3.8.0). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/360/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
351845423 | MDU6SXNzdWUzNTE4NDU0MjM= | 3 | Experiment with contentless FTS tables | 9599 | closed | 0 | 1 | 2018-08-18T19:31:01Z | 2019-07-22T20:58:55Z | 2019-07-22T20:58:55Z | OWNER | Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/ | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
349850687 | MDU6SXNzdWUzNDk4NTA2ODc= | 2 | Mechanism for adding foreign keys to an existing table | 9599 | closed | 0 | 1 | 2018-08-12T22:50:56Z | 2019-02-24T21:34:41Z | 2019-02-24T21:34:41Z | OWNER | SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running `UPDATE sqlite_master SET sql=... WHERE type='table' AND name='X';` Here's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125 And here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page) | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
349827640 | MDU6SXNzdWUzNDk4Mjc2NDA= | 359 | Faceted browse against a JSON list of tags | 9599 | closed | 0 | 6 | 2018-08-12T17:01:14Z | 2019-05-29T21:39:12Z | 2019-05-03T00:21:44Z | OWNER | If a table has a `["foo", "bar", "baz"]` JSON column allow that to be faceted against. - [x] Support `?column__arraycontains=x` filter queries - [x] Support `?_facet_array=column` faceting | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
348043884 | MDU6SXNzdWUzNDgwNDM4ODQ= | 357 | Plugin hook for loading metadata.json | 9599 | open | 0 | 6 | 2018-08-06T19:00:01Z | 2020-06-21T22:19:58Z | OWNER | For https://github.com/simonw/russian-ira-facebook-ads-datasette/tree/af6d956995e14afd585c35a6a06bb01da32043ba I wrote a script to convert YAML to JSON because YAML is a better format for embedding multi-line HTML descriptions and canned SQL statements. Example yaml metadata file: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/af6d956995e14afd585c35a6a06bb01da32043ba/russian-ads-metadata.yaml It would be useful if Datasette could be fed a YAML file directly: datasette -m metadata.yaml Question is... should this be a native feature (hence adding a YAML dependency) or should it be handled by a `datasette-metadata-yaml` plugin, using a new plugin hook for loading metadata? If so, what would other use-cases for that plugin hook be? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
346028655 | MDU6SXNzdWUzNDYwMjg2NTU= | 356 | Ability to display facet counts for many-to-many relationships | 9599 | closed | 0 | 4 | 2018-07-31T04:14:26Z | 2019-05-29T21:39:12Z | 2019-05-25T16:30:09Z | OWNER | Parent: #354 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
346027040 | MDU6SXNzdWUzNDYwMjcwNDA= | 355 | Table view should support filtering via many-to-many relationships | 9599 | open | 0 | 10 | 2018-07-31T04:04:16Z | 2019-05-23T06:04:03Z | OWNER | Parent: #354 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
346026869 | MDU6SXNzdWUzNDYwMjY4Njk= | 354 | Handle many-to-many relationships | 9599 | open | 0 | 0 | 2018-07-31T04:03:13Z | 2020-11-24T19:51:18Z | OWNER | This is a master tracking ticket for various many-2-many features. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
345821500 | MDU6SXNzdWUzNDU4MjE1MDA= | 352 | render_cell(value) plugin hook | 9599 | closed | 0 | 4 | 2018-07-30T15:56:20Z | 2020-02-10T16:18:58Z | 2018-08-05T00:14:57Z | OWNER | To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/352/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
345469355 | MDU6SXNzdWUzNDU0NjkzNTU= | 351 | Automatically create a GitHub release linking to release notes for every tagged release | 9599 | closed | 0 | 1 | 2018-07-28T18:31:12Z | 2020-05-28T18:56:16Z | 2020-05-28T18:56:15Z | OWNER | Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
344701755 | MDU6SXNzdWUzNDQ3MDE3NTU= | 350 | Don't list default plugins on /-/plugins | 9599 | closed | 0 | 2 | 2018-07-26T05:38:00Z | 2018-08-28T17:13:50Z | 2018-08-28T16:48:19Z | OWNER | https://dbbe707.datasette.io/-/plugins is showing "datasette.publish.now" and "datasette.publish.heroku" | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
344656114 | MDU6SXNzdWUzNDQ2NTYxMTQ= | 348 | Unit tests for "datasette publish" | 9599 | closed | 0 | 1 | 2018-07-26T00:52:23Z | 2018-07-26T05:46:10Z | 2018-07-26T05:46:10Z | OWNER | The datasette publish family of commands all work by shelling out to heroku/now/docker from subprocess import call, check_output So in tests I should be able to mock those calls: @mock.patch('subprocess.call') | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/348/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
344654623 | MDU6SXNzdWUzNDQ2NTQ2MjM= | 347 | Rename "datasette package" to "datasette publish docker" | 9599 | open | 0 | 0 | 2018-07-26T00:42:46Z | 2018-07-26T00:42:46Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/347/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
340282796 | MDU6SXNzdWUzNDAyODI3OTY= | 338 | Only load vegaEmbed if charting tools are enabled | 9599 | closed | 0 | 1 | 2018-07-11T15:02:14Z | 2018-07-11T15:21:47Z | 2018-07-11T15:21:47Z | OWNER | vegaEmbed is a LOT of code (it bundles d3) Inspired by this tweet: https://twitter.com/thelarkinn/status/1017053567641948162 - it would be great if we loaded that code on demand the first time the "Show chart options" button was clicked, or when the page loads with #g. options in the URL. Even better: avoid the overhead if loading React unless the chart options need to be displayed. This would be a pretty major refactoring though. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340065374 | MDU6SXNzdWUzNDAwNjUzNzQ= | 337 | Documentation for datasette publish and datasette package | 9599 | closed | 0 | 1 | 2018-07-11T02:04:06Z | 2018-07-11T02:07:32Z | 2018-07-11T02:05:56Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
340039409 | MDU6SXNzdWUzNDAwMzk0MDk= | 336 | Ensure --help examples in docs are always up to date | 9599 | closed | 0 | 3 | 2018-07-10T23:20:01Z | 2018-07-24T16:01:29Z | 2018-07-24T16:01:29Z | OWNER | Ideally I would automatically generate the --help output shown in our docs, but I don't think I can get that working with readthedocs. Instead, I'm going to add a unit test that checks that those extracts in the documentation match the current output of the --help command. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/336/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
340730961 | MDU6SXNzdWUzNDA3MzA5NjE= | 340 | Embrace black | 9599 | closed | 0 | 1 | 2018-07-12T17:32:29Z | 2019-06-24T06:50:27Z | 2019-06-24T06:50:26Z | OWNER | Run [black](https://github.com/ambv/black) against everything. Then set up CI to fail if code doesn't conform to black's style. Here's how Starlette does this: * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/.travis.yml#L14 * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/scripts/lint - essentially runs `black starlette tests --check` And here's an example of a test run that failed: https://travis-ci.org/encode/starlette/jobs/403172478 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/340/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
339505204 | MDU6SXNzdWUzMzk1MDUyMDQ= | 335 | Package datasette for installation using homebrew | 9599 | closed | 0 | 12 | 2018-07-09T15:45:03Z | 2020-08-11T16:54:06Z | 2020-08-11T16:54:06Z | OWNER | https://docs.brew.sh/Python-for-Formula-Authors describes how. > Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa. It recommends using https://github.com/tdsmith/homebrew-pypi-poet | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
338768551 | MDU6SXNzdWUzMzg3Njg1NTE= | 333 | Datasette on Zeit Now returns http URLs for facet and next links | 9599 | closed | 0 | 4 | 2018-07-06T00:40:49Z | 2018-07-24T04:53:20Z | 2018-07-24T01:51:53Z | OWNER | e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0 ``` { "facet_results": { "lg_id": { "name": "lg_id", "results": [ { "value": "NBA", "label": "NBA", "count": 118016, "toggle_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=NBA", "selected": false }, { "value": "ABA", "label": "ABA", "count": 8298, "toggle_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=ABA", "selected": false } ], "truncated": false } }, "suggested_facets": [ { "name": "_iscopy", "toggle_url": "/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_facet=_iscopy" } ], "next_url": "http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_next=1", } ``` `next_url` and `facet_results` both link to `http://` when they should link to `https://`. Note that suggested facets doesn't include the full URL at all, which is a consistency bug. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
337141108 | MDU6SXNzdWUzMzcxNDExMDg= | 332 | Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1 | 9599 | closed | 0 | 12 | 2018-06-29T21:21:27Z | 2018-07-24T04:53:20Z | 2018-07-24T03:08:30Z | OWNER | It turns out if you load this CSV using `csvs-to-sqlite` you get an Infinity value in SQLite: ``` name,num sasha,10 terry,Inf cathy,0.5 ``` `csvs-to-sqlite infinity-bug.csv infinity-bug.db` I deployed this using: ``` datasette publish now infinity-bug.db --name=datasette-infinity-bug --install=datasette-vega ``` Datasette outputs that as `Infinity` in the JSON format, which causes JavaScript errors. Demo * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug - HTML view works * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug.json?_shape=array - this outputs the following: ``` [ { "rowid": 1, "name": "sasha", "num": 10.0 }, { "rowid": 2, "name": "terry", "num": Infinity }, { "rowid": 3, "name": "cathy", "num": 0.5 } ] ``` But... in Firefox that gets rendered like this: ![2018-06-29 at 4 20 pm](https://user-images.githubusercontent.com/9599/42115408-5d30f630-7bb8-11e8-8370-c8484801c49b.png) And if you click the "Show charting options" button you get this error in the console: ``` SyntaxError: JSON.parse: unexpected character at line 1 column 83 of the JSON data ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
336465018 | MDU6SXNzdWUzMzY0NjUwMTg= | 329 | Travis should push tagged images to Docker Hub for each release | 9599 | closed | 0 | 7 | 2018-06-28T04:01:31Z | 2018-11-05T06:54:10Z | 2018-11-05T06:53:28Z | OWNER | https://sebest.github.io/post/using-travis-ci-to-build-docker-images/ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/329/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
336464733 | MDU6SXNzdWUzMzY0NjQ3MzM= | 328 | Installation instructions, including how to use the docker image | 9599 | closed | 0 | 4 | 2018-06-28T03:59:33Z | 2023-09-05T14:10:39Z | 2018-06-28T04:02:10Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/328/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
335200136 | MDU6SXNzdWUzMzUyMDAxMzY= | 327 | Explore if SquashFS can be used to shrink size of packaged Docker containers | 9599 | open | 0 | 4 | 2018-06-24T18:15:16Z | 2022-02-17T23:37:24Z | OWNER | Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed https://en.wikipedia.org/wiki/SquashFS is "a compressed read-only file system for Linux" - which means it could be a really nice fit for Datasette and its read-only SQLite databases. It would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by `datasette package` and friends. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/327/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
335141434 | MDU6SXNzdWUzMzUxNDE0MzQ= | 326 | CSV should respect --cors and return cors headers | 9599 | closed | 0 | 1 | 2018-06-24T00:44:07Z | 2021-06-17T18:14:24Z | 2018-06-24T00:59:45Z | OWNER | Otherwise tools like Vega can't load data via CSV. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/326/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
335064777 | MDU6SXNzdWUzMzUwNjQ3Nzc= | 325 | Error on row page if table has slashes in the name and ends in .csv | 9599 | closed | 0 | 1 | 2018-06-23T03:43:42Z | 2018-07-09T17:28:27Z | 2018-07-08T05:21:59Z | OWNER | https://v0-23-1.datasette.io/fixtures-e14e080/table%252Fwith%252Fslashes.csv/3 > no such table: table%252Fwith%252Fslashes.csv From clicking the row link on https://v0-23-1.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/325/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
334169932 | MDU6SXNzdWUzMzQxNjk5MzI= | 320 | Need unit tests covering the different states for the advanced export box | 9599 | closed | 0 | 1 | 2018-06-20T17:03:40Z | 2018-07-24T04:53:20Z | 2018-07-24T03:38:40Z | OWNER | There are quite a few variants of this box: ![2018-06-20 at 10 02 am](https://user-images.githubusercontent.com/9599/41673229-1d423adc-7471-11e8-99d4-4251f7d03aa5.png) Test coverage should exercise all of them, since the logic is a little unclear. https://github.com/simonw/datasette/blob/fdfbbbb9ee0d02fd4d43dfc42382252fa2287d6d/datasette/templates/table.html#L140-L159 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/320/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
334149717 | MDU6SXNzdWUzMzQxNDk3MTc= | 319 | Incorrect display of compound primary keys with foreign key relationships | 9599 | closed | 0 | 3439337 | 2 | 2018-06-20T16:09:36Z | 2018-06-21T15:58:15Z | 2018-06-21T14:56:41Z | OWNER | https://registry.datasette.io/registry-7d4f81f/datasette_tags ![2018-06-20 at 9 07 am](https://user-images.githubusercontent.com/9599/41670542-68cc4dec-7469-11e8-9521-3bbc6465eccb.png) Underlying JSON looks [like this](https://registry.datasette.io/registry-7d4f81f/datasette_tags.json?_labels=on): ``` { "database": "registry", "table": "datasette_tags", "is_view": false, "human_description_en": "", "rows": [ { "datasette_id": { "value": 1, "label": "Global Power Plant Database" }, "tag": { "value": "geospatial", "label": "geospatial" } }, ```` Bug is likely somewhere in here: https://github.com/simonw/datasette/blob/e04f5b0d348ef7275a0a5ab9eb53527105132885/datasette/views/table.py#L143-L207 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/319/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
334148669 | MDU6SXNzdWUzMzQxNDg2Njk= | 318 | Facets with value of 0 displayed incorrectly | 9599 | closed | 0 | 3439337 | 1 | 2018-06-20T16:06:46Z | 2019-05-29T21:39:12Z | 2018-06-21T04:30:45Z | OWNER | https://registry.datasette.io/registry-7d4f81f/tables?_facet=is_hidden#facet-is_hidden ![2018-06-20 at 9 05 am](https://user-images.githubusercontent.com/9599/41670448-2c06e642-7469-11e8-86be-4664269582b1.png) Displays correctly if you select it: https://registry.datasette.io/registry-7d4f81f/tables?_facet=is_hidden&is_hidden=0 ![2018-06-20 at 9 06 am](https://user-images.githubusercontent.com/9599/41670471-3e61e486-7469-11e8-8710-5da90ef65787.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/318/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
334698969 | MDU6SXNzdWUzMzQ2OTg5Njk= | 323 | Speed up Travis CI builds | 9599 | closed | 0 | 1 | 2018-06-21T23:55:27Z | 2018-07-10T15:03:37Z | 2018-07-10T15:03:36Z | OWNER | They've got a bit slow. Part of this is the Zeit Now deploy, but the build-and-test cycle is taking at least a couple of minutes. ![2018-06-21 at 4 54 pm](https://user-images.githubusercontent.com/9599/41751010-e48c823e-7573-11e8-88f3-7aa8a7e53917.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/323/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
333326107 | MDU6SXNzdWUzMzMzMjYxMDc= | 317 | Travis CI fails to upload new releases to PyPI | 9599 | closed | 0 | 3439337 | 2 | 2018-06-18T15:44:26Z | 2018-06-21T15:45:47Z | 2018-06-21T15:45:47Z | OWNER | https://travis-ci.org/simonw/datasette/jobs/393684139 ``` ... removing build/bdist.linux-x86_64/wheel Uploading distributions to https://upload.pypi.org/legacy/ Uploading datasette-0.23-py3-none-any.whl 100%|██████████| 201k/201k [00:00<00:00, 1.02MB/s] HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/ ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/317/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
333096176 | MDU6SXNzdWUzMzMwOTYxNzY= | 314 | HTML table does not correctly display entirely blank rows | 9599 | closed | 0 | 3439337 | 1 | 2018-06-17T21:58:06Z | 2018-06-21T16:04:59Z | 2018-06-21T15:26:26Z | OWNER | https://958b75c.datasette.io/fixtures-35b6eb6/simple_view ![2018-06-17 at 2 56 pm](https://user-images.githubusercontent.com/9599/41512541-b52e90be-723e-11e8-95c9-7d091738d5cc.png) https://958b75c.datasette.io/fixtures-35b6eb6/simple_view.json shows the underlying data: ``` "rows": [ [ "hello", "HELLO" ], [ "world", "WORLD" ], [ "", "" ] ] ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/314/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
333086005 | MDU6SXNzdWUzMzMwODYwMDU= | 313 | Deploy demo of Datasette on every commit that passes tests | 9599 | closed | 0 | 6 | 2018-06-17T19:19:12Z | 2018-06-17T21:52:58Z | 2018-06-17T21:52:58Z | OWNER | We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/313/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
333000163 | MDU6SXNzdWUzMzMwMDAxNjM= | 312 | HTML, CSV and JSON views should support ?_col=&_col= | 9599 | closed | 0 | 1 | 2018-06-16T16:53:35Z | 2021-06-17T18:14:24Z | 2018-06-16T17:00:12Z | OWNER | To support whitelisting columns to display. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
332830309 | MDU6SXNzdWUzMzI4MzAzMDk= | 310 | datasette publish now is broken in master | 9599 | closed | 0 | 0 | 2018-06-15T16:01:14Z | 2018-06-16T16:29:50Z | 2018-06-16T16:29:50Z | OWNER | ``` > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2 > unable to execute 'gcc': No such file or directory > error: command 'gcc' failed with exit status 1 > > ---------------------------------------- > Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-s73273rj/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-yha7dxqq/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-s73273rj/httptools/ ``` Turns out the `python-slim` base image I introduced in b18e4515855c3f1eeca3dfcccdbb6df05869084a doesn't include gcc: https://github.com/docker-library/python/issues/60#issuecomment-134322383 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/310/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
331343824 | MDU6SXNzdWUzMzEzNDM4MjQ= | 309 | On 404s with a trailing slash redirect to that page without a trailing slash | 9599 | closed | 0 | 3439337 | 2 | 2018-06-11T20:46:49Z | 2018-06-21T15:22:02Z | 2018-06-21T15:13:15Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/309/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
329147284 | MDU6SXNzdWUzMjkxNDcyODQ= | 305 | Add contributor guidelines to docs | 9599 | closed | 0 | 2 | 2018-06-04T17:25:30Z | 2019-06-24T06:40:19Z | 2019-06-24T06:40:19Z | OWNER | https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/305/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
329661905 | MDU6SXNzdWUzMjk2NjE5MDU= | 306 | Custom URL routing with independent tests | 9599 | closed | 0 | 5 | 2018-06-05T23:40:08Z | 2018-06-07T15:29:28Z | 2018-06-07T15:29:28Z | OWNER | The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching. If we go custom, tests should live in `test_routing.py` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/306/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
328229224 | MDU6SXNzdWUzMjgyMjkyMjQ= | 304 | Ability to configure SQLite cache_size | 9599 | closed | 0 | 3 | 2018-05-31T17:28:07Z | 2018-06-04T16:13:32Z | 2018-06-04T16:03:19Z | OWNER | See https://www.sqlite.org/pragma.html#pragma_cache_size Let's call the config setting `cache_size_kb` to emphasize that we're using the negative option. Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10 > If the argument N is positive then the suggested cache size is set to N. If the argument N is negative, then the number of cache pages is adjusted to use approximately abs(N*1024) bytes of memory. Backwards compatibility note: The behavior of cache_size with a negative N was different in prior to version 3.7.10 (2012-01-16). In version 3.7.9 and earlier, the number of pages in the cache was set to the absolute value of N. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
328172521 | MDU6SXNzdWUzMjgxNzI1MjE= | 303 | Support table names ending with .json or .csv | 9599 | closed | 0 | 4 | 2018-05-31T14:53:23Z | 2018-06-15T06:55:50Z | 2018-06-15T06:55:50Z | OWNER | This is needed for #266 - if a table name ends with `.json` or `.csv` right now our URL pattern matching will do the wrong thing. We should be smarter about this. This does mean we will have some URLs that look like this: http://localhost:8001/dbname/weird.json - returning HTML, not JSON http://localhost:8001/dbname/weird.json.json - returning JSON http://localhost:8001/dbname/weird.json.csv - returning CSV | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/303/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
328171513 | MDU6SXNzdWUzMjgxNzE1MTM= | 302 | test-2.3.sqlite database filename throws a 404 | 9599 | closed | 0 | 3439337 | 2 | 2018-05-31T14:50:58Z | 2018-06-21T15:21:17Z | 2018-06-21T15:21:16Z | OWNER | The following almost works: datasette test-2.3.sqlite http://127.0.0.1:8001test-2.3-c88bc35/HighWays loads OK, but http://127.0.0.1:8001test-2.3-c88bc35 throws a 404: ![2018-05-31 at 7 50 am](https://user-images.githubusercontent.com/9599/40789434-447ae934-64a7-11e8-9a07-4eeba87147d5.png) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/302/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
328155946 | MDU6SXNzdWUzMjgxNTU5NDY= | 301 | --spatialite option for "datasette publish heroku" | 9599 | open | 0 | 1 | 2018-05-31T14:13:09Z | 2022-01-20T21:28:50Z | OWNER | Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/301/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
327395270 | MDU6SXNzdWUzMjczOTUyNzA= | 296 | Per-database and per-table /-/ URL namespace | 9599 | open | 0 | 3 | 2018-05-29T16:23:13Z | 2019-06-28T16:46:34Z | OWNER | Initially this will be for subsets of `/-/inspect` and `/-/metadata` but it will also give us a URL namespace for future features like `/-/facet` (expanded list of a specific facet, linked to from `...`) and `/-/graph` To start: * `/dbname/-/inspect` * `/dbname/-/metadata` * `/dbname/tablename/-/inspect` * `/dbname/tablename/-/metadata` This means we will no longer allow databases or tables to have the name `"-"` - I think that's OK We will continue to support rows with a primary key of `"-"` at the following URL: * `/dbname/tablename/-` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/296/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
327383759 | MDU6SXNzdWUzMjczODM3NTk= | 295 | Extract unit tests for inspect out to test_inspect.py | 9599 | closed | 0 | 2 | 2018-05-29T15:55:04Z | 2019-05-11T21:40:32Z | 2019-05-11T21:40:32Z | OWNER | Right now they are bundled up as API unit tests for a relatively unimportant endpoint. They should be their own thing. Blocks #294 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/295/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327365110 | MDU6SXNzdWUzMjczNjUxMTA= | 294 | inspect should record column types | 9599 | open | 0 | 7 | 2018-05-29T15:10:41Z | 2019-06-28T16:45:28Z | OWNER | For each table we want to know the columns, their order and what type they are. I'm going to break with SQLite defaults a little on this one and allow datasette to define additional types - to start with just a `geometry` type for columns that are detected as SpatiaLite geometries. Possible JSON design: "columns": [{ "name": "title", "type": "text" }, ...] Refs #276 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/294/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
327541975 | MDU6SXNzdWUzMjc1NDE5NzU= | 300 | Hide sort select box on larger screens | 9599 | closed | 0 | 0 | 2018-05-30T01:34:59Z | 2018-05-31T14:43:13Z | 2018-05-31T14:43:13Z | OWNER | I'm larger screens you can sort by clicking column headers, so no need to show the select box (which was added for the small screen layout that doesn't show headers) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/300/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327461381 | MDU6SXNzdWUzMjc0NjEzODE= | 299 | Documentation covering ALL datasette URLs | 9599 | closed | 0 | 1 | 2018-05-29T19:46:15Z | 2018-07-28T04:24:05Z | 2018-07-28T04:22:30Z | OWNER | Relates to #296. We need a single page of the docs listing all of the URL patterns Datasette responds to, also detailing which templates are used to render them and linking to examples of the JSON they output when called with `.json`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327459829 | MDU6SXNzdWUzMjc0NTk4Mjk= | 298 | URLify URLs in results from custom SQL statements / views | 9599 | closed | 0 | 2 | 2018-05-29T19:41:07Z | 2018-07-24T04:53:20Z | 2018-07-24T03:56:50Z | OWNER | Consider this custom query: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+user%2C+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+user%29+as+user_url%2C+created_at%2C+text%2C+url+from+%5Btwitter-ratio%2Fsenators%5D+limit+10%3B ```select user, ('https://twitter.com/' || user) as user_url, created_at, text, url from [twitter-ratio/senators] limit 10;``` ![2018-05-29 at 12 38 pm](https://user-images.githubusercontent.com/9599/40681177-44a36d5c-633d-11e8-935b-c49dad7ac682.png) It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators ![2018-05-29 at 12 39 pm](https://user-images.githubusercontent.com/9599/40681206-5c69c47c-633d-11e8-9f3a-08899f8659b8.png) This currently does not happen because the table view render logic takes a different path through `display_columns_and_rows()` which includes this bit: https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/views/table.py#L195-L202 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/298/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
327420945 | MDU6SXNzdWUzMjc0MjA5NDU= | 297 | datasette publish Dockerfile should use python:3.6-slim-stretch | 9599 | closed | 0 | 1 | 2018-05-29T17:40:08Z | 2018-05-31T14:44:37Z | 2018-05-31T14:44:37Z | OWNER | Right now the Dockerfile generated by `datasette package` and `datasette publish` uses this: https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/utils.py#L269 This appears to result in a SQLite version of `3.8.7.1` - https://parlgov.datasettes.com/-/versions ``` "sqlite": { "extensions": {}, "fts_versions": [ "FTS4", "FTS3" ], "version": "3.8.7.1" } ``` Meanwhile, https://fivethirtyeight.datasettes.com/-/versions is deployed with this Dockerfile https://github.com/simonw/fivethirtyeight-datasette/blob/0849901cae06e957fe04892cd4033bdcd1fcf966/Dockerfile which uses `FROM python:3.6-slim-stretch` and results in the following version report: ``` "sqlite": { "extensions": { "json1": null }, "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "version": "3.16.2" } ``` So not only do we get a more recent SQLite (including https://www.sqlite.org/rowvalue.html added in 3.15) but we also get `FTS5` and `json1` as well. Refs #191 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/297/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326189744 | MDU6SXNzdWUzMjYxODk3NDQ= | 285 | num_threads and cache_max_age should be --config options | 9599 | closed | 0 | 2 | 2018-05-24T16:04:51Z | 2018-05-27T00:53:35Z | 2018-05-27T00:43:33Z | OWNER | https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/app.py#L106 And https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/views/base.py#L325 Refs #275 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/285/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326182814 | MDU6SXNzdWUzMjYxODI4MTQ= | 284 | Ability to enable/disable specific features via --config | 9599 | closed | 0 | 5 | 2018-05-24T15:47:56Z | 2018-05-25T06:05:02Z | 2018-05-25T05:51:09Z | OWNER | `--config` settings from #274 can currently only be integers. I'd like them to be available as boooeans too. Then we can use them to have that are turned on by default but can be turned off. First features to get this treatment: - [x] `allow_sql` - whether or not the `?sql=` parameter is allowed and form is displayed - [X] `allow_facet` - is `?_facet=` allowed or do we only run facets defined in `metadata.json` - [X] `allow_download` - do we let users download the full SQLite database file? - [X] `suggest_facets` - do we attempt to calculate suggested facets? Refs #275 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/284/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326800219 | MDU6SXNzdWUzMjY4MDAyMTk= | 292 | Mechanism for customizing the SQL used to select specific columns in the table view | 9599 | closed | 0 | 15 | 2018-05-27T09:05:52Z | 2021-05-27T04:25:01Z | 2021-05-27T04:25:01Z | OWNER | Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow. We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice. I think there are two features here: * the ability to request a subset of columns on the table view * the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example Both features should be available via both querystring arguments and in `metadata.json` The querystring argument for custom SQL should only work if `allow_sql` config is turned on. Refs #276 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/292/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326783670 | MDU6SXNzdWUzMjY3ODM2NzA= | 291 | Avoid plugins accidentally loading dependencies twice | 9599 | closed | 0 | 3 | 2018-05-27T03:15:21Z | 2020-09-30T20:36:12Z | 2018-05-28T20:42:02Z | OWNER | Plugins that include JavaScript files risk loading the same code twice. In particular: I want to build a second plugin that uses the Leaflet mapping library (the first was [datasette-cluster-map](https://pypi.org/project/datasette-cluster-map/)). But I don't want the two plugins to load duplicate copies of Leaflet. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/291/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326778161 | MDU6SXNzdWUzMjY3NzgxNjE= | 290 | Consider increasing the default for num_sql_threads (currently 3) | 9599 | open | 0 | 0 | 2018-05-27T00:52:41Z | 2018-05-27T00:52:41Z | OWNER | I ran a very rough micro-benchmark on the new `num_sql_threads` config option (added in #285) datasette --config num_sql_threads:1 fivethirtyeight.db Then ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators' | Number of threads | Requests/second | |---|---| | 1 | 4.57 | | 3 | 9.77 | | 10 | 13.53 | | 20 | 15.24 | 50 | 8.21 | This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/290/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
326768188 | MDU6SXNzdWUzMjY3NjgxODg= | 289 | ?_ttl= parameter to control caching | 9599 | closed | 0 | 3 | 2018-05-26T21:22:55Z | 2018-05-26T22:22:47Z | 2018-05-26T22:17:48Z | OWNER | This would allow clients to specify the max-age caching header that should be returned with the query. Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/289/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326767626 | MDU6SXNzdWUzMjY3Njc2MjY= | 288 | Support multiple filters of the same type | 9599 | closed | 0 | 3 | 2018-05-26T21:13:12Z | 2019-04-15T23:45:04Z | 2019-04-15T23:44:26Z | OWNER | This should work for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/biopics%2Fbiopics?year_release__not=2014&year_release__not=2015 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/288/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326617744 | MDU6SXNzdWUzMjY2MTc3NDQ= | 287 | ?_shape=arrayfirst | 9599 | closed | 0 | 1 | 2018-05-25T18:11:03Z | 2018-05-27T00:32:53Z | 2018-05-27T00:32:29Z | OWNER | Return an array of single items (the first item in each row returned from the SQL query). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/287/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326599525 | MDU6SXNzdWUzMjY1OTk1MjU= | 286 | Database hash should include current datasette version | 9599 | open | 0 | 2 | 2018-05-25T17:03:42Z | 2018-05-25T17:07:36Z | OWNER | Right now deploying a new version of datasette doesn't invalidate existing URLs, so users may still see a cached copy of the old templates. We can fix this by including the current datasette version in the input to the hash function (which currently just the database file contents). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/286/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
325294102 | MDU6SXNzdWUzMjUyOTQxMDI= | 278 | Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0 | 9599 | closed | 0 | 3 | 2018-05-22T13:28:40Z | 2018-05-23T17:43:36Z | 2018-05-23T17:43:36Z | OWNER | A Dockerfile that does the following: * Bundles Datasette master * Python 3.6 most recent version (or 3.7 if it has been released) * SQLite 3.23.1 (or most recent release) such that "import sqite3" in Python gets that version. Ideally with the json1 module baked in by default, but having it loadable as an optional module is fine too * SpatiaLite 4.4.0-RC0 (or most recent version) such that it can be loaded as an optional module * Uses multi-stage builds to stay as small as possible Note that the current "release" of SpatiaLite is 4.3.0 which is missing key features like https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN - 4.4.0 probably needs to be compiled from source. I don't know the best way to get a current SQLite version bundled for Python 3. Maybe https://github.com/coleifer/pysqlite3 ? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/278/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
325958506 | MDU6SXNzdWUzMjU5NTg1MDY= | 283 | Support cross-database joins | 9599 | closed | 0 | 26 | 2018-05-24T04:18:39Z | 2021-06-06T09:40:18Z | 2021-02-18T22:16:46Z | OWNER | SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases. Since Datasette supports more than one database, this would make a pretty neat feature. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/283/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
325705981 | MDU6SXNzdWUzMjU3MDU5ODE= | 282 | Faceting breaks pagination | 9599 | closed | 0 | 1 | 2018-05-23T13:29:47Z | 2018-05-23T13:53:39Z | 2018-05-23T13:42:07Z | OWNER | e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id#facet-lg_id - click the "next page" link: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id&_next=100 Invalid SQL: near "and": syntax error | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/282/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
324188953 | MDU6SXNzdWUzMjQxODg5NTM= | 272 | Port Datasette to ASGI | 9599 | closed | 0 | 9599 | 3268330 | 42 | 2018-05-17T21:16:32Z | 2019-06-24T04:54:15Z | 2019-06-24T03:33:06Z | OWNER | Datasette doesn't take much advantage of Sanic, and I'm increasingly having to work around parts of it because of idiosyncrasies that are specific to Datasette - caring about the exact order of querystring arguments for example. Since Datasette is GET-only our needs from a web framework are actually pretty slim. This becomes more important as I expand the plugins #14 framework. Am I sure I want the plugin ecosystem to depend on a Sanic if I might move away from it in the future? If Datasette wasn't all about async/await I would use WSGI, but today it makes more sense to use ASGI. I'd like to be confident that switching to ASGI would still give me the excellent performance that Sanic provides. https://github.com/django/asgiref/blob/master/specs/asgi.rst | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/272/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||
324162476 | MDU6SXNzdWUzMjQxNjI0NzY= | 271 | Mechanism for automatically picking up changes when on-disk .db file changes | 9599 | closed | 0 | 4 | 2018-05-17T19:53:15Z | 2019-01-10T21:35:18Z | 2019-01-10T21:35:18Z | OWNER | It would be useful if Datasette could spot when a SQLite database file changes on disk and restart itself (hence re-running .inspect() and picking up the new content hash). Ideally this could happen in an atomic way so no requests get dropped during the switch-over. This may not play well with SQLite opening databases in immutable mode. Research required. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/271/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
324720095 | MDU6SXNzdWUzMjQ3MjAwOTU= | 275 | "config" section in metadata.json (root, database and table level) | 9599 | closed | 0 | 3 | 2018-05-20T16:02:28Z | 2023-08-23T01:28:37Z | 2023-08-23T01:28:37Z | OWNER | Split off from #274 Metadata should an optional `"config"` section at root, table or database level. The TableView and RowView and DatabaseView and BaseView classes could all have a `.config("key")` method which knows how to resolve the hierarchy of configs. This will allow individual tables (or databases) to set their own config settings for things like `sql_time_limit_ms` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/275/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
324652142 | MDU6SXNzdWUzMjQ2NTIxNDI= | 274 | Rename --limit to --config, add --help-config | 9599 | closed | 0 | 2 | 2018-05-19T18:57:42Z | 2018-05-20T17:04:55Z | 2018-05-20T17:04:11Z | OWNER | #270 introduced `--limit` but on further thought it should be called `--config` instead. `--page_size` should becomes `--config default_page_size:1000` Add `--help-config` to show full help showing all config settings. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/274/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
324451322 | MDU6SXNzdWUzMjQ0NTEzMjI= | 273 | Figure out a way to have /-/version return current git commit hash | 9599 | closed | 0 | 2 | 2018-05-18T15:16:56Z | 2018-05-22T19:35:22Z | 2018-05-22T19:35:22Z | OWNER | https://fivethirtyeight.datasettes.com/-/versions reports Datasette version `0.21` This isn't actually correct. The deploy script for that site actually deploys current master using `https://github.com/simonw/datasette/archive/master.zip`: https://github.com/simonw/fivethirtyeight-datasette/blob/66b4b0dfedd7237bc8c02d3e26d905bca7b84069/Dockerfile#L9 Ideally this would show the current commit hash, but I'm not at all sure if it's possible to derive that from `pip install https://github.com/simonw/datasette/archive/master.zip`. Is there another mechanism that could be used to reliably `pip install` current master but still provide access to the most recent commit hash? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/273/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323223872 | MDU6SXNzdWUzMjMyMjM4NzI= | 260 | Validate metadata.json on startup | 9599 | open | 0 | 7 | 2018-05-15T13:42:56Z | 2023-06-21T12:51:22Z | OWNER | It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail. To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/260/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
323830051 | MDU6SXNzdWUzMjM4MzAwNTE= | 270 | --limit= CLI option for setting limits | 9599 | closed | 0 | 1 | 2018-05-17T00:14:24Z | 2018-05-18T06:19:31Z | 2018-05-18T06:16:39Z | OWNER | #264 calls for four new datasette limit options, on top of the two existing ones: * `--max_returned_rows` * `--sql_time_limit_ms` These are already clogging up `datasette serve --help` a bit. How about this syntax instead? datasette --limit max_returned_rows:100 \ --limit facet_timeout_ms:500 demo.db Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/270/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323726888 | MDU6SXNzdWUzMjM3MjY4ODg= | 269 | If a facet fails due to timing out, let the user know somehow | 9599 | closed | 0 | 0 | 2018-05-16T18:01:47Z | 2018-05-18T06:11:46Z | 2018-05-18T06:11:46Z | OWNER | Refs #255 - right now facets fail silently if the user requested them but they take longer than 200ms to calculate - see also #264 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/269/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323718842 | MDU6SXNzdWUzMjM3MTg4NDI= | 268 | Mechanism for ranking results from SQLite full-text search | 9599 | open | 0 | 12 | 2018-05-16T17:36:40Z | 2022-01-13T22:21:28Z | OWNER | This isn't particularly straight-forward - all the more reason for Datasette to implement it for you. This article is helpful: http://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/268/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
323716411 | MDU6SXNzdWUzMjM3MTY0MTE= | 267 | Documentation for URL hashing, redirects and cache policy | 9599 | closed | 0 | 3 | 2018-05-16T17:29:01Z | 2019-06-24T06:41:02Z | 2019-06-24T06:41:02Z | OWNER | See my comments on #258 for a starting point | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/267/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323681589 | MDU6SXNzdWUzMjM2ODE1ODk= | 266 | Export to CSV | 9599 | closed | 0 | 27 | 2018-05-16T15:50:24Z | 2021-06-17T18:14:24Z | 2018-06-18T06:05:25Z | OWNER | Datasette needs to be able to export data to CSV. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/266/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323677499 | MDU6SXNzdWUzMjM2Nzc0OTk= | 265 | Add links to example Datasette instances to appropiate places in docs | 9599 | closed | 0 | 5 | 2018-05-16T15:40:20Z | 2018-06-18T15:52:15Z | 2018-06-18T15:52:15Z | OWNER | Links to working examples would really help, especially on these pages: * http://datasette.readthedocs.io/en/latest/json_api.html * http://datasette.readthedocs.io/en/latest/sql_queries.html * http://datasette.readthedocs.io/en/latest/facets.html * http://datasette.readthedocs.io/en/latest/full_text_search.html | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/265/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323673899 | MDU6SXNzdWUzMjM2NzM4OTk= | 264 | Make it possible to customize various facet settings | 9599 | closed | 0 | 1 | 2018-05-16T15:31:34Z | 2018-05-18T06:18:00Z | 2018-05-18T05:11:52Z | OWNER | The new Facets implementation from #255 includes several hard-coded settings which should be made configurable somehow: Number of rows to return in a facet (maybe this should also be an option that can be set via quersytring argument, e.g. `?_facet=qSpecies:40`): https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L539 Time limit for executing a facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L559-L562 Maximum unique values returned in order for a column to be suggested as a facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L646-L647 Time limit for calculating if a column should be a suggested facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L664-L667 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/264/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323671577 | MDU6SXNzdWUzMjM2NzE1Nzc= | 263 | Facets should not execute for ?shape=array|object | 9599 | closed | 0 | 3 | 2018-05-16T15:26:13Z | 2021-06-02T02:54:34Z | 2021-06-02T02:54:34Z | OWNER | Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/263/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323658641 | MDU6SXNzdWUzMjM2NTg2NDE= | 262 | Add ?_extra= mechanism for requesting extra properties in JSON | 9599 | open | 0 | 3268330 | 27 | 2018-05-16T14:55:42Z | 2023-03-29T06:22:22Z | OWNER | Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional `template_data()` function which is called if the view is being rendered as HTML. This `template_data()` function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON. Example of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704 With features like Facets in #255 I'm beginning to want to move more items into the `template_data()` - in the case of facets it's the `suggested_facets` array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used. But... as an API user, I want to still optionally be able to access that information. Solution: Add a `?_extra=suggested_facets&_extra=table_metadata` argument which can be used to optionally request additional blocks to be added to the JSON API. Then redefine as many of the current `template_data()` features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates. This could allow the JSON representation to be slimmed down further (removing e.g. the `table_definition` and `view_definition` keys) while still making that information available to API users who need it. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/262/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
322787470 | MDU6SXNzdWUzMjI3ODc0NzA= | 259 | inspect() should detect many-to-many relationships | 9599 | closed | 0 | 6 | 2018-05-14T12:03:58Z | 2019-05-23T03:55:37Z | 2019-05-23T03:55:37Z | OWNER | Relates to #255 - in particular supporting facets across M2M relationships. It should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables. When rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side. Since SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/259/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
322551723 | MDU6SXNzdWUzMjI1NTE3MjM= | 256 | Break up app.py into separate view modules | 9599 | closed | 0 | 1 | 2018-05-12T23:56:33Z | 2018-05-14T03:05:37Z | 2018-05-14T03:05:37Z | OWNER | `views/table.py` and `views/database.py` and `views/utils.py` as a starting point. Likewise, create `tests/test_views_table.py` and `tests/test_views_database.py` - these will contain both HTML and API test for those views. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/256/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
322477187 | MDU6SXNzdWUzMjI0NzcxODc= | 255 | Facets | 9599 | closed | 0 | 16 | 2018-05-12T03:00:07Z | 2019-05-29T21:39:12Z | 2018-05-16T15:32:12Z | OWNER | Ability to display facets and facet counts on the table view. Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `"facets": [...]` property in `metadata.json` - [x] Implement `?_facet=` - [x] Implement `metadata.json` `facets` key - [x] Design for how facets should be presented - [x] Facets should be able to toggle off as well as on - [x] Expand labels for facets that are foreign keys - [x] Suggest potential facets (if we can do so within a tight time limit) - [x] Documentation | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/255/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
321631020 | MDU6SXNzdWUzMjE2MzEwMjA= | 253 | Documentation explaining how to use SQLite FTS with Datasette | 9599 | closed | 0 | 1 | 2018-05-09T16:02:08Z | 2018-05-12T12:09:02Z | 2018-05-12T12:06:51Z | OWNER | In particular how to work with https://www.sqlite.org/fts3.html#_external_content_fts4_tables_ - which Datasette can automatically detect and use to add a search UI to your page. Examples of basic search setup like this: ``` CREATE VIRTUAL TABLE "interests_fts" USING FTS4 (name, content="interests"); INSERT INTO "interests_fts" (rowid, name) SELECT rowid, name FROM interests; ``` And complex join-based search setup like this: ``` CREATE VIRTUAL TABLE "interests_fts" USING FTS4 (name, category, member, content="interests"); INSERT INTO "interests_fts" (rowid, name, category, member) SELECT interests.rowid, interests.name, interest_categories.name, members.name FROM interests JOIN interest_categories ON interests.category_id = interest_categories.id JOIN members ON interests.member_id = members.id; ``` Also mention how `csvs-to-sqlite` can be used to do this easily. This will benefit from #252 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/253/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
321624016 | MDU6SXNzdWUzMjE2MjQwMTY= | 252 | /-/versions should report the FTS version supported by SQLite | 9599 | closed | 0 | 0 | 2018-05-09T15:43:47Z | 2018-05-11T13:19:52Z | 2018-05-11T13:19:52Z | OWNER | I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/252/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
320132682 | MDU6SXNzdWUzMjAxMzI2ODI= | 250 | Setup some issue templates | 9599 | open | 0 | 0 | 2018-05-04T01:49:07Z | 2018-05-04T01:49:07Z | OWNER | https://twitter.com/left_pad/status/99216385740464537 I like the idea of using these to help people understand some of the ways I want to use issues. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/250/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
320090329 | MDU6SXNzdWUzMjAwOTAzMjk= | 249 | ?_size=max argument | 9599 | closed | 0 | 1 | 2018-05-03T21:42:04Z | 2018-05-04T18:26:30Z | 2018-05-04T18:05:04Z | OWNER | For plugins that want to load the most data allowable, having `?_size=max` would be useful. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/249/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
320592643 | MDU6SXNzdWUzMjA1OTI2NDM= | 251 | Explore "distinct values for column" in inspect() | 9599 | closed | 0 | 4 | 2018-05-06T13:27:24Z | 2018-05-14T22:47:55Z | 2018-05-14T22:47:55Z | OWNER | A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B Detecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns. The problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/251/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
319358200 | MDU6SXNzdWUzMTkzNTgyMDA= | 245 | ?_shape=array option | 9599 | closed | 0 | 1 | 2018-05-01T23:11:07Z | 2018-05-03T14:14:33Z | 2018-05-02T00:12:20Z | OWNER | Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects. `?_shape=array` should do just that While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays` And validate that `_shape` is a valid option And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/245/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
319954545 | MDU6SXNzdWUzMTk5NTQ1NDU= | 248 | /-/plugins should show version of each installed plugin | 9599 | closed | 0 | 2 | 2018-05-03T14:50:45Z | 2018-05-04T18:25:40Z | 2018-05-04T18:05:04Z | OWNER | Refs #244 https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules ``` >>> import pkg_resources >>> pkg_resources.get_distribution('datasette_cluster_map').version '0.4' ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/248/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
318738000 | MDU6SXNzdWUzMTg3MzgwMDA= | 244 | /-/versions page | 9599 | closed | 0 | 1 | 2018-04-29T18:22:15Z | 2018-05-03T14:13:49Z | 2018-05-03T14:09:53Z | OWNER | Displays the current version of: * datasette * Python * SQLite * Spatialite (if available) Installed plugin versions should be shown on /-/plugins | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/244/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
318737808 | MDU6SXNzdWUzMTg3Mzc4MDg= | 243 | --spatialite option for datasette publish commands | 9599 | closed | 0 | 2 | 2018-04-29T18:19:32Z | 2018-05-31T14:17:53Z | 2018-05-31T14:17:53Z | OWNER | Performs the necessary incantations to install Spatialite on Zeit Now or Heroku and sets the corresponding environment variable to ensure the module is correctly loaded by datasette serve. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/243/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
318692953 | MDU6SXNzdWUzMTg2OTI5NTM= | 242 | Rename ?_sql_time_limit_ms= to ?_timelimit= | 9599 | closed | 0 | 0 | 2018-04-29T06:11:35Z | 2018-05-02T00:20:42Z | 2018-05-02T00:20:42Z | OWNER | It's a bit of a mouthful at the moment. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/242/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
318490133 | MDU6SXNzdWUzMTg0OTAxMzM= | 241 | Default datasette logging format should be JSON | 9599 | open | 0 | 0 | 2018-04-27T17:32:48Z | 2018-07-10T17:45:40Z | OWNER | Structured logs are better. Datasette should default to outputting it's HTTP access log lines as newline delimited JSON instead of the Sanic default format it uses at the moment. For improved greppability these logs should have keys ordered in a consistent way. Python's JSON module can do this with ordered dictionaries. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/241/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
317001500 | MDU6SXNzdWUzMTcwMDE1MDA= | 236 | datasette publish lambda plugin | 9599 | open | 0 | 11 | 2018-04-23T22:10:30Z | 2023-03-12T14:04:15Z | OWNER | Refs #217 - create a publish plugin that can deploy to AWS Lambda. https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it). Lambdas do get a 512 MB `/tmp` directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the `.db` file there and then have the lambda download it on startup. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/236/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
317900587 | MDU6SXNzdWUzMTc5MDA1ODc= | 240 | FTS table detection should be part of .inspect() | 9599 | closed | 0 | 0 | 2018-04-26T06:58:10Z | 2018-04-29T00:04:44Z | 2018-04-29T00:04:44Z | OWNER | The code that detects if specific tables have a corresponding FTS column is currently called from TableView - it should instead be handled as part of `.inspect()`. This will make it easier to build other features that need to behave differently depending on whether a table can be searched, e.g. an autocomplete widget for selecting filters from foreign key tables. Current code: https://github.com/simonw/datasette/blob/f188ceaa2a3a5b2eab83425ad0f00cb0d364e24a/datasette/app.py#L728-L733 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/240/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
317760361 | MDU6SXNzdWUzMTc3NjAzNjE= | 239 | Support for hidden tables in metadata.json | 9599 | closed | 0 | 2 | 2018-04-25T19:21:17Z | 2018-04-26T03:45:12Z | 2018-04-26T03:43:10Z | OWNER | Since we already have a hidden feature, let's expose it more to our users | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/239/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
317714268 | MDU6SXNzdWUzMTc3MTQyNjg= | 238 | External metadata.json | 9599 | closed | 0 | 3 | 2018-04-25T17:02:30Z | 2019-06-24T06:52:55Z | 2019-06-24T06:52:45Z | OWNER | A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json` Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/238/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
317475156 | MDU6SXNzdWUzMTc0NzUxNTY= | 237 | Support for ?_search_colname=blah searches | 9599 | closed | 0 | 2 | 2018-04-25T04:29:53Z | 2018-05-05T22:56:42Z | 2018-05-05T22:33:23Z | OWNER | Right now the `_search=` argument searches across all fields in a full-text index, for example: https://san-francisco.datasettes.com/sf-film-locations-84594a7/Film_Locations_in_San_Francisco?_search=justin SQLite FTS also supports searches within a specified field, for example: https://san-francisco.datasettes.com/sf-film-locations-84594a7?sql=select+rowid%2C+*+from+Film_Locations_in_San_Francisco+where+rowid+in+%28select+rowid+from+%5BFilm_Locations_in_San_Francisco_fts%5D+where+%5BLocations%5D+match+%3Asearch%29+order+by+rowid+limit+101&search=justin ``` select rowid, * from Film_Locations_in_San_Francisco where rowid in ( select rowid from [Film_Locations_in_San_Francisco_fts] where [Locations] match :search ) order by rowid limit 101 ``` The `_search=` parameter could be extended to support this using `_search_colname=`. This should also be able to support columns with spaces and special characters in their names, something like this: `_search_Column%20With%20Spaces=foo` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/237/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
316323336 | MDU6SXNzdWUzMTYzMjMzMzY= | 231 | metadata.json support for plugin configuration options | 9599 | closed | 0 | 4 | 2018-04-20T15:58:47Z | 2019-05-13T18:56:21Z | 2019-05-13T18:56:21Z | OWNER | My [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin currently works by detecting `latitude` and `longitude` columns. I'd like to be able to configure it to look for different column names. One way to do this could be to support optional plugin configuration as part of `metadata.json`. Something like this: { "title": "Polar Bear Ear Tags, 2009-2011", "source": "USGS Alaska Science Center, Polar Bear Research Program", "source_url": "https://alaska.usgs.gov/products/data.php?dataid=130", "plugins": { "datasette_cluster_map": { "latitude_columns": [ "latitude", "Capture Latitude" ], "longitude_columns": [ "longitude", "Capture Longitude" ] } } } These settings should be supported at the root level or at the individual database or table level. They could also be exposed in the https://datasette-cluster-map-demo.now.sh/-/plugins debug tool. Refs #14 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |