github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
906385991 | MDU6SXNzdWU5MDYzODU5OTE= | 1349 | CSV ?_stream=on redundantly calculates facets for every page | 9599 | closed | 0 | 9 | 2021-05-29T06:11:23Z | 2021-06-17T18:12:32Z | 2021-06-01T15:52:53Z | OWNER | I'm trying to figure out why a full CSV export from https://covid-19.datasettes.com/covid/ny_times_us_counties runs unbearably slowly. It's because the streaming endpoint works by scrolling through every page, and it turns out every page calculates facets and suggested facets! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906993731 | MDU6SXNzdWU5MDY5OTM3MzE= | 1351 | Get `?_trace=1` working with CSV and streaming CSVs | 9599 | closed | 0 | 1 | 2021-05-31T03:02:15Z | 2021-06-17T18:12:32Z | 2021-06-01T15:50:09Z | OWNER | > I think it's worth getting `?_trace=1` to work with streaming CSV - this would have helped me spot this issue a long time ago. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1349#issuecomment-851133125_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
908446997 | MDU6SXNzdWU5MDg0NDY5OTc= | 1353 | ?_nocount=1 for opting out of table counts | 9599 | closed | 0 | 2 | 2021-06-01T15:53:27Z | 2021-06-01T16:18:54Z | 2021-06-01T16:17:04Z | OWNER | Running a trace against a CSV streaming export with the new `_trace=1` feature from #1351 shows that the following code is executing a `select count(*) from table` for every page of results returned: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/table.py#L700-L705 This is inefficient - a new `?_nocount=1` option would let us disable this count in the same way as #1349: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/base.py#L264-L276 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
908465747 | MDU6SXNzdWU5MDg0NjU3NDc= | 1354 | Update help in tests for latest Click | 9599 | closed | 0 | 1 | 2021-06-01T16:14:31Z | 2021-06-01T16:17:04Z | 2021-06-01T16:17:04Z | OWNER | Now that Uvicorn 0.14 is out with an unpinned Click dependency - https://github.com/encode/uvicorn/pull/1033 - our test suite runs against Click 8.0 - which subtly changes the output of `--help` causing test failures: https://github.com/simonw/datasette/runs/2720383031?check_suite_focus=true ``` def test_help_includes(name, filename): expected = (docs_path / filename).read_text() runner = CliRunner() result = runner.invoke(cli, name.split() + ["--help"], terminal_width=88) actual = f"$ datasette {name} --help\n\n{result.output}" # actual has "Usage: cli package [OPTIONS] FILES" # because it doesn't know that cli will be aliased to datasette expected = expected.replace("Usage: datasette", "Usage: cli") > assert expected == actual E AssertionError: assert '$ datasette ...e and exit.\n' == '$ datasette ...e and exit.\n' E Skipping 848 identical leading characters in diff, use -v to show E nt_id xxx E + E --version-note TEXT Additional note to show on /-/versions E --secret TEXT Secret used for signing secure values, such as signed E cookies E + E --title TEXT Title for metadata ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |