github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
268469569 | MDU6SXNzdWUyNjg0Njk1Njk= | 39 | Protect against malicious SQL that causes damage even though our DB is immutable | 9599 | closed | 0 | 2857392 | 4 | 2017-10-25T16:44:27Z | 2021-08-17T23:52:07Z | 2017-11-05T02:53:47Z | OWNER | I’m currently operating under the assumption that it’s safe to allow arbitrary SQL statements because we are dealing with an immutable database. But this might not be the case - there are some pretty weird SQLite language extensions (ATTACH, PRAGMA etc) and I’m not certain they cannot be used to break things in a way that would affect future requests to the API. Solution: provide a “safe mode” option which disables the ?sql= mechanism. This still leaves the URL filter lookups, so I need to make sure that those are “safe”. In the future I may also implement a whitelist option where datasets can be configured to only allow specific filters against specific columns. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/39/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
970626625 | MDU6SXNzdWU5NzA2MjY2MjU= | 1435 | Turn off suggest facets on tables with large numbers of columns | 9599 | open | 0 | 0 | 2021-08-13T18:30:48Z | 2021-08-13T18:30:48Z | OWNER | If a table has 200 columns it will take multiple seconds to try and suggest facets. I should either quit after the first 20 or not suggest facets at all. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1435/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
969548935 | MDU6SXNzdWU5Njk1NDg5MzU= | 1429 | UI for setting `?_size=max` on table page | 9599 | open | 0 | 2 | 2021-08-12T20:52:09Z | 2021-08-13T04:37:41Z | OWNER | It defaults to 100 per page, but you can increase that to 1000 per page using `?_size=max` (or higher if `max_returned_rows` is set higher than that). But... that's only available to people who know how to hack URLs. Solution: add a link that sets that option to the pagination block at the bottom of the table: <img width="565" alt="data__states__28_999_rows" src="https://user-images.githubusercontent.com/9599/129268119-989dd835-3e04-4baa-8e9e-28e56ad59154.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1429/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
969840302 | MDU6SXNzdWU5Njk4NDAzMDI= | 1431 | `--help-config` should be called `--help-settings` | 9599 | closed | 0 | 1 | 2021-08-13T00:46:48Z | 2021-08-13T01:01:58Z | 2021-08-13T01:01:58Z | OWNER | Follow-on from #1105 rebranding exercise. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1431/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
969758038 | MDExOlB1bGxSZXF1ZXN0NzExNzgzNjE2 | 1430 | Column metadata | 9599 | closed | 0 | 1 | 2021-08-12T23:34:39Z | 2021-08-12T23:53:23Z | 2021-08-12T23:53:23Z | OWNER | simonw/datasette/pulls/1430 | Refs #942 Still needs: - [x] Tests - [x] Documentation | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1430/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
963528457 | MDU6SXNzdWU5NjM1Mjg0NTc= | 1425 | render_cell() hook should support returning an awaitable | 9599 | closed | 0 | 11 | 2021-08-08T22:32:29Z | 2021-08-09T07:14:35Z | 2021-08-09T03:00:37Z | OWNER | Many of the plugin hooks can return an awaitable - e.g. https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-extra-template-vars - but `render_cell()` doesn't support this. I recently found myself wanting to execute an additional SQL query from that hook, but it wasn't possible to do that since I couldn't use `await`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1425/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
963527045 | MDU6SXNzdWU5NjM1MjcwNDU= | 1424 | Document exceptions that can be raised by db.execute() and friends | 9599 | open | 0 | 4 | 2021-08-08T22:23:25Z | 2021-08-08T22:27:31Z | OWNER | Not currently covered here: https://docs.datasette.io/en/stable/internals.html#await-db-execute-sql | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1424/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
961367843 | MDU6SXNzdWU5NjEzNjc4NDM= | 1422 | Ability to default to hiding the SQL for a canned query | 9599 | closed | 0 | 4 | 2021-08-05T02:51:39Z | 2021-08-07T05:32:29Z | 2021-08-07T05:32:29Z | OWNER | I'm working on a project with some HUGE (400+ lines of SQL) canned queries right now. Any time you land on the canned query page you have to scroll down a long distance to get to the results! Would be useful to be able to default to https://latest.datasette.io/fixtures/magic_parameters?_hide_sql=1 without needing the parameter. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1422/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
959898166 | MDU6SXNzdWU5NTk4OTgxNjY= | 1420 | `datasette publish cloudrun --cpu X` option | 9599 | closed | 0 | 5 | 2021-08-04T05:04:31Z | 2021-08-05T00:54:59Z | 2021-08-04T05:33:48Z | OWNER | For setting the number of vCPUs - current valid values are 1, 2 or 4: https://cloud.google.com/run/docs/configuring/cpu Pass that through to `gcloud run deploy --image IMAGE_URL --cpu CPU` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1420/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
959278472 | MDU6SXNzdWU5NTkyNzg0NzI= | 1417 | Use codespell in CI to spot spelling errors | 9599 | closed | 0 | 1 | 2021-08-03T16:14:15Z | 2021-08-03T16:36:40Z | 2021-08-03T16:36:40Z | OWNER | I noticed Rich is using this: https://github.com/willmcgugan/rich/commit/9c12a4537499797c43725fff5276ef0da62423ef#diff-ce84a1b2c9eb4ab3ea22f610cad7111cb9a2f66365c3b24679901376a2a73ab2 Ran it against the Datasette docs and found a bunch of obvious fixes, surprisingly with no false positives. ``` datasette % codespell docs/*.rst docs/authentication.rst:63: perfom ==> perform docs/authentication.rst:76: perfom ==> perform docs/changelog.rst:429: repsonse ==> response docs/changelog.rst:503: permissons ==> permissions docs/changelog.rst:717: compatibilty ==> compatibility docs/changelog.rst:1172: browseable ==> browsable docs/deploying.rst:191: similiar ==> similar docs/internals.rst:434: Respons ==> Response, respond docs/internals.rst:440: Respons ==> Response, respond docs/internals.rst:717: tha ==> than, that, the docs/performance.rst:42: databse ==> database docs/plugin_hooks.rst:667: utilites ==> utilities docs/publish.rst:168: countainer ==> container docs/settings.rst:352: inalid ==> invalid docs/sql_queries.rst:406: preceeded ==> preceded, proceeded ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1417/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
959284434 | MDExOlB1bGxSZXF1ZXN0NzAyNDIyMjYz | 1418 | Spelling corrections plus CI job for codespell | 9599 | closed | 0 | 2 | 2021-08-03T16:21:19Z | 2021-08-03T16:36:39Z | 2021-08-03T16:36:38Z | OWNER | simonw/datasette/pulls/1418 | Refs #1417. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1418/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
959276629 | MDU6SXNzdWU5NTkyNzY2Mjk= | 1416 | Use rich to render tracebacks on errors, if available | 9599 | closed | 0 | 0 | 2021-08-03T16:12:08Z | 2021-08-03T16:12:51Z | 2021-08-03T16:12:51Z | OWNER | > Now thinking I should try adding Rich as an optional dependency to Datasette - if it's there, show tracebacks using it. Could be really handy for development > https://twitter.com/simonw/status/1422576091055616003 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1416/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
810394616 | MDU6SXNzdWU4MTAzOTQ2MTY= | 1227 | Configure sphinx.ext.extlinks for issues | 9599 | closed | 0 | 1 | 2021-02-17T17:38:02Z | 2021-08-02T21:38:39Z | 2021-02-18T01:20:33Z | OWNER | Spotted this in the aspw documentation: https://github.com/rogerbinns/apsw/blob/3.34.0-r1/doc/conf.py#L29-L36 ```python extlinks={ 'cvstrac': ('https://sqlite.org/cvstrac/tktview?tn=%s', 'SQLite ticket #'), 'sqliteapi': ('https://sqlite.org/c3ref/%s.html', 'XXYouShouldNotSeeThisXX'), 'issue': ('https://github.com/rogerbinns/apsw/issues/%s', 'APSW issue '), 'source': ('https://github.com/rogerbinns/apsw/blob/master/%s', ''), } ``` Which lets you link to issues like this: :issue:`268` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1227/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957345476 | MDU6SXNzdWU5NTczNDU0NzY= | 1411 | Canned query ?sql= is pointlessly echoed in query string starting from hidden mode | 9599 | closed | 0 | 1 | 2021-08-01T00:17:13Z | 2021-08-01T03:27:30Z | 2021-08-01T00:58:17Z | OWNER | Example: https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 Submitting that form again results in this: https://latest.datasette.io/fixtures/neighborhood_search?sql=%0D%0Aselect+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities%0D%0A++++++++on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B%0D%0A&_hide_sql=1&text=cork Because the HTML on https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 includes this: ```html <h3>Custom SQL query returning 1 row <span class="show-hide-sql">(<a href="/fixtures/neighborhood_search?text=cork">show</a>)</span></h3> <input type="hidden" name="sql" value=" select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood; "> <input type="hidden" name="_hide_sql" value="1"> ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1411/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957347860 | MDU6SXNzdWU5NTczNDc4NjA= | 1412 | Mention WAL mode in the documentation (plus backup tips) | 9599 | open | 0 | 0 | 2021-08-01T00:27:11Z | 2021-08-01T00:27:11Z | OWNER | This is useful for people who are deploying Datasette with any write functionality, especially if they might be using something like EFS. I can add a section about this to the bottom of https://docs.datasette.io/en/stable/deploying.html and then link to that section from both https://docs.datasette.io/en/stable/sql_queries.html#writable-canned-queries and https://docs.datasette.io/en/stable/internals.html#await-db-execute-write-sql-params-none-block-false Also useful: mention that just copying a SQLite database file while it is being written to may not get a consistent file, so tell people to use one of the SQLite backup mechanisms instead. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1412/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
957315684 | MDU6SXNzdWU5NTczMTU2ODQ= | 1410 | Rename settings to `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream` | 9599 | open | 0 | 3268330 | 0 | 2021-07-31T20:27:12Z | 2021-07-31T20:27:49Z | OWNER | > If I was prone to over-thinking (which I am) I'd note that `allow_facet` and `allow_download` and `allow_csv_stream` are all settings that do NOT have an equivalent in the newer permissions system, which is itself a little weird and inconsistent. > > So maybe there's a future task where I introduce those as both permissions and metadata `"allow_x"` blocks, then rename the settings themselves to be called `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`. > > If I was going to do that I should get it in before Datasette 1.0. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1409#issuecomment-890400425_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1410/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
957298475 | MDU6SXNzdWU5NTcyOTg0NzU= | 1407 | OSError: AF_UNIX path too long in ds_unix_domain_socket_server | 9599 | closed | 0 | 2 | 2021-07-31T18:36:06Z | 2021-07-31T19:03:44Z | 2021-07-31T19:03:44Z | OWNER | Got this exception while working on #1406. ``` @pytest.fixture(scope="session") def ds_unix_domain_socket_server(tmp_path_factory): socket_folder = tmp_path_factory.mktemp("uds") uds = str(socket_folder / "datasette.sock") ds_proc = subprocess.Popen( ["datasette", "--memory", "--uds", uds], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=tempfile.gettempdir(), ) # Give the server time to start time.sleep(1.5) # Check it started successfully > assert not ds_proc.poll(), ds_proc.stdout.read().decode("utf-8") E AssertionError: INFO: Started server process [48453] E INFO: Waiting for application startup. E INFO: Application startup complete. E Traceback (most recent call last): E File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette", line 33, in <module> E sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')()) E File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py", line 1137, in __call__ E return self.main(*args, **kwargs) E File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py", line 1062, in main E rv = self.invoke(ctx) E File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py", line 1668, in invoke E return _process_result(sub_ctx.command.invoke(sub_ctx)) E File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py", line 1404, in invoke E return ctx.invoke(self.callback, **ctx.params) E File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py", line 763, in invoke E return __callback(*args, **kwargs) E … | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1407/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
957302085 | MDU6SXNzdWU5NTczMDIwODU= | 1408 | Review places in codebase that use os.chdir(), in particularly relating to tests | 9599 | open | 0 | 2 | 2021-07-31T18:57:06Z | 2021-07-31T19:00:32Z | OWNER | > To clarify: the core problem here is that an error is thrown any time you call `os.getcwd()` but the directory you are currently in has been deleted. > > `runner.isolated_filesystem()` assumes that the current directory in has not been deleted. But the various temporary directory utilities in `pytest` work by creating directories and then deleting them. > > Maybe there's a larger problem here that I play a bit fast and loose with `os.chdir()` in both the test suite and in various lines of code in Datasette itself (in particular in the publish commands)? _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1406#issuecomment-890390198_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1408/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
956303470 | MDU6SXNzdWU5NTYzMDM0NzA= | 1406 | Tests failing with FileNotFoundError in runner.isolated_filesystem | 9599 | closed | 0 | 8 | 2021-07-30T00:39:00Z | 2021-07-31T18:56:35Z | 2021-07-31T18:56:35Z | OWNER | e.g. https://github.com/simonw/datasette/runs/3197141955 I've seen this error before, but I don't yet have a good workaround for it. ``` @contextlib.contextmanager def isolated_filesystem( self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None ) -> t.Iterator[str]: """A context manager that creates a temporary directory and changes the current working directory to it. This isolates tests that affect the contents of the CWD to prevent them from interfering with each other. :param temp_dir: Create the temporary directory under this directory. If given, the created directory is not removed when exiting. .. versionchanged:: 8.0 Added the ``temp_dir`` parameter. """ > cwd = os.getcwd() E FileNotFoundError: [Errno 2] No such file or directory /opt/hostedtoolcache/Python/3.6.14/x64/lib/python3.6/site-packages/click/testing.py:466: FileNotFoundError =========================== short test summary info ============================ FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_apt_get_install FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[---setting force_https_urls on] FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[--setting base_url /foo---setting base_url /foo --setting force_https_urls on] FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[--setting force_https_urls off---setting force_https_urls off] FAILED tests/test_publish_heroku.py::test_publish_heroku_requires_heroku - Fi... FAILED tests/test_publish_heroku.py::test_publish_heroku_installs_plugin - Fi... FAILED tests/test_publish_heroku.py::test_publish_heroku - FileNotFoundError:... FAILED tests/test_publish_heroku.py::test_publish_heroku_plugin_secrets - Fil... ================== 8 failed, 920 passed in 188.22s (0:03:08) =================== ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1406/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
947044667 | MDU6SXNzdWU5NDcwNDQ2Njc= | 1398 | Documentation on using Datasette as a library | 9599 | open | 0 | 1 | 2021-07-18T14:15:27Z | 2021-07-30T03:21:49Z | OWNER | Instantiating `Datasette()` directly is an increasingly interesting pattern. I do it in tests all the time, but thanks to `datasette.client` there are plenty of neat things you can do with it in a library context. Maybe support `from datasette import Datasette` for this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1398/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
955316250 | MDU6SXNzdWU5NTUzMTYyNTA= | 1405 | utils.parse_metadata() should be a documented internal function | 9599 | closed | 0 | 3 | 2021-07-28T23:51:39Z | 2021-07-29T23:33:30Z | 2021-07-29T23:30:24Z | OWNER | Because it's used by this plugin: https://github.com/simonw/datasette-remote-metadata | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1405/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
953352015 | MDU6SXNzdWU5NTMzNTIwMTU= | 1404 | `register_routes()` hook should take `datasette` argument | 9599 | closed | 0 | 1 | 2021-07-26T23:00:33Z | 2021-07-26T23:27:07Z | 2021-07-26T23:26:00Z | OWNER | Currently that plugin hook takes no arguments at all. This means it's not possible to conditionally register routes based on Datasette plugin configuration. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1404/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
275125561 | MDU6SXNzdWUyNzUxMjU1NjE= | 123 | Datasette serve should accept paths/URLs to CSVs and other file formats | 9599 | open | 0 | 9 | 2017-11-19T02:05:48Z | 2021-07-19T00:04:32Z | OWNER | This would remove the csvs-to-sqlite step which I end up using for almost everything. I'm hesitant to introduce pandas as a required dependency though since it require compiling numpy. Could build it so this option is only available if you have pandas installed. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/123/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
792652391 | MDU6SXNzdWU3OTI2NTIzOTE= | 1199 | Experiment with PRAGMA mmap_size=N | 9599 | open | 0 | 2 | 2021-01-23T21:24:09Z | 2021-07-17T17:39:17Z | OWNER | https://sqlite.org/mmap.html - SQLite supports memory-mapped I/O but it's disabled by default. The `PRAGMA mmap_size=N` option can be used to enable it. It would be very interesting to understand the impact this could have on Datasette performance for various different shapes of data. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1199/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
946553953 | MDExOlB1bGxSZXF1ZXN0NjkxNzA3NDA5 | 1397 | Fix for race condition in refresh_schemas(), closes #1231 | 9599 | closed | 0 | 0 | 2021-07-16T19:44:43Z | 2021-07-16T19:45:00Z | 2021-07-16T19:44:58Z | OWNER | simonw/datasette/pulls/1397 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1397/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
811367257 | MDU6SXNzdWU4MTEzNjcyNTc= | 1231 | Race condition errors in new refresh_schemas() mechanism | 9599 | closed | 0 | 11 | 2021-02-18T18:49:54Z | 2021-07-16T19:44:59Z | 2021-07-16T19:44:59Z | OWNER | I tried running a Locust load test against Datasette and hit an error message about a failure to create tables because they already existed. I think this means there are race conditions in the new `refresh_schemas()` mechanism added in #1150. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1231/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
944870799 | MDU6SXNzdWU5NDQ4NzA3OTk= | 1394 | Big performance boost on faceting: skip the inner order by | 9599 | closed | 0 | 4 | 2021-07-14T23:32:29Z | 2021-07-16T02:23:32Z | 2021-07-15T00:05:50Z | OWNER | I just noticed something that could make for a huge performance improvement in faceting. The default query used by Datasette when faceting looks like this: ```sql select country_long, count(*) from ( select * from [global-power-plants] order by rowid ) where country_long is not null group by country_long order by count(*) desc ``` Here it takes 53ms: https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D+order+by+rowid%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc Note that there's a `order by rowid` in there which isn't necessary - the order on that inner query doesn't matter since we're grouping and counting. I had assumed SQLite would optimize this away - but it turns out it doesn't! Consider this version of the query, with that pointless order by removed: ``` select country_long, count(*) from ( select * from [global-power-plants] ) where country_long is not null group by country_long order by count(*) desc ``` https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc runs in 7.2ms! I tried this optimization on a table with 2.5m rows in it - without the optimization it took 5 seconds, with the optimization it took 450ms. So this is a very significant improvement! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1394/reactions", "total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
456578474 | MDU6SXNzdWU0NTY1Nzg0NzQ= | 511 | Get Datasette tests passing on Windows in GitHub Actions | 9599 | open | 0 | 13 | 2019-06-15T21:41:58Z | 2021-07-11T17:23:05Z | OWNER | This should almost happen as a side-effect or moving from Sanic to Uvicorn during the port to ASGI: #272 Additional steps: - test it manually - update documentation - set up some form of Windows CI | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/511/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
466996584 | MDExOlB1bGxSZXF1ZXN0Mjk2NzM1MzIw | 557 | Get tests running on Windows using Travis CI | 9599 | closed | 0 | 4 | 2019-07-11T16:36:57Z | 2021-07-10T23:39:48Z | 2021-07-10T23:39:48Z | OWNER | simonw/datasette/pulls/557 | Refs #511 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/557/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
941300946 | MDU6SXNzdWU5NDEzMDA5NDY= | 1391 | Stop using generated columns in fixtures.db | 9599 | closed | 0 | 5 | 2021-07-10T18:26:11Z | 2021-07-10T19:26:58Z | 2021-07-10T19:26:00Z | OWNER | Refs #1376 - but I also keep running into this myself, where I try to run something against `fixtures.db` and get this confusing error: sqlite3.DatabaseError: malformed database schema (generated_columns) - near "AS": syntax error I'm going to stop using generated columns in `fixtures.db` and instead dynamically generate the generated column table for the duration of the relevant test. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1391/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
940077168 | MDU6SXNzdWU5NDAwNzcxNjg= | 1389 | "searchmode": "raw" in table metadata | 9599 | closed | 0 | 6 | 2021-07-08T17:32:10Z | 2021-07-10T18:33:13Z | 2021-07-10T18:33:13Z | OWNER | > http://localhost:8001/index/summary?_search=language%3Aeng&_sort=title&_searchmode=raw > > But I'm not able to manage it in the metadata file. Here is mine (note that the sort column is taken into account) > Here it is: > > ``` > { > "databases": { > "index": { > "tables": { > "summary": { > "sort": "title", > "searchmode": "raw" > } > } > } > } > } _Originally posted by @Krazybug in https://github.com/simonw/datasette/issues/759#issuecomment-624860451_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1389/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
940891698 | MDU6SXNzdWU5NDA4OTE2OTg= | 1390 | Mention restarting systemd in documentation | 9599 | closed | 0 | 2 | 2021-07-09T16:05:15Z | 2021-07-09T16:32:57Z | 2021-07-09T16:32:33Z | OWNER | https://docs.datasette.io/en/stable/deploying.html#running-datasette-using-systemd Need to clarify that if you add a new database or change metadata you need to restart systemd. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1390/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
935930820 | MDU6SXNzdWU5MzU5MzA4MjA= | 1387 | absolute_url() behind a proxy assembles incorrect http://127.0.0.1:8001/ URLs | 9599 | closed | 0 | 8 | 2021-07-02T16:58:25Z | 2021-07-02T17:58:23Z | 2021-07-02T17:33:05Z | OWNER | Reported in the wild on https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-3d4a4b7/bib?_facet=bib_level_callnumber - the "next page" link links to https://127.0.0.1:8010/collection-analysis/current_collection-3d4a4b7/bib?_facet=bib_level_callnumber&_next=100 That installation uses `"base_url": "/collection-analysis/"` Weirdly all of the other links on that page - to facet results, sort orders, row permalinks etc - work fine. It's JUST the `next_url` one that is broken. Also broken in their JSON: https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-3d4a4b7/bib.json?_size=1 returns ```json "suggested_facets": [], "next": "1", "next_url": "https://127.0.0.1:8010/collection-analysis/current_collection-3d4a4b7/bib.json?_size=1&_next=1", "private": false, ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1387/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
920884085 | MDU6SXNzdWU5MjA4ODQwODU= | 1377 | Mechanism for plugins to exclude certain paths from CSRF checks | 9599 | closed | 0 | 3 | 2021-06-15T00:48:20Z | 2021-06-23T22:51:33Z | 2021-06-23T22:51:33Z | OWNER | I need this for a plugin I'm building that offers a POST API. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1377/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
925491857 | MDU6SXNzdWU5MjU0OTE4NTc= | 1383 | Improve test coverage for `inspect.py` | 9599 | open | 0 | 0 | 2021-06-20T00:22:43Z | 2021-06-20T00:22:49Z | OWNER | https://codecov.io/gh/simonw/datasette/src/main/datasette/inspect.py shows only 36% coverage for that module at the moment. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1383/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
268176505 | MDU6SXNzdWUyNjgxNzY1MDU= | 34 | Support CSV export with a .csv extension | 9599 | closed | 0 | 1 | 2017-10-24T20:34:43Z | 2021-06-17T18:14:48Z | 2018-05-28T20:45:34Z | OWNER | Maybe do this using streaming with multiple pagination SQL queries so we can support arbritrarily large exports. How would this work against a view which doesn’t have an obvious efficient pagination mechanism? Maybe limit views to up to 1000 exported records? Relates to #5 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/34/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323681589 | MDU6SXNzdWUzMjM2ODE1ODk= | 266 | Export to CSV | 9599 | closed | 0 | 27 | 2018-05-16T15:50:24Z | 2021-06-17T18:14:24Z | 2018-06-18T06:05:25Z | OWNER | Datasette needs to be able to export data to CSV. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/266/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
333000163 | MDU6SXNzdWUzMzMwMDAxNjM= | 312 | HTML, CSV and JSON views should support ?_col=&_col= | 9599 | closed | 0 | 1 | 2018-06-16T16:53:35Z | 2021-06-17T18:14:24Z | 2018-06-16T17:00:12Z | OWNER | To support whitelisting columns to display. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
335141434 | MDU6SXNzdWUzMzUxNDE0MzQ= | 326 | CSV should respect --cors and return cors headers | 9599 | closed | 0 | 1 | 2018-06-24T00:44:07Z | 2021-06-17T18:14:24Z | 2018-06-24T00:59:45Z | OWNER | Otherwise tools like Vega can't load data via CSV. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/326/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
725184645 | MDU6SXNzdWU3MjUxODQ2NDU= | 1034 | Better way of representing binary data in .csv output | 9599 | closed | 0 | 6026070 | 19 | 2020-10-20T04:28:58Z | 2021-06-17T18:13:21Z | 2020-10-29T22:47:46Z | OWNER | I just noticed this: https://latest.datasette.io/fixtures/binary_data.csv ```csv rowid,data 1,b'\x15\x1c\x02\xc7\xad\x05\xfe' 2,b'\x15\x1c\x03\xc7\xad\x05\xfe' ``` There's no good way to represent binary data in a CSV file, but this seems like one of the more-bad options. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1034/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
503190241 | MDU6SXNzdWU1MDMxOTAyNDE= | 584 | Codec error in some CSV exports | 9599 | closed | 0 | 2 | 2019-10-07T01:15:34Z | 2021-06-17T18:13:20Z | 2019-10-18T05:23:16Z | OWNER | Got this exploring my Swarm checkins: ![448DBFC4-71F8-4846-83C0-BEA511B2157A](https://user-images.githubusercontent.com/9599/66279259-3af53480-e865-11e9-9651-04fd2d895392.jpeg) `/swarm/stickers.csv?stickerType=messageOnly&_size=max` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/584/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516748849 | MDU6SXNzdWU1MTY3NDg4NDk= | 612 | CSV export is broken for tables with null foreign keys | 9599 | closed | 0 | 2 | 2019-11-02T22:52:47Z | 2021-06-17T18:13:20Z | 2019-11-02T23:12:53Z | OWNER | Following on from #406 - this CSV export appears to be broken: https://14da705.datasette.io/fixtures/foreign_key_references.csv?_labels=on&_size=max ```csv pk,foreign_key_with_label,foreign_key_with_label_label,foreign_key_with_no_label,foreign_key_with_no_label_label 1,1,hello,1,1 2,, ``` That second row should have 5 values, but it only has 4. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/612/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
775666296 | MDU6SXNzdWU3NzU2NjYyOTY= | 1160 | "datasette insert" command and plugin hook | 9599 | open | 0 | 23 | 2020-12-29T02:37:03Z | 2021-06-17T18:12:32Z | OWNER | Tools for loading data into Datasette currently mostly exist as separate utilities - `yaml-to-sqlite` and `csvs-to-sqlite` and suchlike. Bringing these into Datasette could have some interesting properties: - A `datasette insert` command could be extended with plugins to handle more formats - Any format that can be inserted on the command-line could also be inserted using a web UI or web API - which would benefit from new format plugin hooks - If Datasette ever grows beyond SQLite (see #670) a built-in import mechanism could work for those other databases as well - without me needing to write `yaml-to-postgresql` and suchlike | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1160/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128269 | MDU6SXNzdWU3NzYxMjgyNjk= | 1162 | First working version of "datasette insert data.db file.csv" | 9599 | open | 0 | 0 | 2020-12-29T23:20:11Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1162/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
776128565 | MDU6SXNzdWU3NzYxMjg1NjU= | 1163 | "datasette insert data.db url-to-csv" | 9599 | open | 0 | 1 | 2020-12-29T23:21:21Z | 2021-06-17T18:12:32Z | OWNER | Refs #1160 - get filesystem imports working first for #1162, then add import-from-URL. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1163/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
906385991 | MDU6SXNzdWU5MDYzODU5OTE= | 1349 | CSV ?_stream=on redundantly calculates facets for every page | 9599 | closed | 0 | 9 | 2021-05-29T06:11:23Z | 2021-06-17T18:12:32Z | 2021-06-01T15:52:53Z | OWNER | I'm trying to figure out why a full CSV export from https://covid-19.datasettes.com/covid/ny_times_us_counties runs unbearably slowly. It's because the streaming endpoint works by scrolling through every page, and it turns out every page calculates facets and suggested facets! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906993731 | MDU6SXNzdWU5MDY5OTM3MzE= | 1351 | Get `?_trace=1` working with CSV and streaming CSVs | 9599 | closed | 0 | 1 | 2021-05-31T03:02:15Z | 2021-06-17T18:12:32Z | 2021-06-01T15:50:09Z | OWNER | > I think it's worth getting `?_trace=1` to work with streaming CSV - this would have helped me spot this issue a long time ago. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1349#issuecomment-851133125_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
736365306 | MDU6SXNzdWU3MzYzNjUzMDY= | 1083 | Advanced CSV export for arbitrary queries | 9599 | open | 0 | 2 | 2020-11-04T19:23:05Z | 2021-06-17T18:12:31Z | OWNER | There's no link to download the CSV file - the table page has that as an advanced export option, but this is missing from the query page. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1083/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
743359646 | MDU6SXNzdWU3NDMzNTk2NDY= | 1096 | TSV should be a default export option | 9599 | open | 0 | 1 | 2020-11-15T22:24:02Z | 2021-06-17T18:12:31Z | OWNER | Refs #1095 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1096/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
759695780 | MDU6SXNzdWU3NTk2OTU3ODA= | 1133 | Option to omit header row in CSV export | 9599 | closed | 0 | 2 | 2020-12-08T18:54:46Z | 2021-06-17T18:12:31Z | 2020-12-10T23:28:51Z | OWNER | `?_header=off` - for symmetry with existing option `?_nl=on`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1133/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763361458 | MDU6SXNzdWU3NjMzNjE0NTg= | 1142 | "Stream all rows" is not at all obvious | 9599 | open | 0 | 9 | 2020-12-12T06:24:57Z | 2021-06-17T18:12:31Z | OWNER | Got a question about how to download all rows - the current option isn't at all clear. <img width="668" alt="loans__ppp_loans__9_511_rows_where_where_search_matches__tech__sorted_by_rowid" src="https://user-images.githubusercontent.com/9599/101977057-ac660b00-3bff-11eb-88f4-c93ffd03d3e0.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1142/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
732685643 | MDU6SXNzdWU3MzI2ODU2NDM= | 1063 | .csv should link to .blob downloads | 9599 | closed | 0 | 6026070 | 3 | 2020-10-29T21:45:58Z | 2021-06-17T18:12:30Z | 2020-10-29T22:47:45Z | OWNER | - [x] Update `.csv` output to link to these things (and get that `xfail` test to pass) - ~~Add a `.csv?_blob_base64=1` argument that causes them to be output in base64 in the CSV~~ > Moving the CSV work to a separate ticket. _Originally posted by @simonw in https://github.com/simonw/datasette/pull/1061#issuecomment-719042601_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1063/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
924203783 | MDU6SXNzdWU5MjQyMDM3ODM= | 1379 | Idea: ?_end=1 option for streaming CSV responses | 9599 | open | 0 | 0 | 2021-06-17T18:11:21Z | 2021-06-17T18:11:30Z | OWNER | As discussed in this thread: https://twitter.com/simonw/status/1405554676993433605 - one of the disadvantages of Datasette's streaming CSV feature is that it's hard to tell if you got the whole file or if the connection ended early - or if an error occurred. Idea: offer an optional `?_end=1` parameter which, if enabled, adds a single row to the end of the CSV file that looks like this: `END,,,,,,,,,` For however many columns the CSV file usually has. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1379/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
915455228 | MDU6SXNzdWU5MTU0NTUyMjg= | 1371 | Menu plugin hooks should include the request | 9599 | closed | 0 | 1 | 2021-06-08T20:23:35Z | 2021-06-10T04:46:01Z | 2021-06-10T04:46:01Z | OWNER | https://docs.datasette.io/en/stable/plugin_hooks.html#menu-links-datasette-actor - `menu_links(datasette, actor)` - `table_actions(datasette, actor, database, table)` - `database_actions(datasette, actor, database)` All three of these should optionally also accept the `request` object. This would allow them to take into account additional cookies, `Authorization` headers or the current request URL (including the domain/subdomain) - or even access `request.scope` for extra context that might have been passed down from ASGI middleware. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1371/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
915488244 | MDU6SXNzdWU5MTU0ODgyNDQ= | 1372 | Add section to "writing plugins" about security, e.g. avoiding XSS | 9599 | open | 0 | 0 | 2021-06-08T20:49:33Z | 2021-06-08T20:49:46Z | OWNER | https://docs.datasette.io/en/stable/writing_plugins.html should have tips on writing secure plugins. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1372/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
913900374 | MDU6SXNzdWU5MTM5MDAzNzQ= | 1369 | Don't show foreign key IDs twice if no label | 9599 | open | 0 | 1 | 2021-06-07T19:47:02Z | 2021-06-07T19:47:24Z | OWNER | ![B5B54D94-A768-4544-A88D-CDCAB417CD3C](https://user-images.githubusercontent.com/9599/121078979-6e9d0600-c78e-11eb-8b70-20e6d29b48b1.jpeg) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1369/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
913823889 | MDU6SXNzdWU5MTM4MjM4ODk= | 1367 | Navigation menu display bug | 9599 | closed | 0 | 1 | 2021-06-07T18:18:08Z | 2021-06-07T18:24:19Z | 2021-06-07T18:24:19Z | OWNER | With Datasette 0.57 the navigation menu looks like this: <img width="991" alt="Datasette_Fixtures___memory___internal__fixtures__extra_database" src="https://user-images.githubusercontent.com/9599/121069137-ffb9b000-c781-11eb-962a-b250a2772590.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1367/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
913809802 | MDU6SXNzdWU5MTM4MDk4MDI= | 1366 | Get rid of this `restore_working_directory` hack entirely | 9599 | open | 0 | 2 | 2021-06-07T18:01:21Z | 2021-06-07T18:03:03Z | OWNER | > That seems to have fixed it. I'd love to get rid of this `restore_working_directory` hack entirely. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1361#issuecomment-855308811_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1366/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
912959264 | MDU6SXNzdWU5MTI5NTkyNjQ= | 1364 | Don't truncate columns on the list of databases | 9599 | closed | 0 | 0 | 2021-06-06T22:01:56Z | 2021-06-06T22:07:50Z | 2021-06-06T22:07:50Z | OWNER | https://covid-19.datasettes.com/covid currently truncates at 9 database columns: <img width="860" alt="covid" src="https://user-images.githubusercontent.com/9599/120941536-11467d80-c6d8-11eb-970a-ce469623f92c.png"> Django SQL Dashboard showed me that this is a bad idea - having the full list of columns is actually really useful documentation for crafting custom SQL queries. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1364/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
325958506 | MDU6SXNzdWUzMjU5NTg1MDY= | 283 | Support cross-database joins | 9599 | closed | 0 | 26 | 2018-05-24T04:18:39Z | 2021-06-06T09:40:18Z | 2021-02-18T22:16:46Z | OWNER | SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases. Since Datasette supports more than one database, this would make a pretty neat feature. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/283/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
912485040 | MDU6SXNzdWU5MTI0ODUwNDA= | 1361 | Intermittent CI failure: restore_working_directory FileNotFoundError | 9599 | closed | 0 | 4 | 2021-06-05T22:48:13Z | 2021-06-05T23:16:24Z | 2021-06-05T23:16:24Z | OWNER | e.g. in https://github.com/simonw/datasette/runs/2754772233 - this is an intermittent error: ``` __________ ERROR at setup of test_hook_register_routes_render_message __________ [gw0] linux -- Python 3.8.10 /opt/hostedtoolcache/Python/3.8.10/x64/bin/python tmpdir = local('/tmp/pytest-of-runner/pytest-0/popen-gw0/test_hook_register_routes_rend0') request = <SubRequest 'restore_working_directory' for <Function test_hook_register_routes_render_message>> @pytest.fixture def restore_working_directory(tmpdir, request): > previous_cwd = os.getcwd() E FileNotFoundError: [Errno 2] No such file or directory ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1361/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
912464443 | MDU6SXNzdWU5MTI0NjQ0NDM= | 1360 | Security flaw, to be fixed in 0.56.1 and 0.57 | 9599 | closed | 0 | 2 | 2021-06-05T21:53:51Z | 2021-06-05T22:23:23Z | 2021-06-05T22:22:06Z | OWNER | See security advisory here for details: https://github.com/simonw/datasette/security/advisories/GHSA-xw7c-jx9m-xh5g - the `?_trace=1` debugging option was not correctly escaping its JSON output, resulting in a [reflected cross-site scripting](https://owasp.org/www-community/attacks/xss/#reflected-xss-attacks) vulnerability. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1360/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
912418094 | MDU6SXNzdWU5MTI0MTgwOTQ= | 1358 | Release Datasette 0.57 | 9599 | closed | 0 | 3 | 2021-06-05T19:56:13Z | 2021-06-05T22:20:07Z | 2021-06-05T22:20:07Z | OWNER | Need release notes. Changes are here: https://github.com/simonw/datasette/compare/0.56...368aa5f1b16ca35f82d90ff747023b9a2bfa27c1 Partial release notes already exist for the two alphas, https://github.com/simonw/datasette/releases/tag/0.57a0 and https://github.com/simonw/datasette/releases/tag/0.57a1 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1358/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
912419349 | MDU6SXNzdWU5MTI0MTkzNDk= | 1359 | `?_trace=1` should only be available with a new `trace_debug` setting | 9599 | closed | 0 | 0 | 2021-06-05T19:59:27Z | 2021-06-05T20:18:46Z | 2021-06-05T20:18:46Z | OWNER | Just like template debug mode is controlled by this off-by-default setting: https://github.com/simonw/datasette/blob/368aa5f1b16ca35f82d90ff747023b9a2bfa27c1/datasette/app.py#L160-L164 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
912394511 | MDExOlB1bGxSZXF1ZXN0NjYyNTU3MjQw | 1357 | Make custom pages compatible with base_url setting | 9599 | closed | 0 | 1 | 2021-06-05T18:54:39Z | 2021-06-05T18:59:54Z | 2021-06-05T18:59:54Z | OWNER | simonw/datasette/pulls/1357 | Refs #1238. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
904537568 | MDExOlB1bGxSZXF1ZXN0NjU1Njg0NDc3 | 1346 | Re-display user's query with an error message if an error occurs | 9599 | closed | 0 | 3 | 2021-05-28T02:04:20Z | 2021-06-02T03:46:21Z | 2021-06-02T03:46:21Z | OWNER | simonw/datasette/pulls/1346 | Refs #619 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1346/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
828811618 | MDU6SXNzdWU4Mjg4MTE2MTg= | 1257 | Table names containing single quotes break things | 9599 | closed | 0 | 2 | 2021-03-11T06:29:38Z | 2021-06-02T03:28:29Z | 2021-06-02T03:28:29Z | OWNER | e.g. I found a table called `Yesterday's ELRs by County` It threw an error inside the `detect_fts()` function attempting to run this SQL query: ```sql select name from sqlite_master where rootpage = 0 and ( sql like '%VIRTUAL TABLE%USING FTS%content="Yesterday's ELRs by County"%' or sql like '%VIRTUAL TABLE%USING FTS%content=[Yesterday's ELRs by County]%' or ( tbl_name = "Yesterday's ELRs by County" and sql like '%VIRTUAL TABLE%USING FTS%' ) ) ``` Here's the code at fault: https://github.com/simonw/datasette/blob/640ac7071b73111ba4423812cd683756e0e1936b/datasette/utils/__init__.py#L534-L548 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1257/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
800669347 | MDU6SXNzdWU4MDA2NjkzNDc= | 1216 | /-/databases should reflect connection order, not alphabetical order | 9599 | closed | 0 | 1 | 2021-02-03T20:20:23Z | 2021-06-02T03:10:19Z | 2021-06-02T03:10:19Z | OWNER | The order in which databases are attached to Datasette matters - it affects the homepage, and it's beginning to influence how certain plugins work (see https://github.com/simonw/datasette-tiles/issues/8). Two years ago in cccea85be6aaaeadb31f3b588ec7f732628815f5 I made `/-/databases` return things in alphabetical order, to fix a test failure in Python 3.5. Python 3.5 is no longer supported, so this is no longer necessary - and this behaviour should now be treated as a bug. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1216/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
323671577 | MDU6SXNzdWUzMjM2NzE1Nzc= | 263 | Facets should not execute for ?shape=array|object | 9599 | closed | 0 | 3 | 2018-05-16T15:26:13Z | 2021-06-02T02:54:34Z | 2021-06-02T02:54:34Z | OWNER | Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/263/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
906977719 | MDU6SXNzdWU5MDY5Nzc3MTk= | 1350 | ?_nofacets=1 query string argument for disabling facets and suggested facets | 9599 | closed | 0 | 2 | 2021-05-31T02:22:29Z | 2021-06-01T16:19:38Z | 2021-05-31T02:39:18Z | OWNER | This is needed as an internal option for #1349. `datasette-graphql` can benefit from this too - maybe can even use it so that if you pass `?_shape=array` it gets automatically added, fixing #263. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
908446997 | MDU6SXNzdWU5MDg0NDY5OTc= | 1353 | ?_nocount=1 for opting out of table counts | 9599 | closed | 0 | 2 | 2021-06-01T15:53:27Z | 2021-06-01T16:18:54Z | 2021-06-01T16:17:04Z | OWNER | Running a trace against a CSV streaming export with the new `_trace=1` feature from #1351 shows that the following code is executing a `select count(*) from table` for every page of results returned: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/table.py#L700-L705 This is inefficient - a new `?_nocount=1` option would let us disable this count in the same way as #1349: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/base.py#L264-L276 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
908465747 | MDU6SXNzdWU5MDg0NjU3NDc= | 1354 | Update help in tests for latest Click | 9599 | closed | 0 | 1 | 2021-06-01T16:14:31Z | 2021-06-01T16:17:04Z | 2021-06-01T16:17:04Z | OWNER | Now that Uvicorn 0.14 is out with an unpinned Click dependency - https://github.com/encode/uvicorn/pull/1033 - our test suite runs against Click 8.0 - which subtly changes the output of `--help` causing test failures: https://github.com/simonw/datasette/runs/2720383031?check_suite_focus=true ``` def test_help_includes(name, filename): expected = (docs_path / filename).read_text() runner = CliRunner() result = runner.invoke(cli, name.split() + ["--help"], terminal_width=88) actual = f"$ datasette {name} --help\n\n{result.output}" # actual has "Usage: cli package [OPTIONS] FILES" # because it doesn't know that cli will be aliased to datasette expected = expected.replace("Usage: datasette", "Usage: cli") > assert expected == actual E AssertionError: assert '$ datasette ...e and exit.\n' == '$ datasette ...e and exit.\n' E Skipping 848 identical leading characters in diff, use -v to show E nt_id xxx E + E --version-note TEXT Additional note to show on /-/versions E --secret TEXT Secret used for signing secure values, such as signed E cookies E + E --title TEXT Title for metadata ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
904071938 | MDU6SXNzdWU5MDQwNzE5Mzg= | 1345 | ?_nocol= does not interact well with default facets | 9599 | closed | 0 | 7 | 2021-05-27T18:39:55Z | 2021-05-31T02:40:44Z | 2021-05-31T02:31:21Z | OWNER | Clicking "Hide this column" on `fips` on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error: https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips > ## Invalid SQL > no such column: fips The reason is that https://covid-19.datasettes.com/-/metadata sets up the following: ```json "ny_times_us_counties": { "sort_desc": "date", "facets": [ "state", "county", "fips" ], ``` It's setting `fips` as a default facet, which breaks if you attempt to remove the column using `?_nocol`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1345/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
903978133 | MDU6SXNzdWU5MDM5NzgxMzM= | 1343 | Figure out how to publish alpha/beta releases to Docker Hub | 9599 | closed | 0 | 4 | 2021-05-27T16:42:17Z | 2021-05-27T16:46:37Z | 2021-05-27T16:45:41Z | OWNER | > It looks like all I need to do to ship an alpha version to Docker Hub is NOT point the `latest` tag at it after it goes live: https://github.com/simonw/datasette/blob/1a8972f9c012cd22b088c6b70661a9c3d3847853/.github/workflows/publish.yml#L75-L77 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1319#issuecomment-849780481_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1343/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
898904402 | MDU6SXNzdWU4OTg5MDQ0MDI= | 1337 | "More" link for facets that shows _facet_size=max results | 9599 | closed | 0 | 7 | 2021-05-23T00:08:51Z | 2021-05-27T16:14:14Z | 2021-05-27T16:01:03Z | OWNER | _Original title: "More" link for facets that shows the full set of results_ The simplest way to do this will be to have it link to a generated SQL query. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1332#issuecomment-846479062_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
903902495 | MDU6SXNzdWU5MDM5MDI0OTU= | 1342 | Improve `path_with_replaced_args()` and friends and document them | 9599 | open | 0 | 3 | 2021-05-27T15:18:28Z | 2021-05-27T15:23:02Z | OWNER | > In order to cleanly implement this I need to expose the `path_with_replaced_args` utility function to Datasette's template engine. This is the first time this will become an exposed (and hence should-by-documented) API and I don't like its shape much. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1337#issuecomment-849721280_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1342/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
903200328 | MDU6SXNzdWU5MDMyMDAzMjg= | 1341 | "Show all columns" cog menu item should show if ?_col= is used | 9599 | closed | 0 | 1 | 2021-05-27T04:28:17Z | 2021-05-27T04:31:16Z | 2021-05-27T04:31:16Z | OWNER | On https://latest.datasette.io/fixtures/sortable?_col=sortable the "Show all columns" item (from #615) is not shown (it should be): <img width="630" alt="fixtures__sortable__201_rows" src="https://user-images.githubusercontent.com/9599/119766097-44337a80-be69-11eb-8674-8031e72cb17d.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1341/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
517451234 | MDU6SXNzdWU1MTc0NTEyMzQ= | 615 | ?_col= and ?_nocol= support for toggling columns on table view | 9599 | closed | 0 | 16 | 2019-11-04T22:55:41Z | 2021-05-27T04:26:10Z | 2021-05-27T04:17:44Z | OWNER | Split off from #292 (I guess this is a re-opening of #312). | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/615/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
326800219 | MDU6SXNzdWUzMjY4MDAyMTk= | 292 | Mechanism for customizing the SQL used to select specific columns in the table view | 9599 | closed | 0 | 15 | 2018-05-27T09:05:52Z | 2021-05-27T04:25:01Z | 2021-05-27T04:25:01Z | OWNER | Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow. We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice. I think there are two features here: * the ability to request a subset of columns on the table view * the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example Both features should be available via both querystring arguments and in `metadata.json` The querystring argument for custom SQL should only work if `allow_sql` config is turned on. Refs #276 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/292/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
899851083 | MDExOlB1bGxSZXF1ZXN0NjUxNDkyODg4 | 1339 | ?_col=/?_nocol= to show/hide columns on the table page | 9599 | closed | 0 | 1 | 2021-05-24T17:15:20Z | 2021-05-27T04:17:44Z | 2021-05-27T04:17:43Z | OWNER | simonw/datasette/pulls/1339 | See #615. Still to do: - [x] Allow combination of `?_col=` and `?_nocol=` (`_nocol` wins) - [x] Deduplicate same column if passed in `?_col=` multiple times - [x] Validate that user did not try to remove a primary key - [x] Add tests - [x] Ensure this works correctly for SQL views - [x] Add documentation | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1339/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
901009787 | MDU6SXNzdWU5MDEwMDk3ODc= | 1340 | Research: Cell action menu (like column action but for individual cells) | 9599 | open | 0 | 1 | 2021-05-25T15:49:16Z | 2021-05-26T18:59:58Z | OWNER | Had an idea today that it might be useful to select an individual cell and say things like "show me all other rows with the same value" - maybe even a set of other menu options against cells as well. Mocked up a show-on-hover ellipses demo using the CSS inspector: ![idea](https://user-images.githubusercontent.com/9599/119528316-f0744480-bd35-11eb-8eb4-1deea6d60cce.gif) | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1340/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
899169307 | MDU6SXNzdWU4OTkxNjkzMDc= | 1338 | Fix jinja2 warnings | 9599 | closed | 0 | 0 | 2021-05-24T01:38:23Z | 2021-05-24T01:41:55Z | 2021-05-24T01:41:55Z | OWNER | Lots of these in the test suite now, after the Jinja upgrade in #1331: ``` tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:45: DeprecationWarning: 'jinja2.escape' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.escape' instead. label=jinja2.escape(data["label"] or "") or " ", tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:41: DeprecationWarning: 'jinja2.Markup' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.Markup' instead. return jinja2.Markup( ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
642296989 | MDU6SXNzdWU2NDIyOTY5ODk= | 856 | Consider pagination of canned queries | 9599 | open | 0 | 3 | 2020-06-20T03:15:59Z | 2021-05-21T14:22:41Z | OWNER | The new `canned_queries()` plugin hook from #852 combined with plugins like https://github.com/simonw/datasette-saved-queries could mean that some installations end up with hundreds or even thousands of canned queries. I should consider pagination or some other way of ensuring that this doesn't cause performance problems for Datasette. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/856/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
812228314 | MDU6SXNzdWU4MTIyMjgzMTQ= | 1236 | Ability to increase size of the SQL editor window | 9599 | closed | 0 | 9 | 2021-02-19T18:09:27Z | 2021-05-18T03:28:25Z | 2021-02-22T21:05:21Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1236/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||||
842862708 | MDU6SXNzdWU4NDI4NjI3MDg= | 1280 | Ability to run CI against multiple SQLite versions | 9599 | open | 0 | 2 | 2021-03-28T23:54:50Z | 2021-05-10T19:07:46Z | OWNER | Issue #1276 happened because I didn't run tests against a SQLite version prior to 3.16.0 (released 2017-01-02). Glitch is a deployment target and runs SQLite 3.11.0 from 2016-02-15. If CI ran against that version of SQLite this bug could have been avoided. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1280/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
871304967 | MDU6SXNzdWU4NzEzMDQ5Njc= | 1315 | settings.json should be picked up by "datasette publish cloudrun" | 9599 | open | 0 | 0 | 2021-04-29T18:16:41Z | 2021-04-29T18:16:41Z | OWNER | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||||
866668415 | MDU6SXNzdWU4NjY2Njg0MTU= | 1308 | Columns named "link" display in bold | 9599 | closed | 0 | 3 | 2021-04-24T05:58:11Z | 2021-04-24T06:07:49Z | 2021-04-24T06:07:49Z | OWNER | Reported in office hours today. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1308/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
856895291 | MDU6SXNzdWU4NTY4OTUyOTE= | 1299 | Design better empty states | 9599 | open | 0 | 0 | 2021-04-13T12:06:12Z | 2021-04-13T12:06:12Z | OWNER | Inspiration here: https://emptystat.es/ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1299/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
520667773 | MDU6SXNzdWU1MjA2Njc3NzM= | 620 | Mechanism for indicating foreign key relationships in the table and query page URLs | 9599 | open | 0 | 6 | 2019-11-10T22:26:27Z | 2021-04-05T03:57:22Z | OWNER | Datasette currently only inflates foreign keys (into names hyperlinks) if it detects them as foreign key constraints in the underlying database. It would be useful if you could specify additional "foreign keys" using both `metadata.json` and the querystring - similar time how you can pass `?_fts_table=x` https://datasette.readthedocs.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/620/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1 } |
||||||||
672421411 | MDU6SXNzdWU2NzI0MjE0MTE= | 916 | Support reverse pagination (previous page, has-previous-items) | 9599 | open | 0 | 7 | 2020-08-04T00:32:06Z | 2021-04-03T23:43:11Z | OWNER | I need this for `datasette-graphql` for full compatibility with the way Relay likes to paginate - using cursors for paginating backwards as well as for paginating forwards. > This may be the kick I need to get Datasette pagination to work in reverse too. _Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-668305853_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/916/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
849396758 | MDU6SXNzdWU4NDkzOTY3NTg= | 1287 | Upgrade to Python 3.9.4 | 9599 | open | 0 | 5 | 2021-04-02T18:43:15Z | 2021-04-03T22:38:39Z | OWNER | Has some security fixes https://pythoninsider.blogspot.com/2021/04/python-393-and-389-are-now-available.html | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1287/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
849568079 | MDExOlB1bGxSZXF1ZXN0NjA4MzIzMDI4 | 1290 | Use pytest-xdist to speed up tests | 9599 | closed | 0 | 1 | 2021-04-03T03:34:36Z | 2021-04-03T03:42:29Z | 2021-04-03T03:42:28Z | OWNER | simonw/datasette/pulls/1290 | Closes #1289, refs #1212. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/1290/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
849543502 | MDU6SXNzdWU4NDk1NDM1MDI= | 1289 | Speed up tests with pytest-xdist | 9599 | closed | 0 | 3 | 2021-04-03T00:47:39Z | 2021-04-03T03:42:28Z | 2021-04-03T03:42:28Z | OWNER | I think I can get this working for almost every test, then use the pattern in https://github.com/pytest-dev/pytest-xdist/issues/385#issuecomment-444545641 to opt specific tests out of being run in parallel. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1289/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
817544251 | MDU6SXNzdWU4MTc1NDQyNTE= | 1245 | Sticky table column headers would be useful, especially on the query page | 9599 | open | 0 | 1 | 2021-02-26T17:42:51Z | 2021-04-02T20:53:35Z | OWNER | Suggestion from office hours. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1245/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
576722115 | MDU6SXNzdWU1NzY3MjIxMTU= | 696 | Single failing unit test when run inside the Docker image | 9599 | closed | 0 | 3268330 | 2 | 2020-03-06T06:16:36Z | 2021-03-29T17:04:19Z | 2021-03-07T07:41:18Z | OWNER | ``` docker run -it -v `pwd`:/mnt datasetteproject/datasette:latest /bin/bash root@0e1928cfdf79:/# cd /mnt root@0e1928cfdf79:/mnt# pip install -e .[test] root@0e1928cfdf79:/mnt# pytest ``` I get one failure! It was for `test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]` ``` def test_searchable(app_client, path, expected_rows): response = app_client.get(path) > assert expected_rows == response.json["rows"] E AssertionError: assert [[1, 'barry c...sel', 'puma']] == [] E Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther'] E Full diff: E + [] E - [[1, 'barry cat', 'terry dog', 'panther'], E - [2, 'terry dog', 'sara weasel', 'puma']] ``` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/695#issuecomment-595614469_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/696/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
842881221 | MDU6SXNzdWU4NDI4ODEyMjE= | 1281 | Latest Datasette tags missing from Docker Hub | 9599 | closed | 0 | 7 | 2021-03-29T00:58:30Z | 2021-03-29T01:41:48Z | 2021-03-29T01:41:48Z | OWNER | Spotted this while testing https://github.com/simonw/datasette/issues/1249#issuecomment-808998719_ https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated isn't showing the tags for any version more recent than 0.54.1 - we are up to 0.56 now. But the `:latest` tag is for the new 0.56 release. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1281/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
763207948 | MDU6SXNzdWU3NjMyMDc5NDg= | 1141 | Default styling for bullet point lists | 9599 | closed | 0 | 0 | 2020-12-12T02:49:33Z | 2021-03-29T00:14:05Z | 2021-03-29T00:14:05Z | OWNER | I just noticed that https://datasette.io/content/recent_releases (which uses `datasette-render-markdown`) is missing its bullet points: <img width="864" alt="content__recent_releases__399_rows" src="https://user-images.githubusercontent.com/9599/101970823-95b0bb80-3be1-11eb-9512-9274cef64cf3.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1141/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
741862364 | MDU6SXNzdWU3NDE4NjIzNjQ= | 1090 | Custom widgets for canned query forms | 9599 | open | 0 | 3 | 2020-11-12T19:21:07Z | 2021-03-27T16:25:25Z | OWNER | This is an idea that was cut from the first version of writable canned queries: > I really want the option to use a `<textarea>` for a specific value. > > Idea: metadata syntax like this: > > ```json > { > "databases": { > "my-database": { > "queries": { > "add_twitter_handle": { > "sql": "insert into twitter_handles (username) values (:username)", > "write": true, > "params": { > "username": { > "widget": "textarea" > } > } > } > } > } > } > } > ``` > > I can ship with some default widgets and provide a plugin hook for registering extra widgets. > > This opens up some really exciting possibilities for things like map widgets that let you draw polygons. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-608125928_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1090/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
842212586 | MDU6SXNzdWU4NDIyMTI1ODY= | 1277 | Facet by array breaks if table name contains a space | 9599 | closed | 0 | 1 | 2021-03-26T18:38:19Z | 2021-03-27T03:49:38Z | 2021-03-27T03:49:34Z | OWNER | It breaks when you try to select a filtered item. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1277/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
839367451 | MDU6SXNzdWU4MzkzNjc0NTE= | 1275 | Idea: long-running query mode | 9599 | open | 0 | 0 | 2021-03-24T05:23:20Z | 2021-03-24T05:23:20Z | OWNER | It would be cool if you could run Datasette in a long-running query mode, for use with trusted users - something like this: datasette --unlimited my.db This would disable the query limit, but would also enable a feature where if a query takes longer than e.g. 1s to return Datasette returns an HTML page to the browser with a progress indicator and polls the server until the query is complete.... but also provides the user with a "cancel" button. This relates to the `.interrupt()` research in #1270. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1275/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
837350092 | MDU6SXNzdWU4MzczNTAwOTI= | 1270 | Try implementing SQLite timeouts using .interrupt() instead of using .set_progress_handler() | 9599 | open | 0 | 3 | 2021-03-22T06:00:17Z | 2021-03-23T16:45:39Z | OWNER | > Maybe I could implement SQLite query timeouts using the `interrupt()` method instead of the progress handler hack I'm currently using? > > https://stackoverflow.com/questions/43240496/python-sqlite3-how-to-quickly-and-cleanly-interrupt-long-running-query-with-e has some tips. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1268#issuecomment-803764919_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/1270/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |