{"id": 836273891, "node_id": "MDU6SXNzdWU4MzYyNzM4OTE=", "number": 1266, "title": "Documentation for Response.asgi_send(send) method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-03-19T18:52:49Z", "updated_at": "2021-03-20T21:35:00Z", "closed_at": "2021-03-20T21:32:28Z", "author_association": "OWNER", "pull_request": null, "body": "I found myself wanting to use this method for https://github.com/simonw/datasette-auth-passwords/issues/15 - but it's not documented. It should be documented.\r\n\r\nhttps://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/asgi.py#L320-L340", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1266/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 836123030, "node_id": "MDU6SXNzdWU4MzYxMjMwMzA=", "number": 1265, "title": "Support for HTTP Basic Authentication", "user": {"value": 468612, "label": "yunzheng"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-03-19T15:31:09Z", "updated_at": "2021-03-19T22:05:12Z", "closed_at": "2021-03-19T21:03:09Z", "author_association": "NONE", "pull_request": null, "body": "It would be nice if datasette could support [HTTP Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication).\r\n\r\nFor now I could ofcourse leverage Nginx for basic authentication, but it would be nice to have support for this in datasette by default or via a plugin like datasette-auth-github.\r\n\r\nMy main usecase is to put the whole datasette instance behind a username/password prompt via Basic Auth and not specific urls.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1265/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 824067604, "node_id": "MDU6SXNzdWU4MjQwNjc2MDQ=", "number": 1250, "title": "Research: Plugin hook for alternative database connections", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-08T00:28:15Z", "updated_at": "2021-03-12T01:01:25Z", "closed_at": "2021-03-12T01:01:17Z", "author_association": "OWNER", "pull_request": null, "body": "The `Database` class is a natural looking fit for a plugin hook to load custom database connections... potentially even databases other than SQLite. DuckDB (refs #968) could make for a great starting point, since it looks very compatible with the existing SQLite code.\r\n\r\nThe real win would be if this could lead to running Datasette against PostgreSQL. I made some initial explorations in that direction a while ago in #670.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1250/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 797649915, "node_id": "MDExOlB1bGxSZXF1ZXN0NTY0NjA4MjY0", "number": 1211, "title": "Use context manager instead of plain open", "user": {"value": 4488943, "label": "kbaikov"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-01-31T07:58:10Z", "updated_at": "2021-03-11T16:15:50Z", "closed_at": "2021-03-11T16:15:50Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1211", "body": "Context manager with open closes the files after usage. Fixes: https://github.com/simonw/datasette/issues/1208\r\n\r\nWhen the object is already a pathlib.Path i used read_text\r\nwrite_text functions\r\n\r\nIn some cases pathlib.Path.open were used in context manager,\r\nit is basically the same as builtin open.\r\n\r\nTests are passing: 850 passed, 5 xfailed, 10 xpassed", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1211/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 794554881, "node_id": "MDU6SXNzdWU3OTQ1NTQ4ODE=", "number": 1208, "title": "A lot of open(file) functions are used without a context manager thus producing ResourceWarning: unclosed file <_io.TextIOWrapper", "user": {"value": 4488943, "label": "kbaikov"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-26T20:56:28Z", "updated_at": "2021-03-11T16:15:49Z", "closed_at": "2021-03-11T16:15:49Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Your code is full of open files that are never closed, especially when you deal with reading/writing json/yaml files.\r\n\r\nIf you run python with warnings enabled this problem becomes evident.\r\nThis probably contributes to some memory leaks in long running datasettes if the GC will not 'collect' those resources properly.\r\n\r\nThis is easily fixed by using a context manager instead of just using open:\r\n```python\r\nwith open('some_file', 'w') as opened_file:\r\n    opened_file.write('string')\r\n```\r\n\r\nIn some newer parts of the code you use Path objects 'read_text' and 'write_text' functions which close the file properly and are prefered in some cases.\r\n\r\n\r\nIf you want I can create a PR for all places i found this pattern in.\r\n\r\n\r\nBellow is a fraction of places where i found a ResourceWarning:\r\n```python\r\n\r\nupdate-docs-help.py:\r\n  20          actual = actual.replace(\"Usage: cli \", \"Usage: datasette \")\r\n  21:         open(docs_path / filename, \"w\").write(actual)\r\n  22  \r\n\r\ndatasette\\app.py:\r\n  210          ):\r\n  211:             inspect_data = json.load((config_dir / \"inspect-data.json\").open())\r\n  212              if immutables is None:\r\n\r\n  266          if config_dir and (config_dir / \"settings.json\").exists() and not config:\r\n  267:             config = json.load((config_dir / \"settings.json\").open())\r\n  268          self._settings = dict(DEFAULT_SETTINGS, **(config or {}))\r\n\r\n  445              self._app_css_hash = hashlib.sha1(\r\n  446:                 open(os.path.join(str(app_root), \"datasette/static/app.css\"))\r\n  447                  .read()\r\n\r\ndatasette\\cli.py:\r\n  130      else:\r\n  131:         out = open(inspect_file, \"w\")\r\n  132      loop = asyncio.get_event_loop()\r\n\r\n  459      if inspect_file:\r\n  460:         inspect_data = json.load(open(inspect_file))\r\n  461  \r\n\r\n```\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1208/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 826613352, "node_id": "MDExOlB1bGxSZXF1ZXN0NTg4NjAxNjI3", "number": 1254, "title": "Update Docker Spatialite version to 5.0.1 + add support for Spatialite topology functions", "user": {"value": 3200608, "label": "durkie"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-03-09T20:49:08Z", "updated_at": "2021-03-10T18:27:45Z", "closed_at": "2021-03-09T22:04:23Z", "author_association": "NONE", "pull_request": "simonw/datasette/pulls/1254", "body": "This requires adding the RT Topology library (Spatialite changed to RT Topology from LWGEOM between 4.4 and 5.0), as well as upgrading the GEOS version (which is the reason for switching to `python:3.7.10-slim-buster` as the base image.)\r\n\r\n`autoconf` and `libtool` are added to build RT Topology, and Spatialite is now built with `--disable-minizip` (minizip wasn't an option in 4.4 and I didn't want to add another dependency) and `--disable-dependency-tracking` which, according to Spatialite, \"speeds up one-time builds\"", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1254/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 827341657, "node_id": "MDExOlB1bGxSZXF1ZXN0NTg5MjYzMjk3", "number": 1256, "title": "Minor type in IP adress", "user": {"value": 6371750, "label": "JBPressac"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-03-10T08:28:22Z", "updated_at": "2021-03-10T18:26:46Z", "closed_at": "2021-03-10T18:26:40Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1256", "body": "127.0.01 replaced by 127.0.0.1", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1256/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 823035080, "node_id": "MDU6SXNzdWU4MjMwMzUwODA=", "number": 1248, "title": "duckdb database (very low performance in SQLite)", "user": {"value": 15836677, "label": "verajosemanuel"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-03-05T12:20:29Z", "updated_at": "2021-03-08T00:25:27Z", "closed_at": "2021-03-08T00:25:27Z", "author_association": "NONE", "pull_request": null, "body": "My sqlite is getting too big to be processed by datasette (more than 10 minutes waiting to load) so I am working with duckdb and is waaaaay faster. I think the fastest embeddable database actually.\r\n\r\nhttps://duckdb.org/\r\n\r\nTaking into account DuckDb is SQLite based it would be GREAT to use it with datasette.\r\n\r\nis that possible?\r\n\r\nRegards and thanks for a superb job", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1248/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 806918878, "node_id": "MDExOlB1bGxSZXF1ZXN0NTcyMjU0MTAz", "number": 1223, "title": "Add compile option to Dockerfile to fix failing test (fixes #696)", "user": {"value": 7476523, "label": "bobwhitelock"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-02-12T03:38:05Z", "updated_at": "2021-03-07T12:01:12Z", "closed_at": "2021-03-07T07:41:17Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1223", "body": "This test was failing when run inside the Docker container: `test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]`,\r\n\r\nwith this error:\r\n\r\n```\r\n    def test_searchable(app_client, path, expected_rows):\r\n        response = app_client.get(path)\r\n>       assert expected_rows == response.json[\"rows\"]\r\nE       AssertionError: assert [[1, 'barry c...sel', 'puma']] == []\r\nE         Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']\r\nE         Full diff:\r\nE         + []\r\nE         - [[1, 'barry cat', 'terry dog', 'panther'],\r\nE         -  [2, 'terry dog', 'sara weasel', 'puma']]\r\n```\r\n\r\nThe issue was that the version of sqlite3 built inside the Docker container was built with FTS3 and FTS4 enabled, but without the\r\n`SQLITE_ENABLE_FTS3_PARENTHESIS` compile option passed, which adds support for using `AND` and `NOT` within `match` expressions (see https://sqlite.org/fts3.html#compiling_and_enabling_fts3_and_fts4 and https://www.sqlite.org/compile.html).\r\n\r\nWithout this, the `AND` used in the search in this test was being interpreted as a literal string, and so no matches were found. Adding this compile option fixes this.\r\n\r\n---\r\n\r\nI actually ran into this issue because the same test was failing when I ran the test suite on my own machine, outside of Docker, and so I eventually tracked this down to my system sqlite3 also being compiled without this option.\r\n\r\nI wonder if this is a sign of a slightly deeper issue, that Datasette can silently behave differently based on the version and compilation of sqlite3 it is being used with. On my own system I fixed the test suite by running `pip install pysqlite3-binary`, so that this would be picked up instead of the `sqlite` package, as this seems to be compiled using this option, . Maybe using `pysqlite3-binary` could be installed/recommended by default so a more deterministic version of sqlite is used? Or there could be some feature detection done on the available sqlite version, to know what features are available and can be used/tested?", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1223/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 815955014, "node_id": "MDExOlB1bGxSZXF1ZXN0NTc5Njk3ODMz", "number": 1243, "title": "fix small typo", "user": {"value": 306240, "label": "UtahDave"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-02-25T00:22:34Z", "updated_at": "2021-03-04T05:46:10Z", "closed_at": "2021-03-04T05:46:10Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1243", "body": "", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1243/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 818430405, "node_id": "MDU6SXNzdWU4MTg0MzA0MDU=", "number": 1247, "title": "datasette.add_memory_database() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-01T03:48:38Z", "updated_at": "2021-03-01T04:02:26Z", "closed_at": "2021-03-01T04:02:26Z", "author_association": "OWNER", "pull_request": null, "body": "I just wrote this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/47eb885cc2c3aafa03645c330c6f597bee9b3b25/tests/test_facets.py#L334-L335\r\n\r\nIt would be nice if you didn't have to separately instantiate a database object here.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1247/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 817597268, "node_id": "MDU6SXNzdWU4MTc1OTcyNjg=", "number": 1246, "title": "Suggest for ArrayFacet possibly confused by blank values", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-26T19:11:52Z", "updated_at": "2021-03-01T03:46:11Z", "closed_at": "2021-03-01T03:46:11Z", "author_association": "OWNER", "pull_request": null, "body": "I sometimes don't get the suggestion for facet-by-array for columns that contain arrays. I think it may be because they have empty spaces in them - or perhaps it's because the null detection doesn't actually work.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1246/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 718259202, "node_id": "MDU6SXNzdWU3MTgyNTkyMDI=", "number": 1005, "title": "Remove xfail tests when new httpx is released", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 3, "created_at": "2020-10-09T16:00:19Z", "updated_at": "2021-02-28T22:41:08Z", "closed_at": "2021-02-28T22:41:08Z", "author_association": "OWNER", "pull_request": null, "body": "> My `httpx` pull request adding `raw_path` support was just merged: https://github.com/encode/httpx/pull/1357 - but it's not in a release yet.\r\n>\r\n> I'm going to mark these tests as `xfail` so I can land this change - I'll remove that once an `httpx` release comes out that I can use to get the tests passing.\r\n>\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1000#issuecomment-706263157_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1005/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 814591962, "node_id": "MDU6SXNzdWU4MTQ1OTE5NjI=", "number": 1240, "title": "Allow facetting on custom queries", "user": {"value": 7107523, "label": "Kabouik"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-23T15:52:19Z", "updated_at": "2021-02-26T18:19:46Z", "closed_at": "2021-02-26T18:18:18Z", "author_association": "NONE", "pull_request": null, "body": "Facets are a tremendously useful feature, especially for people peeking at the database for the first time and still having little knowledge about the details of the data. It is of great assistance to discover interesting features to explore futher in advanced queries.\r\n\r\nYet, it seems it's impossible to use facets when running a custom SQL query, be it from the little gear icons in column names, the facet suggestions at the top (hidden when performing a custom query), or by appending a facet code to the URL. \r\n\r\nIs there a technical limitation, or is this something that could be unlocked easily?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1240/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 817528452, "node_id": "MDU6SXNzdWU4MTc1Mjg0NTI=", "number": 1244, "title": "Plugin tip: look at the examples linked from the hooks page", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-26T17:18:27Z", "updated_at": "2021-02-26T17:30:38Z", "closed_at": "2021-02-26T17:27:15Z", "author_association": "OWNER", "pull_request": null, "body": "Someone asked \"what are good example plugins I can look at?\" and I realized that the answer is to look through the example links on https://docs.datasette.io/en/stable/plugin_hooks.html - but that tip should be written down somewhere on the https://docs.datasette.io/en/stable/writing_plugins.html page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1244/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 815554385, "node_id": "MDU6SXNzdWU4MTU1NTQzODU=", "number": 237, "title": "db[\"my_table\"].drop(ignore=True) parameter, plus sqlite-utils drop-table --ignore and drop-view --ignore", "user": {"value": 649467, "label": "mhalle"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-24T14:55:06Z", "updated_at": "2021-02-25T17:11:41Z", "closed_at": "2021-02-25T17:11:41Z", "author_association": "NONE", "pull_request": null, "body": "When I'm generating a derived table in python, I often drop the table and create it from scratch. However, the first time I generate the table, it doesn't exist, so the drop raises an exception.  That means more boilerplate.\r\n\r\nI was going to submit a pull request that adds an \"if_exists\" option to the `drop` method of tables and views. \r\n\r\nHowever, for a utility like sqlite_utils, perhaps the \"IF EXISTS\" SQL semantics is what you want most of the time, and thus should be the default.\r\n\r\nWhat do you think?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/237/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 816523763, "node_id": "MDU6SXNzdWU4MTY1MjM3NjM=", "number": 238, "title": ".add_foreign_key() corrupts database if column contains a space", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-25T15:07:20Z", "updated_at": "2021-02-25T16:54:02Z", "closed_at": "2021-02-25T16:54:02Z", "author_association": "OWNER", "pull_request": null, "body": "I ran this:\r\n\r\n    db[\"Reports\"].add_foreign_key(\"Reported by ID\", \"Reporters\", \"id\")\r\n\r\nAnd got this:\r\n\r\n```\r\n~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys)\r\n    616         # Have to VACUUM outside the transaction to ensure .foreign_keys property\r\n    617         # can see the newly created foreign key.\r\n--> 618         self.vacuum()\r\n    619 \r\n    620     def index_foreign_keys(self):\r\n\r\n~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self)\r\n    629 \r\n    630     def vacuum(self):\r\n--> 631         self.execute(\"VACUUM;\")\r\n    632 \r\n    633 \r\n\r\n~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters)\r\n    234             return self.conn.execute(sql, parameters)\r\n    235         else:\r\n--> 236             return self.conn.execute(sql)\r\n    237 \r\n    238     def executescript(self, sql):\r\n\r\nDatabaseError: database disk image is malformed\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 816560819, "node_id": "MDU6SXNzdWU4MTY1NjA4MTk=", "number": 240, "title": "table.pks_and_rows_where() method returning primary keys along with the rows", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2021-02-25T15:49:28Z", "updated_at": "2021-02-25T16:39:23Z", "closed_at": "2021-02-25T16:28:23Z", "author_association": "OWNER", "pull_request": null, "body": "*Original title: Easier way to update a row returned from .rows*\r\n\r\nHere's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by `db[table].rows` how can you update that row?\r\n\r\nThe problem is that the `db[table].update(...)` method requires a primary key. But if you have a row from the `db[table].rows` iterator it might not even contain the primary key - provided the table is a `rowid` table.\r\n\r\nInstead, currently, you need to introspect the table and, if `rowid` is a primary key, explicitly include that in the `select=`  argument to `table.rows_where(...)` - otherwise it will not be returned.\r\n\r\nA utility mechanism to make this easier would be very welcome.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 813978858, "node_id": "MDU6SXNzdWU4MTM5Nzg4NTg=", "number": 1239, "title": "JSON filter fails if column contains spaces", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-23T00:18:07Z", "updated_at": "2021-02-23T00:22:53Z", "closed_at": "2021-02-23T00:22:53Z", "author_association": "OWNER", "pull_request": null, "body": "Got this exception:\r\n\r\n`ERROR: conn=<sqlite3.Connection object at 0x10ea68e40>, sql = 'select Address, Affiliation, County, [Has Report], [Latest report notes], [Latest report yes], Latitude, [Location Type], Longitude, Name, id, [Appointment scheduling instructions], [Availability Info], [Latest report] from locations where rowid in (\\n            select locations.rowid from locations, json_each(locations.Availability Info) j\\n            where j.value = :p0\\n        ) and \"Latest report yes\" = :p1 order by id limit 101', params = {'p0': 'Yes: appointment required', 'p1': '1'}: near \"Info\": syntax error`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1239/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 783778672, "node_id": "MDU6SXNzdWU3ODM3Nzg2NzI=", "number": 220, "title": "Better error message for *_fts methods against views", "user": {"value": 649467, "label": "mhalle"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-01-11T23:24:00Z", "updated_at": "2021-02-22T20:44:51Z", "closed_at": "2021-02-14T22:34:26Z", "author_association": "NONE", "pull_request": null, "body": "enable_fts and its related methods only work on tables, not views. \r\n\r\nCould those methods and possibly others move up to the Queryable superclass?\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/220/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 797651831, "node_id": "MDU6SXNzdWU3OTc2NTE4MzE=", "number": 1212, "title": "Tests are very slow. ", "user": {"value": 4488943, "label": "kbaikov"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-31T08:06:16Z", "updated_at": "2021-02-19T22:54:13Z", "closed_at": "2021-02-19T22:54:13Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Working on my PR i noticed that tests are very slow.\r\n\r\nThe plain pytest run took about 37 minutes for me.\r\nHowever i could shave of about 10 minutes from that if i used pytest-xdist to parallelize execution.\r\n`pytest -n 8` is run only in 28 minutes on my machine.\r\n\r\nI can create a PR to mention that in your documentation.\r\nThis will be a simple change to add pytest-xdist to requirements and change a command to run pytest in documentation.\r\n\r\nDoes that make sense to you?\r\n\r\nAfter a bit more investigation it looks like python-xdist is not an answer. It creates a race condition for tests that try to clead temp dir before run.\r\n\r\nProfiling shows that most time is spent on conn.executescript(TABLES) in make_app_client function. Which makes sense.\r\n\r\nPerhaps the better approach would be look at the app_client fixture which is already session scoped, but not used by all test cases.\r\nAnd/or use conn = sqlite3.connect(\":memory:\") which is much faster.\r\nAnd/or truncate tables after each TC instead of deleting the file and re-creating them.\r\n\r\nI can take a look which is the best approach if you give the go-ahead. ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1212/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811680502, "node_id": "MDU6SXNzdWU4MTE2ODA1MDI=", "number": 236, "title": "--attach command line option for attaching extra databases", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-19T04:38:30Z", "updated_at": "2021-02-19T05:10:41Z", "closed_at": "2021-02-19T05:08:43Z", "author_association": "OWNER", "pull_request": null, "body": "This will enable cross-database joins, as seen in https://github.com/simonw/datasette/issues/283\r\n\r\nAlso refs #113", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/236/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 621286870, "node_id": "MDU6SXNzdWU2MjEyODY4NzA=", "number": 113, "title": "Syntactic sugar for ATTACH DATABASE", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-19T21:10:00Z", "updated_at": "2021-02-19T05:09:12Z", "closed_at": "2021-02-19T04:56:36Z", "author_association": "OWNER", "pull_request": null, "body": "https://www.sqlite.org/lang_attach.html\r\n\r\nMaybe something like this:\r\n```python\r\ndb.attach(\"other_db\", \"other_db.db\")\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/113/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811589344, "node_id": "MDU6SXNzdWU4MTE1ODkzNDQ=", "number": 1235, "title": "Upgrade Python version used by official Datasette Docker image", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-02-19T00:47:40Z", "updated_at": "2021-02-19T01:48:31Z", "closed_at": "2021-02-19T01:48:30Z", "author_association": "OWNER", "pull_request": null, "body": "Currently uses 3.7.2:\r\n\r\nhttps://github.com/simonw/datasette/blob/73bed175631a79e13a521eee82f8451dd0477eb3/Dockerfile#L1\r\n\r\nThere's a security fix for Python which it would be good to ship in this image (even though I'm reasonably confident it doesn't affect Datasette): https://bugs.python.org/issue42938", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1235/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 811407131, "node_id": "MDExOlB1bGxSZXF1ZXN0NTc1OTQwMTkz", "number": 1232, "title": "--crossdb option for joining across databases", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2021-02-18T19:48:50Z", "updated_at": "2021-02-18T22:09:13Z", "closed_at": "2021-02-18T22:09:12Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1232", "body": "Refs #283. Still needs:\r\n\r\n- [x] Unit test for --crossdb queries\r\n- [x] Show warning on console if it truncates at ten databases (or on web interface)\r\n- [x] Show connected databases on the `/_memory` database page\r\n- [x] Documentation\r\n- [x] https://latest.datasette.io/ demo should demonstrate this feature", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1232/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 808843401, "node_id": "MDU6SXNzdWU4MDg4NDM0MDE=", "number": 1226, "title": "--port option should validate port is between 0 and 65535", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-15T22:01:33Z", "updated_at": "2021-02-18T18:41:27Z", "closed_at": "2021-02-18T18:41:27Z", "author_association": "OWNER", "pull_request": null, "body": "Currently throws an ugly error message:\r\n```\r\n(datasette-graphql) datasette-graphql % datasette fivethirtyeight.db -p 80094\r\nINFO:     Started server process [45497]\r\nINFO:     Waiting for application startup.\r\nINFO:     Application startup complete.\r\nTraceback (most recent call last):\r\n  File \"/Users/simon/.local/share/virtualenvs/datasette-graphql-n1OSJCS8/bin/datasette\", line 8, in <module>\r\n    sys.exit(cli())\r\n...\r\n    server = await loop.create_server(\r\n  File \"/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py\", line 1461, in create_server\r\n    sock.bind(sa)\r\nOverflowError: bind(): port must be 0-65535.\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1226/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 807174161, "node_id": "MDU6SXNzdWU4MDcxNzQxNjE=", "number": 227, "title": "Error reading csv files with large column data", "user": {"value": 295329, "label": "camallen"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-12T11:51:47Z", "updated_at": "2021-02-16T11:48:03Z", "closed_at": "2021-02-14T21:17:19Z", "author_association": "NONE", "pull_request": null, "body": "*Feel free to close this issue - I mostly added it for reference for future folks that run into this :)*\r\n\r\nI have a CSV file with one column that has very long strings.  When i try to import this file via the `insert` command I get the following error: \r\n```\r\nsqlite-utils insert database.db table_name file_with_large_column.csv\r\n\r\nTraceback (most recent call last):\r\n  File \"/usr/local/bin/sqlite-utils\", line 10, in <module>\r\n    sys.exit(cli())\r\n  File \"/usr/local/lib/python3.7/site-packages/click/core.py\", line 829, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"/usr/local/lib/python3.7/site-packages/click/core.py\", line 782, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/usr/local/lib/python3.7/site-packages/click/core.py\", line 1259, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"/usr/local/lib/python3.7/site-packages/click/core.py\", line 1066, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/usr/local/lib/python3.7/site-packages/click/core.py\", line 610, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py\", line 774, in insert\r\n    default=default,\r\n  File \"/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py\", line 705, in insert_upsert_implementation\r\n    docs, pk=pk, batch_size=batch_size, alter=alter, **extra_kwargs\r\n  File \"/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1852, in insert_all\r\n    first_record = next(records)\r\n  File \"/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py\", line 703, in <genexpr>\r\n    docs = (decode_base64_values(doc) for doc in docs)\r\n  File \"/usr/local/lib/python3.7/site-packages/sqlite_utils/cli.py\", line 681, in <genexpr>\r\n    docs = (dict(zip(headers, row)) for row in reader)\r\n_csv.Error: field larger than field limit (131072)\r\n```\r\nBuilt with the docker image `datasetteproject/datasette:0.54` with the following versions:\r\n```\r\n# sqlite-utils --version\r\nsqlite-utils, version 3.4.1\r\n\r\n# datasette --version\r\ndatasette, version 0.54\r\n```\r\nIt appears this is a [known issue](https://stackoverflow.com/a/54517228/2761423) reading in csv files in python and [doesn't look to be modifiable](https://github.com/python/cpython/blob/ea46579067fd2d4e164d6605719ffec690c4d621/Modules/_csv.c#L1685) through system / env vars (i may be very wrong on this).\r\n\r\nNoting that using sqlite3 `import` command work without error (not using the python csv reader)\r\n```\r\nsqlite3 database.db\r\nsqlite> .mode csv\r\nsqlite> .import file_with_large_column.csv table_name\r\n```\r\nSadly I couldn't see an easy way around this while using the cli as it appears this value needs to be changed in python code. FWIW I've switched to using https://datasette.io/tools/csvs-to-sqlite for importing csv data and it's working well. \r\n\r\nFinally, I'm loving https://datasette.io/ thank you very much for an amazing tool and data ecosytem \ud83d\ude47\u200d\u2640\ufe0f ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/227/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 807817197, "node_id": "MDU6SXNzdWU4MDc4MTcxOTc=", "number": 229, "title": "Hitting `_csv.Error: field larger than field limit (131072)`", "user": {"value": 631242, "label": "frosencrantz"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-13T19:52:44Z", "updated_at": "2021-02-14T21:33:33Z", "closed_at": "2021-02-14T21:33:33Z", "author_association": "NONE", "pull_request": null, "body": "I have a csv file where one of the fields is so large it is throwing an exception with this error and stops loading:\r\n\t```\r\n\t_csv.Error: field larger than field limit (131072)\r\n\t```\r\n\r\nThe stack trace occurs here: https://github.com/simonw/sqlite-utils/blob/3.1/sqlite_utils/cli.py#L633\r\n\r\n\r\nThere is a way to handle this that helps:\r\nhttps://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072\r\n\r\nOne issue I had with this problem was sqlite-utils only provides limited context as to where the problem line is.\r\nThere is the progress bar, but that is by percent rather than by line number.  It would have been helpful if it could have provided a line number.\r\n\r\nAlso, it would have been useful if it had allowed the loading to continue with later lines.\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/229/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 808008305, "node_id": "MDU6SXNzdWU4MDgwMDgzMDU=", "number": 230, "title": "--sniff option for sniffing delimiters", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2021-02-14T17:43:54Z", "updated_at": "2021-02-14T21:15:33Z", "closed_at": "2021-02-14T19:24:32Z", "author_association": "OWNER", "pull_request": null, "body": "> I just spotted that `csv.Sniffer` in the Python standard library has a `.has_header(sample)` method which detects if the first row appears to be a header or not, which is interesting. https://docs.python.org/3/library/csv.html#csv.Sniffer\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/230/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 797159961, "node_id": "MDExOlB1bGxSZXF1ZXN0NTY0MjE1MDEx", "number": 225, "title": "fix for problem in Table.insert_all on search for columns per chunk of rows", "user": {"value": 261237, "label": "nieuwenhoven"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-29T20:16:07Z", "updated_at": "2021-02-14T21:04:13Z", "closed_at": "2021-02-14T21:04:13Z", "author_association": "NONE", "pull_request": "simonw/sqlite-utils/pulls/225", "body": "Hi,\r\n\r\nI ran into a problem when trying to create a database from my Apple Healthkit data using [healthkit-to-sqlite](https://github.com/dogsheep/healthkit-to-sqlite). The program crashed because of an invalid insert statement that was generated for table `rDistanceCycling`. \r\n\r\nThe actual problem turned out to be in [sqlite-utils](https://github.com/simonw/sqlite-utils). `Table.insert_all` processes the data to be inserted in chunks of rows and checks for every chunk which columns are used, and it will collect all column names in the variable `all_columns`.  The collection of columns is done using a nested list comprehension that is not completely correct. \r\n\r\nI'm using a Windows machine and had to make a few adjustments to the tests in order to be able to run them because they had a posix dependency.\r\n\r\nThanks, kind regards,\r\n\r\nFrans\r\n\r\n```\r\n# this is a (condensed) chunk of data from my Apple healthkit export that caused the problem.\r\n# the 3 last items in the chunk have additional keys: metadata_HKMetadataKeySyncVersion and metadata_HKMetadataKeySyncIdentifier\r\n\r\nchunk = [{'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '7.0.1',\r\n          'device': '<<HKDevice: 0x281cf6c70>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:7.0.1>',\r\n          'unit': 'km', 'creationDate': '2020-10-10 12:29:09 +0100', 'startDate': '2020-10-10 12:29:06 +0100',\r\n          'endDate': '2020-10-10 12:29:07 +0100', 'value': '0.00518016'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '7.0.1',\r\n          'device': '<<HKDevice: 0x281cf6c70>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:7.0.1>',\r\n          'unit': 'km', 'creationDate': '2020-10-10 12:29:10 +0100', 'startDate': '2020-10-10 12:29:07 +0100',\r\n          'endDate': '2020-10-10 12:29:08 +0100', 'value': '0.00544049'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '6.2.6',\r\n          'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>',\r\n          'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:40:50 +0100',\r\n          'endDate': '2020-07-15 16:42:49 +0100', 'value': '0.952092', 'metadata_HKMetadataKeySyncVersion': '1',\r\n          'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520450.99823:616520569.99360:119'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '6.2.6',\r\n          'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>',\r\n          'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:42:49 +0100',\r\n          'endDate': '2020-07-15 16:44:51 +0100', 'value': '0.848983', 'metadata_HKMetadataKeySyncVersion': '1',\r\n          'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520569.99360:616520691.98826:119'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '6.2.6',\r\n          'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>',\r\n          'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:44:51 +0100',\r\n          'endDate': '2020-07-15 16:46:50 +0100', 'value': '0.834403', 'metadata_HKMetadataKeySyncVersion': '1',\r\n          'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520691.98826:616520810.98305:119'}]\r\n\r\n\r\n\r\ndef all_columns_old():\r\n    all_columns = [col for col in chunk[0]]\r\n    all_columns += [column for record in chunk\r\n                           for column in record if column not in all_columns]\r\n    return all_columns\r\n\r\n\r\ndef all_columns_new():\r\n    all_columns = [col for col in chunk[0]]\r\n    for record in chunk:\r\n        all_columns += [column for column in record if column not in all_columns]\r\n    return all_columns\r\n\r\n\r\n\r\nif __name__ == '__main__':\r\n    from pprint import pprint\r\n\r\n    print('problem: ')\r\n    pprint(all_columns_old())\r\n    print('\\nfix: ')\r\n    pprint(all_columns_new())\r\n\r\n```\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/225/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 808046597, "node_id": "MDU6SXNzdWU4MDgwNDY1OTc=", "number": 234, "title": ".insert_all() fails if subsequent chunks contain additional columns", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-02-14T21:01:51Z", "updated_at": "2021-02-14T21:03:40Z", "closed_at": "2021-02-14T21:03:40Z", "author_association": "OWNER", "pull_request": null, "body": "Reported by @nieuwenhoven in #225 along with a proposed fix.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 808036774, "node_id": "MDU6SXNzdWU4MDgwMzY3NzQ=", "number": 232, "title": "Run tests against Windows in GitHub Actions", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-14T20:09:45Z", "updated_at": "2021-02-14T20:39:55Z", "closed_at": "2021-02-14T20:39:55Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to try and get the test suite to run in Windows on GitHub Actions.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 808037010, "node_id": "MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4", "number": 233, "title": "Run tests against Ubuntu, macOS and Windows", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-14T20:11:02Z", "updated_at": "2021-02-14T20:39:54Z", "closed_at": "2021-02-14T20:39:54Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/233", "body": "Refs #232", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/233/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 808028757, "node_id": "MDU6SXNzdWU4MDgwMjg3NTc=", "number": 231, "title": "limit=X, offset=Y parameters for more Python methods", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-02-14T19:31:23Z", "updated_at": "2021-02-14T20:03:08Z", "closed_at": "2021-02-14T20:03:08Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to add a `offset=` parameter to support this case. Thanks for the suggestion!\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/224#issuecomment-778828495_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/231/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792297010, "node_id": "MDExOlB1bGxSZXF1ZXN0NTYwMjA0MzA2", "number": 224, "title": "Add fts offset docs.", "user": {"value": 37962604, "label": "polyrand"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-22T20:50:58Z", "updated_at": "2021-02-14T19:31:06Z", "closed_at": "2021-02-14T19:31:06Z", "author_association": "NONE", "pull_request": "simonw/sqlite-utils/pulls/224", "body": "The limit can be passed as a string to the query builder to have an offset. I have tested it using the shorthand `limit=f\"15, 30\"`, the standard syntax should work too.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/224/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 806743116, "node_id": "MDU6SXNzdWU4MDY3NDMxMTY=", "number": 1220, "title": "Installing datasette via docker: Path 'fixtures.db' does not exist", "user": {"value": 30607, "label": "aborruso"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-02-11T21:09:14Z", "updated_at": "2021-02-12T21:35:17Z", "closed_at": "2021-02-12T21:35:17Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nIf I run\r\n\r\n```\r\ndocker run -p 8001:8001 -v `pwd`:/mnt \\                                                                         1 \u21b5\r\n    datasetteproject/datasette \\\r\n    datasette -p 8001 -h 0.0.0.0 fixtures.db\r\n```\r\n\r\nI have \r\n\r\n```\r\nError: Invalid value for '[FILES]...': Path 'fixtures.db' does not exist.\r\n```\r\n\r\nIf I run `test -f fixtures.db && echo \"it exists.\"` I have `it exists.`.\r\n\r\nWhat's my error?\r\n\r\nThank you", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1220/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 806861312, "node_id": "MDExOlB1bGxSZXF1ZXN0NTcyMjA5MjQz", "number": 1222, "title": "--ssl-keyfile and --ssl-certfile, refs #1221", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-12T00:45:58Z", "updated_at": "2021-02-12T00:52:18Z", "closed_at": "2021-02-12T00:52:17Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1222", "body": "", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1222/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 770712149, "node_id": "MDExOlB1bGxSZXF1ZXN0NTQyNDA2OTEw", "number": 10, "title": "BugFix for encoding and not update info.", "user": {"value": 1277270, "label": "riverzhou"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-12-18T08:58:54Z", "updated_at": "2021-02-11T22:37:56Z", "closed_at": "2021-02-11T22:37:56Z", "author_association": "NONE", "pull_request": "dogsheep/evernote-to-sqlite/pulls/10", "body": "Bugfix 1:\r\n\r\nTraceback (most recent call last):\r\n  File \"d:\\anaconda3\\lib\\runpy.py\", line 194, in _run_module_as_main\r\n    return _run_code(code, main_globals, None,\r\n  File \"d:\\anaconda3\\lib\\runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"D:\\Anaconda3\\Scripts\\evernote-to-sqlite.exe\\__main__.py\", line 7, in <module>\r\n  File \"d:\\anaconda3\\lib\\site-packages\\click\\core.py\", line 829, in __call__\r\n  File \"d:\\anaconda3\\lib\\site-packages\\click\\core.py\", line 782, in main\r\n    rv = self.invoke(ctx)\r\n  File \"d:\\anaconda3\\lib\\site-packages\\click\\core.py\", line 1259, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"d:\\anaconda3\\lib\\site-packages\\click\\core.py\", line 610, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"d:\\anaconda3\\lib\\site-packages\\evernote_to_sqlite\\cli.py\", line 30, in enex\r\n    for tag, note in find_all_tags(fp, [\"note\"], progress_callback=bar.update):\r\n  File \"d:\\anaconda3\\lib\\site-packages\\evernote_to_sqlite\\utils.py\", line 11, in find_all_tags\r\n    chunk = fp.read(1024 * 1024)\r\nUnicodeDecodeError: 'gbk' codec can't decode byte 0xa4 in position 383: illegal multibyte sequence\r\n\r\nBugfix 2:\r\n\r\nTraceback (most recent call last):\r\n  File \"D:\\Anaconda3\\Scripts\\evernote-to-sqlite-script.py\", line 33, in <module>\r\n    sys.exit(load_entry_point('evernote-to-sqlite==0.3', 'console_scripts', 'evernote-to-sqlite')())\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\click\\core.py\", line 829, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\click\\core.py\", line 782, in main\r\n    rv = self.invoke(ctx)\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\click\\core.py\", line 1259, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\click\\core.py\", line 1066, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\click\\core.py\", line 610, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\evernote_to_sqlite-0.3-py3.8.egg\\evernote_to_sqlite\\cli.py\", line 31, in enex\r\n  File \"D:\\Anaconda3\\lib\\site-packages\\evernote_to_sqlite-0.3-py3.8.egg\\evernote_to_sqlite\\utils.py\", line 28, in save_note\r\nAttributeError: 'NoneType' object has no attribute 'text'", "repo": {"value": 303218369, "label": "evernote-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/10/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 748370021, "node_id": "MDExOlB1bGxSZXF1ZXN0NTI1MzcxMDI5", "number": 8, "title": "fix import error if note has no \"updated\" element", "user": {"value": 4028322, "label": "mkorosec"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-11-22T22:51:05Z", "updated_at": "2021-02-11T22:34:06Z", "closed_at": "2021-02-11T22:34:06Z", "author_association": "CONTRIBUTOR", "pull_request": "dogsheep/evernote-to-sqlite/pulls/8", "body": "I got the following error when executing evernote-to-sqlite enex evernote.db evernote.enex\r\n```   \r\n... \r\n  File \"evernote_to_sqlite/cli.py\", line 31, in enex\r\n    save_note(db, note)\r\n  File \"evernote_to_sqlite/utils.py\", line 28, in save_note\r\n    updated = note.find(\"updated\").text\r\nAttributeError: 'NoneType' object has no attribute 'text'\r\n```    \r\n\r\nSeems that in some cases the updated element is not added to the note, this is a part of the problematic note:\r\n\r\n```    \r\n<created>20201019T074518Z</created>\r\n<note-attributes>\r\n   <source>web.clip7</source>\r\n   <source-application>webclipper.evernote</source-application>\r\n</note-attributes>\r\n```", "repo": {"value": 303218369, "label": "evernote-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/8/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 743297582, "node_id": "MDU6SXNzdWU3NDMyOTc1ODI=", "number": 7, "title": "evernote-to-sqlite on windows 10 give this error: TypeError: insert() got an unexpected keyword argument 'replace'", "user": {"value": 42387931, "label": "martinvanwieringen"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-11-15T16:57:28Z", "updated_at": "2021-02-11T22:13:17Z", "closed_at": "2021-02-11T22:13:17Z", "author_association": "NONE", "pull_request": null, "body": "running evernote-to-sqlite 0.2 on windows 10. Command:\r\n\r\nevernote-to-sqlite enex evernote.db MyNotes.enex\r\n\r\nI get the followinng error:\r\n\r\nFile \"C:\\Users\\marti\\AppData\\Roaming\\Python\\Python38\\site-packages\\evernote_to_sqlite\\utils.py\", line 46, in save_note\r\n    note_id = db[\"notes\"].insert(row, hash_id=\"id\", replace=True, alter=True).last_pk\r\nTypeError: insert() got an unexpected keyword argument 'replace'\r\n\r\nRemoving replace=True,\r\n\r\nLeads to below error:\r\n\r\n  note_id = db[\"notes\"].insert(row, hash_id=\"id\", alter=True).last_pk\r\n  File \"C:\\Users\\marti\\AppData\\Roaming\\Python\\Python38\\site-packages\\sqlite_utils\\db.py\", line 924, in insert\r\n    return self.insert_all(\r\n  File \"C:\\Users\\marti\\AppData\\Roaming\\Python\\Python38\\site-packages\\sqlite_utils\\db.py\", line 1046, in insert_all\r\n    result = self.db.conn.execute(sql, values)\r\nsqlite3.IntegrityError: UNIQUE constraint failed: notes.id", "repo": {"value": 303218369, "label": "evernote-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 748372469, "node_id": "MDU6SXNzdWU3NDgzNzI0Njk=", "number": 9, "title": "ParseError: undefined entity &scaron;", "user": {"value": 4028322, "label": "mkorosec"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-11-22T23:04:35Z", "updated_at": "2021-02-11T22:10:55Z", "closed_at": "2021-02-11T22:10:55Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I encountered a parse error if the enex file contained &scaron; or &nbsp; \r\n\r\nRun command: \r\nevernote-to-sqlite enex evernote.db evernote.enex\r\n\r\n```\r\nTraceback (most recent call last):\r\n...\r\n  File \"evernote_to_sqlite/cli.py\", line 31, in enex\r\n    save_note(db, note)\r\n  File \"evernote_to_sqlite/utils.py\", line 35, in save_note\r\n    content = ET.tostring(ET.fromstring(content_xml)).decode(\"utf-8\")\r\n  File \"/usr/lib/python3.8/xml/etree/ElementTree.py\", line 1320, in XML\r\n    parser.feed(text)\r\nxml.etree.ElementTree.ParseError: undefined entity &scaron;: line 3, column 35\r\n```\r\n\r\nWorkaround:\r\n```\r\nsed -i 's/&scaron;//g' evernote.enex\r\nsed -i 's/&nbsp;//g' evernote.enex\r\n```", "repo": {"value": 303218369, "label": "evernote-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/9/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792851444, "node_id": "MDU6SXNzdWU3OTI4NTE0NDQ=", "number": 11, "title": "XML parse error", "user": {"value": 3613583, "label": "dskrad"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-24T17:38:54Z", "updated_at": "2021-02-11T21:18:58Z", "closed_at": "2021-02-11T21:18:48Z", "author_association": "NONE", "pull_request": null, "body": "I am on Windows 10 using Windows Subsystem for Linux, Python 3.8. I installed evernote-to-sqlite via pipx (in a venv). I tried using enex files from the latest version of Evernote for Windows (10.6.9 which only lets you export 50 notes at a time) and from Legacy Evernote (6.25.2.9198 which lets you export all your notes at once). The enex file from latest evernote gives this error:\r\n\r\n    File \"/usr/lib/python3.8/xml/etree/ElementTree.py\", line 1320, in XML parser.feed(text)\r\n    xml.etree.ElementTree.ParseError: XML or text declaration not at start of entity: line 2, column 6\r\n\r\nThe enex file from Legacy Evernote gives this error:\r\n\r\n    File \"/home/david/.local/pipx/venvs/evernote-to-sqlite/lib/python3.8/site-packages/evernote_to_sqlite/utils.py\", line 28, in save_note\r\n    updated = note.find(\"updated\").text\r\n    AttributeError: 'NoneType' object has no attribute 'text'", "repo": {"value": 303218369, "label": "evernote-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 802583450, "node_id": "MDU6SXNzdWU4MDI1ODM0NTA=", "number": 226, "title": "3.4 release is broken - includes a rogue line", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-02-06T02:08:01Z", "updated_at": "2021-02-06T02:10:26Z", "closed_at": "2021-02-06T02:10:26Z", "author_association": "OWNER", "pull_request": null, "body": "I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3\r\n\r\nThat was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199\r\n\r\nI'm surprised the tests didn't catch this!", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 788527932, "node_id": "MDU6SXNzdWU3ODg1Mjc5MzI=", "number": 223, "title": "--delimiter option for CSV import", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-18T20:25:03Z", "updated_at": "2021-02-06T01:39:47Z", "closed_at": "2021-02-06T01:34:54Z", "author_association": "OWNER", "pull_request": null, "body": "https://bruxellesdata.opendatasoft.com/explore/dataset/dog-toilets/export/?location=12,50.85802,4.38054 says:\r\n\r\n> CSV uses semicolon (;) as a separator.\r\n\r\nWould be useful to be able to do this:\r\n\r\n    sqlite-utils insert places.db places places.csv --delimiter ';'\r\n\r\n`--delimiter` could imply `--csv`", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/223/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 796234313, "node_id": "MDU6SXNzdWU3OTYyMzQzMTM=", "number": 1210, "title": "Immutable Database w/ Canned Queries", "user": {"value": 525780, "label": "heyarne"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-28T18:08:29Z", "updated_at": "2021-02-05T11:30:34Z", "closed_at": "2021-02-05T11:30:34Z", "author_association": "NONE", "pull_request": null, "body": "I have a database that I only want to read from; when instructing datasette to treat the database as immutable my defined canned queries disappear. Are these two features incompatible or have I hit an unintended bug? Thanks for datasette in any way, it's a joy to use!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1210/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 796736607, "node_id": "MDU6SXNzdWU3OTY3MzY2MDc=", "number": 56, "title": "Not all quoted statuses get fetched?", "user": {"value": 42315895, "label": "gsajko"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-01-29T09:48:44Z", "updated_at": "2021-02-03T10:36:36Z", "closed_at": "2021-02-03T10:36:36Z", "author_association": "NONE", "pull_request": null, "body": "\r\n![image](https://user-images.githubusercontent.com/42315895/106259325-5f75dc80-621f-11eb-8311-db8f2fe2a257.png)\r\n\r\nIn my database I have 13300 quote tweets, but eta 3600 have `quoted_status` empty.\r\n\r\nI fetched some of them using `https://api.twitter.com/1.1/statuses/show.json?id=xx` and they did have ids of quoted tweets.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 799693777, "node_id": "MDU6SXNzdWU3OTk2OTM3Nzc=", "number": 1214, "title": "Re-submitting filter form duplicates _x querystring arguments", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-02-02T21:13:35Z", "updated_at": "2021-02-02T21:28:53Z", "closed_at": "2021-02-02T21:21:13Z", "author_association": "OWNER", "pull_request": null, "body": "Really nasty bug, caused by #1194 fix in 07e163561592c743e4117f72102fcd350a600909\r\n\r\nNavigate to this page: https://github-to-sqlite.dogsheep.net/github/labels?_search=help&_sort=id\r\n\r\nClick \"Apply\" to submit the form and the resulting URL is https://github-to-sqlite.dogsheep.net/github/labels?_search=help&_sort=id&_search=help&_sort=id\r\n\r\nThat's because the (truncated) HTML for the form looks like this:\r\n\r\n```html\r\n    ... <input id=\"_search\" type=\"search\" name=\"_search\" value=\"help\">\r\n    ...\r\n            <div class=\"select-wrapper small-screen-only\">\r\n                <select name=\"_sort\" id=\"sort_by\">\r\n                    <option value=\"\">Sort...</option>\r\n                            <option value=\"id\" selected>Sort by id</option>\r\n                            <option value=\"node_id\">Sort by node_id</option>\r\n                            ...\r\n                </select>\r\n            </div>\r\n            ...\r\n            <input type=\"hidden\" name=\"_search\" value=\"help\">\r\n            <input type=\"hidden\" name=\"_sort\" value=\"id\">\r\n        <input type=\"submit\" value=\"Apply\">\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1214/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 793881756, "node_id": "MDU6SXNzdWU3OTM4ODE3NTY=", "number": 1207, "title": "Document the Datasette(..., pdb=True) testing pattern", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-26T02:48:10Z", "updated_at": "2021-01-29T02:37:19Z", "closed_at": "2021-01-29T02:12:34Z", "author_association": "OWNER", "pull_request": null, "body": "If you're writing tests for a Datasette plugin and you get a 500 error from inside Datasette, you can cause Datasette to open a PDB session within the application server code by doing this:\r\n\r\n```python\r\nds = Datasette([db_path], pdb=True)\r\nresponse = await ds.client.get(\"/\")\r\n```\r\n\r\nYou'll need to run `pytest -s` to interact with the debugger, otherwise you'll get an error.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1207/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 793027837, "node_id": "MDU6SXNzdWU3OTMwMjc4Mzc=", "number": 1205, "title": "Rename /:memory: to /_memory", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 3, "created_at": "2021-01-25T05:04:56Z", "updated_at": "2021-01-28T22:55:02Z", "closed_at": "2021-01-28T22:51:42Z", "author_association": "OWNER", "pull_request": null, "body": "For consistency with `/_internal` - and because then we don't need to escape the `:` characters.\r\n\r\nThis change would need to be in before Datasette 1.0. I could land it earlier and set up redirects from the old URLs though.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1205/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 770448622, "node_id": "MDU6SXNzdWU3NzA0NDg2MjI=", "number": 1151, "title": "Database class mechanism for cross-connection in-memory databases", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 11, "created_at": "2020-12-17T23:25:43Z", "updated_at": "2021-01-26T19:07:44Z", "closed_at": "2020-12-18T01:01:26Z", "author_association": "OWNER", "pull_request": null, "body": "> Next challenge: figure out how to use the `Database` class from https://github.com/simonw/datasette/blob/0.53/datasette/database.py for an in-memory database which persists data for the duration of the lifetime of the server, and allows access to that in-memory database from multiple threads in a way that lets them see each other's changes.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747768112_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1151/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792904595, "node_id": "MDU6SXNzdWU3OTI5MDQ1OTU=", "number": 1201, "title": "Release notes for Datasette 0.54", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 5, "created_at": "2021-01-24T21:22:28Z", "updated_at": "2021-01-25T17:42:21Z", "closed_at": "2021-01-25T17:42:21Z", "author_association": "OWNER", "pull_request": null, "body": "These will incorporate the release notes from the alpha, much expanded: https://github.com/simonw/datasette/releases/tag/0.54a0", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1201/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 793086333, "node_id": "MDExOlB1bGxSZXF1ZXN0NTYwODMxNjM4", "number": 1206, "title": "Release 0.54", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-01-25T06:45:47Z", "updated_at": "2021-01-25T17:33:30Z", "closed_at": "2021-01-25T17:33:29Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1206", "body": "Refs #1201", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1206/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 788447787, "node_id": "MDU6SXNzdWU3ODg0NDc3ODc=", "number": 1194, "title": "?_size= argument is not persisted by hidden form fields in the table filters", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-18T17:41:52Z", "updated_at": "2021-01-25T03:10:23Z", "closed_at": "2021-01-25T03:10:23Z", "author_association": "OWNER", "pull_request": null, "body": "Click \"Apply\" on https://covid-19.datasettes.com/covid/ny_times_us_counties?_size=1000&county__exact=San+Francisco&state__exact=California&_sort_desc=date#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=cases&g.y_type=quantitative and the `?_size=1000` parameter from the URL will no longer apply on the reloaded page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1194/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777145954, "node_id": "MDU6SXNzdWU3NzcxNDU5NTQ=", "number": 1167, "title": "Add Prettier to contributing documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2020-12-31T22:00:55Z", "updated_at": "2021-01-25T02:01:19Z", "closed_at": "2021-01-25T01:58:28Z", "author_association": "OWNER", "pull_request": null, "body": "Following #1166 - the docs at https://docs.datasette.io/en/stable/contributing.html should include a section about JavaScript, and it should document how to run Prettier.\r\n\r\nI run it in VS Code but it can be run on the command-line too:\r\n\r\n    npx prettier 'datasette/static/*[!.min].js' --write\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1167/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792958773, "node_id": "MDExOlB1bGxSZXF1ZXN0NTYwNzI1NzE0", "number": 1203, "title": "Easier way to run Prettier locally", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-25T01:39:06Z", "updated_at": "2021-01-25T01:41:46Z", "closed_at": "2021-01-25T01:41:46Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1203", "body": "Refs #1167", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1203/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 771208009, "node_id": "MDU6SXNzdWU3NzEyMDgwMDk=", "number": 1154, "title": "Documentation for new _internal database and tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 2, "created_at": "2020-12-18T22:34:52Z", "updated_at": "2021-01-25T00:09:22Z", "closed_at": "2021-01-25T00:08:41Z", "author_association": "OWNER", "pull_request": null, "body": "> Needs documentation, but I can wait to write that until I've tested out the feature a bit more.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-748352106_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1154/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792931244, "node_id": "MDU6SXNzdWU3OTI5MzEyNDQ=", "number": 1202, "title": "Documentation convention for marking unstable APIs.", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 2, "created_at": "2021-01-24T23:47:18Z", "updated_at": "2021-01-25T00:01:02Z", "closed_at": "2021-01-25T00:01:02Z", "author_association": "OWNER", "pull_request": null, "body": "> I'm going to document this but mark it as unstable, using a new documentation convention for marking unstable APIs.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1154#issuecomment-766462197_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1202/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 785588942, "node_id": "MDU6SXNzdWU3ODU1ODg5NDI=", "number": 1187, "title": "extra_body_script() support for script type=\"module\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2021-01-14T02:01:47Z", "updated_at": "2021-01-24T21:21:44Z", "closed_at": "2021-01-14T02:14:39Z", "author_association": "OWNER", "pull_request": null, "body": "Follows #1186. The `extra_body_script()` plugin hook should provide a mechanism for specifying that the script should use `<script type=\"module\">`.\r\n\r\nRelevant docs: https://docs.datasette.io/en/stable/plugin_hooks.html#extra-body-script-template-database-table-columns-view-name-request-datasette", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1187/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 785573793, "node_id": "MDU6SXNzdWU3ODU1NzM3OTM=", "number": 1186, "title": "script type=\"module\" support", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2021-01-14T01:17:47Z", "updated_at": "2021-01-24T21:21:41Z", "closed_at": "2021-01-14T01:50:58Z", "author_association": "OWNER", "pull_request": null, "body": "Custom JavaScript can be loaded in `metadata.json` like this:\r\n```json\r\n{\r\n    \"extra_js_urls\": [\r\n        {\r\n            \"url\": \"https://code.jquery.com/jquery-3.2.1.slim.min.js\",\r\n            \"sri\": \"sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g=\"\r\n        }\r\n    ]\r\n}\r\n```\r\nAdd a `\"module\": true` option which causes the resulting script element to use `<script type=\"module\">`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1186/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 784628163, "node_id": "MDU6SXNzdWU3ODQ2MjgxNjM=", "number": 1185, "title": "\"Statement may not contain PRAGMA\" error is not strictly true", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-12T22:07:10Z", "updated_at": "2021-01-24T21:21:37Z", "closed_at": "2021-01-12T22:26:26Z", "author_association": "OWNER", "pull_request": null, "body": "Consider https://latest.datasette.io/fixtures?sql=select+%27select%0D%0A%27+%7C%7C+group_concat%28%27++++case+when+%5B%27+%7C%7C+name+%7C%7C+%27%5D+is+not+null+then+%27+%7C%7C+quote%28name+%7C%7C+%27%2C+%27%29+%7C%7C+%27+else+%27%27%27%27+end%27%2C+%27+%7C%7C%0D%0A%27%29+%7C%7C+%27%0D%0A++as+columns%2C%0D%0A++count%28*%29+as+num_rows%0D%0Afrom%0D%0A++%5B%27+%7C%7C+%3Atable+%7C%7C+%27%5D%0D%0Agroup+by%0D%0A++columns%0D%0Aorder+by%0D%0A++num_rows+desc%27+as+query+from+pragma_ytable_info%28%3Atable%29&table=facetable\r\n\r\nIt says \"Statement may not contain PRAGMA\" - but that's not actually true. Datasette has an allow-list of PRAGMA that are OK - in this case there was a typo in `pragma_ytable_info` which caused the error, but pragma_table_info` would have been OK.\r\n\r\nSo the error message is misleading.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1185/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 783714076, "node_id": "MDU6SXNzdWU3ODM3MTQwNzY=", "number": 1184, "title": "request.full_path property", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 0, "created_at": "2021-01-11T21:21:58Z", "updated_at": "2021-01-24T21:21:16Z", "closed_at": "2021-01-11T21:34:47Z", "author_association": "OWNER", "pull_request": null, "body": "> I'll also add `request.full_path` for consistency with these: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/asgi.py#L77-L90\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1179#issuecomment-755495387_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1184/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 782692159, "node_id": "MDU6SXNzdWU3ODI2OTIxNTk=", "number": 1182, "title": "Retire \"Ecosystem\" page in favour of datasette.io/plugins and /tools", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-09T21:54:47Z", "updated_at": "2021-01-24T21:21:09Z", "closed_at": "2021-01-09T22:17:28Z", "author_association": "OWNER", "pull_request": null, "body": "https://docs.datasette.io/en/stable/ecosystem.html is no longer needed.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1182/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 780267857, "node_id": "MDU6SXNzdWU3ODAyNjc4NTc=", "number": 1178, "title": "Use force_https_urls on when deploying with Cloud Run", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 9, "created_at": "2021-01-06T08:20:55Z", "updated_at": "2021-01-24T21:21:05Z", "closed_at": "2021-01-06T18:24:47Z", "author_association": "OWNER", "pull_request": null, "body": "_Original title: datasette.absolute_url() should return https:// not http:// on Cloud Run_\r\n\r\nhttps://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_labels=on currently provides `http://` links, which break in Observable since it won't load `http://` content.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1178/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 778126516, "node_id": "MDExOlB1bGxSZXF1ZXN0NTQ4MjcxNDcy", "number": 1170, "title": "Install Prettier via package.json", "user": {"value": 3637, "label": "benpickles"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 3, "created_at": "2021-01-04T14:18:03Z", "updated_at": "2021-01-24T21:21:01Z", "closed_at": "2021-01-04T19:52:34Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1170", "body": "This adds a package.json with Prettier and means that developers/CI will use the same version. It also ensures that NPM packages are cached on GitHub Actions which fixes #1169.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1170/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 773913793, "node_id": "MDExOlB1bGxSZXF1ZXN0NTQ0OTIzNDM3", "number": 1158, "title": "Modernize code to Python 3.6+", "user": {"value": 6774676, "label": "eumiro"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 4, "created_at": "2020-12-23T16:21:38Z", "updated_at": "2021-01-24T21:20:50Z", "closed_at": "2020-12-23T17:04:32Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1158", "body": "- compact dict and set building\r\n- remove redundant parentheses\r\n- simplify chained conditions\r\n- change method name to lowercase\r\n- use triple double quotes for docstrings\r\n\r\nplease feel free to accept/reject any of these independent commits", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1158/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 772438273, "node_id": "MDU6SXNzdWU3NzI0MzgyNzM=", "number": 1157, "title": "Use time.perf_counter() instead of time.time() to measure performance", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2020-12-21T20:21:41Z", "updated_at": "2021-01-24T21:20:42Z", "closed_at": "2020-12-21T21:49:20Z", "author_association": "OWNER", "pull_request": null, "body": "I do that in a bunch of places: https://ripgrep.datasette.io/-/ripgrep?pattern=time%28%29&literal=on&glob=datasette%2F%2A%2A\r\n\r\nhttps://docs.python.org/3/library/time.html#time.perf_counter\r\n\r\n> `time.``perf_counter`() \u2192 float\r\n> \r\n> Return the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep and is system-wide. The reference point of the returned value is undefined, so that only the difference between the results of consecutive calls is valid.\r\n> \r\n> _New in version 3.3._", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1157/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 772408750, "node_id": "MDU6SXNzdWU3NzI0MDg3NTA=", "number": 1156, "title": "Rename _schemas to _internal", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2020-12-21T19:27:58Z", "updated_at": "2021-01-24T21:20:39Z", "closed_at": "2020-12-21T19:51:18Z", "author_association": "OWNER", "pull_request": null, "body": "I like `_internal` as the name for the in-memory Datasette schema database better.\r\n\r\nRefs #1154 #1150 #1155", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1156/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 766494367, "node_id": "MDExOlB1bGxSZXF1ZXN0NTM5NDg5NTI1", "number": 1145, "title": "Update pytest requirement from <6.2.0,>=5.2.2 to >=5.2.2,<6.3.0", "user": {"value": 27856297, "label": "dependabot-preview[bot]"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 1, "created_at": "2020-12-14T14:22:16Z", "updated_at": "2021-01-24T21:20:29Z", "closed_at": "2020-12-16T21:44:39Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1145", "body": "Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.\n<details>\n<summary>Release notes</summary>\n<p><em>Sourced from <a href=\"https://github.com/pytest-dev/pytest/releases\">pytest's releases</a>.</em></p>\n<blockquote>\n<h2>6.2.0</h2>\n<h1>pytest 6.2.0 (2020-12-12)</h1>\n<h2>Breaking Changes</h2>\n<ul>\n<li><a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7808\">#7808</a>: pytest now supports python3.6+ only.</li>\n</ul>\n<h2>Deprecations</h2>\n<ul>\n<li>\n<p><a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7469\">#7469</a>: Directly constructing/calling the following classes/functions is now deprecated:</p>\n<ul>\n<li><code>_pytest.cacheprovider.Cache</code></li>\n<li><code>_pytest.cacheprovider.Cache.for_config()</code></li>\n<li><code>_pytest.cacheprovider.Cache.clear_cache()</code></li>\n<li><code>_pytest.cacheprovider.Cache.cache_dir_from_config()</code></li>\n<li><code>_pytest.capture.CaptureFixture</code></li>\n<li><code>_pytest.fixtures.FixtureRequest</code></li>\n<li><code>_pytest.fixtures.SubRequest</code></li>\n<li><code>_pytest.logging.LogCaptureFixture</code></li>\n<li><code>_pytest.pytester.Pytester</code></li>\n<li><code>_pytest.pytester.Testdir</code></li>\n<li><code>_pytest.recwarn.WarningsRecorder</code></li>\n<li><code>_pytest.recwarn.WarningsChecker</code></li>\n<li><code>_pytest.tmpdir.TempPathFactory</code></li>\n<li><code>_pytest.tmpdir.TempdirFactory</code></li>\n</ul>\n<p>These have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 7.0.0.</p>\n</li>\n<li>\n<p><a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7530\">#7530</a>: The <code>--strict</code> command-line option has been deprecated, use <code>--strict-markers</code> instead.</p>\n<p>We have plans to maybe in the future to reintroduce <code>--strict</code> and make it an encompassing flag for all strictness\nrelated options (<code>--strict-markers</code> and <code>--strict-config</code> at the moment, more might be introduced in the future).</p>\n</li>\n<li>\n<p><a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7988\">#7988</a>: The <code>@pytest.yield_fixture</code> decorator/function is now deprecated. Use pytest.fixture instead.</p>\n<p><code>yield_fixture</code> has been an alias for <code>fixture</code> for a very long time, so can be search/replaced safely.</p>\n</li>\n</ul>\n<h2>Features</h2>\n<ul>\n<li>\n<p><a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/5299\">#5299</a>: pytest now warns about unraisable exceptions and unhandled thread exceptions that occur in tests on Python&gt;=3.8.\nSee unraisable for more information.</p>\n</li>\n<li>\n<p><a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7425\">#7425</a>: New pytester fixture, which is identical to testdir but its methods return pathlib.Path when appropriate instead of <code>py.path.local</code>.</p>\n<p>This is part of the movement to use pathlib.Path objects internally, in order to remove the dependency to <code>py</code> in the future.</p>\n<p>Internally, the old Testdir &lt;_pytest.pytester.Testdir&gt; is now a thin wrapper around Pytester &lt;_pytest.pytester.Pytester&gt;, preserving the old interface.</p>\n</li>\n</ul>\n<!-- raw HTML omitted -->\n</blockquote>\n</details>\n<details>\n<summary>Changelog</summary>\n<p><em>Sourced from <a href=\"https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst\">pytest's changelog</a>.</em></p>\n</details>\n<details>\n<summary>Commits</summary>\n<ul>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/e7073afe6e2373175130511135020af8b4e3a670\"><code>e7073af</code></a> Prepare release version 6.2.0</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/683f29f84df61e9dab92970a57190d20ca58385e\"><code>683f29f</code></a> Merge pull request <a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8129\">#8129</a> from bluetech/docs-pygments-workaround</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/0feeddf8edb87052402fafe690d019e3eb75dfa4\"><code>0feeddf</code></a> doc: temporary workaround for pytest-pygments lexing error</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/b4782757772373a2ea00802606407e9399b98031\"><code>b478275</code></a> Merge pull request <a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8128\">#8128</a> from bluetech/skip-reason-empty</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/3302ff994959964c3ec2ab35ec1fe98c038684c0\"><code>3302ff9</code></a> terminal: when the skip/xfail is empty, don't show it as &quot;()&quot;</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/59bd0f6912fd9ec2b4c3e3bbc989636bb5f54dbf\"><code>59bd0f6</code></a> Merge pull request <a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8126\">#8126</a> from bluetech/tox-regen-pretend-scm2</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/6298ff1f4e21307627b59aa86437145ddcf2bafe\"><code>6298ff1</code></a> tox: use pip legacy resolver for regen job</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/d51ecbd44d7e2f4f8c21f246fe9ed034bc8bae5b\"><code>d51ecbd</code></a> Merge pull request <a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8125\">#8125</a> from bluetech/tox-rm-pip-req</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/f237b077fc83fe834589e619be1e898f28026eb2\"><code>f237b07</code></a> tox: remove requires: pip&gt;=20.3.1</li>\n<li><a href=\"https://github.com/pytest-dev/pytest/commit/95e0e19b8d5b55b0462d420af7454f3bdcbe7fbc\"><code>95e0e19</code></a> Merge pull request <a href=\"https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8124\">#8124</a> from bluetech/s0undt3ch-feature/skip-context-hook</li>\n<li>Additional commits viewable in <a href=\"https://github.com/pytest-dev/pytest/compare/5.2.2...6.2.0\">compare view</a></li>\n</ul>\n</details>\n<br />\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n<details>\n<summary>Dependabot commands and options</summary>\n<br />\n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot merge` will merge this PR after your CI passes on it\n- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it\n- `@dependabot cancel merge` will cancel a previously requested merge and block automerging\n- `@dependabot reopen` will reopen this PR if it is closed\n- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language\n- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language\n- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language\n- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language\n- `@dependabot badge me` will comment on this PR with code to add a \"Dependabot enabled\" badge to your readme\n\nAdditionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):\n- Update frequency (including time of day and day of week)\n- Pull request limits (per update run and/or open at any time)\n- Out-of-range updates (receive only lockfile updates, if desired)\n- Security updates (receive only security updates, if desired)\n\n\n\n</details>", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1145/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 742011049, "node_id": "MDU6SXNzdWU3NDIwMTEwNDk=", "number": 1091, "title": ".json and .csv exports fail to apply base_url", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6346396, "label": "Datasette 0.54"}, "comments": 22, "created_at": "2020-11-12T23:45:16Z", "updated_at": "2021-01-24T21:20:24Z", "closed_at": "2021-01-09T22:19:29Z", "author_association": "OWNER", "pull_request": null, "body": "> Just tested with the latest Docker image, and it works pretty much everywhere!  THANK YOU!\r\n> \r\n> I did notice that if I try to export json or csv, the base is not applied.  Not sure if I should reopen this issue or open a new one.\r\n> \r\n> To see this, go here: https://corpora.tika.apache.org/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES\r\n> \r\n> Click/hover over json or CSV and you'll see that the 'datasette' base is not included.\r\n\r\n_Originally posted by @tballison in https://github.com/simonw/datasette/issues/865#issuecomment-726385422_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1091/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 760605882, "node_id": "MDU6SXNzdWU3NjA2MDU4ODI=", "number": 1135, "title": "Feature: --create option to create database file if it does not yet exist", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-12-09T19:23:58Z", "updated_at": "2021-01-24T21:19:39Z", "closed_at": "2020-12-09T19:45:52Z", "author_association": "OWNER", "pull_request": null, "body": "I'd like to be able to tell people to run the following in the Datasette documentation to get started:\r\n\r\n    brew install datasette\r\n    datasette install datasette-upload-csvs\r\n    datasette data.db --create --root --open\r\n\r\nThis would give them a local Datasette instance with the ability to drag-and-drop CSV files directly into it.\r\n\r\nJust one catch: I don't want to have to talk them through creating an empty SQLite database file. So I want to add a new `--create` option which means \"If any of the database files passed on the command-line do not yet exist, create them\".", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1135/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 791381623, "node_id": "MDU6SXNzdWU3OTEzODE2MjM=", "number": 1197, "title": "DB size limit for publishing with Heroku", "user": {"value": 1186275, "label": "mtdukes"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-21T18:08:43Z", "updated_at": "2021-01-24T20:53:44Z", "closed_at": "2021-01-24T20:53:44Z", "author_association": "NONE", "pull_request": null, "body": "Hello,\r\nI tried searching for this, but can't seem to get a great answer: Does anybody know the size limit for databases deploying to Heroku? The files I'm working with are pretty large, but I might be able to pare them down if I have a limit in mind.\r\n\r\n I'm getting the following error when running `datasette heroku publish`:\r\n\r\n`RangeError [ERR_INVALID_OPT_VALUE]: The value \"14504095744\" is invalid for option \"size\"`", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1197/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 792625812, "node_id": "MDU6SXNzdWU3OTI2MjU4MTI=", "number": 1198, "title": "Plugin testing documentation on using pytest-httpx", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-23T18:46:16Z", "updated_at": "2021-01-24T20:40:38Z", "closed_at": "2021-01-24T20:38:43Z", "author_association": "OWNER", "pull_request": null, "body": "I keep on having to figure this out: if you use the https://pypi.org/project/pytest-httpx/ fixture to write tests against mocked external APIs, they will fail because that module will break Datasette's own `datasette.client` testing mechanism.\r\n\r\nYou can fix this using:\r\n\r\n```python\r\n@pytest.fixture\r\ndef non_mocked_hosts():\r\n    return [\"localhost\"]\r\n```\r\n\r\nSee https://github.com/simonw/datasette-indieauth/blob/1.2/tests/test_indieauth.py\r\n\r\nI can add this tip to the https://docs.datasette.io/en/stable/testing_plugins.html page.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1198/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 725996507, "node_id": "MDU6SXNzdWU3MjU5OTY1MDc=", "number": 1036, "title": "Make it possible to download BLOB data from the Datasette UI", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 16, "created_at": "2020-10-20T22:47:56Z", "updated_at": "2021-01-18T17:45:00Z", "closed_at": "2020-10-25T00:14:52Z", "author_association": "OWNER", "pull_request": null, "body": "Currently you can only extract binary BLOB data as base64-encoded JSON, which is not user friendly at all. It should always be possible for end-users to get the binary data out.\r\n\r\nI'm worried about XSS vulnerabilities here, but hopefully sending `Content-Type: application/octet-stream` helps there? Need to research that.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1036/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 743400216, "node_id": "MDU6SXNzdWU3NDM0MDAyMTY=", "number": 11, "title": "Error thrown: sqlite3.OperationalError: table users has no column named lastName", "user": {"value": 61791, "label": "beaugunderson"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-11-16T01:21:18Z", "updated_at": "2021-01-18T04:35:22Z", "closed_at": "2021-01-18T04:35:22Z", "author_association": "NONE", "pull_request": null, "body": "Just installed `swarm-to-sqlite-0.3.2` and tried according to the docs:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/usr/local/bin/swarm-to-sqlite\", line 8, in <module>\r\n    sys.exit(cli())\r\n  File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 829, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 782, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1066, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 610, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/usr/local/lib/python3.9/site-packages/swarm_to_sqlite/cli.py\", line 73, in cli\r\n    save_checkin(checkin, db)\r\n  File \"/usr/local/lib/python3.9/site-packages/swarm_to_sqlite/utils.py\", line 82, in save_checkin\r\n    checkins_table.m2m(\"users\", user, m2m_table=\"likes\", pk=\"id\")\r\n  File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1914, in m2m\r\n    id = other_table.insert(record, pk=pk, replace=True).last_pk\r\n  File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1647, in insert\r\n    return self.insert_all(\r\n  File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1765, in insert_all\r\n    self.insert_chunk(\r\n  File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1575, in insert_chunk\r\n    result = self.db.execute(query, params)\r\n  File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 200, in execute\r\n    return self.conn.execute(sql, parameters)\r\nsqlite3.OperationalError: table users has no column named lastName\r\n```", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 787900412, "node_id": "MDU6SXNzdWU3ODc5MDA0MTI=", "number": 222, "title": ".m2m() should accept alter=True parameter", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-18T04:15:43Z", "updated_at": "2021-01-18T04:26:10Z", "closed_at": "2021-01-18T04:26:10Z", "author_association": "OWNER", "pull_request": null, "body": "Needed by https://github.com/dogsheep/swarm-to-sqlite/issues/11", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 783910901, "node_id": "MDU6SXNzdWU3ODM5MTA5MDE=", "number": 221, "title": ".add_missing_columns() does not take case insensitivity into account", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-12T05:01:00Z", "updated_at": "2021-01-12T23:17:33Z", "closed_at": "2021-01-12T23:17:33Z", "author_association": "OWNER", "pull_request": null, "body": "SQLite columns are case insensitive - but the `.add_missing_columns()` method doesn't know that. This means that it can crash if it identifies a column that is a case-insensitive duplicate of an existing column. https://github.com/simonw/sqlite-utils/blob/4cc82fd0bccc9d2eeb3510beb4e691d7da099f84/sqlite_utils/db.py#L1974-L1980", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 497170355, "node_id": "MDU6SXNzdWU0OTcxNzAzNTU=", "number": 576, "title": "Documented internals API for use in plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 10, "created_at": "2019-09-23T15:28:50Z", "updated_at": "2021-01-05T23:12:51Z", "closed_at": "2021-01-05T23:12:37Z", "author_association": "OWNER", "pull_request": null, "body": "Quite a few of the plugin hooks make a `datasette\u201d`instance of the Datasette class available to the plugins, so that they can look up configuration settings and execute database queries.\r\n\r\nThis means it should provide a documented, stable API so that plugin authors can rely on it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/576/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 435819321, "node_id": "MDU6SXNzdWU0MzU4MTkzMjE=", "number": 436, "title": "400 Error when trying to register new user via https://publish.datasettes.com/", "user": {"value": 317694, "label": "nniiicc"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-04-22T17:55:00Z", "updated_at": "2021-01-04T20:15:42Z", "closed_at": "2021-01-04T20:15:41Z", "author_association": "NONE", "pull_request": null, "body": "Behavior: When registering a new user via Zeit - confirmation is sent and screen acknowledges registered user... When clicking grant access the next screen is a white 400 error message. \r\n\r\nReplicated: Chrome and Firefox; 2 different email accounts", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/436/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 459537047, "node_id": "MDU6SXNzdWU0NTk1MzcwNDc=", "number": 517, "title": "Add unit test for \"static\" mechanism in plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-06-23T05:03:31Z", "updated_at": "2021-01-04T20:15:19Z", "closed_at": "2021-01-04T20:15:19Z", "author_association": "OWNER", "pull_request": null, "body": "Split out from #272 - this is actually quite tricky. Here's the relevant code:\r\n\r\nhttps://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/utils.py#L602-L614", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/517/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 377156339, "node_id": "MDU6SXNzdWUzNzcxNTYzMzk=", "number": 371, "title": "datasette publish digitalocean plugin", "user": {"value": 82988, "label": "psychemedia"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2018-11-04T14:07:41Z", "updated_at": "2021-01-04T20:14:28Z", "closed_at": "2021-01-04T20:14:28Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Provide support for launching `datasette` on Digital Ocean.\r\n\r\nExample: [Deploy Docker containers into Digital Ocean](https://blog.machinebox.io/deploy-machine-box-in-digital-ocean-385265fbeafd).\r\n\r\nDigital Ocean also has a preconfigured VM running Docker that can be launched from the command line via the Digital Ocean API: [Docker One-Click Application](https://www.digitalocean.com/docs/one-clicks/docker/).\r\n\r\nRelated:\r\n- Launching containers in Digital Ocean servers running docker: [How To Provision and Manage Remote Docker Hosts with Docker Machine on Ubuntu 16.04](https://www.digitalocean.com/community/tutorials/how-to-provision-and-manage-remote-docker-hosts-with-docker-machine-on-ubuntu-16-04)\r\n- [How To Use Doctl, the Official DigitalOcean Command-Line Client](https://www.digitalocean.com/community/tutorials/how-to-use-doctl-the-official-digitalocean-command-line-client)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/371/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 274264175, "node_id": "MDU6SXNzdWUyNzQyNjQxNzU=", "number": 102, "title": "datasette publish elasticbeanstalk", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2017-11-15T18:48:31Z", "updated_at": "2021-01-04T20:13:20Z", "closed_at": "2021-01-04T20:13:19Z", "author_association": "OWNER", "pull_request": null, "body": "It looks like Elastic Beanstalk is the most convenient way to deploy a docker container to AWS without first deploying a cluster.\r\n\r\nhttps://aws.amazon.com/blogs/devops/dockerizing-a-python-web-app/ looks helpful. We would need to automate the deployment with Boto: http://boto3.readthedocs.io/en/latest/reference/services/elasticbeanstalk.html", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/102/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 315142414, "node_id": "MDU6SXNzdWUzMTUxNDI0MTQ=", "number": 221, "title": "Allow plugins to add new cli sub commands ", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2018-04-17T16:40:13Z", "updated_at": "2021-01-04T20:12:14Z", "closed_at": "2021-01-04T20:12:14Z", "author_association": "OWNER", "pull_request": null, "body": "I could then test this out by having https://github.com/simonw/csvs-to-sqlite register itself as a plugin", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/221/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 267739593, "node_id": "MDU6SXNzdWUyNjc3Mzk1OTM=", "number": 18, "title": "See if I can get a websockets interface working", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2017-10-23T16:46:41Z", "updated_at": "2021-01-04T20:05:52Z", "closed_at": "2021-01-04T20:05:48Z", "author_association": "OWNER", "pull_request": null, "body": "Since I am already running on Sanic, how hard would it be to add a websocket ebdpoint that lets you talk to sqlite interactively?\r\n\r\nCould this be used to efficiently support streaming in answers to giant queries?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/18/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 274265878, "node_id": "MDU6SXNzdWUyNzQyNjU4Nzg=", "number": 103, "title": "datasette publish appengine", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2017-11-15T18:54:18Z", "updated_at": "2021-01-04T20:05:14Z", "closed_at": "2021-01-04T20:05:14Z", "author_association": "OWNER", "pull_request": null, "body": "Similar approach to Heroku, discussed in #90 \r\n\r\nLooks like this could be pretty easy:  https://cloud.google.com/appengine/docs/flexible/python/quickstart", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/103/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777677671, "node_id": "MDU6SXNzdWU3Nzc2Nzc2NzE=", "number": 1169, "title": "Prettier package not actually being cached", "user": {"value": 3637, "label": "benpickles"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-03T17:04:41Z", "updated_at": "2021-01-04T19:52:34Z", "closed_at": "2021-01-04T19:52:33Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "With the current configuration Prettier seems to be installed on every run - which can been [seen from the output](https://github.com/simonw/datasette/runs/1631686028?check_suite_focus=true#step:4:4):\r\n\r\n```\r\nnpx: installed 1 in 5.166s\r\n```\r\n\r\nPrettier isn't explicitly being installed (it's surprising that actually installing the dependencies isn't included in the [actions/cache docs](https://github.com/actions/cache/blob/main/examples.md#macos-and-ubuntu)) but it turns out that `npx` will automatically install the package for the specified command (it actually _guesses_ the package name from the name of the command). I'm not sure where Prettier ends up being installed but it doesn't appear to be in `~/.npm` according to the [post-cache output](https://github.com/simonw/datasette/runs/1631686028#step:7:2) (or `./node_modules` when I tested locally):\r\n\r\n```\r\nCache hit occurred on the primary key Linux-npm-565329898f77080e58b14d45cf816ab94877e6f2ece9d395c369c533548a7ee7, not saving cache.\r\n```\r\n\r\nI think there are a couple of approaches to tackling this, you could manually install/cache Prettier within the action, or add a `package.json` with Prettier. I would go with the latter because it's a more standard and maintainable approach and it will also ensure that, along with CI, anyone working on the project will run the same version of Prettier (you'll also get Dependabot JavaScript updates).\r\n\r\nI've tested the [`package.json` approach on a branch](https://github.com/simonw/datasette/compare/main...benpickles:cache-prettier) and am happy to turn it into a pull request if you fancy.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1169/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777707544, "node_id": "MDU6SXNzdWU3Nzc3MDc1NDQ=", "number": 219, "title": "reset_counts() method and command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-01-03T20:08:28Z", "updated_at": "2021-01-03T20:59:37Z", "closed_at": "2021-01-03T20:59:37Z", "author_association": "OWNER", "pull_request": null, "body": "> Thought: maybe there should be a `.reset_counts()` method too, for if the table gets out of date with the triggers.\r\n>\r\n> One way that could happen is if a table is dropped and recreated - the counts in the `_counts` table would likely no longer match the number of rows in that table.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/219/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777535402, "node_id": "MDU6SXNzdWU3Nzc1MzU0MDI=", "number": 215, "title": "Use _counts to speed up counts", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2021-01-02T22:30:17Z", "updated_at": "2021-01-03T20:19:40Z", "closed_at": "2021-01-03T20:19:40Z", "author_association": "OWNER", "pull_request": null, "body": "Utility mechanism for taking advantage of the new `_counts` table from #212 would be nice.\r\n\r\nThese can trigger automatically if the `_counts` table exists, but since `sqlite-utils` needs to work against any existing database there should be a way of opting out of this optimization.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/215/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 738514367, "node_id": "MDU6SXNzdWU3Mzg1MTQzNjc=", "number": 202, "title": "sqlite-utils insert -f colname - for configuring full-text search", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-11-08T17:30:09Z", "updated_at": "2021-01-03T05:00:36Z", "closed_at": "2021-01-03T05:00:27Z", "author_association": "OWNER", "pull_request": null, "body": "A mechanism for specifying columns that should be configured for full-text search as part of the initial data import:\r\n\r\n    sqlite-utils insert mydb.db articles articles.csv --csv -f title -f body", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/202/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777543336, "node_id": "MDU6SXNzdWU3Nzc1NDMzMzY=", "number": 217, "title": "Rename .escape() to .quote()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-02T23:40:52Z", "updated_at": "2021-01-03T04:27:38Z", "closed_at": "2021-01-03T04:15:23Z", "author_association": "OWNER", "pull_request": null, "body": "`.quote()` is a better name because it reflects that the method adds quotes around the value.\r\n\r\nThis method has never been documented so I'm going to rename it without a major version bump. ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/217/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777540352, "node_id": "MDU6SXNzdWU3Nzc1NDAzNTI=", "number": 216, "title": "database.triggers_dict introspection property", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-02T23:13:00Z", "updated_at": "2021-01-03T04:27:14Z", "closed_at": "2021-01-03T04:25:36Z", "author_association": "OWNER", "pull_request": null, "body": "Following #211", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777530107, "node_id": "MDU6SXNzdWU3Nzc1MzAxMDc=", "number": 214, "title": "sqlite-utils enable-counts command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-02T21:45:48Z", "updated_at": "2021-01-03T04:26:44Z", "closed_at": "2021-01-03T04:26:44Z", "author_association": "OWNER", "pull_request": null, "body": "The CLI version of #212 and #213.\r\n\r\n    # Enable counts for all tables:\r\n    sqlite-utils enable-counts data.db\r\n\r\n    # Enable counts for specific tables:\r\n    sqlite-utils enable-counts data.db table1 table2", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777560474, "node_id": "MDU6SXNzdWU3Nzc1NjA0NzQ=", "number": 218, "title": "\"sqlite-utils triggers\" command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-03T02:34:50Z", "updated_at": "2021-01-03T03:49:51Z", "closed_at": "2021-01-03T03:03:35Z", "author_association": "OWNER", "pull_request": null, "body": "A command to list the triggers in the database.\r\n\r\n    sqlite-utils triggers my.db\r\n\r\nCan optionally take one or more tables:\r\n\r\n    sqlite-utils triggers my.db table1 table2", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777529979, "node_id": "MDU6SXNzdWU3Nzc1Mjk5Nzk=", "number": 213, "title": "db.enable_counts() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-02T21:44:55Z", "updated_at": "2021-01-02T22:04:02Z", "closed_at": "2021-01-02T22:04:02Z", "author_association": "OWNER", "pull_request": null, "body": "Following #212 it would be useful if there was a utility method for enabling counts for ALL tables in a database:\r\n\r\n```python\r\ndb.enable_counts()\r\n```\r\n\r\nOpen question: should this setup triggers for virtual tables such as FTS tables? Could that break things?\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/213/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777392020, "node_id": "MDU6SXNzdWU3NzczOTIwMjA=", "number": 212, "title": "Mechanism for maintaining cache of table counts using triggers", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-01-02T02:58:53Z", "updated_at": "2021-01-02T21:40:27Z", "closed_at": "2021-01-02T21:40:27Z", "author_association": "OWNER", "pull_request": null, "body": "Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases.\r\n\r\nCarefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted.\r\n\r\n`sqlite-utils` already has a mechanism for creating triggers for FTS - the `table.enable_fts(..., create_triggers=True)` method. How about a similar mechanism for setting up triggers to maintain a count of table rows?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 777386465, "node_id": "MDU6SXNzdWU3NzczODY0NjU=", "number": 211, "title": "table.triggers_dict introspection property", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-01-02T02:04:00Z", "updated_at": "2021-01-02T02:10:10Z", "closed_at": "2021-01-02T02:10:10Z", "author_association": "OWNER", "pull_request": null, "body": "`table.triggers` currently returns a list of `Trigger` values. A `table.triggers_dict` property could behave like `columns_dict`, returning a dictionary mapping trigger names to their SQL definitions for that table.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 767685961, "node_id": "MDU6SXNzdWU3Njc2ODU5NjE=", "number": 210, "title": "Support of RData files", "user": {"value": 23739126, "label": "PeterBailey"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-12-15T15:04:14Z", "updated_at": "2021-01-02T00:02:40Z", "closed_at": "2021-01-02T00:02:40Z", "author_association": "NONE", "pull_request": null, "body": "Hi Simon,\r\n\r\nWould be great if you could ingest RData files!\r\n\r\nI could do this in a few lines of code but I am too lazy - sorry!\r\n\r\nPeter", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/210/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 766156875, "node_id": "MDU6SXNzdWU3NjYxNTY4NzU=", "number": 209, "title": "Test failure with sqlite 3.34 in test_cli.py::test_optimize", "user": {"value": 191622, "label": "meatcar"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-12-14T08:58:18Z", "updated_at": "2021-01-01T23:52:46Z", "closed_at": "2021-01-01T23:52:46Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "pytest output:\r\n```\r\n...\r\n============================== short test summary info ===============================\r\nFAILED tests/test_cli.py::test_optimize[tables0] - assert 1662976 < 1662976\r\nFAILED tests/test_cli.py::test_optimize[tables1] - assert 1667072 < 1662976\r\n===================== 2 failed, 538 passed, 3 skipped in 34.32s ======================\r\n```\r\n\r\nCame across this while packaging `sqlite-utils` for NixOS, but it can be recreated it using the `alpine:edge` docker image as well as follows:\r\n\r\n```\r\ndocker run --rm -it alpine:edge /bin/sh\r\n# apk update && apk add git sqlite python3 gcc python3-dev musl-dev && python3 -m ensurepip\r\n# git clone https://github.com/simonw/sqlite-utils.git\r\n# cd sqlite-utils/\r\n# pip3 install -e .[test]\r\n# pytest\r\n```\r\n\r\nThis definitely works on sqlite v3.33.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/209/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 770436876, "node_id": "MDU6SXNzdWU3NzA0MzY4NzY=", "number": 1150, "title": "Maintain an in-memory SQLite table of connected databases and their tables", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 32, "created_at": "2020-12-17T23:02:13Z", "updated_at": "2020-12-27T14:51:39Z", "closed_at": "2020-12-18T22:34:12Z", "author_association": "OWNER", "pull_request": null, "body": "I want Datasette to have its own internal metadata about connected tables, to power features like a paginated searchable homepage in #461. I want this to be a SQLite table.\r\n\r\nThis could also be part of the directory scanning mechanism prototyped in #672 - where Datasette can be set to continually scan a directory for new database files that it can serve.\r\n\r\nAlso relevant to the Datasette Library concept in #417.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1150/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 456568880, "node_id": "MDU6SXNzdWU0NTY1Njg4ODA=", "number": 509, "title": "Support opening multiple databases with the same stem", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2019-06-15T19:32:00Z", "updated_at": "2020-12-22T20:04:35Z", "closed_at": "2020-12-22T20:04:35Z", "author_association": "OWNER", "pull_request": null, "body": "e.g. I should be able to do this:\r\n\r\n    datasette App/data.db Other_App/data.db\r\n\r\nThis currently errors because you can't have two databases taking the `/data` URL path.\r\n\r\nInstead, how about in this particular case assigning the second database `/data-1`?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/509/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 771216293, "node_id": "MDU6SXNzdWU3NzEyMTYyOTM=", "number": 1155, "title": "Better internal database_name for _internal database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2020-12-18T22:47:27Z", "updated_at": "2020-12-22T20:04:35Z", "closed_at": "2020-12-22T20:04:35Z", "author_association": "OWNER", "pull_request": null, "body": "https://latest.datasette.io/login-as-root then https://latest.datasette.io/_internal\r\n\r\n<img width=\"1627\" alt=\"_schemas__columns__132_rows\" src=\"https://user-images.githubusercontent.com/9599/102668086-e70bfe00-413f-11eb-927d-e37ec9238e12.png\">\r\n\r\nThat `database_name` is ugly. It should just be `_internal`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1155/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}