{"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688482355", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688482355, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ4MjM1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T19:22:51Z", "updated_at": "2020-09-07T19:22:51Z", "author_association": "OWNER", "body": "And the SQLite documentation says:\r\n> When the REPLACE conflict resolution strategy deletes rows in order to satisfy a constraint, [delete triggers](https://www.sqlite.org/lang_createtrigger.html) fire if and only if [recursive triggers](https://www.sqlite.org/pragma.html#pragma_recursive_triggers) are enabled.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688482055", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688482055, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ4MjA1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T19:21:42Z", "updated_at": "2020-09-07T19:21:42Z", "author_association": "OWNER", "body": "Using `replace=True` there executes `INSERT OR REPLACE` - and Dan Kennedy (SQLite maintainer) on the SQLite forums said this:\r\n> Are you using \"REPLACE INTO\", or \"UPDATE OR REPLACE\" on the \"licenses\" table without having first executed \"PRAGMA recursive_triggers = 1\"? The docs note that delete triggers will not be fired in this case, which would explain things. Second paragraph under \"REPLACE\" here:\r\n>\r\n> https://www.sqlite.org/lang_conflict.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688481374", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688481374, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ4MTM3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T19:19:08Z", "updated_at": "2020-09-07T19:19:08Z", "author_association": "OWNER", "body": "reading through the code for `github-to-sqlite repos` - one of the things it does is calls `save_license` for each repo:\r\n\r\nhttps://github.com/dogsheep/github-to-sqlite/blob/39b2234253096bd579feed4e25104698b8ccd2ba/github_to_sqlite/utils.py#L259-L262\r\n\r\n```python\r\ndef save_license(db, license):\r\n if license is None:\r\n return None\r\n return db[\"licenses\"].insert(license, pk=\"key\", replace=True).last_pk\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688480665", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688480665, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ4MDY2NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T19:16:20Z", "updated_at": "2020-09-07T19:16:20Z", "author_association": "OWNER", "body": "Aha! I have managed to replicate the bug:\r\n```\r\n(github-to-sqlite) /tmp % sqlite-utils tables --counts github.db | grep licenses\r\n {\"table\": \"licenses\", \"count\": 7},\r\n {\"table\": \"licenses_fts_data\", \"count\": 35},\r\n {\"table\": \"licenses_fts_idx\", \"count\": 16},\r\n {\"table\": \"licenses_fts_docsize\", \"count\": 9151},\r\n {\"table\": \"licenses_fts_config\", \"count\": 1},\r\n {\"table\": \"licenses_fts\", \"count\": 7},\r\n(github-to-sqlite) /tmp % github-to-sqlite repos github.db dogsheep \r\n(github-to-sqlite) /tmp % sqlite-utils tables --counts github.db | grep licenses\r\n {\"table\": \"licenses\", \"count\": 7},\r\n {\"table\": \"licenses_fts_data\", \"count\": 45},\r\n {\"table\": \"licenses_fts_idx\", \"count\": 26},\r\n {\"table\": \"licenses_fts_docsize\", \"count\": 9161},\r\n {\"table\": \"licenses_fts_config\", \"count\": 1},\r\n {\"table\": \"licenses_fts\", \"count\": 7},\r\n```\r\nNote that the number of records in `licenses_fts_docsize` went from 9151 to 9161.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688464181", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688464181, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ2NDE4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T18:19:54Z", "updated_at": "2020-09-07T18:19:54Z", "author_association": "OWNER", "body": "Even though that table doesn't declare an integer primary key it does have a `rowid` column: https://github-to-sqlite.dogsheep.net/github?sql=select+rowid%2C+%5Bkey%5D%2C+name%2C+spdx_id%2C+url%2C+node_id+from+licenses+order+by+%5Bkey%5D+limit+101\r\n\r\n| rowid | key | name | spdx_id | url | node_id |\r\n| --- | --- | --- | --- | --- | --- |\r\n| 9150 | apache-2.0 | Apache License 2.0 | Apache-2.0 | | MDc6TGljZW5zZTI= |\r\n| 112 | bsd-3-clause | BSD 3-Clause \"New\" or \"Revised\" License | BSD-3-Clause | | MDc6TGljZW5zZTU= |\r\n\r\nhttps://www.sqlite.org/rowidtable.html explains has this clue:\r\n\r\n> If the rowid is not aliased by INTEGER PRIMARY KEY then it is not persistent and might change. In particular the VACUUM command will change rowids for tables that do not declare an INTEGER PRIMARY KEY. Therefore, applications should not normally access the rowid directly, but instead use an INTEGER PRIMARY KEY. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688460865", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688460865, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ2MDg2NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T18:07:14Z", "updated_at": "2020-09-07T18:07:14Z", "author_association": "OWNER", "body": "Another likely culprit: `licenses` has a text primary key, so it's not using `rowid`:\r\n```sql\r\nCREATE TABLE [licenses] (\r\n [key] TEXT PRIMARY KEY,\r\n [name] TEXT,\r\n [spdx_id] TEXT,\r\n [url] TEXT,\r\n [node_id] TEXT\r\n);\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688460729", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/149", "id": 688460729, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQ2MDcyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T18:06:44Z", "updated_at": "2020-09-07T18:06:44Z", "author_association": "OWNER", "body": "First posted on SQLite forum here but I'm pretty sure this is a bug in how `sqlite-utils` created those tables: https://sqlite.org/forum/forumpost/51aada1b45", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695319258, "label": "FTS table with 7 rows has _fts_docsize table with 9,141 rows"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/148#issuecomment-688434226", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/148", "id": 688434226, "node_id": "MDEyOklzc3VlQ29tbWVudDY4ODQzNDIyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-07T16:50:33Z", "updated_at": "2020-09-07T16:50:33Z", "author_association": "OWNER", "body": "This may be as easy as applying `textwrap.dedent()` to this: https://github.com/simonw/sqlite-utils/blob/0e62744da9a429093e3409575c1f881376b0361f/sqlite_utils/db.py#L778-L787\r\n\r\nI could apply that to a few other queries in that code as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 695276328, "label": "More attractive indentation of created FTS table schema"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/147#issuecomment-683528149", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/147", "id": 683528149, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzUyODE0OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-31T03:17:26Z", "updated_at": "2020-08-31T03:17:26Z", "author_association": "OWNER", "body": "+1 to making this something that users can customize. An optional argument to the `Database` constructor would be a neat way to do this.\r\n\r\nI think there's a terrifying way that we could find this value... we could perform a binary search for it! Open up a memory connection and try running different bulk inserts against it and catch the exceptions - then adjust and try again.\r\n\r\nMy hunch is that we could perform just 2 or 3 probes (maybe against carefully selected values) to find the highest value that works. If this process took less than a few ms to run I'd be happy to do it automatically when the class is instantiated (and let users disable that automatic proving by setting a value using the constructor argument).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688670158, "label": "SQLITE_MAX_VARS maybe hard-coded too low"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-683448569", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 683448569, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ0ODU2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T17:39:09Z", "updated_at": "2020-08-30T18:34:34Z", "author_association": "OWNER", "body": "So the steps needed are:\r\n\r\n- Download and extract latest CodeMirror zip file\r\n- Rename `lib/codemirror.js` to `codemirror-5.57.0.js`\r\n- Rename `lib/codemirror.css` to `codemirror-5.57.0.css`\r\n- Rename `mode/sql/sql.js` to `codemirror-5.57.0-sql.js`\r\n- Edit both JS files to make the top comment a `/* */` block\r\n- Minify JavaScript files like this:\r\n - `npx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js --comments '/LICENSE/'`\r\n - `npx uglify-js codemirror-5.57.0-sql.js -o codemirror-5.57.0-sql.min.js --comments '/LICENSE/'`\r\n- Check that the LICENSE comment did indeed survive minification\r\n- Minify CSS file like this:\r\n - `npx clean-css-cli codemirror-5.57.0.css -o codemirror-5.57.0.min.css`\r\n- Edit the `_codemirror.html` template to reference the new files\r\n- `git rm` the old files, `git add` the new files", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-683452613", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 683452613, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ1MjYxMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T18:16:28Z", "updated_at": "2020-08-30T18:16:28Z", "author_association": "OWNER", "body": "I added documentation on how to upgrade CodeMirror for the future here: https://docs.datasette.io/en/latest/contributing.html#upgrading-codemirror", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/655#issuecomment-683449837", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/655", "id": 683449837, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ0OTgzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T17:51:38Z", "updated_at": "2020-08-30T17:51:38Z", "author_association": "OWNER", "body": "I think was fixed by #948", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 542553350, "label": "Copy and paste doesn't work reliably on iPhone for SQL editor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-683449804", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 683449804, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ0OTgwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T17:51:18Z", "updated_at": "2020-08-30T17:51:18Z", "author_association": "OWNER", "body": "Copy and paste on mobile safari seems to work now. #655 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-683448635", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 683448635, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ0ODYzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T17:39:54Z", "updated_at": "2020-08-30T17:39:54Z", "author_association": "OWNER", "body": "I'll wait for this to deploy to https://latest.datasette.io/ and then test it in various desktop and mobile browsers.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-683445704", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 683445704, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ0NTcwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T17:11:58Z", "updated_at": "2020-08-30T17:33:30Z", "author_association": "OWNER", "body": "One catch: this stripped the license information from the top of the JS.\r\n\r\nI fixed this by editing the license to be a single `/* ... */` block comment instead of multiple `//` lines and running this:\r\n\r\n npx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js --comments '/LICENSE/'\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-683445114", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 683445114, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzQ0NTExNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T17:06:39Z", "updated_at": "2020-08-30T17:06:39Z", "author_association": "OWNER", "body": "Minifying using `npx`:\r\n```\r\nnpx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js\r\nnpx uglify-js codemirror-5.57.0-sql.js -o codemirror-5.57.0-sql.min.js\r\nnpx clean-css-cli codemirror-5.57.0.css -o codemirror-5.57.0.min.css\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/957#issuecomment-683357092", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/957", "id": 683357092, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzM1NzA5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T00:15:51Z", "updated_at": "2020-08-30T00:16:02Z", "author_association": "OWNER", "body": "Weirdly even removing this single `datasette` import from `utils/asgi.py` didn't fix the circular import:\r\n\r\nhttps://github.com/simonw/datasette/blob/44cf424a94a85b74552075272660bb96a7432661/datasette/utils/asgi.py#L1-L3", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688622148, "label": "Simplify imports of common classes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/957#issuecomment-683356440", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/957", "id": 683356440, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzM1NjQ0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T00:08:18Z", "updated_at": "2020-08-30T00:10:26Z", "author_association": "OWNER", "body": "Annoyingly this seems to be the line that causes the circular import:\r\n```python\r\nfrom .utils.asgi import Forbidden, NotFound, Response\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688622148, "label": "Simplify imports of common classes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/957#issuecomment-683355993", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/957", "id": 683355993, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzM1NTk5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-30T00:02:11Z", "updated_at": "2020-08-30T00:04:18Z", "author_association": "OWNER", "body": "I tried doing this and got this error:\r\n```\r\n(datasette) datasette % pytest\r\n==================================================================== test session starts =====================================================================\r\nplatform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1\r\nrootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini\r\nplugins: asyncio-0.14.0, timeout-1.4.2\r\ncollected 1 item / 23 errors \r\n\r\n=========================================================================== ERRORS ===========================================================================\r\n_____________________________________________________________ ERROR collecting tests/test_api.py _____________________________________________________________\r\nImportError while importing test module '/Users/simon/Dropbox/Development/datasette/tests/test_api.py'.\r\nHint: make sure your test modules/packages have valid Python names.\r\nTraceback:\r\n/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py:127: in import_module\r\n return _bootstrap._gcd_import(name[level:], package, level)\r\ntests/test_api.py:1: in \r\n from datasette.plugins import DEFAULT_PLUGINS\r\ndatasette/__init__.py:2: in \r\n from .app import Datasette\r\ndatasette/app.py:30: in \r\n from .views.base import DatasetteError, ureg\r\ndatasette/views/base.py:12: in \r\n from datasette.plugins import pm\r\ndatasette/plugins.py:26: in \r\n mod = importlib.import_module(plugin)\r\n/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py:127: in import_module\r\n return _bootstrap._gcd_import(name[level:], package, level)\r\ndatasette/publish/heroku.py:2: in \r\n from datasette import hookimpl\r\nE ImportError: cannot import name 'hookimpl' from partially initialized module 'datasette' (most likely due to a circular import) (/Users/simon/Dropbox/Development/datasette/datasette/__init__.py)\r\n```\r\nThat's with `datasette/__init__.py` looking like this:\r\n```python\r\nfrom datasette.version import __version_info__, __version__ # noqa\r\nfrom .app import Datasette\r\nfrom .utils.asgi import Forbidden, NotFound, Response\r\nfrom .utils import actor_matches_allow, QueryInterrupted\r\nfrom .hookspecs import hookimpl # noqa\r\nfrom .hookspecs import hookspec # noqa\r\n\r\n\r\n__all__ = [\r\n \"actor_matches_allow\",\r\n \"hookimpl\",\r\n \"hookspec\",\r\n \"QueryInterrupted\",\r\n \"Forbidden\",\r\n \"NotFound\",\r\n \"Response\",\r\n \"Datasette\",\r\n]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688622148, "label": "Simplify imports of common classes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/957#issuecomment-683355598", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/957", "id": 683355598, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzM1NTU5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T23:55:10Z", "updated_at": "2020-08-29T23:55:34Z", "author_association": "OWNER", "body": "Of these I think I'm going to promote the following to being importable directly `from datasette`:\r\n\r\n- `from datasette.app import Datasette`\r\n- `from datasette.utils import QueryInterrupted`\r\n- `from datasette.utils.asgi import Response, Forbidden, NotFound`\r\n- `from datasette.utils import actor_matches_allow`\r\n\r\nAll of the rest are infrequently used enough (or clearly named enough) that I'm happy to leave them as-is.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688622148, "label": "Simplify imports of common classes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/957#issuecomment-683355508", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/957", "id": 683355508, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzM1NTUwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T23:54:01Z", "updated_at": "2020-08-29T23:54:01Z", "author_association": "OWNER", "body": "Reviewing https://github.com/search?q=user%3Asimonw+%22from+datasette%22&type=Code I spotted these others:\r\n\r\n```python\r\n# Various:\r\nfrom datasette.utils import path_with_replaced_args\r\nfrom datasette.plugins import pm\r\nfrom datasette.utils import QueryInterrupted\r\nfrom datasette.utils.asgi import Response, Forbidden, NotFound\r\n\r\n# datasette-publish-vercel:\r\nfrom datasette.publish.common import (\r\n add_common_publish_arguments_and_options,\r\n fail_if_publish_binary_not_installed\r\n)\r\nfrom datasette.utils import temporary_docker_directory\r\n\r\n# datasette-insert\r\nfrom datasette.utils import actor_matches_allow, sqlite3\r\n\r\n# obsolete: russian-ira-facebook-ads-datasette \r\nfrom datasette.utils import TableFilter\r\n\r\n# simonw/museums\r\nfrom datasette.utils.asgi import asgi_send\r\n\r\n# datasette-media\r\nfrom datasette.utils.asgi import Response, asgi_send_file\r\n\r\n# datasette/tests/plugins/my_plugin.py\r\nfrom datasette.facets import Facet\r\n\r\n# datasette-graphql\r\nfrom datasette.views.table import TableView\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688622148, "label": "Simplify imports of common classes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/956#issuecomment-683214102", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/956", "id": 683214102, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzIxNDEwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T01:32:21Z", "updated_at": "2020-08-29T01:32:21Z", "author_association": "OWNER", "body": "Maybe the bug here is the double colon?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688427751, "label": "Push to Docker Hub failed - but it shouldn't run for alpha releases anyway"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/956#issuecomment-683213973", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/956", "id": 683213973, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzIxMzk3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T01:31:39Z", "updated_at": "2020-08-29T01:31:39Z", "author_association": "OWNER", "body": "Here's how the old Travis mechanism worked: https://github.com/simonw/datasette/blob/52eabb019d4051084b21524bd0fd9c2731126985/.travis.yml#L41-L47\r\n\r\nSo I was assuming that the eqivalent of `$REPO:$TRAVIS_TAG` in GitHub Actions is `$REPO::${GITHUB_REF#refs/tags/}`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688427751, "label": "Push to Docker Hub failed - but it shouldn't run for alpha releases anyway"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/956#issuecomment-683212960", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/956", "id": 683212960, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzIxMjk2MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T01:25:34Z", "updated_at": "2020-08-29T01:25:34Z", "author_association": "OWNER", "body": "So I guess this bit is wrong: https://github.com/simonw/datasette/blob/c36e287d71d68ecb2a45e9808eede15f19f931fb/.github/workflows/publish.yml#L71-L73", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688427751, "label": "Push to Docker Hub failed - but it shouldn't run for alpha releases anyway"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/956#issuecomment-683212421", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/956", "id": 683212421, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzIxMjQyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T01:22:23Z", "updated_at": "2020-08-29T01:22:23Z", "author_association": "OWNER", "body": "Here's the error message again:\r\n\r\n> invalid argument `\"***/datasette::0.49a0\"` for `\"-t, --tag\"` flag: invalid reference format", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688427751, "label": "Push to Docker Hub failed - but it shouldn't run for alpha releases anyway"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/956#issuecomment-683212246", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/956", "id": 683212246, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzIxMjI0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-29T01:21:26Z", "updated_at": "2020-08-29T01:21:26Z", "author_association": "OWNER", "body": "I added this but I have no idea if I got it right or not: https://github.com/simonw/datasette/blob/c36e287d71d68ecb2a45e9808eede15f19f931fb/.github/workflows/publish.yml#L58-L63", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688427751, "label": "Push to Docker Hub failed - but it shouldn't run for alpha releases anyway"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/955#issuecomment-683189334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/955", "id": 683189334, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE4OTMzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T23:30:48Z", "updated_at": "2020-08-28T23:30:48Z", "author_association": "OWNER", "body": "Also https://github.com/simonw/datasette-copyable", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 687711713, "label": "Release updated datasette-atom and datasette-ics"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/955#issuecomment-683185861", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/955", "id": 683185861, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE4NTg2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T23:17:09Z", "updated_at": "2020-08-28T23:17:09Z", "author_association": "OWNER", "body": "I released 0.49a0 which means I can update the main branches of those two plugins - I'll push a release of them once 0.49 is fully shipped.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 687711713, "label": "Release updated datasette-atom and datasette-ics"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/144#issuecomment-683180581", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/144", "id": 683180581, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE4MDU4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:57:04Z", "updated_at": "2020-08-28T22:57:04Z", "author_association": "OWNER", "body": "That worked! https://github.com/simonw/sqlite-utils/runs/1043640785?check_suite_focus=true\r\n\r\n\"Add_numpy_to_the_matrix__refs__144_\u00b7_simonw_sqlite-utils_cbc22ef\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688395275, "label": "Run some tests against numpy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/144#issuecomment-683179678", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/144", "id": 683179678, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3OTY3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:53:17Z", "updated_at": "2020-08-28T22:53:17Z", "author_association": "OWNER", "body": "I'm going to try doing this as a GitHub Actions test matrix.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688395275, "label": "Run some tests against numpy"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/139#issuecomment-683178570", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139", "id": 683178570, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3ODU3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:48:51Z", "updated_at": "2020-08-28T22:48:51Z", "author_association": "OWNER", "body": "Thanks @simonwiles, this is now released in 2.16.1: https://sqlite-utils.readthedocs.io/en/stable/changelog.html", "reactions": "{\"total_count\": 2, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 686978131, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/143#issuecomment-683175491", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/143", "id": 683175491, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3NTQ5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:37:15Z", "updated_at": "2020-08-28T22:37:15Z", "author_association": "OWNER", "body": "I'm going to start running black exclusively in the GitHub Actions workflow, rather than having it run by the unit tests themselves.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688389933, "label": "Move to GitHub Actions CI"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683173375", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142", "id": 683173375, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3MzM3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:29:02Z", "updated_at": "2020-08-28T22:29:02Z", "author_association": "OWNER", "body": "Yeah I think that failure is actually because there's a brand new release of Black out and it subtly changes some of the formatting rules. I'll merge this and then run Black against the entire codebase.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688386219, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683172829", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142", "id": 683172829, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3MjgyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:27:05Z", "updated_at": "2020-08-28T22:27:05Z", "author_association": "OWNER", "body": "Looks like it failed the \"black\" formatting test - possibly because there's a new release if black out. I'm going to merge despite that failure.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688386219, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683172082", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142", "id": 683172082, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3MjA4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:24:25Z", "updated_at": "2020-08-28T22:24:25Z", "author_association": "OWNER", "body": "Thanks very much!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688386219, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/119#issuecomment-683146200", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/119", "id": 683146200, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE0NjIwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T21:05:37Z", "updated_at": "2020-08-28T21:05:37Z", "author_association": "OWNER", "body": "Maybe use `transform_table()` in #114 for this? Would be less efficient as it would copy the whole table but it would reduce library complexity a bit.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 652700770, "label": "Ability to remove a foreign key"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682771226", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139", "id": 682771226, "node_id": "MDEyOklzc3VlQ29tbWVudDY4Mjc3MTIyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T15:57:42Z", "updated_at": "2020-08-28T15:57:42Z", "author_association": "OWNER", "body": "That pull request should update this section of the documentation too:\r\n\r\n> If you have more than one record to insert, the insert_all() method is a much more efficient way of inserting them. Just like insert() it will automatically detect the columns that should be created, but it will inspect the first batch of 100 items to help decide what those column types should be.\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/ea87c2b943fdd162c42a900ac0aea5ecc2f4b9d9/docs/python-api.rst#L393", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 686978131, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682762911", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139", "id": 682762911, "node_id": "MDEyOklzc3VlQ29tbWVudDY4Mjc2MjkxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T15:54:57Z", "updated_at": "2020-08-28T15:55:20Z", "author_association": "OWNER", "body": "Here's a suggested test update:\r\n```diff\r\ndiff --git a/sqlite_utils/db.py b/sqlite_utils/db.py\r\nindex a8791c3..12fa2f2 100644\r\n--- a/sqlite_utils/db.py\r\n+++ b/sqlite_utils/db.py\r\n@@ -1074,6 +1074,13 @@ class Table(Queryable):\r\n all_columns = list(sorted(all_columns))\r\n if hash_id:\r\n all_columns.insert(0, hash_id)\r\n+ else:\r\n+ all_columns += [\r\n+ column\r\n+ for record in chunk\r\n+ for column in record\r\n+ if column not in all_columns\r\n+ ]\r\n validate_column_names(all_columns)\r\n first = False\r\n # values is the list of insert data that is passed to the\r\ndiff --git a/tests/test_create.py b/tests/test_create.py\r\nindex a84eb8d..3a7fafc 100644\r\n--- a/tests/test_create.py\r\n+++ b/tests/test_create.py\r\n@@ -707,13 +707,15 @@ def test_insert_thousands_using_generator(fresh_db):\r\n assert 10000 == fresh_db[\"test\"].count\r\n \r\n \r\n-def test_insert_thousands_ignores_extra_columns_after_first_100(fresh_db):\r\n+def test_insert_thousands_adds_extra_columns_after_first_100(fresh_db):\r\n+ # https://github.com/simonw/sqlite-utils/issues/139\r\n fresh_db[\"test\"].insert_all(\r\n [{\"i\": i, \"word\": \"word_{}\".format(i)} for i in range(100)]\r\n- + [{\"i\": 101, \"extra\": \"This extra column should cause an exception\"}]\r\n+ + [{\"i\": 101, \"extra\": \"Should trigger ALTER\"}],\r\n+ alter=True,\r\n )\r\n rows = fresh_db.execute_returning_dicts(\"select * from test where i = 101\")\r\n- assert [{\"i\": 101, \"word\": None}] == rows\r\n+ assert [{\"i\": 101, \"word\": None, \"extra\": \"Should trigger ALTER\"}] == rows\r\n \r\n \r\n def test_insert_ignore(fresh_db):\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 686978131, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/954#issuecomment-682312736", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/954", "id": 682312736, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MjMxMjczNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T04:05:01Z", "updated_at": "2020-08-28T04:05:10Z", "author_association": "OWNER", "body": "> It can also return a dictionary with the following keys. This format is **deprecated** as-of Datasette 0.49 and will be removed by Datasette 1.0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 687694947, "label": "Remove old register_output_renderer dict mechanism in Datasette 1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/953#issuecomment-682312494", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/953", "id": 682312494, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MjMxMjQ5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T04:03:56Z", "updated_at": "2020-08-28T04:03:56Z", "author_association": "OWNER", "body": "Documentation says that the old dictionary mechanism will be deprecated by 1.0:\r\n\r\nhttps://github.com/simonw/datasette/blob/799ecae94824640bdff21f86997f69844048d5c3/docs/plugin_hooks.rst#L460", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 687681018, "label": "register_output_renderer render function should be able to return a Response"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682285212", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139", "id": 682285212, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MjI4NTIxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T02:12:51Z", "updated_at": "2020-08-28T02:12:51Z", "author_association": "OWNER", "body": "I'd be happy to accept a PR for this, provided it included updated unit tests that illustrate it working. I think this is a really good improvement.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 686978131, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682284908", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/139", "id": 682284908, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MjI4NDkwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T02:11:40Z", "updated_at": "2020-08-28T02:11:40Z", "author_association": "OWNER", "body": "This is deliberate behaviour, but I'm not at all attached to it - you're right in pointing out that it's actually pretty unexpected.\r\n\r\nI'd be happy to change this behaviour so if you pass `alter=True` and then use `.insert_all()` on more than 100 rows it works as you would expect, instead of silently ignoring new columns past the first 100 rows. I don't expect that anyone would be depending on the current behaviour (ignore new columns after the first 100) such that this should be considered a backwards incompatible change.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 686978131, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/950#issuecomment-680374196", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/950", "id": 680374196, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MDM3NDE5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-26T00:43:50Z", "updated_at": "2020-08-26T00:43:50Z", "author_association": "OWNER", "body": "The problem with the term \"private\" is that it could be confused with the concept of databases that aren't visible to the public due to the permissions system - the ones that are displayed with the padlock icon e.g. on https://datasette-auth-passwords-demo.datasette.io/\r\n\r\n\"datasette-auth-passwords_demo__public__private\"\r\n\r\nSo I think \"secret\" is a better term for these.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 685806511, "label": "Private/secret databases: database files that are only visible to plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/950#issuecomment-680264202", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/950", "id": 680264202, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MDI2NDIwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-25T20:53:13Z", "updated_at": "2020-08-25T20:53:13Z", "author_association": "OWNER", "body": "Forcing people to spell out `datasette github.db --private private.db` isn't terrible though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 685806511, "label": "Private/secret databases: database files that are only visible to plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/950#issuecomment-680263999", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/950", "id": 680263999, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MDI2Mzk5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-25T20:52:47Z", "updated_at": "2020-08-25T20:52:47Z", "author_association": "OWNER", "body": "Naming challenge: secret databases or private databases?\r\n\r\nI prefer private. But `datasette -p` is already taken by `--port`. `datasette -s` is currently available.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 685806511, "label": "Private/secret databases: database files that are only visible to plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/950#issuecomment-680263427", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/950", "id": 680263427, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MDI2MzQyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-25T20:51:30Z", "updated_at": "2020-08-25T20:52:13Z", "author_association": "OWNER", "body": "`datasette-graphql` currently dispatches requests through the `TableView` class, so if that couldn't access private databases then it would not be able to either. See also the concept for `datasette.get(...)` as an internal API in #943 - that might need to have a mechanism for also being able to query private databases, maybe `datasette.get(path, allow_private=True)`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 685806511, "label": "Private/secret databases: database files that are only visible to plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/949#issuecomment-679367931", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/949", "id": 679367931, "node_id": "MDEyOklzc3VlQ29tbWVudDY3OTM2NzkzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-24T21:09:50Z", "updated_at": "2020-08-24T21:09:50Z", "author_association": "OWNER", "body": "I'm attracted to this because of how good GraphiQL is for auto-completing queries. But I realize there's a problem here: GraphQL is designed to be autocomplete-friendly, but SQL is not. If you type `select ` and it doesn't know what's going in the `from` clause it can't give you good column autocomplete, for example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684961449, "label": "Try out CodeMirror SQL hints"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/949#issuecomment-679363710", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/949", "id": 679363710, "node_id": "MDEyOklzc3VlQ29tbWVudDY3OTM2MzcxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-24T21:00:43Z", "updated_at": "2020-08-24T21:00:43Z", "author_association": "OWNER", "body": "I think this requires three extra files from https://github.com/codemirror/CodeMirror/tree/5.57.0/addon/hint\r\n\r\n- `show-hint.css`\r\n- `show-hint.js`\r\n- `sql-hint.js`\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684961449, "label": "Try out CodeMirror SQL hints"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-679355426", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 679355426, "node_id": "MDEyOklzc3VlQ29tbWVudDY3OTM1NTQyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-24T20:43:07Z", "updated_at": "2020-08-24T20:43:07Z", "author_association": "OWNER", "body": "It would also be interesting to try out the SQL hint mode, which can autocomplete against tables and columns. This demo shows how to configure that: https://codemirror.net/mode/sql/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/948#issuecomment-679333717", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/948", "id": 679333717, "node_id": "MDEyOklzc3VlQ29tbWVudDY3OTMzMzcxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-24T19:55:59Z", "updated_at": "2020-08-24T19:55:59Z", "author_association": "OWNER", "body": "CodeMirror 6 is in pre-release at the moment and is a complete rewrite. I'll stick with the 5.x series for now. https://github.com/codemirror/codemirror.next/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684925907, "label": "Upgrade CodeMirror"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/138#issuecomment-678732667", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/138", "id": 678732667, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODczMjY2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-23T05:46:10Z", "updated_at": "2020-08-23T05:46:10Z", "author_association": "OWNER", "body": "Actually the `TEXT` column thing wasn't a `sqlite-utils` issue, it was unique to how `shapefile-to-spatialite` was creating the table when using the SpatiaLite extension.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684118950, "label": "extracts= doesn't configure foreign keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/136#issuecomment-678508056", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/136", "id": 678508056, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODUwODA1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T21:13:41Z", "updated_at": "2020-08-21T21:13:41Z", "author_association": "OWNER", "body": "The `--spatialite` option should be available for other useful commands too, refs #137.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683812642, "label": "--load-extension=spatialite shortcut option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/137#issuecomment-678507502", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/137", "id": 678507502, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODUwNzUwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T21:13:19Z", "updated_at": "2020-08-21T21:13:19Z", "author_association": "OWNER", "body": "Adding `--spatialite` too would be great for usability: #136", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683830416, "label": "--load-extension for other sqlite-utils commands"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678497497", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/134", "id": 678497497, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ5NzQ5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T21:06:26Z", "updated_at": "2020-08-21T21:06:26Z", "author_association": "OWNER", "body": "Ended up needing two skipIfs:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/7e9aad7e1c09d1cf80d0b4d17d6157212a4b857d/tests/test_cli.py#L888-L893", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683804172, "label": "--load-extension option for sqlite-utils query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/136#issuecomment-678480969", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/136", "id": 678480969, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ4MDk2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:33:45Z", "updated_at": "2020-08-21T20:33:45Z", "author_association": "OWNER", "body": "I think this should initialize SpatiaLite against the current database if it has not been initialized already.\r\n\r\nRelevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126\r\n\r\n```python\r\ndef init_spatialite(db, lib):\r\n db.conn.enable_load_extension(True)\r\n db.conn.load_extension(lib)\r\n # Initialize SpatiaLite if not yet initialized\r\n if \"spatial_ref_sys\" in db.table_names():\r\n return\r\n db.conn.execute(\"select InitSpatialMetadata(1)\")\r\n\r\n\r\ndef ensure_table_has_geometry(db, table, table_srid):\r\n if \"geometry\" not in db[table].columns_dict:\r\n db.conn.execute(\r\n \"SELECT AddGeometryColumn(?, 'geometry', ?, 'GEOMETRY', 2);\",\r\n [table, table_srid],\r\n )\r\n```\r\nNot sure if I should add a utility function or CLI command for that `ensure_table_has_geometry` bit.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683812642, "label": "--load-extension=spatialite shortcut option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/135#issuecomment-678479741", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/135", "id": 678479741, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ3OTc0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:30:37Z", "updated_at": "2020-08-21T20:30:37Z", "author_association": "OWNER", "body": "Docs: https://github.com/simonw/sqlite-utils/blob/bf4c6b7c82fab6b2400e48424f8dac1ae2f0a2dc/docs/python-api.rst#finding-spatialite", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683805434, "label": "Code for finding SpatiaLite in the usual locations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/135#issuecomment-678476842", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/135", "id": 678476842, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ3Njg0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:23:13Z", "updated_at": "2020-08-21T20:23:13Z", "author_association": "OWNER", "body": "I'm going to start with just the first two - I'm not convinced I understand the `.so.5` variants.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683805434, "label": "Code for finding SpatiaLite in the usual locations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678476338", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/134", "id": 678476338, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ3NjMzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:22:02Z", "updated_at": "2020-08-21T20:22:02Z", "author_association": "OWNER", "body": "I think that adds it as `/usr/lib/x86_64-linux-gnu/mod_spatialite.so`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683804172, "label": "--load-extension option for sqlite-utils query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/135#issuecomment-678475578", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/135", "id": 678475578, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ3NTU3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:20:05Z", "updated_at": "2020-08-21T20:20:05Z", "author_association": "OWNER", "body": "https://github.com/simonw/cryptozoology/blob/2ad69168f3b78ebd90a2cbeea8136c9115e2a9b7/build_cryptids_database.py#L16-L22\r\n```python\r\ntry_these = (\r\n \"mod_spatialite\",\r\n \"/usr/local/lib/mod_spatialite.dylib\",\r\n \"/usr/lib/x86_64-linux-gnu/mod_spatialite.so\",\r\n \"/usr/lib/x86_64-linux-gnu/libspatialite.so.5\",\r\n \"/usr/lib/x86_64-linux-gnu/libspatialite.so.7\",\r\n)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683805434, "label": "Code for finding SpatiaLite in the usual locations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678474928", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/134", "id": 678474928, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ3NDkyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:18:33Z", "updated_at": "2020-08-21T20:18:33Z", "author_association": "OWNER", "body": "This should get me SpatiaLite in the GitHub Actions Ubuntu:\r\n```\r\napt install libsqlite3-mod-spatialite\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683804172, "label": "--load-extension option for sqlite-utils query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678474018", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/134", "id": 678474018, "node_id": "MDEyOklzc3VlQ29tbWVudDY3ODQ3NDAxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-21T20:16:20Z", "updated_at": "2020-08-21T20:16:20Z", "author_association": "OWNER", "body": "Trickiest part of this is how to write a test for it.\r\n\r\nI'll do a `pytest.skipIf` that only executes the test if SpatiaLite is available.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683804172, "label": "--load-extension option for sqlite-utils query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/945#issuecomment-676556377", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/945", "id": 676556377, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NjU1NjM3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T17:21:16Z", "updated_at": "2020-08-19T17:21:16Z", "author_association": "OWNER", "body": "Documented here: https://docs.datasette.io/en/latest/plugins.html#installing-plugins", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 682005535, "label": "datasette install -U for upgrading packages"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675889865", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675889865, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTg4OTg2NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T06:57:00Z", "updated_at": "2020-08-19T06:57:00Z", "author_association": "OWNER", "body": "Maybe `.get` vs `.get_html`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675889551", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675889551, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTg4OTU1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T06:56:06Z", "updated_at": "2020-08-19T06:56:17Z", "author_association": "OWNER", "body": "I'm leaning towards defaulting to JSON as the requested format - you can pass `format=\"html\"` if you want HTML.\r\n\r\nBut weird that it's different from the web UI.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675884980", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675884980, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTg4NDk4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T06:44:26Z", "updated_at": "2020-08-19T06:44:26Z", "author_association": "OWNER", "body": "Need to decide what to do about JSON responses.\r\n\r\nWhen called from a template it's likely the intent will be to further loop through the JSON data returned. It would be annoying to have to run `json.loads` here.\r\n\r\nMaybe a `.get_json()` method then? Or even return a response that has `.json()` and `.text` similar to `httpx` - or just return an `httpx` response.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/944#issuecomment-675830678", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/944", "id": 675830678, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTgzMDY3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T03:30:10Z", "updated_at": "2020-08-19T03:30:10Z", "author_association": "OWNER", "body": "These templates will need a way to raise a 404 - so that if the template itself is deciding if the page exists (for example using `datasette-template-sql` or the proposed `datasette.get()` method from #943 or the `graphql()` template function in https://github.com/simonw/datasette-graphql/issues/50) it can return a regular 404 page.\r\n\r\nThis can imitate the `custom_redirect()` function from https://docs.datasette.io/en/stable/custom_templates.html#custom-redirects:\r\n```html+jinja\r\n{{ custom_redirect(\"https://github.com/simonw/datasette\", 301) }}\r\n```\r\nIt could be as simple as this:\r\n```\r\n{{ raise_404(\"Museum not found\") }}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681516976, "label": "Path parameters for custom pages"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/944#issuecomment-675829942", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/944", "id": 675829942, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTgyOTk0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T03:27:25Z", "updated_at": "2020-08-19T03:27:25Z", "author_association": "OWNER", "body": "I created a template file called `templates/pages/museums/{slug}.html` and used the debugger to see if Jinja could see it. This worked:\r\n```\r\n(Pdb) self.ds.jinja_env.list_templates()\r\n['500.html', '_codemirror.html', '_codemirror_foot.html', '_description_source_license.html', '_footer.html',\r\n'_table.html', 'allow_debug.html', 'base.html', 'database.html', 'default:500.html', 'default:_codemirror.html',\r\n'default:_codemirror_foot.html', 'default:_description_source_license.html', 'default:_footer.html',\r\n'default:_table.html', 'default:allow_debug.html', 'default:base.html', 'default:database.html',\r\n'default:index.html', 'default:logout.html', 'default:messages_debug.html', 'default:patterns.html',\r\n'default:permissions_debug.html', 'default:query.html', 'default:row.html', 'default:show_json.html',\r\n'default:table.html', 'forbidden.html', 'index.html', 'logout.html', 'messages_debug.html',\r\n'pages/about.html', 'pages/museums/{slug}.html', 'patterns.html', 'permissions_debug.html',\r\n'query.html', 'row.html', 'show_json.html', 'table.html']\r\n```\r\nThe `pages/museums/{slug}.html` template is in that list.\r\n\r\nHere's the implementation of that `list_templates()` method - it does some filesystem walking so it may be a bit expensive to run it on every request: https://github.com/pallets/jinja/blob/ca8b0b0287e320fe1f4a74f36910ef7ae3303d99/src/jinja2/loaders.py#L197-L212\r\n\r\nBut caching it would be pretty easy - either until the server is restarted or as an in-memory cache for a few seconds.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681516976, "label": "Path parameters for custom pages"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675788203", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675788203, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc4ODIwMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T00:46:08Z", "updated_at": "2020-08-19T00:46:23Z", "author_association": "OWNER", "body": "Also fun: the inevitable plugin that exposes this to the template language - so Datasette templates can stitch together data from multiple other internal API calls. Fun way to take advantage of `async` support in Jinja.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675787416", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675787416, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc4NzQxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-19T00:42:38Z", "updated_at": "2020-08-19T00:42:38Z", "author_association": "OWNER", "body": "I just realised that this mechanism is kind of like being able to use microservices - make API calls within your application - except that everything runs in the same process against SQLite databases so calls will be _lightning fast_.\r\n\r\nIt also means that a plugin can add a new internal API to Datasette that's accessible to other plugins by registering a new route with `register_routes`!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675753114", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675753114, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc1MzExNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:34:55Z", "updated_at": "2020-08-18T22:34:55Z", "author_association": "OWNER", "body": "Maybe allow this:\r\n\r\n response = await datasette.get(\"/{database}/{table}.json\", database=database, table=table)\r\n\r\nThis could cause problems if users ever need to pass literal `{` in their paths. Maybe allow this too:\r\n\r\n response = await datasette.get(\"/{database}/{table}.json\", interpolate=False)\r\n\r\nNot convinced this is useful - it's a bit unintuitive.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675752436", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675752436, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc1MjQzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:32:44Z", "updated_at": "2020-08-18T22:32:44Z", "author_association": "OWNER", "body": "One thing to consider here: Datasette's table and database name escaping rules can be a little bit convoluted.\r\n\r\nIf a plugin wants to get back the first five rows of a table, it will need to construct a URL `/dbname/tablename?_size=5` - but it will need to know how to turn the database and table names into the correctly escaped `dbname` and `tablename` values.\r\n\r\nHere's how the `row.html` table handles that right now: https://github.com/simonw/datasette/blob/b21ed237ab940768574c834aa5a7130724bd3a2d/datasette/templates/row.html#L19-L23\r\n\r\nIt would be an improvement to have this logic abstracted out somewhere and documented so plugins can use it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675751719", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675751719, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc1MTcxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:30:27Z", "updated_at": "2020-08-18T22:30:27Z", "author_association": "OWNER", "body": "Right now calling `datasette.app()` instantiates an ASGI application - complete with a bunch of routes and wrappers - and returns that application object. Calling it twice instantiates another ASGI application.\r\n\r\nI think a single `Datasette` instance should only ever create a single ASGI app - so the `.app()` method should cache the ASGI app that it returns the first time and return the same application again on future calls.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/915#issuecomment-675751136", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/915", "id": 675751136, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc1MTEzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:28:36Z", "updated_at": "2020-08-18T22:28:36Z", "author_association": "OWNER", "body": "I'm closing this in favour of an internal requests mechanism in #943.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 671763164, "label": "Refactor TableView class so things like datasette-graphql can reuse the logic"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675750845", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675750845, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc1MDg0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:27:43Z", "updated_at": "2020-08-18T22:27:43Z", "author_association": "OWNER", "body": "What about authentication checks etc? Won't they run twice?\r\n\r\nI think that's OK too, in fact it's desirable: think of the case of `datasette-graphql` where a bunch of different TableView calls are being made as part of the same GraphQL queries. Having those calls take advantage of finely grained per-table authentication and permission checks seems like a good feature.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675750382", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675750382, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc1MDM4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:26:15Z", "updated_at": "2020-08-18T22:26:15Z", "author_association": "OWNER", "body": "Should internal requests executed in this way be handled by plugins that used the `asgi_wrapper()` hook?\r\n\r\nHard to be sure one way or the other. I'm worried about logging middleware triggering twice - but actually anyone doing serious logging of their Datasette instance is probably doing it in a different layer (uvicorn logs or nginx proxy or whatever) so they wouldn't be affected. There aren't any ASGI logging middlewares out there that I've seen.\r\n\r\nAlso: if you run into a situation where your stuff is breaking because `datasette.get()` is calling ASGI middleware twice you can fix it by running your ASGI middleware outside of the `asgi_wrapper` plugin hook mechanism.\r\n\r\nSo I think it DOES execute `asgi_wrapper()` middleware.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675749319", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675749319, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc0OTMxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:23:01Z", "updated_at": "2020-08-18T22:23:01Z", "author_association": "OWNER", "body": "Actually no - `requests.get()` and `httpx.get()` prove that having a `.get()` method for an HTTP-related API isn't confusing to people at all.\r\n\r\n`datasette.get()` it is.\r\n\r\n(I'll probably add `datasette.post()` in the future too).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675749076", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675749076, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc0OTA3Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:22:21Z", "updated_at": "2020-08-18T22:22:21Z", "author_association": "OWNER", "body": "Alternative name possibilities:\r\n\r\n- `datasette.http_get(...)` - slightly misleading since it's not going over the HTTP protocol\r\n- `datasette.internal_get(...)` - the `internal_` might suggest its not an API for external use, which isn't true - it's for plugins\r\n- `datasette.get(...)` - clashes with `dict.get()` but I'm not at all sure that's a good reason not to use it", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675748573", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675748573, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc0ODU3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:20:52Z", "updated_at": "2020-08-18T22:20:52Z", "author_association": "OWNER", "body": "Should it default to treating things as if they had the `.json` extension? There are use-cases for the non-JSON method, such as https://github.com/natbat/tidepools_near_me/commit/ec102c6da5a5d86f17628740d90b6365b671b5e1\r\n\r\nI think I'm OK with people having to add `.json` to their internal calls. Maybe they could use `format=\"json\"`) as an optional parameter which would automatically handle the very weird edge-cases where you need to use `?_format=json` instead of `.json` (due to table names existing with a `.json` suffix).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-675747878", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 675747878, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc0Nzg3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:18:46Z", "updated_at": "2020-08-18T22:19:12Z", "author_association": "OWNER", "body": "Could be as simple as `response = await datasette.get(\"/path/blah\")` - which could also be re-used by the implementation of the `datasette --get /` CLI option introduced in #927.\r\n\r\nBit weird calling it `.get()` since that clashes with Python's dictionary `.get()` method.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/915#issuecomment-675746544", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/915", "id": 675746544, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTc0NjU0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T22:14:41Z", "updated_at": "2020-08-18T22:14:41Z", "author_association": "OWNER", "body": "I'm actually pretty happy with how `datasette-graphql` works now - maybe the trick here is to redesign the JSON format in #782 such that it can be used as a documented interface by things like `datasette-graphql` and then ensure Datasette has a documented mechanism for dispatching internal requests.\r\n\r\nI just did a horrible hack here that simulates an internal request, so supporting them as a feature would definitely make sense: https://github.com/natbat/tidepools_near_me/commit/ec102c6da5a5d86f17628740d90b6365b671b5e1", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 671763164, "label": "Refactor TableView class so things like datasette-graphql can reuse the logic"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/268#issuecomment-675725464", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/268", "id": 675725464, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTcyNTQ2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T21:18:07Z", "updated_at": "2020-08-18T21:18:35Z", "author_association": "OWNER", "body": "I want this on the table page - but that means that the table page will need to run a slightly more complex query since it needs access to a `rank` column to sort by - which it gets from running a join.\r\n\r\nBUT... that join needs to be constructed in a way that keeps existing filters, `?_where=` clauses etc intact.\r\n\r\nHere's a prototype using SQLite CTEs: https://register-of-members-interests.datasettes.com/regmem?sql=with+original+as+%28select+rowid%2C+*+from+items%29%0D%0Aselect%0D%0A++original.*%2C%0D%0A++items_fts.rank+as+items_fts_rank%0D%0Afrom%0D%0A++original+join+items_fts+on+original.rowid+%3D+items_fts.rowid%0D%0Awhere%0D%0A++items_fts+match+escape_fts%28%3Asearch%29%0D%0Aorder+by+items_fts_rank+desc+limit+10&search=hotel\r\n\r\n```sql\r\nwith original as (\r\n select\r\n rowid,\r\n *\r\n from\r\n items\r\n)\r\nselect\r\n original.*,\r\n items_fts.rank as items_fts_rank\r\nfrom\r\n original\r\n join items_fts on original.rowid = items_fts.rowid\r\nwhere\r\n items_fts match escape_fts(:search)\r\norder by\r\n items_fts_rank desc\r\nlimit\r\n 10\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323718842, "label": "Mechanism for ranking results from SQLite full-text search"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-675720040", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 675720040, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTcyMDA0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T21:05:24Z", "updated_at": "2020-08-18T21:05:24Z", "author_association": "OWNER", "body": "Is `columns` the right key for this in the table metadata block? I might want to use that for initial values for `?_col=` in #615.\r\n\r\nAlternative names:\r\n\r\n- `column_descriptions`\r\n- `column_info`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-675718593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 675718593, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTcxODU5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T21:02:11Z", "updated_at": "2020-08-18T21:02:24Z", "author_association": "OWNER", "body": "Easiest solution: if you provide column metadata it gets displayed above the table, something like on https://fivethirtyeight.datasettes.com/fivethirtyeight/antiquities-act%2Factions_under_antiquities_act\r\n\r\n\"fivethirtyeight__antiquities-act_actions_under_antiquities_act__344_rows\"\r\n\r\nHTML `title=` tooltips are also added to the table headers, which won't be visible on touch devices but that's OK because the information is visible on the page already.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-675715472", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 675715472, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTcxNTQ3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T20:55:02Z", "updated_at": "2020-08-18T20:55:02Z", "author_association": "OWNER", "body": "Could display these as tooltips on icons something like this (from the experimental `datasette-inspect-columns` plugin):\r\n\r\n\"fixtures__facetable__15_rows_and_NOAA_tides_second_attempt_-_Jupyter_Notebook\"\r\n\r\nThis would need to take accessibility into account, and would need a different display for the mobile web layout. Need to consider how it will interact with the column menu suggested in #690.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/873#issuecomment-675610275", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/873", "id": 675610275, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTYxMDI3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T17:24:05Z", "updated_at": "2020-08-18T17:26:10Z", "author_association": "OWNER", "body": "Maybe I can do this with ASGI after all. Here's the output of `/-/asgi-scope` with `datasette-debug-asgi` installed:\r\n```\r\n{'asgi': {'spec_version': '2.1', 'version': '3.0'},\r\n 'client': ('127.0.0.1', 62035),\r\n 'headers': [(b'host', b'127.0.0.1:62029'),\r\n (b'user-agent',\r\n b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko'\r\n b'/20100101 Firefox/79.0'),\r\n (b'accept',\r\n b'text/html,application/xhtml+xml,application/xml;q=0.9,image/'\r\n b'webp,*/*;q=0.8'),\r\n (b'accept-language', b'en-US,en;q=0.5'),\r\n (b'accept-encoding', b'gzip, deflate'),\r\n (b'dnt', b'1'),\r\n (b'connection', b'keep-alive'),\r\n (b'upgrade-insecure-requests', b'1'),\r\n (b'cache-control', b'max-age=0')],\r\n 'http_version': '1.1',\r\n 'method': 'GET',\r\n 'path': '/-/asgi-scope',\r\n 'query_string': b'',\r\n 'raw_path': b'/-/asgi-scope',\r\n 'root_path': '',\r\n 'scheme': 'http',\r\n 'server': ('127.0.0.1', 62029),\r\n 'type': 'http'}\r\n```\r\nThat `'server': ('127.0.0.1', 62029)` bit has the correct port. Question is, can I access that programmatically on server startup?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 647095487, "label": "\"datasette -p 0 --root\" gives the wrong URL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/873#issuecomment-675609109", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/873", "id": 675609109, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTYwOTEwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T17:21:51Z", "updated_at": "2020-08-18T17:21:51Z", "author_association": "OWNER", "body": "Asked about this on the encode gitter here: https://gitter.im/encode/community?at=5f3c0dcaa8c17801765940c0", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 647095487, "label": "\"datasette -p 0 --root\" gives the wrong URL"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-675538586", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 675538586, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTUzODU4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T15:11:36Z", "updated_at": "2020-08-18T15:11:36Z", "author_association": "OWNER", "body": "I tested this new publish pattern (running the tests in parallel before the deploy step) on `github-to-sqlite` - skipping the Docker step - and it worked: https://github.com/dogsheep/github-to-sqlite/actions/runs/213809864", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-675253373", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 675253373, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTI1MzM3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T05:10:17Z", "updated_at": "2020-08-18T05:10:17Z", "author_association": "OWNER", "body": "I'll close this after the next release successfully goes out.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-675251613", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 675251613, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTI1MTYxMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T05:05:15Z", "updated_at": "2020-08-18T05:05:15Z", "author_association": "OWNER", "body": "I think this is ready. I'll only know for sure the first time I push a release through it though!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-674590583", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 674590583, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU5MDU4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T23:15:51Z", "updated_at": "2020-08-18T05:04:43Z", "author_association": "OWNER", "body": "This example of jobs depending on each other and sharing data via artifacts looks relevant: https://docs.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts#passing-data-between-jobs-in-a-workflow", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-675250280", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 675250280, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTI1MDI4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T05:01:34Z", "updated_at": "2020-08-18T05:01:42Z", "author_association": "OWNER", "body": "I think `${GITHUB_REF#refs/tags/}` is the equivalent of `$TRAVIS_TAG`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-674589472", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 674589472, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU4OTQ3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T23:05:57Z", "updated_at": "2020-08-16T23:05:57Z", "author_association": "OWNER", "body": "When I figure this out I'll update the https://github.com/simonw/datasette-plugin/blob/main/datasette-%7B%7Bcookiecutter.hyphenated%7D%7D/.github/workflows/publish.yml default workflow to do this - right now it runs the tests once on just a single version of Python as part of the package deploy to PyPI step.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-674589321", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 674589321, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU4OTMyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T23:04:34Z", "updated_at": "2020-08-16T23:04:34Z", "author_association": "OWNER", "body": "https://docs.github.com/en/actions/getting-started-with-github-actions/core-concepts-for-github-actions#job\r\n\r\n> A set of steps that execute on the same runner. You can define the dependency rules for how jobs run in a workflow file. Jobs can run at the same time in parallel or run sequentially depending on the status of a previous job. For example, a workflow can have two sequential jobs that build and test code, where the test job is dependent on the status of the build job. If the build job fails, the test job will not run.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-674589035", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 674589035, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU4OTAzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T23:02:23Z", "updated_at": "2020-08-16T23:02:23Z", "author_association": "OWNER", "body": "I'd like to set these up as different workflows that depend on each other, if that's possible.\r\n\r\nI want to start three test runs in parallel (on three different Python versions), then if all three pass kick off the PyPI push (without running more tests), then if that passes do the Docker build and push.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/914#issuecomment-674578388", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/914", "id": 674578388, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU3ODM4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T21:10:27Z", "updated_at": "2020-08-16T21:10:27Z", "author_association": "OWNER", "body": "Demo of the fix: https://latest.datasette.io/fixtures/binary_data.json?_sort_desc=data&_shape=array&_nl=on\r\n\r\n```\r\n{\"rowid\": 2, \"data\": {\"$base64\": true, \"encoded\": \"FRwDx60F/g==\"}}\r\n{\"rowid\": 1, \"data\": {\"$base64\": true, \"encoded\": \"FRwCx60F/g==\"}}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 671056788, "label": "\"Object of type bytes is not JSON serializable\" for _nl=on"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/940#issuecomment-674566618", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/940", "id": 674566618, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU2NjYxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T19:20:58Z", "updated_at": "2020-08-16T19:20:58Z", "author_association": "OWNER", "body": "I need to figure out how to build and push the Docker image on releases. Here's the Travis code for that: https://github.com/simonw/datasette/blob/52eabb019d4051084b21524bd0fd9c2731126985/.travis.yml#L38-L47", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679808124, "label": "Move CI to GitHub Issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/938#issuecomment-674558631", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/938", "id": 674558631, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU1ODYzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T18:10:23Z", "updated_at": "2020-08-16T18:10:23Z", "author_association": "OWNER", "body": "Documentation: https://github.com/simonw/datasette/blob/3a4c8ed36aa97211e46849d32a09f2f386f342dd/docs/plugin_hooks.rst#extra-template-vars-template-database-table-columns-view-name-request-datasette", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679700269, "label": "Pass columns to extra CSS/JS/etc plugin hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/938#issuecomment-674551826", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/938", "id": 674551826, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU1MTgyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T17:07:57Z", "updated_at": "2020-08-16T17:07:57Z", "author_association": "OWNER", "body": "extra_tenplate_vars should he documented first and should be the only one to document the arguments and the async return feature. Others should refer back to it rather than duplicating that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679700269, "label": "Pass columns to extra CSS/JS/etc plugin hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/939#issuecomment-674548163", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/939", "id": 674548163, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU0ODE2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T16:34:30Z", "updated_at": "2020-08-16T16:34:30Z", "author_association": "OWNER", "body": "Docs also need to note that `request` may be `None` for all of these hooks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679779797, "label": "extra_ plugin hooks should take the same arguments"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/939#issuecomment-674547811", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/939", "id": 674547811, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NDU0NzgxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-16T16:31:20Z", "updated_at": "2020-08-16T16:31:20Z", "author_association": "OWNER", "body": "And the docs need to be updated too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 679779797, "label": "extra_ plugin hooks should take the same arguments"}, "performed_via_github_app": null}