html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688482355,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688482355,MDEyOklzc3VlQ29tbWVudDY4ODQ4MjM1NQ==,9599,2020-09-07T19:22:51Z,2020-09-07T19:22:51Z,OWNER,"And the SQLite documentation says: > When the REPLACE conflict resolution strategy deletes rows in order to satisfy a constraint, [delete triggers](https://www.sqlite.org/lang_createtrigger.html) fire if and only if [recursive triggers](https://www.sqlite.org/pragma.html#pragma_recursive_triggers) are enabled.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688482055,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688482055,MDEyOklzc3VlQ29tbWVudDY4ODQ4MjA1NQ==,9599,2020-09-07T19:21:42Z,2020-09-07T19:21:42Z,OWNER,"Using `replace=True` there executes `INSERT OR REPLACE` - and Dan Kennedy (SQLite maintainer) on the SQLite forums said this: > Are you using ""REPLACE INTO"", or ""UPDATE OR REPLACE"" on the ""licenses"" table without having first executed ""PRAGMA recursive_triggers = 1""? The docs note that delete triggers will not be fired in this case, which would explain things. Second paragraph under ""REPLACE"" here: > > https://www.sqlite.org/lang_conflict.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688481374,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688481374,MDEyOklzc3VlQ29tbWVudDY4ODQ4MTM3NA==,9599,2020-09-07T19:19:08Z,2020-09-07T19:19:08Z,OWNER,"reading through the code for `github-to-sqlite repos` - one of the things it does is calls `save_license` for each repo: https://github.com/dogsheep/github-to-sqlite/blob/39b2234253096bd579feed4e25104698b8ccd2ba/github_to_sqlite/utils.py#L259-L262 ```python def save_license(db, license): if license is None: return None return db[""licenses""].insert(license, pk=""key"", replace=True).last_pk ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688480665,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688480665,MDEyOklzc3VlQ29tbWVudDY4ODQ4MDY2NQ==,9599,2020-09-07T19:16:20Z,2020-09-07T19:16:20Z,OWNER,"Aha! I have managed to replicate the bug: ``` (github-to-sqlite) /tmp % sqlite-utils tables --counts github.db | grep licenses {""table"": ""licenses"", ""count"": 7}, {""table"": ""licenses_fts_data"", ""count"": 35}, {""table"": ""licenses_fts_idx"", ""count"": 16}, {""table"": ""licenses_fts_docsize"", ""count"": 9151}, {""table"": ""licenses_fts_config"", ""count"": 1}, {""table"": ""licenses_fts"", ""count"": 7}, (github-to-sqlite) /tmp % github-to-sqlite repos github.db dogsheep (github-to-sqlite) /tmp % sqlite-utils tables --counts github.db | grep licenses {""table"": ""licenses"", ""count"": 7}, {""table"": ""licenses_fts_data"", ""count"": 45}, {""table"": ""licenses_fts_idx"", ""count"": 26}, {""table"": ""licenses_fts_docsize"", ""count"": 9161}, {""table"": ""licenses_fts_config"", ""count"": 1}, {""table"": ""licenses_fts"", ""count"": 7}, ``` Note that the number of records in `licenses_fts_docsize` went from 9151 to 9161.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688464181,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688464181,MDEyOklzc3VlQ29tbWVudDY4ODQ2NDE4MQ==,9599,2020-09-07T18:19:54Z,2020-09-07T18:19:54Z,OWNER,"Even though that table doesn't declare an integer primary key it does have a `rowid` column: https://github-to-sqlite.dogsheep.net/github?sql=select+rowid%2C+%5Bkey%5D%2C+name%2C+spdx_id%2C+url%2C+node_id+from+licenses+order+by+%5Bkey%5D+limit+101 | rowid | key | name | spdx_id | url | node_id | | --- | --- | --- | --- | --- | --- | | 9150 | apache-2.0 | Apache License 2.0 | Apache-2.0 | | MDc6TGljZW5zZTI= | | 112 | bsd-3-clause | BSD 3-Clause ""New"" or ""Revised"" License | BSD-3-Clause | | MDc6TGljZW5zZTU= | https://www.sqlite.org/rowidtable.html explains has this clue: > If the rowid is not aliased by INTEGER PRIMARY KEY then it is not persistent and might change. In particular the VACUUM command will change rowids for tables that do not declare an INTEGER PRIMARY KEY. Therefore, applications should not normally access the rowid directly, but instead use an INTEGER PRIMARY KEY. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688460865,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688460865,MDEyOklzc3VlQ29tbWVudDY4ODQ2MDg2NQ==,9599,2020-09-07T18:07:14Z,2020-09-07T18:07:14Z,OWNER,"Another likely culprit: `licenses` has a text primary key, so it's not using `rowid`: ```sql CREATE TABLE [licenses] ( [key] TEXT PRIMARY KEY, [name] TEXT, [spdx_id] TEXT, [url] TEXT, [node_id] TEXT ); ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688460729,https://api.github.com/repos/simonw/sqlite-utils/issues/149,688460729,MDEyOklzc3VlQ29tbWVudDY4ODQ2MDcyOQ==,9599,2020-09-07T18:06:44Z,2020-09-07T18:06:44Z,OWNER,First posted on SQLite forum here but I'm pretty sure this is a bug in how `sqlite-utils` created those tables: https://sqlite.org/forum/forumpost/51aada1b45,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695319258, https://github.com/simonw/sqlite-utils/issues/148#issuecomment-688434226,https://api.github.com/repos/simonw/sqlite-utils/issues/148,688434226,MDEyOklzc3VlQ29tbWVudDY4ODQzNDIyNg==,9599,2020-09-07T16:50:33Z,2020-09-07T16:50:33Z,OWNER,"This may be as easy as applying `textwrap.dedent()` to this: https://github.com/simonw/sqlite-utils/blob/0e62744da9a429093e3409575c1f881376b0361f/sqlite_utils/db.py#L778-L787 I could apply that to a few other queries in that code as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695276328, https://github.com/simonw/sqlite-utils/issues/147#issuecomment-683528149,https://api.github.com/repos/simonw/sqlite-utils/issues/147,683528149,MDEyOklzc3VlQ29tbWVudDY4MzUyODE0OQ==,9599,2020-08-31T03:17:26Z,2020-08-31T03:17:26Z,OWNER,"+1 to making this something that users can customize. An optional argument to the `Database` constructor would be a neat way to do this. I think there's a terrifying way that we could find this value... we could perform a binary search for it! Open up a memory connection and try running different bulk inserts against it and catch the exceptions - then adjust and try again. My hunch is that we could perform just 2 or 3 probes (maybe against carefully selected values) to find the highest value that works. If this process took less than a few ms to run I'd be happy to do it automatically when the class is instantiated (and let users disable that automatic proving by setting a value using the constructor argument).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688670158, https://github.com/simonw/datasette/issues/948#issuecomment-683448569,https://api.github.com/repos/simonw/datasette/issues/948,683448569,MDEyOklzc3VlQ29tbWVudDY4MzQ0ODU2OQ==,9599,2020-08-30T17:39:09Z,2020-08-30T18:34:34Z,OWNER,"So the steps needed are: - Download and extract latest CodeMirror zip file - Rename `lib/codemirror.js` to `codemirror-5.57.0.js` - Rename `lib/codemirror.css` to `codemirror-5.57.0.css` - Rename `mode/sql/sql.js` to `codemirror-5.57.0-sql.js` - Edit both JS files to make the top comment a `/* */` block - Minify JavaScript files like this: - `npx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js --comments '/LICENSE/'` - `npx uglify-js codemirror-5.57.0-sql.js -o codemirror-5.57.0-sql.min.js --comments '/LICENSE/'` - Check that the LICENSE comment did indeed survive minification - Minify CSS file like this: - `npx clean-css-cli codemirror-5.57.0.css -o codemirror-5.57.0.min.css` - Edit the `_codemirror.html` template to reference the new files - `git rm` the old files, `git add` the new files","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/948#issuecomment-683452613,https://api.github.com/repos/simonw/datasette/issues/948,683452613,MDEyOklzc3VlQ29tbWVudDY4MzQ1MjYxMw==,9599,2020-08-30T18:16:28Z,2020-08-30T18:16:28Z,OWNER,I added documentation on how to upgrade CodeMirror for the future here: https://docs.datasette.io/en/latest/contributing.html#upgrading-codemirror,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/655#issuecomment-683449837,https://api.github.com/repos/simonw/datasette/issues/655,683449837,MDEyOklzc3VlQ29tbWVudDY4MzQ0OTgzNw==,9599,2020-08-30T17:51:38Z,2020-08-30T17:51:38Z,OWNER,I think was fixed by #948,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",542553350, https://github.com/simonw/datasette/issues/948#issuecomment-683449804,https://api.github.com/repos/simonw/datasette/issues/948,683449804,MDEyOklzc3VlQ29tbWVudDY4MzQ0OTgwNA==,9599,2020-08-30T17:51:18Z,2020-08-30T17:51:18Z,OWNER,Copy and paste on mobile safari seems to work now. #655 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/948#issuecomment-683448635,https://api.github.com/repos/simonw/datasette/issues/948,683448635,MDEyOklzc3VlQ29tbWVudDY4MzQ0ODYzNQ==,9599,2020-08-30T17:39:54Z,2020-08-30T17:39:54Z,OWNER,I'll wait for this to deploy to https://latest.datasette.io/ and then test it in various desktop and mobile browsers.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/948#issuecomment-683445704,https://api.github.com/repos/simonw/datasette/issues/948,683445704,MDEyOklzc3VlQ29tbWVudDY4MzQ0NTcwNA==,9599,2020-08-30T17:11:58Z,2020-08-30T17:33:30Z,OWNER,"One catch: this stripped the license information from the top of the JS. I fixed this by editing the license to be a single `/* ... */` block comment instead of multiple `//` lines and running this: npx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js --comments '/LICENSE/' ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/948#issuecomment-683445114,https://api.github.com/repos/simonw/datasette/issues/948,683445114,MDEyOklzc3VlQ29tbWVudDY4MzQ0NTExNA==,9599,2020-08-30T17:06:39Z,2020-08-30T17:06:39Z,OWNER,"Minifying using `npx`: ``` npx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js npx uglify-js codemirror-5.57.0-sql.js -o codemirror-5.57.0-sql.min.js npx clean-css-cli codemirror-5.57.0.css -o codemirror-5.57.0.min.css ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/957#issuecomment-683357092,https://api.github.com/repos/simonw/datasette/issues/957,683357092,MDEyOklzc3VlQ29tbWVudDY4MzM1NzA5Mg==,9599,2020-08-30T00:15:51Z,2020-08-30T00:16:02Z,OWNER,"Weirdly even removing this single `datasette` import from `utils/asgi.py` didn't fix the circular import: https://github.com/simonw/datasette/blob/44cf424a94a85b74552075272660bb96a7432661/datasette/utils/asgi.py#L1-L3","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688622148, https://github.com/simonw/datasette/issues/957#issuecomment-683356440,https://api.github.com/repos/simonw/datasette/issues/957,683356440,MDEyOklzc3VlQ29tbWVudDY4MzM1NjQ0MA==,9599,2020-08-30T00:08:18Z,2020-08-30T00:10:26Z,OWNER,"Annoyingly this seems to be the line that causes the circular import: ```python from .utils.asgi import Forbidden, NotFound, Response ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688622148, https://github.com/simonw/datasette/issues/957#issuecomment-683355993,https://api.github.com/repos/simonw/datasette/issues/957,683355993,MDEyOklzc3VlQ29tbWVudDY4MzM1NTk5Mw==,9599,2020-08-30T00:02:11Z,2020-08-30T00:04:18Z,OWNER,"I tried doing this and got this error: ``` (datasette) datasette % pytest ==================================================================== test session starts ===================================================================== platform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 rootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini plugins: asyncio-0.14.0, timeout-1.4.2 collected 1 item / 23 errors =========================================================================== ERRORS =========================================================================== _____________________________________________________________ ERROR collecting tests/test_api.py _____________________________________________________________ ImportError while importing test module '/Users/simon/Dropbox/Development/datasette/tests/test_api.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_api.py:1: in from datasette.plugins import DEFAULT_PLUGINS datasette/__init__.py:2: in from .app import Datasette datasette/app.py:30: in from .views.base import DatasetteError, ureg datasette/views/base.py:12: in from datasette.plugins import pm datasette/plugins.py:26: in mod = importlib.import_module(plugin) /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) datasette/publish/heroku.py:2: in from datasette import hookimpl E ImportError: cannot import name 'hookimpl' from partially initialized module 'datasette' (most likely due to a circular import) (/Users/simon/Dropbox/Development/datasette/datasette/__init__.py) ``` That's with `datasette/__init__.py` looking like this: ```python from datasette.version import __version_info__, __version__ # noqa from .app import Datasette from .utils.asgi import Forbidden, NotFound, Response from .utils import actor_matches_allow, QueryInterrupted from .hookspecs import hookimpl # noqa from .hookspecs import hookspec # noqa __all__ = [ ""actor_matches_allow"", ""hookimpl"", ""hookspec"", ""QueryInterrupted"", ""Forbidden"", ""NotFound"", ""Response"", ""Datasette"", ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688622148, https://github.com/simonw/datasette/issues/957#issuecomment-683355598,https://api.github.com/repos/simonw/datasette/issues/957,683355598,MDEyOklzc3VlQ29tbWVudDY4MzM1NTU5OA==,9599,2020-08-29T23:55:10Z,2020-08-29T23:55:34Z,OWNER,"Of these I think I'm going to promote the following to being importable directly `from datasette`: - `from datasette.app import Datasette` - `from datasette.utils import QueryInterrupted` - `from datasette.utils.asgi import Response, Forbidden, NotFound` - `from datasette.utils import actor_matches_allow` All of the rest are infrequently used enough (or clearly named enough) that I'm happy to leave them as-is.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688622148, https://github.com/simonw/datasette/issues/957#issuecomment-683355508,https://api.github.com/repos/simonw/datasette/issues/957,683355508,MDEyOklzc3VlQ29tbWVudDY4MzM1NTUwOA==,9599,2020-08-29T23:54:01Z,2020-08-29T23:54:01Z,OWNER,"Reviewing https://github.com/search?q=user%3Asimonw+%22from+datasette%22&type=Code I spotted these others: ```python # Various: from datasette.utils import path_with_replaced_args from datasette.plugins import pm from datasette.utils import QueryInterrupted from datasette.utils.asgi import Response, Forbidden, NotFound # datasette-publish-vercel: from datasette.publish.common import ( add_common_publish_arguments_and_options, fail_if_publish_binary_not_installed ) from datasette.utils import temporary_docker_directory # datasette-insert from datasette.utils import actor_matches_allow, sqlite3 # obsolete: russian-ira-facebook-ads-datasette from datasette.utils import TableFilter # simonw/museums from datasette.utils.asgi import asgi_send # datasette-media from datasette.utils.asgi import Response, asgi_send_file # datasette/tests/plugins/my_plugin.py from datasette.facets import Facet # datasette-graphql from datasette.views.table import TableView ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688622148, https://github.com/simonw/datasette/issues/956#issuecomment-683214102,https://api.github.com/repos/simonw/datasette/issues/956,683214102,MDEyOklzc3VlQ29tbWVudDY4MzIxNDEwMg==,9599,2020-08-29T01:32:21Z,2020-08-29T01:32:21Z,OWNER,Maybe the bug here is the double colon?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688427751, https://github.com/simonw/datasette/issues/956#issuecomment-683213973,https://api.github.com/repos/simonw/datasette/issues/956,683213973,MDEyOklzc3VlQ29tbWVudDY4MzIxMzk3Mw==,9599,2020-08-29T01:31:39Z,2020-08-29T01:31:39Z,OWNER,"Here's how the old Travis mechanism worked: https://github.com/simonw/datasette/blob/52eabb019d4051084b21524bd0fd9c2731126985/.travis.yml#L41-L47 So I was assuming that the eqivalent of `$REPO:$TRAVIS_TAG` in GitHub Actions is `$REPO::${GITHUB_REF#refs/tags/}`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688427751, https://github.com/simonw/datasette/issues/956#issuecomment-683212960,https://api.github.com/repos/simonw/datasette/issues/956,683212960,MDEyOklzc3VlQ29tbWVudDY4MzIxMjk2MA==,9599,2020-08-29T01:25:34Z,2020-08-29T01:25:34Z,OWNER,So I guess this bit is wrong: https://github.com/simonw/datasette/blob/c36e287d71d68ecb2a45e9808eede15f19f931fb/.github/workflows/publish.yml#L71-L73,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688427751, https://github.com/simonw/datasette/issues/956#issuecomment-683212421,https://api.github.com/repos/simonw/datasette/issues/956,683212421,MDEyOklzc3VlQ29tbWVudDY4MzIxMjQyMQ==,9599,2020-08-29T01:22:23Z,2020-08-29T01:22:23Z,OWNER,"Here's the error message again: > invalid argument `""***/datasette::0.49a0""` for `""-t, --tag""` flag: invalid reference format","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688427751, https://github.com/simonw/datasette/issues/956#issuecomment-683212246,https://api.github.com/repos/simonw/datasette/issues/956,683212246,MDEyOklzc3VlQ29tbWVudDY4MzIxMjI0Ng==,9599,2020-08-29T01:21:26Z,2020-08-29T01:21:26Z,OWNER,I added this but I have no idea if I got it right or not: https://github.com/simonw/datasette/blob/c36e287d71d68ecb2a45e9808eede15f19f931fb/.github/workflows/publish.yml#L58-L63,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688427751, https://github.com/simonw/datasette/issues/955#issuecomment-683189334,https://api.github.com/repos/simonw/datasette/issues/955,683189334,MDEyOklzc3VlQ29tbWVudDY4MzE4OTMzNA==,9599,2020-08-28T23:30:48Z,2020-08-28T23:30:48Z,OWNER,Also https://github.com/simonw/datasette-copyable,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",687711713, https://github.com/simonw/datasette/issues/955#issuecomment-683185861,https://api.github.com/repos/simonw/datasette/issues/955,683185861,MDEyOklzc3VlQ29tbWVudDY4MzE4NTg2MQ==,9599,2020-08-28T23:17:09Z,2020-08-28T23:17:09Z,OWNER,I released 0.49a0 which means I can update the main branches of those two plugins - I'll push a release of them once 0.49 is fully shipped.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",687711713, https://github.com/simonw/sqlite-utils/issues/144#issuecomment-683180581,https://api.github.com/repos/simonw/sqlite-utils/issues/144,683180581,MDEyOklzc3VlQ29tbWVudDY4MzE4MDU4MQ==,9599,2020-08-28T22:57:04Z,2020-08-28T22:57:04Z,OWNER,"That worked! https://github.com/simonw/sqlite-utils/runs/1043640785?check_suite_focus=true ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688395275, https://github.com/simonw/sqlite-utils/issues/144#issuecomment-683179678,https://api.github.com/repos/simonw/sqlite-utils/issues/144,683179678,MDEyOklzc3VlQ29tbWVudDY4MzE3OTY3OA==,9599,2020-08-28T22:53:17Z,2020-08-28T22:53:17Z,OWNER,I'm going to try doing this as a GitHub Actions test matrix.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688395275, https://github.com/simonw/sqlite-utils/issues/139#issuecomment-683178570,https://api.github.com/repos/simonw/sqlite-utils/issues/139,683178570,MDEyOklzc3VlQ29tbWVudDY4MzE3ODU3MA==,9599,2020-08-28T22:48:51Z,2020-08-28T22:48:51Z,OWNER,"Thanks @simonwiles, this is now released in 2.16.1: https://sqlite-utils.readthedocs.io/en/stable/changelog.html","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",686978131, https://github.com/simonw/sqlite-utils/issues/143#issuecomment-683175491,https://api.github.com/repos/simonw/sqlite-utils/issues/143,683175491,MDEyOklzc3VlQ29tbWVudDY4MzE3NTQ5MQ==,9599,2020-08-28T22:37:15Z,2020-08-28T22:37:15Z,OWNER,"I'm going to start running black exclusively in the GitHub Actions workflow, rather than having it run by the unit tests themselves.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688389933, https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683173375,https://api.github.com/repos/simonw/sqlite-utils/issues/142,683173375,MDEyOklzc3VlQ29tbWVudDY4MzE3MzM3NQ==,9599,2020-08-28T22:29:02Z,2020-08-28T22:29:02Z,OWNER,Yeah I think that failure is actually because there's a brand new release of Black out and it subtly changes some of the formatting rules. I'll merge this and then run Black against the entire codebase.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688386219, https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683172829,https://api.github.com/repos/simonw/sqlite-utils/issues/142,683172829,MDEyOklzc3VlQ29tbWVudDY4MzE3MjgyOQ==,9599,2020-08-28T22:27:05Z,2020-08-28T22:27:05Z,OWNER,"Looks like it failed the ""black"" formatting test - possibly because there's a new release if black out. I'm going to merge despite that failure.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688386219, https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683172082,https://api.github.com/repos/simonw/sqlite-utils/issues/142,683172082,MDEyOklzc3VlQ29tbWVudDY4MzE3MjA4Mg==,9599,2020-08-28T22:24:25Z,2020-08-28T22:24:25Z,OWNER,Thanks very much!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",688386219, https://github.com/simonw/sqlite-utils/issues/119#issuecomment-683146200,https://api.github.com/repos/simonw/sqlite-utils/issues/119,683146200,MDEyOklzc3VlQ29tbWVudDY4MzE0NjIwMA==,9599,2020-08-28T21:05:37Z,2020-08-28T21:05:37Z,OWNER,Maybe use `transform_table()` in #114 for this? Would be less efficient as it would copy the whole table but it would reduce library complexity a bit.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",652700770, https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682771226,https://api.github.com/repos/simonw/sqlite-utils/issues/139,682771226,MDEyOklzc3VlQ29tbWVudDY4Mjc3MTIyNg==,9599,2020-08-28T15:57:42Z,2020-08-28T15:57:42Z,OWNER,"That pull request should update this section of the documentation too: > If you have more than one record to insert, the insert_all() method is a much more efficient way of inserting them. Just like insert() it will automatically detect the columns that should be created, but it will inspect the first batch of 100 items to help decide what those column types should be. https://github.com/simonw/sqlite-utils/blob/ea87c2b943fdd162c42a900ac0aea5ecc2f4b9d9/docs/python-api.rst#L393","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",686978131, https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682762911,https://api.github.com/repos/simonw/sqlite-utils/issues/139,682762911,MDEyOklzc3VlQ29tbWVudDY4Mjc2MjkxMQ==,9599,2020-08-28T15:54:57Z,2020-08-28T15:55:20Z,OWNER,"Here's a suggested test update: ```diff diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index a8791c3..12fa2f2 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -1074,6 +1074,13 @@ class Table(Queryable): all_columns = list(sorted(all_columns)) if hash_id: all_columns.insert(0, hash_id) + else: + all_columns += [ + column + for record in chunk + for column in record + if column not in all_columns + ] validate_column_names(all_columns) first = False # values is the list of insert data that is passed to the diff --git a/tests/test_create.py b/tests/test_create.py index a84eb8d..3a7fafc 100644 --- a/tests/test_create.py +++ b/tests/test_create.py @@ -707,13 +707,15 @@ def test_insert_thousands_using_generator(fresh_db): assert 10000 == fresh_db[""test""].count -def test_insert_thousands_ignores_extra_columns_after_first_100(fresh_db): +def test_insert_thousands_adds_extra_columns_after_first_100(fresh_db): + # https://github.com/simonw/sqlite-utils/issues/139 fresh_db[""test""].insert_all( [{""i"": i, ""word"": ""word_{}"".format(i)} for i in range(100)] - + [{""i"": 101, ""extra"": ""This extra column should cause an exception""}] + + [{""i"": 101, ""extra"": ""Should trigger ALTER""}], + alter=True, ) rows = fresh_db.execute_returning_dicts(""select * from test where i = 101"") - assert [{""i"": 101, ""word"": None}] == rows + assert [{""i"": 101, ""word"": None, ""extra"": ""Should trigger ALTER""}] == rows def test_insert_ignore(fresh_db): ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",686978131, https://github.com/simonw/datasette/issues/954#issuecomment-682312736,https://api.github.com/repos/simonw/datasette/issues/954,682312736,MDEyOklzc3VlQ29tbWVudDY4MjMxMjczNg==,9599,2020-08-28T04:05:01Z,2020-08-28T04:05:10Z,OWNER,> It can also return a dictionary with the following keys. This format is **deprecated** as-of Datasette 0.49 and will be removed by Datasette 1.0.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",687694947, https://github.com/simonw/datasette/issues/953#issuecomment-682312494,https://api.github.com/repos/simonw/datasette/issues/953,682312494,MDEyOklzc3VlQ29tbWVudDY4MjMxMjQ5NA==,9599,2020-08-28T04:03:56Z,2020-08-28T04:03:56Z,OWNER,"Documentation says that the old dictionary mechanism will be deprecated by 1.0: https://github.com/simonw/datasette/blob/799ecae94824640bdff21f86997f69844048d5c3/docs/plugin_hooks.rst#L460","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",687681018, https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682285212,https://api.github.com/repos/simonw/sqlite-utils/issues/139,682285212,MDEyOklzc3VlQ29tbWVudDY4MjI4NTIxMg==,9599,2020-08-28T02:12:51Z,2020-08-28T02:12:51Z,OWNER,"I'd be happy to accept a PR for this, provided it included updated unit tests that illustrate it working. I think this is a really good improvement.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",686978131, https://github.com/simonw/sqlite-utils/issues/139#issuecomment-682284908,https://api.github.com/repos/simonw/sqlite-utils/issues/139,682284908,MDEyOklzc3VlQ29tbWVudDY4MjI4NDkwOA==,9599,2020-08-28T02:11:40Z,2020-08-28T02:11:40Z,OWNER,"This is deliberate behaviour, but I'm not at all attached to it - you're right in pointing out that it's actually pretty unexpected. I'd be happy to change this behaviour so if you pass `alter=True` and then use `.insert_all()` on more than 100 rows it works as you would expect, instead of silently ignoring new columns past the first 100 rows. I don't expect that anyone would be depending on the current behaviour (ignore new columns after the first 100) such that this should be considered a backwards incompatible change.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",686978131, https://github.com/simonw/datasette/issues/950#issuecomment-680374196,https://api.github.com/repos/simonw/datasette/issues/950,680374196,MDEyOklzc3VlQ29tbWVudDY4MDM3NDE5Ng==,9599,2020-08-26T00:43:50Z,2020-08-26T00:43:50Z,OWNER,"The problem with the term ""private"" is that it could be confused with the concept of databases that aren't visible to the public due to the permissions system - the ones that are displayed with the padlock icon e.g. on https://datasette-auth-passwords-demo.datasette.io/ So I think ""secret"" is a better term for these.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511, https://github.com/simonw/datasette/issues/950#issuecomment-680264202,https://api.github.com/repos/simonw/datasette/issues/950,680264202,MDEyOklzc3VlQ29tbWVudDY4MDI2NDIwMg==,9599,2020-08-25T20:53:13Z,2020-08-25T20:53:13Z,OWNER,Forcing people to spell out `datasette github.db --private private.db` isn't terrible though.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511, https://github.com/simonw/datasette/issues/950#issuecomment-680263999,https://api.github.com/repos/simonw/datasette/issues/950,680263999,MDEyOklzc3VlQ29tbWVudDY4MDI2Mzk5OQ==,9599,2020-08-25T20:52:47Z,2020-08-25T20:52:47Z,OWNER,"Naming challenge: secret databases or private databases? I prefer private. But `datasette -p` is already taken by `--port`. `datasette -s` is currently available.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511, https://github.com/simonw/datasette/issues/950#issuecomment-680263427,https://api.github.com/repos/simonw/datasette/issues/950,680263427,MDEyOklzc3VlQ29tbWVudDY4MDI2MzQyNw==,9599,2020-08-25T20:51:30Z,2020-08-25T20:52:13Z,OWNER,"`datasette-graphql` currently dispatches requests through the `TableView` class, so if that couldn't access private databases then it would not be able to either. See also the concept for `datasette.get(...)` as an internal API in #943 - that might need to have a mechanism for also being able to query private databases, maybe `datasette.get(path, allow_private=True)`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511, https://github.com/simonw/datasette/issues/949#issuecomment-679367931,https://api.github.com/repos/simonw/datasette/issues/949,679367931,MDEyOklzc3VlQ29tbWVudDY3OTM2NzkzMQ==,9599,2020-08-24T21:09:50Z,2020-08-24T21:09:50Z,OWNER,"I'm attracted to this because of how good GraphiQL is for auto-completing queries. But I realize there's a problem here: GraphQL is designed to be autocomplete-friendly, but SQL is not. If you type `select ` and it doesn't know what's going in the `from` clause it can't give you good column autocomplete, for example.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684961449, https://github.com/simonw/datasette/issues/949#issuecomment-679363710,https://api.github.com/repos/simonw/datasette/issues/949,679363710,MDEyOklzc3VlQ29tbWVudDY3OTM2MzcxMA==,9599,2020-08-24T21:00:43Z,2020-08-24T21:00:43Z,OWNER,"I think this requires three extra files from https://github.com/codemirror/CodeMirror/tree/5.57.0/addon/hint - `show-hint.css` - `show-hint.js` - `sql-hint.js` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684961449, https://github.com/simonw/datasette/issues/948#issuecomment-679355426,https://api.github.com/repos/simonw/datasette/issues/948,679355426,MDEyOklzc3VlQ29tbWVudDY3OTM1NTQyNg==,9599,2020-08-24T20:43:07Z,2020-08-24T20:43:07Z,OWNER,"It would also be interesting to try out the SQL hint mode, which can autocomplete against tables and columns. This demo shows how to configure that: https://codemirror.net/mode/sql/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/datasette/issues/948#issuecomment-679333717,https://api.github.com/repos/simonw/datasette/issues/948,679333717,MDEyOklzc3VlQ29tbWVudDY3OTMzMzcxNw==,9599,2020-08-24T19:55:59Z,2020-08-24T19:55:59Z,OWNER,CodeMirror 6 is in pre-release at the moment and is a complete rewrite. I'll stick with the 5.x series for now. https://github.com/codemirror/codemirror.next/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684925907, https://github.com/simonw/sqlite-utils/issues/138#issuecomment-678732667,https://api.github.com/repos/simonw/sqlite-utils/issues/138,678732667,MDEyOklzc3VlQ29tbWVudDY3ODczMjY2Nw==,9599,2020-08-23T05:46:10Z,2020-08-23T05:46:10Z,OWNER,"Actually the `TEXT` column thing wasn't a `sqlite-utils` issue, it was unique to how `shapefile-to-spatialite` was creating the table when using the SpatiaLite extension.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",684118950, https://github.com/simonw/sqlite-utils/issues/136#issuecomment-678508056,https://api.github.com/repos/simonw/sqlite-utils/issues/136,678508056,MDEyOklzc3VlQ29tbWVudDY3ODUwODA1Ng==,9599,2020-08-21T21:13:41Z,2020-08-21T21:13:41Z,OWNER,"The `--spatialite` option should be available for other useful commands too, refs #137.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683812642, https://github.com/simonw/sqlite-utils/issues/137#issuecomment-678507502,https://api.github.com/repos/simonw/sqlite-utils/issues/137,678507502,MDEyOklzc3VlQ29tbWVudDY3ODUwNzUwMg==,9599,2020-08-21T21:13:19Z,2020-08-21T21:13:19Z,OWNER,Adding `--spatialite` too would be great for usability: #136,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683830416, https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678497497,https://api.github.com/repos/simonw/sqlite-utils/issues/134,678497497,MDEyOklzc3VlQ29tbWVudDY3ODQ5NzQ5Nw==,9599,2020-08-21T21:06:26Z,2020-08-21T21:06:26Z,OWNER,"Ended up needing two skipIfs: https://github.com/simonw/sqlite-utils/blob/7e9aad7e1c09d1cf80d0b4d17d6157212a4b857d/tests/test_cli.py#L888-L893","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683804172, https://github.com/simonw/sqlite-utils/issues/136#issuecomment-678480969,https://api.github.com/repos/simonw/sqlite-utils/issues/136,678480969,MDEyOklzc3VlQ29tbWVudDY3ODQ4MDk2OQ==,9599,2020-08-21T20:33:45Z,2020-08-21T20:33:45Z,OWNER,"I think this should initialize SpatiaLite against the current database if it has not been initialized already. Relevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126 ```python def init_spatialite(db, lib): db.conn.enable_load_extension(True) db.conn.load_extension(lib) # Initialize SpatiaLite if not yet initialized if ""spatial_ref_sys"" in db.table_names(): return db.conn.execute(""select InitSpatialMetadata(1)"") def ensure_table_has_geometry(db, table, table_srid): if ""geometry"" not in db[table].columns_dict: db.conn.execute( ""SELECT AddGeometryColumn(?, 'geometry', ?, 'GEOMETRY', 2);"", [table, table_srid], ) ``` Not sure if I should add a utility function or CLI command for that `ensure_table_has_geometry` bit.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683812642, https://github.com/simonw/sqlite-utils/issues/135#issuecomment-678479741,https://api.github.com/repos/simonw/sqlite-utils/issues/135,678479741,MDEyOklzc3VlQ29tbWVudDY3ODQ3OTc0MQ==,9599,2020-08-21T20:30:37Z,2020-08-21T20:30:37Z,OWNER,Docs: https://github.com/simonw/sqlite-utils/blob/bf4c6b7c82fab6b2400e48424f8dac1ae2f0a2dc/docs/python-api.rst#finding-spatialite,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683805434, https://github.com/simonw/sqlite-utils/issues/135#issuecomment-678476842,https://api.github.com/repos/simonw/sqlite-utils/issues/135,678476842,MDEyOklzc3VlQ29tbWVudDY3ODQ3Njg0Mg==,9599,2020-08-21T20:23:13Z,2020-08-21T20:23:13Z,OWNER,I'm going to start with just the first two - I'm not convinced I understand the `.so.5` variants.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683805434, https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678476338,https://api.github.com/repos/simonw/sqlite-utils/issues/134,678476338,MDEyOklzc3VlQ29tbWVudDY3ODQ3NjMzOA==,9599,2020-08-21T20:22:02Z,2020-08-21T20:22:02Z,OWNER,I think that adds it as `/usr/lib/x86_64-linux-gnu/mod_spatialite.so`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683804172, https://github.com/simonw/sqlite-utils/issues/135#issuecomment-678475578,https://api.github.com/repos/simonw/sqlite-utils/issues/135,678475578,MDEyOklzc3VlQ29tbWVudDY3ODQ3NTU3OA==,9599,2020-08-21T20:20:05Z,2020-08-21T20:20:05Z,OWNER,"https://github.com/simonw/cryptozoology/blob/2ad69168f3b78ebd90a2cbeea8136c9115e2a9b7/build_cryptids_database.py#L16-L22 ```python try_these = ( ""mod_spatialite"", ""/usr/local/lib/mod_spatialite.dylib"", ""/usr/lib/x86_64-linux-gnu/mod_spatialite.so"", ""/usr/lib/x86_64-linux-gnu/libspatialite.so.5"", ""/usr/lib/x86_64-linux-gnu/libspatialite.so.7"", ) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683805434, https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678474928,https://api.github.com/repos/simonw/sqlite-utils/issues/134,678474928,MDEyOklzc3VlQ29tbWVudDY3ODQ3NDkyOA==,9599,2020-08-21T20:18:33Z,2020-08-21T20:18:33Z,OWNER,"This should get me SpatiaLite in the GitHub Actions Ubuntu: ``` apt install libsqlite3-mod-spatialite ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683804172, https://github.com/simonw/sqlite-utils/issues/134#issuecomment-678474018,https://api.github.com/repos/simonw/sqlite-utils/issues/134,678474018,MDEyOklzc3VlQ29tbWVudDY3ODQ3NDAxOA==,9599,2020-08-21T20:16:20Z,2020-08-21T20:16:20Z,OWNER,"Trickiest part of this is how to write a test for it. I'll do a `pytest.skipIf` that only executes the test if SpatiaLite is available.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",683804172, https://github.com/simonw/datasette/issues/945#issuecomment-676556377,https://api.github.com/repos/simonw/datasette/issues/945,676556377,MDEyOklzc3VlQ29tbWVudDY3NjU1NjM3Nw==,9599,2020-08-19T17:21:16Z,2020-08-19T17:21:16Z,OWNER,Documented here: https://docs.datasette.io/en/latest/plugins.html#installing-plugins,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",682005535, https://github.com/simonw/datasette/issues/943#issuecomment-675889865,https://api.github.com/repos/simonw/datasette/issues/943,675889865,MDEyOklzc3VlQ29tbWVudDY3NTg4OTg2NQ==,9599,2020-08-19T06:57:00Z,2020-08-19T06:57:00Z,OWNER,Maybe `.get` vs `.get_html`?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675889551,https://api.github.com/repos/simonw/datasette/issues/943,675889551,MDEyOklzc3VlQ29tbWVudDY3NTg4OTU1MQ==,9599,2020-08-19T06:56:06Z,2020-08-19T06:56:17Z,OWNER,"I'm leaning towards defaulting to JSON as the requested format - you can pass `format=""html""` if you want HTML. But weird that it's different from the web UI.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675884980,https://api.github.com/repos/simonw/datasette/issues/943,675884980,MDEyOklzc3VlQ29tbWVudDY3NTg4NDk4MA==,9599,2020-08-19T06:44:26Z,2020-08-19T06:44:26Z,OWNER,"Need to decide what to do about JSON responses. When called from a template it's likely the intent will be to further loop through the JSON data returned. It would be annoying to have to run `json.loads` here. Maybe a `.get_json()` method then? Or even return a response that has `.json()` and `.text` similar to `httpx` - or just return an `httpx` response.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/944#issuecomment-675830678,https://api.github.com/repos/simonw/datasette/issues/944,675830678,MDEyOklzc3VlQ29tbWVudDY3NTgzMDY3OA==,9599,2020-08-19T03:30:10Z,2020-08-19T03:30:10Z,OWNER,"These templates will need a way to raise a 404 - so that if the template itself is deciding if the page exists (for example using `datasette-template-sql` or the proposed `datasette.get()` method from #943 or the `graphql()` template function in https://github.com/simonw/datasette-graphql/issues/50) it can return a regular 404 page. This can imitate the `custom_redirect()` function from https://docs.datasette.io/en/stable/custom_templates.html#custom-redirects: ```html+jinja {{ custom_redirect(""https://github.com/simonw/datasette"", 301) }} ``` It could be as simple as this: ``` {{ raise_404(""Museum not found"") }} ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681516976, https://github.com/simonw/datasette/issues/944#issuecomment-675829942,https://api.github.com/repos/simonw/datasette/issues/944,675829942,MDEyOklzc3VlQ29tbWVudDY3NTgyOTk0Mg==,9599,2020-08-19T03:27:25Z,2020-08-19T03:27:25Z,OWNER,"I created a template file called `templates/pages/museums/{slug}.html` and used the debugger to see if Jinja could see it. This worked: ``` (Pdb) self.ds.jinja_env.list_templates() ['500.html', '_codemirror.html', '_codemirror_foot.html', '_description_source_license.html', '_footer.html', '_table.html', 'allow_debug.html', 'base.html', 'database.html', 'default:500.html', 'default:_codemirror.html', 'default:_codemirror_foot.html', 'default:_description_source_license.html', 'default:_footer.html', 'default:_table.html', 'default:allow_debug.html', 'default:base.html', 'default:database.html', 'default:index.html', 'default:logout.html', 'default:messages_debug.html', 'default:patterns.html', 'default:permissions_debug.html', 'default:query.html', 'default:row.html', 'default:show_json.html', 'default:table.html', 'forbidden.html', 'index.html', 'logout.html', 'messages_debug.html', 'pages/about.html', 'pages/museums/{slug}.html', 'patterns.html', 'permissions_debug.html', 'query.html', 'row.html', 'show_json.html', 'table.html'] ``` The `pages/museums/{slug}.html` template is in that list. Here's the implementation of that `list_templates()` method - it does some filesystem walking so it may be a bit expensive to run it on every request: https://github.com/pallets/jinja/blob/ca8b0b0287e320fe1f4a74f36910ef7ae3303d99/src/jinja2/loaders.py#L197-L212 But caching it would be pretty easy - either until the server is restarted or as an in-memory cache for a few seconds.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681516976, https://github.com/simonw/datasette/issues/943#issuecomment-675788203,https://api.github.com/repos/simonw/datasette/issues/943,675788203,MDEyOklzc3VlQ29tbWVudDY3NTc4ODIwMw==,9599,2020-08-19T00:46:08Z,2020-08-19T00:46:23Z,OWNER,Also fun: the inevitable plugin that exposes this to the template language - so Datasette templates can stitch together data from multiple other internal API calls. Fun way to take advantage of `async` support in Jinja.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675787416,https://api.github.com/repos/simonw/datasette/issues/943,675787416,MDEyOklzc3VlQ29tbWVudDY3NTc4NzQxNg==,9599,2020-08-19T00:42:38Z,2020-08-19T00:42:38Z,OWNER,"I just realised that this mechanism is kind of like being able to use microservices - make API calls within your application - except that everything runs in the same process against SQLite databases so calls will be _lightning fast_. It also means that a plugin can add a new internal API to Datasette that's accessible to other plugins by registering a new route with `register_routes`!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675753114,https://api.github.com/repos/simonw/datasette/issues/943,675753114,MDEyOklzc3VlQ29tbWVudDY3NTc1MzExNA==,9599,2020-08-18T22:34:55Z,2020-08-18T22:34:55Z,OWNER,"Maybe allow this: response = await datasette.get(""/{database}/{table}.json"", database=database, table=table) This could cause problems if users ever need to pass literal `{` in their paths. Maybe allow this too: response = await datasette.get(""/{database}/{table}.json"", interpolate=False) Not convinced this is useful - it's a bit unintuitive.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675752436,https://api.github.com/repos/simonw/datasette/issues/943,675752436,MDEyOklzc3VlQ29tbWVudDY3NTc1MjQzNg==,9599,2020-08-18T22:32:44Z,2020-08-18T22:32:44Z,OWNER,"One thing to consider here: Datasette's table and database name escaping rules can be a little bit convoluted. If a plugin wants to get back the first five rows of a table, it will need to construct a URL `/dbname/tablename?_size=5` - but it will need to know how to turn the database and table names into the correctly escaped `dbname` and `tablename` values. Here's how the `row.html` table handles that right now: https://github.com/simonw/datasette/blob/b21ed237ab940768574c834aa5a7130724bd3a2d/datasette/templates/row.html#L19-L23 It would be an improvement to have this logic abstracted out somewhere and documented so plugins can use it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675751719,https://api.github.com/repos/simonw/datasette/issues/943,675751719,MDEyOklzc3VlQ29tbWVudDY3NTc1MTcxOQ==,9599,2020-08-18T22:30:27Z,2020-08-18T22:30:27Z,OWNER,"Right now calling `datasette.app()` instantiates an ASGI application - complete with a bunch of routes and wrappers - and returns that application object. Calling it twice instantiates another ASGI application. I think a single `Datasette` instance should only ever create a single ASGI app - so the `.app()` method should cache the ASGI app that it returns the first time and return the same application again on future calls.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/915#issuecomment-675751136,https://api.github.com/repos/simonw/datasette/issues/915,675751136,MDEyOklzc3VlQ29tbWVudDY3NTc1MTEzNg==,9599,2020-08-18T22:28:36Z,2020-08-18T22:28:36Z,OWNER,I'm closing this in favour of an internal requests mechanism in #943.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",671763164, https://github.com/simonw/datasette/issues/943#issuecomment-675750845,https://api.github.com/repos/simonw/datasette/issues/943,675750845,MDEyOklzc3VlQ29tbWVudDY3NTc1MDg0NQ==,9599,2020-08-18T22:27:43Z,2020-08-18T22:27:43Z,OWNER,"What about authentication checks etc? Won't they run twice? I think that's OK too, in fact it's desirable: think of the case of `datasette-graphql` where a bunch of different TableView calls are being made as part of the same GraphQL queries. Having those calls take advantage of finely grained per-table authentication and permission checks seems like a good feature.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675750382,https://api.github.com/repos/simonw/datasette/issues/943,675750382,MDEyOklzc3VlQ29tbWVudDY3NTc1MDM4Mg==,9599,2020-08-18T22:26:15Z,2020-08-18T22:26:15Z,OWNER,"Should internal requests executed in this way be handled by plugins that used the `asgi_wrapper()` hook? Hard to be sure one way or the other. I'm worried about logging middleware triggering twice - but actually anyone doing serious logging of their Datasette instance is probably doing it in a different layer (uvicorn logs or nginx proxy or whatever) so they wouldn't be affected. There aren't any ASGI logging middlewares out there that I've seen. Also: if you run into a situation where your stuff is breaking because `datasette.get()` is calling ASGI middleware twice you can fix it by running your ASGI middleware outside of the `asgi_wrapper` plugin hook mechanism. So I think it DOES execute `asgi_wrapper()` middleware.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675749319,https://api.github.com/repos/simonw/datasette/issues/943,675749319,MDEyOklzc3VlQ29tbWVudDY3NTc0OTMxOQ==,9599,2020-08-18T22:23:01Z,2020-08-18T22:23:01Z,OWNER,"Actually no - `requests.get()` and `httpx.get()` prove that having a `.get()` method for an HTTP-related API isn't confusing to people at all. `datasette.get()` it is. (I'll probably add `datasette.post()` in the future too).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675749076,https://api.github.com/repos/simonw/datasette/issues/943,675749076,MDEyOklzc3VlQ29tbWVudDY3NTc0OTA3Ng==,9599,2020-08-18T22:22:21Z,2020-08-18T22:22:21Z,OWNER,"Alternative name possibilities: - `datasette.http_get(...)` - slightly misleading since it's not going over the HTTP protocol - `datasette.internal_get(...)` - the `internal_` might suggest its not an API for external use, which isn't true - it's for plugins - `datasette.get(...)` - clashes with `dict.get()` but I'm not at all sure that's a good reason not to use it","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675748573,https://api.github.com/repos/simonw/datasette/issues/943,675748573,MDEyOklzc3VlQ29tbWVudDY3NTc0ODU3Mw==,9599,2020-08-18T22:20:52Z,2020-08-18T22:20:52Z,OWNER,"Should it default to treating things as if they had the `.json` extension? There are use-cases for the non-JSON method, such as https://github.com/natbat/tidepools_near_me/commit/ec102c6da5a5d86f17628740d90b6365b671b5e1 I think I'm OK with people having to add `.json` to their internal calls. Maybe they could use `format=""json""`) as an optional parameter which would automatically handle the very weird edge-cases where you need to use `?_format=json` instead of `.json` (due to table names existing with a `.json` suffix).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/943#issuecomment-675747878,https://api.github.com/repos/simonw/datasette/issues/943,675747878,MDEyOklzc3VlQ29tbWVudDY3NTc0Nzg3OA==,9599,2020-08-18T22:18:46Z,2020-08-18T22:19:12Z,OWNER,"Could be as simple as `response = await datasette.get(""/path/blah"")` - which could also be re-used by the implementation of the `datasette --get /` CLI option introduced in #927. Bit weird calling it `.get()` since that clashes with Python's dictionary `.get()` method.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681375466, https://github.com/simonw/datasette/issues/915#issuecomment-675746544,https://api.github.com/repos/simonw/datasette/issues/915,675746544,MDEyOklzc3VlQ29tbWVudDY3NTc0NjU0NA==,9599,2020-08-18T22:14:41Z,2020-08-18T22:14:41Z,OWNER,"I'm actually pretty happy with how `datasette-graphql` works now - maybe the trick here is to redesign the JSON format in #782 such that it can be used as a documented interface by things like `datasette-graphql` and then ensure Datasette has a documented mechanism for dispatching internal requests. I just did a horrible hack here that simulates an internal request, so supporting them as a feature would definitely make sense: https://github.com/natbat/tidepools_near_me/commit/ec102c6da5a5d86f17628740d90b6365b671b5e1","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",671763164, https://github.com/simonw/datasette/issues/268#issuecomment-675725464,https://api.github.com/repos/simonw/datasette/issues/268,675725464,MDEyOklzc3VlQ29tbWVudDY3NTcyNTQ2NA==,9599,2020-08-18T21:18:07Z,2020-08-18T21:18:35Z,OWNER,"I want this on the table page - but that means that the table page will need to run a slightly more complex query since it needs access to a `rank` column to sort by - which it gets from running a join. BUT... that join needs to be constructed in a way that keeps existing filters, `?_where=` clauses etc intact. Here's a prototype using SQLite CTEs: https://register-of-members-interests.datasettes.com/regmem?sql=with+original+as+%28select+rowid%2C+*+from+items%29%0D%0Aselect%0D%0A++original.*%2C%0D%0A++items_fts.rank+as+items_fts_rank%0D%0Afrom%0D%0A++original+join+items_fts+on+original.rowid+%3D+items_fts.rowid%0D%0Awhere%0D%0A++items_fts+match+escape_fts%28%3Asearch%29%0D%0Aorder+by+items_fts_rank+desc+limit+10&search=hotel ```sql with original as ( select rowid, * from items ) select original.*, items_fts.rank as items_fts_rank from original join items_fts on original.rowid = items_fts.rowid where items_fts match escape_fts(:search) order by items_fts_rank desc limit 10 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",323718842, https://github.com/simonw/datasette/issues/942#issuecomment-675720040,https://api.github.com/repos/simonw/datasette/issues/942,675720040,MDEyOklzc3VlQ29tbWVudDY3NTcyMDA0MA==,9599,2020-08-18T21:05:24Z,2020-08-18T21:05:24Z,OWNER,"Is `columns` the right key for this in the table metadata block? I might want to use that for initial values for `?_col=` in #615. Alternative names: - `column_descriptions` - `column_info`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681334912, https://github.com/simonw/datasette/issues/942#issuecomment-675718593,https://api.github.com/repos/simonw/datasette/issues/942,675718593,MDEyOklzc3VlQ29tbWVudDY3NTcxODU5Mw==,9599,2020-08-18T21:02:11Z,2020-08-18T21:02:24Z,OWNER,"Easiest solution: if you provide column metadata it gets displayed above the table, something like on https://fivethirtyeight.datasettes.com/fivethirtyeight/antiquities-act%2Factions_under_antiquities_act HTML `title=` tooltips are also added to the table headers, which won't be visible on touch devices but that's OK because the information is visible on the page already.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681334912, https://github.com/simonw/datasette/issues/942#issuecomment-675715472,https://api.github.com/repos/simonw/datasette/issues/942,675715472,MDEyOklzc3VlQ29tbWVudDY3NTcxNTQ3Mg==,9599,2020-08-18T20:55:02Z,2020-08-18T20:55:02Z,OWNER,"Could display these as tooltips on icons something like this (from the experimental `datasette-inspect-columns` plugin): This would need to take accessibility into account, and would need a different display for the mobile web layout. Need to consider how it will interact with the column menu suggested in #690.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681334912, https://github.com/simonw/datasette/issues/873#issuecomment-675610275,https://api.github.com/repos/simonw/datasette/issues/873,675610275,MDEyOklzc3VlQ29tbWVudDY3NTYxMDI3NQ==,9599,2020-08-18T17:24:05Z,2020-08-18T17:26:10Z,OWNER,"Maybe I can do this with ASGI after all. Here's the output of `/-/asgi-scope` with `datasette-debug-asgi` installed: ``` {'asgi': {'spec_version': '2.1', 'version': '3.0'}, 'client': ('127.0.0.1', 62035), 'headers': [(b'host', b'127.0.0.1:62029'), (b'user-agent', b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko' b'/20100101 Firefox/79.0'), (b'accept', b'text/html,application/xhtml+xml,application/xml;q=0.9,image/' b'webp,*/*;q=0.8'), (b'accept-language', b'en-US,en;q=0.5'), (b'accept-encoding', b'gzip, deflate'), (b'dnt', b'1'), (b'connection', b'keep-alive'), (b'upgrade-insecure-requests', b'1'), (b'cache-control', b'max-age=0')], 'http_version': '1.1', 'method': 'GET', 'path': '/-/asgi-scope', 'query_string': b'', 'raw_path': b'/-/asgi-scope', 'root_path': '', 'scheme': 'http', 'server': ('127.0.0.1', 62029), 'type': 'http'} ``` That `'server': ('127.0.0.1', 62029)` bit has the correct port. Question is, can I access that programmatically on server startup?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",647095487, https://github.com/simonw/datasette/issues/873#issuecomment-675609109,https://api.github.com/repos/simonw/datasette/issues/873,675609109,MDEyOklzc3VlQ29tbWVudDY3NTYwOTEwOQ==,9599,2020-08-18T17:21:51Z,2020-08-18T17:21:51Z,OWNER,Asked about this on the encode gitter here: https://gitter.im/encode/community?at=5f3c0dcaa8c17801765940c0,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",647095487, https://github.com/simonw/datasette/issues/940#issuecomment-675538586,https://api.github.com/repos/simonw/datasette/issues/940,675538586,MDEyOklzc3VlQ29tbWVudDY3NTUzODU4Ng==,9599,2020-08-18T15:11:36Z,2020-08-18T15:11:36Z,OWNER,I tested this new publish pattern (running the tests in parallel before the deploy step) on `github-to-sqlite` - skipping the Docker step - and it worked: https://github.com/dogsheep/github-to-sqlite/actions/runs/213809864,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-675253373,https://api.github.com/repos/simonw/datasette/issues/940,675253373,MDEyOklzc3VlQ29tbWVudDY3NTI1MzM3Mw==,9599,2020-08-18T05:10:17Z,2020-08-18T05:10:17Z,OWNER,I'll close this after the next release successfully goes out.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-675251613,https://api.github.com/repos/simonw/datasette/issues/940,675251613,MDEyOklzc3VlQ29tbWVudDY3NTI1MTYxMw==,9599,2020-08-18T05:05:15Z,2020-08-18T05:05:15Z,OWNER,I think this is ready. I'll only know for sure the first time I push a release through it though!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-674590583,https://api.github.com/repos/simonw/datasette/issues/940,674590583,MDEyOklzc3VlQ29tbWVudDY3NDU5MDU4Mw==,9599,2020-08-16T23:15:51Z,2020-08-18T05:04:43Z,OWNER,This example of jobs depending on each other and sharing data via artifacts looks relevant: https://docs.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts#passing-data-between-jobs-in-a-workflow,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-675250280,https://api.github.com/repos/simonw/datasette/issues/940,675250280,MDEyOklzc3VlQ29tbWVudDY3NTI1MDI4MA==,9599,2020-08-18T05:01:34Z,2020-08-18T05:01:42Z,OWNER,I think `${GITHUB_REF#refs/tags/}` is the equivalent of `$TRAVIS_TAG`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-674589472,https://api.github.com/repos/simonw/datasette/issues/940,674589472,MDEyOklzc3VlQ29tbWVudDY3NDU4OTQ3Mg==,9599,2020-08-16T23:05:57Z,2020-08-16T23:05:57Z,OWNER,When I figure this out I'll update the https://github.com/simonw/datasette-plugin/blob/main/datasette-%7B%7Bcookiecutter.hyphenated%7D%7D/.github/workflows/publish.yml default workflow to do this - right now it runs the tests once on just a single version of Python as part of the package deploy to PyPI step.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-674589321,https://api.github.com/repos/simonw/datasette/issues/940,674589321,MDEyOklzc3VlQ29tbWVudDY3NDU4OTMyMQ==,9599,2020-08-16T23:04:34Z,2020-08-16T23:04:34Z,OWNER,"https://docs.github.com/en/actions/getting-started-with-github-actions/core-concepts-for-github-actions#job > A set of steps that execute on the same runner. You can define the dependency rules for how jobs run in a workflow file. Jobs can run at the same time in parallel or run sequentially depending on the status of a previous job. For example, a workflow can have two sequential jobs that build and test code, where the test job is dependent on the status of the build job. If the build job fails, the test job will not run.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/940#issuecomment-674589035,https://api.github.com/repos/simonw/datasette/issues/940,674589035,MDEyOklzc3VlQ29tbWVudDY3NDU4OTAzNQ==,9599,2020-08-16T23:02:23Z,2020-08-16T23:02:23Z,OWNER,"I'd like to set these up as different workflows that depend on each other, if that's possible. I want to start three test runs in parallel (on three different Python versions), then if all three pass kick off the PyPI push (without running more tests), then if that passes do the Docker build and push.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/914#issuecomment-674578388,https://api.github.com/repos/simonw/datasette/issues/914,674578388,MDEyOklzc3VlQ29tbWVudDY3NDU3ODM4OA==,9599,2020-08-16T21:10:27Z,2020-08-16T21:10:27Z,OWNER,"Demo of the fix: https://latest.datasette.io/fixtures/binary_data.json?_sort_desc=data&_shape=array&_nl=on ``` {""rowid"": 2, ""data"": {""$base64"": true, ""encoded"": ""FRwDx60F/g==""}} {""rowid"": 1, ""data"": {""$base64"": true, ""encoded"": ""FRwCx60F/g==""}} ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",671056788, https://github.com/simonw/datasette/issues/940#issuecomment-674566618,https://api.github.com/repos/simonw/datasette/issues/940,674566618,MDEyOklzc3VlQ29tbWVudDY3NDU2NjYxOA==,9599,2020-08-16T19:20:58Z,2020-08-16T19:20:58Z,OWNER,I need to figure out how to build and push the Docker image on releases. Here's the Travis code for that: https://github.com/simonw/datasette/blob/52eabb019d4051084b21524bd0fd9c2731126985/.travis.yml#L38-L47,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679808124, https://github.com/simonw/datasette/issues/938#issuecomment-674558631,https://api.github.com/repos/simonw/datasette/issues/938,674558631,MDEyOklzc3VlQ29tbWVudDY3NDU1ODYzMQ==,9599,2020-08-16T18:10:23Z,2020-08-16T18:10:23Z,OWNER,Documentation: https://github.com/simonw/datasette/blob/3a4c8ed36aa97211e46849d32a09f2f386f342dd/docs/plugin_hooks.rst#extra-template-vars-template-database-table-columns-view-name-request-datasette,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679700269, https://github.com/simonw/datasette/issues/938#issuecomment-674551826,https://api.github.com/repos/simonw/datasette/issues/938,674551826,MDEyOklzc3VlQ29tbWVudDY3NDU1MTgyNg==,9599,2020-08-16T17:07:57Z,2020-08-16T17:07:57Z,OWNER,extra_tenplate_vars should he documented first and should be the only one to document the arguments and the async return feature. Others should refer back to it rather than duplicating that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679700269, https://github.com/simonw/datasette/issues/939#issuecomment-674548163,https://api.github.com/repos/simonw/datasette/issues/939,674548163,MDEyOklzc3VlQ29tbWVudDY3NDU0ODE2Mw==,9599,2020-08-16T16:34:30Z,2020-08-16T16:34:30Z,OWNER,Docs also need to note that `request` may be `None` for all of these hooks.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679779797, https://github.com/simonw/datasette/issues/939#issuecomment-674547811,https://api.github.com/repos/simonw/datasette/issues/939,674547811,MDEyOklzc3VlQ29tbWVudDY3NDU0NzgxMQ==,9599,2020-08-16T16:31:20Z,2020-08-16T16:31:20Z,OWNER,And the docs need to be updated too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679779797, https://github.com/simonw/datasette/issues/939#issuecomment-674547788,https://api.github.com/repos/simonw/datasette/issues/939,674547788,MDEyOklzc3VlQ29tbWVudDY3NDU0Nzc4OA==,9599,2020-08-16T16:31:08Z,2020-08-16T16:31:08Z,OWNER,Also: they should all be able to return `async def` inner functions. At the moment only `extra_template_vars` supports that pattern.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",679779797,