html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/1852#issuecomment-1291243333,https://api.github.com/repos/simonw/datasette/issues/1852,1291243333,IC_kwDOBm6k_c5M9s9F,9599,simonw,2022-10-25T23:25:13Z,2022-10-25T23:25:13Z,OWNER,"A `/-/debug-token` page that can take a token and decode it to show you how long until it expires, what actor it represents and the permissions it has will be useful as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421552095,Default API token authentication mechanism, https://github.com/simonw/datasette/issues/1852#issuecomment-1291234262,https://api.github.com/repos/simonw/datasette/issues/1852,1291234262,IC_kwDOBm6k_c5M9qvW,9599,simonw,2022-10-25T23:11:23Z,2022-10-25T23:11:23Z,OWNER,I'm going to build an initial `/-/create-token` interface which just bakes a token with the current actor in it and an optional expiry timestamp. I'll try the limited permissions thing later.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421552095,Default API token authentication mechanism, https://github.com/simonw/datasette/issues/1852#issuecomment-1291233652,https://api.github.com/repos/simonw/datasette/issues/1852,1291233652,IC_kwDOBm6k_c5M9ql0,9599,simonw,2022-10-25T23:10:20Z,2022-10-25T23:10:44Z,OWNER,"In which case the token would need to duplicate the current `actor` and then add extra constraints. So maybe the token design looks like this: ```json { ""a"": { ""copy_of"": ""actor_creating_token""}, ""p"": { ""t"": ""... the thing designed earlier, with those permissions in it"" }, ""e"": ""integer timestamp when token expires"" } ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421552095,Default API token authentication mechanism, https://github.com/simonw/datasette/issues/1852#issuecomment-1291232589,https://api.github.com/repos/simonw/datasette/issues/1852,1291232589,IC_kwDOBm6k_c5M9qVN,9599,simonw,2022-10-25T23:08:37Z,2022-10-25T23:08:37Z,OWNER,"... so maybe there's a way to create a token that inherits the exact permissions of the actor that created the token? That could even be a default mode for tokens, with an option to then further restrict permissions if desired.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421552095,Default API token authentication mechanism, https://github.com/simonw/datasette/issues/1852#issuecomment-1291231651,https://api.github.com/repos/simonw/datasette/issues/1852,1291231651,IC_kwDOBm6k_c5M9qGj,9599,simonw,2022-10-25T23:07:17Z,2022-10-25T23:07:17Z,OWNER,"Interesting challenge: what permissions should users be allowed to grant to tokens? Clearly a user should not be able to create a token with a permission that the user themselves does not have. And should there be a permission that allows people to create tokens? I think so.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421552095,Default API token authentication mechanism, https://github.com/simonw/datasette/issues/1851#issuecomment-1291228502,https://api.github.com/repos/simonw/datasette/issues/1851,1291228502,IC_kwDOBm6k_c5M9pVW,25778,eyeseast,2022-10-25T23:02:10Z,2022-10-25T23:02:10Z,CONTRIBUTOR,That's reasonable. Canned queries and custom endpoints are certainly going to give more room for specific needs. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table, https://github.com/simonw/datasette/issues/1852#issuecomment-1291227942,https://api.github.com/repos/simonw/datasette/issues/1852,1291227942,IC_kwDOBm6k_c5M9pMm,9599,simonw,2022-10-25T23:01:18Z,2022-10-25T23:01:18Z,OWNER,"Datasette currently defaults to having everything public-readable by default, unless a permission plugin changes that default. In thinking more about this API mechanism, I realized that it might be good to have a mode where Datasette _doesn't_ default to public everything. Maybe `datasette --private` to start it like that? Might even be an opportunity to get rid of the current slightly confusing mechanism where permission checks can announce that they should default to true: https://github.com/simonw/datasette/blob/c7dd76c26257ded5bcdfd0570e12412531b8b88f/datasette/views/database.py#L152-L154","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421552095,Default API token authentication mechanism, https://github.com/simonw/datasette/issues/1851#issuecomment-1291226367,https://api.github.com/repos/simonw/datasette/issues/1851,1291226367,IC_kwDOBm6k_c5M9oz_,9599,simonw,2022-10-25T22:58:30Z,2022-10-25T22:58:30Z,OWNER,"The `datasette insert` concept included plugin support, with the idea of being able to support things like SpatiaLite files: - #1160 I think this API mechanism is going to be a bit less exciting than that - it will be low-level for inserting rows, and if you want to do something fancier you can use a canned query that feeds incoming GeoJSON to a SpatiaLite function instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table, https://github.com/simonw/sqlite-utils/issues/505#issuecomment-1291216193,https://api.github.com/repos/simonw/sqlite-utils/issues/505,1291216193,IC_kwDOCGYnMM5M9mVB,9599,simonw,2022-10-25T22:41:16Z,2022-10-25T22:41:16Z,OWNER,Tweeted about it here: https://twitter.com/simonw/status/1585038766678609921,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423182778,Release sqlite-utils 3.30, https://github.com/simonw/sqlite-utils/issues/505#issuecomment-1291203911,https://api.github.com/repos/simonw/sqlite-utils/issues/505,1291203911,IC_kwDOCGYnMM5M9jVH,9599,simonw,2022-10-25T22:21:02Z,2022-10-25T22:21:02Z,OWNER,"- Now tested against Python 3.11. ([#502](https://github.com/simonw/sqlite-utils/issues/502)) - New `table.search_sql(include_rank=True)` option, which adds a `rank` column to the generated SQL. Thanks, Jacob Chapman. ([#480](https://github.com/simonw/sqlite-utils/pull/480)) - Progress bars now display for newline-delimited JSON files using the `--nl` option. Thanks, Mischa Untaga. ([#485](https://github.com/simonw/sqlite-utils/issues/485)) - New `db.close()` method. ([#504](https://github.com/simonw/sqlite-utils/issues/504)) - Conversion functions passed to [table.convert(...)](https://sqlite-utils.datasette.io/en/stable/python-api.html#python-api-convert) can now return lists or dictionaries, which will be inserted into the database as JSON strings. ([#495](https://github.com/simonw/sqlite-utils/issues/495)) - `sqlite-utils install` and `sqlite-utils uninstall` commands for installing packages into the same virtual environment as `sqlite-utils`, [described here](https://sqlite-utils.datasette.io/en/stable/cli.html#cli-install). ([#483](https://github.com/simonw/sqlite-utils/issues/483)) - New [sqlite_utils.utils.flatten()](https://sqlite-utils.datasette.io/en/stable/reference.html#reference-utils-flatten) utility function. ([#500](https://github.com/simonw/sqlite-utils/issues/500)) - Documentation on [using Just](https://sqlite-utils.datasette.io/en/stable/contributing.html#contributing-just) to run tests, linters and build documentation. - Documentation now covers the [Release process](https://sqlite-utils.datasette.io/en/stable/contributing.html#release-process) for this package.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423182778,Release sqlite-utils 3.30, https://github.com/simonw/sqlite-utils/issues/496#issuecomment-1291170072,https://api.github.com/repos/simonw/sqlite-utils/issues/496,1291170072,IC_kwDOCGYnMM5M9bEY,9599,simonw,2022-10-25T21:36:12Z,2022-10-25T21:36:12Z,OWNER,"I was going to suggest using `db.table(name)` instead of `db[name]` - but it looks like that method will have the same problem: https://github.com/simonw/sqlite-utils/blob/defa2974c6d3abc19be28d6b319649b8028dc966/sqlite_utils/db.py#L497-L506 I could change `sqlite-utils` so `db.table(name)` always returns a table and you need to call `db.view(name)` if you want to access a view - that would require bumping to 4.0 though. I'm not convinced that's the best approach here either.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1393202060,devrel/python api: Pylance type hinting, https://github.com/simonw/sqlite-utils/issues/496#issuecomment-1291167887,https://api.github.com/repos/simonw/sqlite-utils/issues/496,1291167887,IC_kwDOCGYnMM5M9aiP,9599,simonw,2022-10-25T21:33:25Z,2022-10-25T21:33:25Z,OWNER,"I do care about this, but I'm not hugely experienced with types yet so I'm open to suggestions about how to do it!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1393202060,devrel/python api: Pylance type hinting, https://github.com/simonw/sqlite-utils/issues/493#issuecomment-1291166273,https://api.github.com/repos/simonw/sqlite-utils/issues/493,1291166273,IC_kwDOCGYnMM5M9aJB,9599,simonw,2022-10-25T21:31:15Z,2022-10-25T21:31:15Z,OWNER,"Based on the docs here I tried the following too: https://docutils.sourceforge.io/docs/user/smartquotes.html#description - `\--` - `\\--` - `\\-\\-` - `\-\-` But none of them had the desired effect in this particular piece of markup: the :ref:`insert \--convert ` I think because this is text inside a `:ref:` block, not regular text. Consider the following: The \--convert and the :ref:`insert \--convert ` and It's rendered like this: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1386562662,Tiny typographical error in install/uninstall docs, https://github.com/simonw/sqlite-utils/issues/495#issuecomment-1291159549,https://api.github.com/repos/simonw/sqlite-utils/issues/495,1291159549,IC_kwDOCGYnMM5M9Yf9,9599,simonw,2022-10-25T21:23:01Z,2022-10-25T21:23:01Z,OWNER,"I've decided not to explicitly document this, since it's consistent with how other parts of the library work already.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1392690202,Support JSON values returned from .convert() functions, https://github.com/simonw/sqlite-utils/issues/495#issuecomment-1291152433,https://api.github.com/repos/simonw/sqlite-utils/issues/495,1291152433,IC_kwDOCGYnMM5M9Wwx,9599,simonw,2022-10-25T21:14:54Z,2022-10-25T21:14:54Z,OWNER,"There is a case where the function can return a dictionary at the moment: `multi=True` ```python table.convert( ""title"", lambda v: {""upper"": v.upper(), ""lower"": v.lower()}, multi=True ) ``` But I think this change is still compatible with that. if you don't use `multi=True` then the return value will be stringified. If you DO use `multi=True` then something like this could work: ```python table.convert( ""title"", lambda v: {""upper"": {""str"": v.upper()}, ""lower"": {""str"": v.lower()}}, multi=True ) ``` This would result in a `upper` and `lower` column, each containing the JSON string `{""str"": ""UPPERCASE""}`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1392690202,Support JSON values returned from .convert() functions, https://github.com/simonw/sqlite-utils/issues/495#issuecomment-1291149509,https://api.github.com/repos/simonw/sqlite-utils/issues/495,1291149509,IC_kwDOCGYnMM5M9WDF,9599,simonw,2022-10-25T21:12:11Z,2022-10-25T21:12:11Z,OWNER,"This makes sense to me. There are other places in the codebase where JSON is automatically stringified: https://github.com/simonw/sqlite-utils/blob/c7e4308e6f49d929704163531632e558f9646e4a/sqlite_utils/db.py#L2759-L2766 I don't see why the return value from a convert function shouldn't do the same thing. Since this will result in previous errors working, I don't think it warrants a major version bump either.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1392690202,Support JSON values returned from .convert() functions, https://github.com/simonw/sqlite-utils/issues/497#issuecomment-1291146850,https://api.github.com/repos/simonw/sqlite-utils/issues/497,1291146850,IC_kwDOCGYnMM5M9VZi,9599,simonw,2022-10-25T21:09:28Z,2022-10-25T21:09:28Z,OWNER,"Yeah, `table.columns` and `table.columns_dict` are meant to handle this: https://sqlite-utils.datasette.io/en/stable/python-api.html#columns","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1393212964,column_names, https://github.com/simonw/sqlite-utils/issues/504#issuecomment-1291136971,https://api.github.com/repos/simonw/sqlite-utils/issues/504,1291136971,IC_kwDOCGYnMM5M9S_L,9599,simonw,2022-10-25T21:00:29Z,2022-10-25T21:00:29Z,OWNER,Documentation: https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Database.close,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423069384,"db.close() method, calling db.conn.close()", https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291124413,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291124413,IC_kwDOCGYnMM5M9P69,9599,simonw,2022-10-25T20:47:34Z,2022-10-25T20:47:34Z,OWNER,TIL about this: https://til.simonwillison.net/python/os-remove-windows,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291122389,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291122389,IC_kwDOCGYnMM5M9PbV,9599,simonw,2022-10-25T20:45:43Z,2022-10-25T20:45:43Z,OWNER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291115986,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291115986,IC_kwDOCGYnMM5M9N3S,9599,simonw,2022-10-25T20:39:24Z,2022-10-25T20:39:24Z,OWNER,"Used `psutil` to confirm that closing a SQLite connection closes the underlying file: https://til.simonwillison.net/python/too-many-open-files-psutil ```pycon >>> import psutil >>> import sqlite3 >>> for f in psutil.Process().open_files(): print(f) ... >>> sqlite3.connect(""/tmp/blah.db"") >>> conn = _ >>> for f in psutil.Process().open_files(): print(f) ... popenfile(path='/private/tmp/blah.db', fd=3) >>> conn.close() >>> for f in psutil.Process().open_files(): print(f) ... >>> ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291111357,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291111357,IC_kwDOCGYnMM5M9Mu9,9599,simonw,2022-10-25T20:36:06Z,2022-10-25T20:36:06Z,OWNER,... or maybe Windows doesn't like attempts to remove a file that the process has opened?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291103021,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291103021,IC_kwDOCGYnMM5M9Kst,9599,simonw,2022-10-25T20:32:01Z,2022-10-25T20:32:01Z,OWNER,"This test reliably fails on Windows with Python 3.11. I'm going to skip the test for the moment to get back to green CI... but I'll leave this issue open. This is definitely concerning, I just don't have the right local environment to solve this at the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291093581,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291093581,IC_kwDOCGYnMM5M9IZN,9599,simonw,2022-10-25T20:23:00Z,2022-10-25T20:23:00Z,OWNER,"I'm not hugely happy with my fix there: https://github.com/simonw/sqlite-utils/blob/c5d7ec1dd71fa1dce829bc8bb82b639018befd63/sqlite_utils/db.py#L321-L328 The problem here was that in the case where the `os.remove()` failed the `self.conn` property was NOT being set to a valid connection - which caused `__repr__` to fail later on. So now I catch the `os.remove()` error, set `self.conn` to a memory connection, then raise the error again.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291088108,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291088108,IC_kwDOCGYnMM5M9HDs,9599,simonw,2022-10-25T20:17:36Z,2022-10-25T20:17:36Z,OWNER,"Now `mypy` is failing: ``` sqlite_utils/db.py:474: error: Item ""None"" of ""Optional[Any]"" has no attribute ""execute"" sqlite_utils/db.py:476: error: Item ""None"" of ""Optional[Any]"" has no attribute ""execute"" sqlite_utils/db.py:486: error: Item ""None"" of ""Optional[Any]"" has no attribute ""executescript"" sqlite_utils/db.py:603: error: Item ""None"" of ""Optional[Any]"" has no attribute ""__enter__"" sqlite_utils/db.py:603: error: Item ""None"" of ""Optional[Any]"" has no attribute ""__exit__"" sqlite_utils/db.py:604: error: Item ""None"" of ""Optional[Any]"" has no attribute ""execute"" sqlite_utils/db.py:607: error: Item ""None"" of ""Optional[Any]"" has no attribute ""execute"" sqlite_utils/db.py:1082: error: Item ""None"" of ""Optional[Any]"" has no attribute ""__enter__"" sqlite_utils/db.py:1082: error: Item ""None"" of ""Optional[Any]"" has no attribute ""__exit__"" sqlite_utils/db.py:1083: error: Item ""None"" of ""Optional[Any]"" has no attribute ""cursor"" sqlite_utils/db.py:1155: error: Item ""None"" of ""Optional[Any]"" has no attribute ""enable_load_extension"" sqlite_utils/db.py:1156: error: Item ""None"" of ""Optional[Any]"" has no attribute ""load_extension"" Found 12 errors in 1 file (checked 51 source files) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291083188,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291083188,IC_kwDOCGYnMM5M9F20,9599,simonw,2022-10-25T20:12:52Z,2022-10-25T20:12:52Z,OWNER,"Failed again, but just noticed this: https://github.com/simonw/sqlite-utils/actions/runs/3323932266/jobs/5494890223 ``` > Database(filepath, recreate=True)[""t2""].insert({""foo"": ""bar""}) tests\test_recreate.py:31: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <[AttributeError(""'Database' object has no attribute 'conn'"") raised in repr()] Database object at 0x29fc125aa90> ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291076031,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291076031,IC_kwDOCGYnMM5M9EG_,9599,simonw,2022-10-25T20:06:28Z,2022-10-25T20:06:28Z,OWNER,"This is the failing test: https://github.com/simonw/sqlite-utils/blob/7b2d1c0ffd0b874e280292b926f328a61cb31e2c/tests/test_recreate.py#L21-L32 I'm going to try a different way of creating the temporary file: https://docs.pytest.org/en/7.1.x/how-to/tmp_path.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/sqlite-utils/issues/503#issuecomment-1291071627,https://api.github.com/repos/simonw/sqlite-utils/issues/503,1291071627,IC_kwDOCGYnMM5M9DCL,9599,simonw,2022-10-25T20:02:18Z,2022-10-25T20:02:18Z,OWNER,Passes on Windows with other Python versions for some reason.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1423000702,test_recreate failing on Windows Python 3.11, https://github.com/simonw/datasette/issues/1854#issuecomment-1291047214,https://api.github.com/repos/simonw/datasette/issues/1854,1291047214,IC_kwDOBm6k_c5M89Eu,9599,simonw,2022-10-25T19:39:36Z,2022-10-25T19:39:48Z,OWNER,"This pattern should work (for the http server at least): ```python # Loop until port 8041 serves traffic while True: try: httpx.get(""http://localhost:8041/"") break except httpx.ConnectError: time.sleep(0.1) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422973111,Flaky test: test_serve_localhost_http, https://github.com/simonw/datasette/issues/1854#issuecomment-1291046958,https://api.github.com/repos/simonw/datasette/issues/1854,1291046958,IC_kwDOBm6k_c5M89Au,9599,simonw,2022-10-25T19:39:22Z,2022-10-25T19:39:22Z,OWNER,"Here's the code that starts those various servers: https://github.com/simonw/datasette/blob/613ad05c095f92653221db267ef53d54d00cdfbb/tests/conftest.py#L104-L177 I don't like those `time.sleep(1.5)` lines much - I'm going to try polling for readiness instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422973111,Flaky test: test_serve_localhost_http, https://github.com/simonw/datasette/issues/1854#issuecomment-1291045997,https://api.github.com/repos/simonw/datasette/issues/1854,1291045997,IC_kwDOBm6k_c5M88xt,9599,simonw,2022-10-25T19:38:28Z,2022-10-25T19:38:28Z,OWNER,"Also: ``` @pytest.mark.serial @pytest.mark.skipif( not hasattr(socket, ""AF_UNIX""), reason=""Requires socket.AF_UNIX support"" ) def test_serve_unix_domain_socket(ds_unix_domain_socket_server): _, uds = ds_unix_domain_socket_server transport = httpx.HTTPTransport(uds=uds) client = httpx.Client(transport=transport) > response = client.get(""http://localhost/_memory.json"") /home/runner/work/datasette/datasette/tests/test_cli_serve_server.py:35: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_client.py:1039: in get return self.request( /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_client.py:815: in request return self.send(request, auth=auth, follow_redirects=follow_redirects) /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_client.py:902: in send response = self._send_handling_auth( /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_client.py:930: in _send_handling_auth response = self._send_handling_redirects( /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_client.py:967: in _send_handling_redirects response = self._send_single_request(request) /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_client.py:1003: in _send_single_request response = transport.handle_request(request) /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/httpx/_transports/default.py:217: in handle_request with map_httpcore_exceptions(): /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/contextlib.py:153: in __exit__ self.gen.throw(typ, value, traceback) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ @contextlib.contextmanager def map_httpcore_exceptions() -> typing.Iterator[None]: try: yield except Exception as exc: # noqa: PIE-786 mapped_exc = None for from_exc, to_exc in HTTPCORE_EXC_MAP.items(): if not isinstance(exc, from_exc): continue # We want to map to the most specific exception we can find. # Eg if `exc` is an `httpcore.ReadTimeout`, we want to map to # `httpx.ReadTimeout`, not just `httpx.TimeoutException`. if mapped_exc is None or issubclass(to_exc, mapped_exc): mapped_exc = to_exc if mapped_exc is None: # pragma: nocover raise message = str(exc) > raise mapped_exc(message) from exc E httpx.ConnectError: [Errno 2] No such file or directory ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422973111,Flaky test: test_serve_localhost_http, https://github.com/simonw/datasette/issues/1853#issuecomment-1291036623,https://api.github.com/repos/simonw/datasette/issues/1853,1291036623,IC_kwDOBm6k_c5M86fP,9599,simonw,2022-10-25T19:28:56Z,2022-10-25T19:28:56Z,OWNER,"Opened an issue here: - https://github.com/coleifer/pysqlite3/issues/43","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/datasette/issues/1853#issuecomment-1291032289,https://api.github.com/repos/simonw/datasette/issues/1853,1291032289,IC_kwDOBm6k_c5M85bh,9599,simonw,2022-10-25T19:24:27Z,2022-10-25T19:24:27Z,OWNER,https://latest.datasette.io/-/versions now shows 3.11.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/sqlite-utils/issues/502#issuecomment-1291029761,https://api.github.com/repos/simonw/sqlite-utils/issues/502,1291029761,IC_kwDOCGYnMM5M840B,9599,simonw,2022-10-25T19:21:44Z,2022-10-25T19:21:44Z,OWNER,"Replicated locally using a fresh virtual environment with Python 3.11 and: pytest -k test_query_invalid_function","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422954582,Fix tests for Python 3.11, https://github.com/simonw/datasette/issues/1853#issuecomment-1291023926,https://api.github.com/repos/simonw/datasette/issues/1853,1291023926,IC_kwDOBm6k_c5M83Y2,9599,simonw,2022-10-25T19:15:49Z,2022-10-25T19:15:49Z,OWNER,"This broke the deploy of `https://latest.datasette.io/` - because it tries to install `pysqlite3-binary` which doesn't have a 3.11 release yet: https://github.com/simonw/datasette/blob/2e9751672d4fe329b3c359d5b7b1992283185820/.github/workflows/deploy-latest.yml#L77 I started using that for the `latest.datasette.io` demo in https://github.com/simonw/datasette/commit/a970276b9999687b96c5e11ea1c817d814f5d267 because I wanted a version of SQLite that supported generated columns. Those were added in [SQLite 3.31.0](https://www.sqlite.org/changes.html#version_3_31_0) - and the SQLite version in the new base image is 3.34.1 - so I don't actually need `pysqlite3-binary` any more.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/datasette/issues/1853#issuecomment-1291012637,https://api.github.com/repos/simonw/datasette/issues/1853,1291012637,IC_kwDOBm6k_c5M80od,9599,simonw,2022-10-25T19:04:03Z,2022-10-25T19:04:09Z,OWNER,"And tested `datasette package` like this: ``` datasette package fixtures.db -t datasette-package-python-upgrade-3-11 ``` Then: ``` docker run -p 8081:8001 datasette-package-python-upgrade-3-11 ``` And tested it like this: ``` curl http://localhost:8081/-/versions.json | jq ``` Output: ``` { ""python"": { ""version"": ""3.11.0"", ""full"": ""3.11.0 (main, Oct 25 2022, 05:00:36) [GCC 10.2.1 20210110]"" }, ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/datasette/issues/1853#issuecomment-1291009987,https://api.github.com/repos/simonw/datasette/issues/1853,1291009987,IC_kwDOBm6k_c5M8z_D,9599,simonw,2022-10-25T19:01:23Z,2022-10-25T19:01:23Z,OWNER,"Also tested by running this locally: datasette publish cloudrun fixtures.db --service issue-1853 https://issue-1853-j7hipcg4aq-uc.a.run.app/-/versions now shows Python 3.11.0.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/datasette/issues/1853#issuecomment-1291006149,https://api.github.com/repos/simonw/datasette/issues/1853,1291006149,IC_kwDOBm6k_c5M8zDF,9599,simonw,2022-10-25T18:57:33Z,2022-10-25T18:57:33Z,OWNER,"Ran the upgrade on the Datasette Cloud image first, works fine there. https://simon.datasette.cloud/-/versions shows me: ``` { ""python"": { ""version"": ""3.11.0"", ""full"": ""3.11.0 (main, Oct 25 2022, 05:00:36) [GCC 10.2.1 20210110]"" }, ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/datasette/issues/1853#issuecomment-1290995178,https://api.github.com/repos/simonw/datasette/issues/1853,1290995178,IC_kwDOBm6k_c5M8wXq,9599,simonw,2022-10-25T18:46:33Z,2022-10-25T18:46:33Z,OWNER,"I ran a very crude benchmark on my laptop using Locust (against the official macOS packages from www.python.org for Python 3.10 and Python 3.11) and saw a substantial speed increase: 533.89 requests/second on 3.11 413.56 requests/second on 3.10 That was from running Locust against this `locustfile.py`: ```python from locust import HttpUser, task class CounterOne(HttpUser): @task def hello(self): self.client.get(""/-/static/app.css"") ``` Using: locust --headless --users 4 --spawn-rate 4 -H http://127.0.0.1:8001","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1422915587,Upgrade Datasette Docker to Python 3.11, https://github.com/simonw/datasette/issues/1851#issuecomment-1290615599,https://api.github.com/repos/simonw/datasette/issues/1851,1290615599,IC_kwDOBm6k_c5M7Tsv,25778,eyeseast,2022-10-25T14:05:12Z,2022-10-25T14:05:12Z,CONTRIBUTOR,"This could use a new plugin hook, too. I don't want to complicate your life too much, but for things like GIS, I'd want a way to turn regular JSON into SpatiaLite geometries or combine X/Y coordinates into point geometries and such. Happy to help however I can.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table, https://github.com/simonw/datasette/issues/1851#issuecomment-1289865317,https://api.github.com/repos/simonw/datasette/issues/1851,1289865317,IC_kwDOBm6k_c5M4chl,9599,simonw,2022-10-25T01:42:47Z,2022-10-25T01:42:47Z,OWNER,"This is going to tie into Datasette's existing permissions mechanism, so plugins will be able to define their own custom mechanisms for tokens to be attached to a specific identity: https://docs.datasette.io/en/stable/authentication.html There's only one plugin for API tokens at the moment, which is this one: https://datasette.io/plugins/datasette-auth-tokens I'm actually planning on adding another, default token mechanism to Datasette itself as part of this work: - #1852 It may well be that `datasette-sandstorm-support` needs to add something custom here too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1421544654,API to insert a single record into an existing table,