html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/issues/842#issuecomment-646271834,https://api.github.com/repos/simonw/datasette/issues/842,646271834,MDEyOklzc3VlQ29tbWVudDY0NjI3MTgzNA==,9599,2020-06-18T19:49:41Z,2020-06-24T18:49:22Z,OWNER,"But then what kind of magic parameters might plugins want to add?
Here's a crazy idea: `_scrapedcontent_url` - it would look for the `url` column on the data being inserted, scrape the content from it and insert that. This does suggest that the magic resolving function `scrapedcontent()` would need to optionally be sent the full row dictionary being inserted too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/842#issuecomment-646270702,https://api.github.com/repos/simonw/datasette/issues/842,646270702,MDEyOklzc3VlQ29tbWVudDY0NjI3MDcwMg==,9599,2020-06-18T19:47:19Z,2020-06-24T18:48:48Z,OWNER,"Brainstorming more potential magic parameters:
* `_actor_id`
* `_actor_name`
* `_request_ip`
* `_request_user_agent`
* `_cookie_cookiename`
* `_signedcookie_cookiename` - reading signed cookies would be cool, not sure how to specify namespace though, maybe always use the same one? Or have the namespace come last, `_signedcookie_cookiename_mynamespace`. Might not need special signed cookie support since `actor` is already usually from a signed cookie.
* `_timestamp_unix` (not happy with these names yet)
* `_timestamp_localtime`
* `_timestamp_datetime`
* `_timestamp_utc`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/842#issuecomment-649000075,https://api.github.com/repos/simonw/datasette/issues/842,649000075,MDEyOklzc3VlQ29tbWVudDY0OTAwMDA3NQ==,9599,2020-06-24T18:46:36Z,2020-06-24T18:47:37Z,OWNER,"Another magic parameter that would be useful would be `_random`. Consider https://github.com/simonw/datasette-auth-tokens/issues/1 for example - I'd like to be able to provide a writable canned query which can create new authentication tokens in the database, but ideally it would automatically populate a secure random secret for each one.
Maybe `_random_chars_128` to create a 128 character long random string (using `os.urandom(64).hex()`).
This would be the first example of a magic parameter where part of the parameter name is used to configure the resulting value. Maybe neater to separate that with a different character? Unfortunately `_random_chars:128` wouldn't work because these parameters are used in a SQLite query where `:` has special meaning: `insert into blah (secret) values (:_random_chars:128)` wouldn't make sense.
Actually this is already supported by the proposed design - `_random_chars_128` would become `random(""chars_128"")` so the `random()` function could split off the 128 itself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/865#issuecomment-648998264,https://api.github.com/repos/simonw/datasette/issues/865,648998264,MDEyOklzc3VlQ29tbWVudDY0ODk5ODI2NA==,9599,2020-06-24T18:43:02Z,2020-06-24T18:43:02Z,OWNER,Thanks for the bug report. Yes I think #838 may be the same issue. Will investigate.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644582921,
https://github.com/simonw/datasette/issues/858#issuecomment-648997857,https://api.github.com/repos/simonw/datasette/issues/858,648997857,MDEyOklzc3VlQ29tbWVudDY0ODk5Nzg1Nw==,9599,2020-06-24T18:42:10Z,2020-06-24T18:42:10Z,OWNER,I really need to get myself a Windows 10 development environment working so I can dig into this kind of bug properly. I have a gaming PC lying around that I could re-task for that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642388564,
https://github.com/simonw/datasette/issues/864#issuecomment-648580556,https://api.github.com/repos/simonw/datasette/issues/864,648580556,MDEyOklzc3VlQ29tbWVudDY0ODU4MDU1Ng==,9599,2020-06-24T04:40:49Z,2020-06-24T04:40:49Z,OWNER,The ideal fix here would be to rework my `BaseView` subclass mechanism to work with `register_routes()` so that those views don't have any special privileges above plugin-provided views.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644309017,
https://github.com/simonw/datasette/issues/864#issuecomment-648580236,https://api.github.com/repos/simonw/datasette/issues/864,648580236,MDEyOklzc3VlQ29tbWVudDY0ODU4MDIzNg==,9599,2020-06-24T04:39:39Z,2020-06-24T04:39:39Z,OWNER,"Urgh, fixing this is going to be a bit of a pain.
Here's where I added that custom `dispatch_request()` method - it was to implement flash messaging in #790: https://github.com/simonw/datasette/blame/1a5b7d318fa923edfcefd3df8f64dae2e9c49d3f/datasette/views/base.py#L85
If I want this to be made available to `register_routes()` views as well, I'm going to have to move the logic somewhere else. In particular I need to make sure that the `request` object is created once and used throughout the whole request cycle.
Currently `register_routes()` view functions get their own separate request object which is created here:
https://github.com/simonw/datasette/blob/1a5b7d318fa923edfcefd3df8f64dae2e9c49d3f/datasette/app.py#L1057-L1068
So I'm going to have to refactor this quite a bit to get that shared request object which can be passed both to `register_routes` views and to my various `BaseView` subclasses.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644309017,
https://github.com/simonw/sqlite-utils/issues/117#issuecomment-648442511,https://api.github.com/repos/simonw/sqlite-utils/issues/117,648442511,MDEyOklzc3VlQ29tbWVudDY0ODQ0MjUxMQ==,9599,2020-06-23T21:39:41Z,2020-06-23T21:39:41Z,OWNER,"So there are two sides to supporting this:
- Being able to sensibly introspect composite foreign keys
- Being able to define composite foreign keys when creating a table","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644161221,
https://github.com/simonw/sqlite-utils/issues/117#issuecomment-648440634,https://api.github.com/repos/simonw/sqlite-utils/issues/117,648440634,MDEyOklzc3VlQ29tbWVudDY0ODQ0MDYzNA==,9599,2020-06-23T21:35:16Z,2020-06-23T21:35:16Z,OWNER,Relevant discussion: https://github.com/simonw/sqlite-generate/issues/8#issuecomment-648438056,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644161221,
https://github.com/simonw/sqlite-utils/issues/117#issuecomment-648440525,https://api.github.com/repos/simonw/sqlite-utils/issues/117,648440525,MDEyOklzc3VlQ29tbWVudDY0ODQ0MDUyNQ==,9599,2020-06-23T21:35:01Z,2020-06-23T21:35:01Z,OWNER,"Here's what's missing:
```
In [11]: db.conn.execute('PRAGMA foreign_key_list(song)').fetchall()
Out[11]:
[(0,
0,
'album',
'songartist',
'albumartist',
'NO ACTION',
'NO ACTION',
'NONE'),
(0, 1, 'album', 'songalbum', 'albumname', 'NO ACTION', 'NO ACTION', 'NONE')]
```
Compare with this code here:
https://github.com/simonw/sqlite-utils/blob/d0cdaaaf00249230e847be3a3b393ee2689fbfe4/sqlite_utils/db.py#L563-L579
The first two columns returned by `PRAGMA foreign_key_list(table)` are `id` and `seq` - these show when two foreign key records are part of the same compound foreign key. `sqlite-utils` entirely ignores those at the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644161221,
https://github.com/simonw/sqlite-utils/issues/116#issuecomment-648434885,https://api.github.com/repos/simonw/sqlite-utils/issues/116,648434885,MDEyOklzc3VlQ29tbWVudDY0ODQzNDg4NQ==,9599,2020-06-23T21:21:33Z,2020-06-23T21:21:33Z,OWNER,New docs: https://github.com/simonw/sqlite-utils/blob/2.10.1/docs/python-api.rst#introspection,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644122661,
https://github.com/simonw/sqlite-utils/issues/116#issuecomment-648403834,https://api.github.com/repos/simonw/sqlite-utils/issues/116,648403834,MDEyOklzc3VlQ29tbWVudDY0ODQwMzgzNA==,9599,2020-06-23T20:36:29Z,2020-06-23T20:36:29Z,OWNER,Should go in this section https://sqlite-utils.readthedocs.io/en/stable/python-api.html#introspection - under `.columns_dict` and before `.foreign_keys`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",644122661,
https://github.com/simonw/datasette/issues/859#issuecomment-648234787,https://api.github.com/repos/simonw/datasette/issues/859,648234787,MDEyOklzc3VlQ29tbWVudDY0ODIzNDc4Nw==,9599,2020-06-23T15:22:51Z,2020-06-23T15:22:51Z,OWNER,"I wonder if this is a SQLite caching issue then?
Datasette has a configuration option for this but I haven't spent much time experimenting with it so I don't know how much of an impact it can have: https://datasette.readthedocs.io/en/stable/config.html#cache-size-kb","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/859#issuecomment-648163272,https://api.github.com/repos/simonw/datasette/issues/859,648163272,MDEyOklzc3VlQ29tbWVudDY0ODE2MzI3Mg==,9599,2020-06-23T13:52:23Z,2020-06-23T13:52:23Z,OWNER,"I'm chunking inserts at 100 at a time right now: https://github.com/simonw/sqlite-utils/blob/4d9a3204361d956440307a57bd18c829a15861db/sqlite_utils/db.py#L1030
I think the performance is more down to using Faker to create the test data - generating millions of entirely fake, randomized records takes a fair bit of time.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/859#issuecomment-647894903,https://api.github.com/repos/simonw/datasette/issues/859,647894903,MDEyOklzc3VlQ29tbWVudDY0Nzg5NDkwMw==,9599,2020-06-23T04:07:59Z,2020-06-23T04:07:59Z,OWNER,"Just to check: are you seeing the problem on this page: https://latest.datasette.io/fixtures (the database page) - or this page (the table page): https://latest.datasette.io/fixtures/compound_three_primary_keys
If it's the table page then the problem may well be #862.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/596#issuecomment-647893140,https://api.github.com/repos/simonw/datasette/issues/596,647893140,MDEyOklzc3VlQ29tbWVudDY0Nzg5MzE0MA==,9599,2020-06-23T03:59:51Z,2020-06-23T03:59:51Z,OWNER,Related: #862 - a time limit on the total time spent considering suggested facets for a table.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",507454958,
https://github.com/simonw/datasette/issues/862#issuecomment-647892930,https://api.github.com/repos/simonw/datasette/issues/862,647892930,MDEyOklzc3VlQ29tbWVudDY0Nzg5MjkzMA==,9599,2020-06-23T03:58:48Z,2020-06-23T03:58:48Z,OWNER,Should this be controlled be a separate configuration setting? I'm inclined to say no - I think instead I'll set the limit to be 10 * whatever `facet_suggest_time_limit_ms` is.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",643510821,
https://github.com/simonw/datasette/issues/859#issuecomment-647890619,https://api.github.com/repos/simonw/datasette/issues/859,647890619,MDEyOklzc3VlQ29tbWVudDY0Nzg5MDYxOQ==,9599,2020-06-23T03:48:21Z,2020-06-23T03:48:21Z,OWNER," sqlite-generate many-cols.db --tables 2 --rows 200000 --columns 50
Looks like that will take 35 minutes to run (it's not a particularly fast tool).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/859#issuecomment-647890378,https://api.github.com/repos/simonw/datasette/issues/859,647890378,MDEyOklzc3VlQ29tbWVudDY0Nzg5MDM3OA==,9599,2020-06-23T03:47:19Z,2020-06-23T03:47:19Z,OWNER,"I generated a 600MB database using [sqlite-generate](https://github.com/simonw/sqlite-generate) just now - with 100 tables at 100,00 rows and 3 tables at 1,000,000 rows - and performance of the database page was fine, 250ms.
Those tables only had 4 columns each though.
You said ""200k+, 50+ rows in a couple of tables"" - does that mean 50+ columns? I'll try with larger numbers of columns and see what difference that makes.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/861#issuecomment-647889674,https://api.github.com/repos/simonw/datasette/issues/861,647889674,MDEyOklzc3VlQ29tbWVudDY0Nzg4OTY3NA==,9599,2020-06-23T03:44:17Z,2020-06-23T03:44:17Z,OWNER,https://github.com/simonw/sqlite-generate is now ready to be used - see also https://pypi.org/project/sqlite-generate/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642652808,
https://github.com/simonw/datasette/issues/861#issuecomment-647822757,https://api.github.com/repos/simonw/datasette/issues/861,647822757,MDEyOklzc3VlQ29tbWVudDY0NzgyMjc1Nw==,9599,2020-06-22T23:40:43Z,2020-06-22T23:40:43Z,OWNER,"I started building that tool here: https://github.com/simonw/sqlite-generate
(I built a new cookiecutter template for that too, https://github.com/simonw/click-app)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642652808,
https://github.com/simonw/datasette/issues/861#issuecomment-647266979,https://api.github.com/repos/simonw/datasette/issues/861,647266979,MDEyOklzc3VlQ29tbWVudDY0NzI2Njk3OQ==,9599,2020-06-22T04:26:25Z,2020-06-22T04:26:25Z,OWNER,I think this is a separate Click utility. I'm going to call it `sqlite-generate`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642652808,
https://github.com/simonw/datasette/issues/687#issuecomment-647258199,https://api.github.com/repos/simonw/datasette/issues/687,647258199,MDEyOklzc3VlQ29tbWVudDY0NzI1ODE5OQ==,9599,2020-06-22T03:55:20Z,2020-06-22T03:55:20Z,OWNER,https://datasette.readthedocs.io/en/latest/testing_plugins.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/687#issuecomment-647237091,https://api.github.com/repos/simonw/datasette/issues/687,647237091,MDEyOklzc3VlQ29tbWVudDY0NzIzNzA5MQ==,9599,2020-06-22T02:44:10Z,2020-06-22T02:44:10Z,OWNER,"Now split into four pages:
- https://datasette.readthedocs.io/en/latest/plugins.html
- https://datasette.readthedocs.io/en/latest/writing_plugins.html
- https://datasette.readthedocs.io/en/latest/plugin_hooks.html
- https://datasette.readthedocs.io/en/latest/internals.html
Still need to add the ""Testing plugins"" page, then I can close this issue.
I should also do #855, documenting the new `datasette-plugin` cookiecutter template. That can go in `writing_plugins.rst`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/687#issuecomment-647203845,https://api.github.com/repos/simonw/datasette/issues/687,647203845,MDEyOklzc3VlQ29tbWVudDY0NzIwMzg0NQ==,9599,2020-06-22T00:32:42Z,2020-06-22T00:32:42Z,OWNER,"Maybe add this to the plugins.rst page near the top:
```
.. toctree::
:caption: See also
:maxdepth: 1
plugin_hooks
internals
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/687#issuecomment-647190177,https://api.github.com/repos/simonw/datasette/issues/687,647190177,MDEyOklzc3VlQ29tbWVudDY0NzE5MDE3Nw==,9599,2020-06-21T22:32:36Z,2020-06-21T22:32:36Z,OWNER,I'm going to break out the plugin hooks first in a single commit to make for a cleaner commit history (since that way git can hopefully detect that the content moved).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/687#issuecomment-647190144,https://api.github.com/repos/simonw/datasette/issues/687,647190144,MDEyOklzc3VlQ29tbWVudDY0NzE5MDE0NA==,9599,2020-06-21T22:32:13Z,2020-06-21T22:32:13Z,OWNER,"So the new plan is NOT to have a `plugins/` folder, but instead have several top-level pages:
- Plugins (exists)
- Writing plugins
- Plugin hooks
- Testing plugins
- Internals for plugins (exists)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/859#issuecomment-647189948,https://api.github.com/repos/simonw/datasette/issues/859,647189948,MDEyOklzc3VlQ29tbWVudDY0NzE4OTk0OA==,9599,2020-06-21T22:30:12Z,2020-06-21T22:30:43Z,OWNER,"I'll write a little script which generates a 300MB SQLite file with a bunch of tables with lots of randomly generated rows in to help test this.
Having a tool like that which can generate larger databases with different gnarly performance characteristics will be useful for other performance work too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/859#issuecomment-647189666,https://api.github.com/repos/simonw/datasette/issues/859,647189666,MDEyOklzc3VlQ29tbWVudDY0NzE4OTY2Ng==,9599,2020-06-21T22:26:55Z,2020-06-21T22:26:55Z,OWNER,"This makes a lot of sense. I implemented the mechanism for the index page because I have my own instance of Datasette that was running slow, but it had a dozen database files attached to it. I've not run into this with a single giant database file but it absolutely makes sense that the same optimization would be necessary for the database page there too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642572841,
https://github.com/simonw/datasette/issues/860#issuecomment-647189535,https://api.github.com/repos/simonw/datasette/issues/860,647189535,MDEyOklzc3VlQ29tbWVudDY0NzE4OTUzNQ==,9599,2020-06-21T22:25:16Z,2020-06-21T22:25:27Z,OWNER,"This is also relevant to #639, and may mean I can close that ticket in place of this one. I'm going to get this at least to a proof-of-concept stage first though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642651572,
https://github.com/simonw/datasette/issues/357#issuecomment-647189045,https://api.github.com/repos/simonw/datasette/issues/357,647189045,MDEyOklzc3VlQ29tbWVudDY0NzE4OTA0NQ==,9599,2020-06-21T22:19:58Z,2020-06-21T22:19:58Z,OWNER,"I'm going to take this in a different direction.
I'm not happy with how `metadata.(json|yaml)` keeps growing new features. Rather than having a single plugin hook for all of `metadata.json` I'm going to split out the feature that shows actual real metadata for tables and databases - `source`, `license` etc - into its own plugin-powered mechanism.
So I'm going to close this ticket and spin up a new one for that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",348043884,
https://github.com/simonw/datasette/issues/687#issuecomment-646938984,https://api.github.com/repos/simonw/datasette/issues/687,646938984,MDEyOklzc3VlQ29tbWVudDY0NjkzODk4NA==,9599,2020-06-20T04:22:25Z,2020-06-20T04:23:02Z,OWNER,"I think I want the ""Plugin hooks"" page to be top-level, parallel to ""Plugins"" and ""Internals for Plugins"". It's the page of documentation refer to most often so I don't want to have to click down a hierarchy from the side navigation to find it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/687#issuecomment-646930455,https://api.github.com/repos/simonw/datasette/issues/687,646930455,MDEyOklzc3VlQ29tbWVudDY0NjkzMDQ1NQ==,9599,2020-06-20T03:22:21Z,2020-06-20T03:22:21Z,OWNER,The tutorial can start by showing how to use the new cookiecutter template from #642.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",572896293,
https://github.com/simonw/datasette/issues/855#issuecomment-646930365,https://api.github.com/repos/simonw/datasette/issues/855,646930365,MDEyOklzc3VlQ29tbWVudDY0NjkzMDM2NQ==,9599,2020-06-20T03:21:48Z,2020-06-20T03:21:48Z,OWNER,"Maybe I should also refactor the plugin documentation, as contemplated in #687.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642127307,
https://github.com/simonw/datasette/issues/642#issuecomment-646930160,https://api.github.com/repos/simonw/datasette/issues/642,646930160,MDEyOklzc3VlQ29tbWVudDY0NjkzMDE2MA==,9599,2020-06-20T03:20:25Z,2020-06-20T03:20:25Z,OWNER,Shipped this today! https://github.com/simonw/datasette-plugin is a cookiecutter template for creating new plugins.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",529429214,
https://github.com/simonw/datasette/issues/642#issuecomment-646930059,https://api.github.com/repos/simonw/datasette/issues/642,646930059,MDEyOklzc3VlQ29tbWVudDY0NjkzMDA1OQ==,9599,2020-06-20T03:19:57Z,2020-06-20T03:19:57Z,OWNER,"@psychemedia sorry I missed your comment before.
Niche Museums is definitely the best example of custom templates at the moment: https://github.com/simonw/museums/tree/master/templates
I want to comprehensively document the variables made available to custom templates before shipping Datasette 1.0 - just filed that as #857.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",529429214,
https://github.com/simonw/datasette/issues/855#issuecomment-646928638,https://api.github.com/repos/simonw/datasette/issues/855,646928638,MDEyOklzc3VlQ29tbWVudDY0NjkyODYzOA==,9599,2020-06-20T03:09:41Z,2020-06-20T03:09:41Z,OWNER,I've shipped the cookiecutter template and used it to build https://github.com/simonw/datasette-saved-queries - it's ready to add to the official documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642127307,
https://github.com/simonw/datasette/issues/852#issuecomment-646905073,https://api.github.com/repos/simonw/datasette/issues/852,646905073,MDEyOklzc3VlQ29tbWVudDY0NjkwNTA3Mw==,9599,2020-06-20T00:21:34Z,2020-06-20T00:22:28Z,OWNER,New repo: https://github.com/simonw/datasette-saved-queries - which I created using the new cookiecutter template at https://github.com/simonw/datasette-plugin,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/852#issuecomment-646760805,https://api.github.com/repos/simonw/datasette/issues/852,646760805,MDEyOklzc3VlQ29tbWVudDY0Njc2MDgwNQ==,9599,2020-06-19T17:07:45Z,2020-06-19T17:07:45Z,OWNER,"Plugin idea: `datasette-saved-queries` - it uses the `startup` hook to initialize a `saved_queries` table, then uses the `canned_queries` hook to add a writable canned query for saving records to that table.
Then it returns any queries from that table as additional canned queries.
Bonus idea: it could write the user's actor_id to a column if they are signed in, and provide a link to see ""just my saved queries"" in that case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/849#issuecomment-646686493,https://api.github.com/repos/simonw/datasette/issues/849,646686493,MDEyOklzc3VlQ29tbWVudDY0NjY4NjQ5Mw==,9599,2020-06-19T15:04:51Z,2020-06-19T15:04:51Z,OWNER,https://twitter.com/jaffathecake/status/1273983493006077952 concerns what happens to open pull requests - they will automatically close when you remove `master` unless you repoint them to `main` first.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639072811,
https://github.com/simonw/datasette/issues/852#issuecomment-646396772,https://api.github.com/repos/simonw/datasette/issues/852,646396772,MDEyOklzc3VlQ29tbWVudDY0NjM5Njc3Mg==,9599,2020-06-19T02:16:47Z,2020-06-19T02:16:47Z,OWNER,I'll close this once I've built a plugin against it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/852#issuecomment-646396690,https://api.github.com/repos/simonw/datasette/issues/852,646396690,MDEyOklzc3VlQ29tbWVudDY0NjM5NjY5MA==,9599,2020-06-19T02:16:24Z,2020-06-19T02:16:24Z,OWNER,Documentation: https://datasette.readthedocs.io/en/latest/plugins.html#canned-queries-datasette-database-actor,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/852#issuecomment-646396499,https://api.github.com/repos/simonw/datasette/issues/852,646396499,MDEyOklzc3VlQ29tbWVudDY0NjM5NjQ5OQ==,9599,2020-06-19T02:15:49Z,2020-06-19T02:15:58Z,OWNER,"Released an alpha preview in https://github.com/simonw/datasette/releases/tag/0.45a1
Wrote about this here: https://simonwillison.net/2020/Jun/19/datasette-alphas/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/852#issuecomment-646350530,https://api.github.com/repos/simonw/datasette/issues/852,646350530,MDEyOklzc3VlQ29tbWVudDY0NjM1MDUzMA==,9599,2020-06-18T23:13:57Z,2020-06-18T23:14:11Z,OWNER,"```python
@hookspec
def canned_queries(datasette, database, actor):
""Return a dictionary of canned query definitions or an awaitable function that returns them""
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/852#issuecomment-646329456,https://api.github.com/repos/simonw/datasette/issues/852,646329456,MDEyOklzc3VlQ29tbWVudDY0NjMyOTQ1Ng==,9599,2020-06-18T22:07:09Z,2020-06-18T22:07:37Z,OWNER,"It would be neat if the queries returned by this hook could be restricted to specific users. I think I can do that by returning an ""allow"" block as part of the query.
But... what if we allow users to save private queries and we might have thousands of users each with hundreds of saved queries?
For that case it would be good if the plugin hook could take an optional `actor` parameter.
This would also allow us to dynamically generate a canned query for ""return the bookmarks belonging to this actor"" or similar!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/807#issuecomment-646320237,https://api.github.com/repos/simonw/datasette/issues/807,646320237,MDEyOklzc3VlQ29tbWVudDY0NjMyMDIzNw==,9599,2020-06-18T21:41:16Z,2020-06-18T21:41:16Z,OWNER,"https://pypi.org/project/datasette/0.45a0/ is the release on PyPI.
And in a fresh virtual environment:
```
$ pip install datasette==0.45a0
...
$ datasette --version
datasette, version 0.45a0
```
But running `pip install datasette` still gets 0.44.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646319315,https://api.github.com/repos/simonw/datasette/issues/807,646319315,MDEyOklzc3VlQ29tbWVudDY0NjMxOTMxNQ==,9599,2020-06-18T21:38:56Z,2020-06-18T21:38:56Z,OWNER,"This worked!
https://pypi.org/project/datasette/#history
![]()
https://github.com/simonw/datasette/releases/tag/0.45a0 is my manually created GitHub prerelease.
https://datasette.readthedocs.io/en/latest/changelog.html#a0-2020-06-18 has the release notes.
A shame Read The Docs doesn't seem to build the docs for these releases -it's not showing the tag in the releases pane here:
![]()
Also the new tag isn't an option in the Build menu on https://readthedocs.org/projects/datasette/builds/
Not a big problem though since the ""latest"" tag on Read The Docs will still carry the in-development documentation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/835#issuecomment-646308467,https://api.github.com/repos/simonw/datasette/issues/835,646308467,MDEyOklzc3VlQ29tbWVudDY0NjMwODQ2Nw==,9599,2020-06-18T21:12:50Z,2020-06-18T21:12:50Z,OWNER,"Problem there is Login CSRF attacks: https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html#login-csrf - I still want to perform CSRF checks on login forms, even though the user may not yet have any cookies.
Maybe I can turn off CSRF checks for cookie-free requests but allow login forms to specifically opt back in to CSRF protection?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646307083,https://api.github.com/repos/simonw/datasette/issues/835,646307083,MDEyOklzc3VlQ29tbWVudDY0NjMwNzA4Mw==,9599,2020-06-18T21:09:35Z,2020-06-18T21:09:35Z,OWNER,So maybe one really easy fix here is to disable CSRF checks entirely for any request that doesn't have any cookies? Also suggested here: https://twitter.com/mrkurt/status/1273682965168603137,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/807#issuecomment-646303240,https://api.github.com/repos/simonw/datasette/issues/807,646303240,MDEyOklzc3VlQ29tbWVudDY0NjMwMzI0MA==,9599,2020-06-18T21:00:41Z,2020-06-18T21:00:41Z,OWNER,New documentation about the alpha/beta releases: https://datasette.readthedocs.io/en/latest/contributing.html#contributing-alpha-beta,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646302909,https://api.github.com/repos/simonw/datasette/issues/807,646302909,MDEyOklzc3VlQ29tbWVudDY0NjMwMjkwOQ==,9599,2020-06-18T21:00:02Z,2020-06-18T21:00:02Z,OWNER,Alpha release is running through Travis now: https://travis-ci.org/github/simonw/datasette/builds/699864168,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646293670,https://api.github.com/repos/simonw/datasette/issues/807,646293670,MDEyOklzc3VlQ29tbWVudDY0NjI5MzY3MA==,9599,2020-06-18T20:38:50Z,2020-06-18T20:38:50Z,OWNER,"https://pypi.org/project/datasette-render-images/#history worked:
![]()
I'm now confident enough that I'll make these changes and ship an alpha of Datasette itself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646293029,https://api.github.com/repos/simonw/datasette/issues/807,646293029,MDEyOklzc3VlQ29tbWVudDY0NjI5MzAyOQ==,9599,2020-06-18T20:37:28Z,2020-06-18T20:37:46Z,OWNER,"Here's the Read The Docs documentation on versioned releases: https://docs.readthedocs.io/en/stable/versions.html
It looks like they do the right thing:
> We in fact are parsing your tag names against the rules given by PEP 440. This spec allows “normal” version numbers like 1.4.2 as well as pre-releases. An alpha version or a release candidate are examples of pre-releases and they look like this: 2.0a1.
>
> We only consider non pre-releases for the stable version of your documentation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646292578,https://api.github.com/repos/simonw/datasette/issues/807,646292578,MDEyOklzc3VlQ29tbWVudDY0NjI5MjU3OA==,9599,2020-06-18T20:36:22Z,2020-06-18T20:36:22Z,OWNER,"https://travis-ci.com/github/simonw/datasette-render-images/builds/172118541 demonstrates that the alpha/beta conditional is working as intended:
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646291309,https://api.github.com/repos/simonw/datasette/issues/807,646291309,MDEyOklzc3VlQ29tbWVudDY0NjI5MTMwOQ==,9599,2020-06-18T20:33:31Z,2020-06-18T20:33:31Z,OWNER,"One more experiment: I'm going to ship `datasette-render-images` 0.2 and see if that works correctly - including printing out the new debug section I put in the Travis config here: https://github.com/simonw/datasette-render-images/blob/6b5f22dab75ca364f671f5597556d2665a251bd8/.travis.yml#L35-L39 - which should demonstrate if my conditional for pushing to Docker Hub will work or not.
In the alpha releasing run on Travis that echo statement did NOT execute: https://travis-ci.com/github/simonw/datasette-render-images/builds/172116625","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646290171,https://api.github.com/repos/simonw/datasette/issues/807,646290171,MDEyOklzc3VlQ29tbWVudDY0NjI5MDE3MQ==,9599,2020-06-18T20:30:48Z,2020-06-18T20:30:48Z,OWNER,"OK, I just shipped 0.2a0 of `datasette-render-images` - https://pypi.org/project/datasette-render-images/ has no indication of that:
![]()
But this page does: https://pypi.org/project/datasette-render-images/#history
![]()
And https://pypi.org/project/datasette-render-images/0.2a0/ exists.
In a fresh virtual environment `pip install datasette-render-images` gets 0.1.
`pip install datasette-render-images==0.2a0` gets 0.2a0.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/835#issuecomment-646288146,https://api.github.com/repos/simonw/datasette/issues/835,646288146,MDEyOklzc3VlQ29tbWVudDY0NjI4ODE0Ng==,9599,2020-06-18T20:26:22Z,2020-06-18T20:26:31Z,OWNER,"Useful tip from Carlton Gibson: https://twitter.com/carltongibson/status/1273680590672453632
> DRF makes ALL views CSRF exempt and then enforces CSRF if you're using Session auth only.
>
> View: https://github.com/encode/django-rest-framework/blob/e18e40d6ae42457f60ca9c68054ad40d15ba8433/rest_framework/views.py#L144
> Auth: https://github.com/encode/django-rest-framework/blob/e18e40d6ae42457f60ca9c68054ad40d15ba8433/rest_framework/authentication.py#L130","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/807#issuecomment-646280134,https://api.github.com/repos/simonw/datasette/issues/807,646280134,MDEyOklzc3VlQ29tbWVudDY0NjI4MDEzNA==,9599,2020-06-18T20:08:15Z,2020-06-18T20:08:15Z,OWNER,https://github.com/simonw/datasette-render-images uses Travis and is low-risk for trying this out.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646279428,https://api.github.com/repos/simonw/datasette/issues/807,646279428,MDEyOklzc3VlQ29tbWVudDY0NjI3OTQyOA==,9599,2020-06-18T20:06:43Z,2020-06-18T20:06:43Z,OWNER,I'm going to try this on a separate repository so I don't accidentally publish a Datasette release I didn't mean to publish!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646279280,https://api.github.com/repos/simonw/datasette/issues/807,646279280,MDEyOklzc3VlQ29tbWVudDY0NjI3OTI4MA==,9599,2020-06-18T20:06:24Z,2020-06-18T20:06:24Z,OWNER,"So maybe this condition is right?
if: (tag IS present) AND NOT (tag =~ [ab])","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646278801,https://api.github.com/repos/simonw/datasette/issues/807,646278801,MDEyOklzc3VlQ29tbWVudDY0NjI3ODgwMQ==,9599,2020-06-18T20:05:18Z,2020-06-18T20:05:18Z,OWNER,"Travis conditions documentation: https://docs.travis-ci.com/user/conditions-v1
These look useful:
```
branch =~ /^(one|two)-three$/
(tag =~ ^v) AND (branch = master)
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646277680,https://api.github.com/repos/simonw/datasette/issues/807,646277680,MDEyOklzc3VlQ29tbWVudDY0NjI3NzY4MA==,9599,2020-06-18T20:02:42Z,2020-06-18T20:02:42Z,OWNER,"So I think if I push a tag of `0.45a0` everything might just work - Travis will build it, push the build to PyPI, PyPI won't treat it as a stable release.
Except... I don't want to push alphas as Docker images - so I need to fix this code:
https://github.com/simonw/datasette/blob/6151c25a5a8d566c109af296244b9267c536bd9a/.travis.yml#L34-L43","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646277155,https://api.github.com/repos/simonw/datasette/issues/807,646277155,MDEyOklzc3VlQ29tbWVudDY0NjI3NzE1NQ==,9599,2020-06-18T20:01:31Z,2020-06-18T20:01:31Z,OWNER,"I thought I might have to update a regex (my CircleCI configs won't match on `a0`, [example](https://github.com/simonw/datasette-publish-now/blob/420f349b278857f62183d8e9835d64f116758be7/.circleci/config.yml#L22)) but it turns out Travis is currently configured to treat ALL tags as potential releases:
https://github.com/simonw/datasette/blob/6151c25a5a8d566c109af296244b9267c536bd9a/.travis.yml#L21-L35","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646276150,https://api.github.com/repos/simonw/datasette/issues/807,646276150,MDEyOklzc3VlQ29tbWVudDY0NjI3NjE1MA==,9599,2020-06-18T19:59:17Z,2020-06-18T19:59:17Z,OWNER,"Relevant PEP: https://www.python.org/dev/peps/pep-0440/
Django's implementation dates back 8 years: https://github.com/django/django/commit/40f0ecc56a23d35c2849f8e79276f6d8931412d1
From the PEP:
> Implicit pre-release number
>
> Pre releases allow omitting the numeral in which case it is implicitly assumed to be 0. The normal form for this is to include the 0 explicitly. This allows versions such as 1.2a which is normalized to 1.2a0.
I'm going to habitually include the 0.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/807#issuecomment-646273035,https://api.github.com/repos/simonw/datasette/issues/807,646273035,MDEyOklzc3VlQ29tbWVudDY0NjI3MzAzNQ==,9599,2020-06-18T19:52:28Z,2020-06-18T19:52:28Z,OWNER,"I'd like this soon, because I want to start experimenting with things like #852 and #842 without shipping those plugin hooks in a full stable release.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",632843030,
https://github.com/simonw/datasette/issues/842#issuecomment-646272627,https://api.github.com/repos/simonw/datasette/issues/842,646272627,MDEyOklzc3VlQ29tbWVudDY0NjI3MjYyNw==,9599,2020-06-18T19:51:32Z,2020-06-18T19:51:32Z,OWNER,I'd be OK with the first version of this not including a plugin hook.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/842#issuecomment-646264051,https://api.github.com/repos/simonw/datasette/issues/842,646264051,MDEyOklzc3VlQ29tbWVudDY0NjI2NDA1MQ==,9599,2020-06-18T19:32:13Z,2020-06-18T19:32:37Z,OWNER,"If every magic parameter has a prefix and suffix, like `_request_ip` and `_actor_id`, then plugins could register a function for a prefix. Register a function to `_actor` and `actor(""id"")`will be called for `_actor_id`.
But does it make sense for every magic parameter to be of form `_a_b`? I think so.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/842#issuecomment-646246062,https://api.github.com/repos/simonw/datasette/issues/842,646246062,MDEyOklzc3VlQ29tbWVudDY0NjI0NjA2Mg==,9599,2020-06-18T18:54:41Z,2020-06-18T18:54:41Z,OWNER,"The `_actor_id` param makes this a bit trickier, because we can't just say ""if you see an unknown parameter called X call this function"" - our magic parameter logic isn't adding single parameters, it might add a whole family of them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/842#issuecomment-646242172,https://api.github.com/repos/simonw/datasette/issues/842,646242172,MDEyOklzc3VlQ29tbWVudDY0NjI0MjE3Mg==,9599,2020-06-18T18:46:06Z,2020-06-18T18:53:31Z,OWNER,"Yes that can work - and using `__missing__` (new in Python 3) is nicer because then the regular dictionary gets checked first:
```python
import sqlite3
conn = sqlite3.connect("":memory:"")
class Magic(dict):
def __missing__(self, key):
return key.upper()
conn.execute(""select :name"", Magic()).fetchall()
```
Outputs:
```
[('NAME',)]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/842#issuecomment-646238702,https://api.github.com/repos/simonw/datasette/issues/842,646238702,MDEyOklzc3VlQ29tbWVudDY0NjIzODcwMg==,9599,2020-06-18T18:39:07Z,2020-06-18T18:39:07Z,OWNER,"It would be nice if Datasette didn't have to do any additional work to find e.g. `_request_ip` if that parameter turned out not to be used by the query.
Could I do this with a custom class that implements `__getitem__()` and then gets passed as SQLite arguments?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,
https://github.com/simonw/datasette/issues/820#issuecomment-646218809,https://api.github.com/repos/simonw/datasette/issues/820,646218809,MDEyOklzc3VlQ29tbWVudDY0NjIxODgwOQ==,9599,2020-06-18T17:58:02Z,2020-06-18T17:58:02Z,OWNER,I had the same idea again ten days later: #852.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",635049296,
https://github.com/simonw/datasette/issues/835#issuecomment-646217766,https://api.github.com/repos/simonw/datasette/issues/835,646217766,MDEyOklzc3VlQ29tbWVudDY0NjIxNzc2Ng==,9599,2020-06-18T17:55:54Z,2020-06-18T17:56:04Z,OWNER,Idea: a mechanism where the `asgi_csrf()` can take an optional `should_protect()` callback function which gets called with the `scope` and decides if the current request should be protected or not. It can then look at headers and paths and suchlike and make its own decisions. Datasette could then provide a `should_protect()` callback which can interact with plugins.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646216934,https://api.github.com/repos/simonw/datasette/issues/835,646216934,MDEyOklzc3VlQ29tbWVudDY0NjIxNjkzNA==,9599,2020-06-18T17:54:14Z,2020-06-18T17:54:14Z,OWNER,"> if you did Origin based CSRF checks, then could the absence of an Origin header be used?
https://twitter.com/cnorthwood/status/1273674392757829632","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646214158,https://api.github.com/repos/simonw/datasette/issues/835,646214158,MDEyOklzc3VlQ29tbWVudDY0NjIxNDE1OA==,9599,2020-06-18T17:48:45Z,2020-06-18T17:48:45Z,OWNER,"I wonder if it's safe to generically say ""Don't do CSRF protection on any request that includes a `Authorization: Bearer...` header - because it's not possible for a regular browser to send that header since the format is different from the header used in browser-based HTTP basic auth?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646209520,https://api.github.com/repos/simonw/datasette/issues/835,646209520,MDEyOklzc3VlQ29tbWVudDY0NjIwOTUyMA==,9599,2020-06-18T17:39:30Z,2020-06-18T17:40:53Z,OWNER,"`datasette-auth-tokens` could switch to using `asgi_wrapper` instead of `actor_from_request` - then it could add a `scope[""skip_csrf""] = True` scope property to indicate that CSRF should not be protected.
Since `asgi_wrapper` wraps the CSRF protection middleware changes made to the `scope` by an `asgi_wrapper` will be visible to the CSRF middleware:
https://github.com/simonw/datasette/blob/d2aef9f7ef30fa20b1450cd181cf803f44fb4e21/datasette/app.py#L877-L888","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646204308,https://api.github.com/repos/simonw/datasette/issues/835,646204308,MDEyOklzc3VlQ29tbWVudDY0NjIwNDMwOA==,9599,2020-06-18T17:32:41Z,2020-06-18T17:32:41Z,OWNER,The only way I can think of for a view to opt-out of CSRF protection is for them to be able to reconfigure the `asgi-csrf` middleware to skip specific URL patterns.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646175055,https://api.github.com/repos/simonw/datasette/issues/835,646175055,MDEyOklzc3VlQ29tbWVudDY0NjE3NTA1NQ==,9599,2020-06-18T17:00:45Z,2020-06-18T17:00:45Z,OWNER,Here's the Rails pattern for this: https://gist.github.com/maxivak/a25957942b6c21a41acd,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646172200,https://api.github.com/repos/simonw/datasette/issues/835,646172200,MDEyOklzc3VlQ29tbWVudDY0NjE3MjIwMA==,9599,2020-06-18T16:57:45Z,2020-06-18T16:57:45Z,OWNER,"I think there are a couple of steps to this one.
The nature of CSRF is that it's about hijacking existing authentication credentials. If your Datasette site runs without any authentication plugins at all CSRF protection isn't actually useful.
Some POST endpoints should be able to opt-out of CSRF protection entirely. A writable canned query that accepts anonymous poll submissions for example might determine that CSRF is not needed.
If a plugin adds `Authorization: Bearer xxx` token support that plugin should also be able to specify that CSRF protection can be skipped. https://github.com/simonw/datasette-auth-tokens could do this.
This means I need two new mechanisms:
- A way for wrapped views to indicate ""actually don't CSRF protect me"". I'm not sure how feasible this is without a major redesign, since the decision to return a 403 forbidden status is made before the wrapped function has even been called.
- A way for authentication plugins like `datasette-auth-tokens` to say ""CSRF protection is not needed for this request"". This is a bit tricky too, since right now the `actor_from_request` hook doesn't have a channel for information other than returning the actor dictionary.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/835#issuecomment-646151706,https://api.github.com/repos/simonw/datasette/issues/835,646151706,MDEyOklzc3VlQ29tbWVudDY0NjE1MTcwNg==,9599,2020-06-18T16:36:23Z,2020-06-18T16:36:23Z,OWNER,Tweeted about this here: https://twitter.com/simonw/status/1273655053170077701,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637363686,
https://github.com/simonw/datasette/issues/853#issuecomment-646140022,https://api.github.com/repos/simonw/datasette/issues/853,646140022,MDEyOklzc3VlQ29tbWVudDY0NjE0MDAyMg==,9599,2020-06-18T16:21:53Z,2020-06-18T16:21:53Z,OWNER,"I have a test that demonstrates this working, but also demonstrates that the CSRF protection from #798 makes this really tricky to work with. I'd like to improve that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640943441,
https://github.com/simonw/datasette/issues/852#issuecomment-645785830,https://api.github.com/repos/simonw/datasette/issues/852,645785830,MDEyOklzc3VlQ29tbWVudDY0NTc4NTgzMA==,9599,2020-06-18T05:37:00Z,2020-06-18T05:37:00Z,OWNER,"The easiest way to do this would be with a new plugin hook:
def canned_queries(datasette, database):
""""""Return a list of canned query definitions
or an awaitable function that returns them""
Another approach would be to make the whole of `metadata.json` customizable by plugins.
I think I like the dedicated `canned_queries` option better. I'm not happy with the way metadata keeps growing - see #493 - so adding a dedicated hook would be more future proof against other changes I might make to the metadata mechanism.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/852#issuecomment-645781482,https://api.github.com/repos/simonw/datasette/issues/852,645781482,MDEyOklzc3VlQ29tbWVudDY0NTc4MTQ4Mg==,9599,2020-06-18T05:24:55Z,2020-06-18T05:25:00Z,OWNER,Question about this on Twitter: https://twitter.com/amjithr/status/1273440766862352384,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",640917326,
https://github.com/simonw/datasette/issues/850#issuecomment-645068128,https://api.github.com/repos/simonw/datasette/issues/850,645068128,MDEyOklzc3VlQ29tbWVudDY0NTA2ODEyOA==,9599,2020-06-16T23:52:16Z,2020-06-16T23:52:16Z,OWNER,https://aws.amazon.com/blogs/compute/announcing-http-apis-for-amazon-api-gateway/ looks very important here: AWS HTTP APIs were introduced in December 2019 and appear to be a third of the price of API Gateway.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/236#issuecomment-645067611,https://api.github.com/repos/simonw/datasette/issues/236,645067611,MDEyOklzc3VlQ29tbWVudDY0NTA2NzYxMQ==,9599,2020-06-16T23:50:12Z,2020-06-16T23:50:59Z,OWNER,"As for your other questions:
> 1. I assume the goal is to have a CORS-friendly HTTPS endpoint that hosts the datasette service + user's db.
Yes, exactly. I know this will limit the size of database that can be deployed (since Lambda has a 50MB total package limit as far as I can tell) but there are plenty of interesting databases that are small enough to fit there.
The new EFS support for Lambda means that theoretically the size of database is now unlimited, which is really interesting. That's what got me inspired to take a look at a proof of concept in #850.
> 2. If that's the goal, I think Lambda alone is insufficient. Lambda provides the compute fabric, but not the HTTP routing. You'd also need to add Application Load Balancer or API Gateway to provide an HTTP endpoint that routes to the lambda function.
>
> Do you have a preference between ALB or API GW? ALB has better economics at scale, but has a minimum monthly cost. API GW has worse per-request economics, but scales to zero when no requests are happening.
I personally like scale-to-zero because many of my projects are likely to receive very little traffic. So API GW first, and maybe ALB as an option later on for people operating at scale?
> 3. Does Datasette have any native components, or is it all pure python? If it has native bits, they'll likely need to be recompiled to work on Amazon Linux 2.
As you've found, the only native component is uvloop which is only needed if uvicorn is being used to serve requests.
> 4. There are a few disparate services that need to be wired together to expose a Python service securely to the web. If I was doing this outside of the datasette publish system, I'd use an AWS CloudFormation template. Even within datasette, I think it still makes sense to use a CloudFormation template and just have the publish plugin invoke it (via the standard `aws` cli) with user-specified parameters. Does that sound reasonable to you?
For the eventual ""datasette publish lambda"" command I want whatever results in the smallest amount of inconvenience for users. I've been trying out Amazon SAM in #850 and it requires users to run Docker on their machines, which is a pretty huge barrier to entry! I don't have much experience with CloudFormation but it's probably a better bet, especially if you can ""pip install"" the dependencies needed to deploy with it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317001500,
https://github.com/simonw/datasette/issues/236#issuecomment-645066486,https://api.github.com/repos/simonw/datasette/issues/236,645066486,MDEyOklzc3VlQ29tbWVudDY0NTA2NjQ4Ng==,9599,2020-06-16T23:45:45Z,2020-06-16T23:45:45Z,OWNER,"Hi Colin,
Sorry I didn't see this sooner! I've just started digging into this myself, to try and play with the new EFS Lambda support: #850.
Yes, uvloop is only needed because of uvicorn. I have a branch here that removes that dependency just for trying out Lambda: https://github.com/simonw/datasette/tree/no-uvicorn - so you can run `pip install https://github.com/simonw/datasette/archive/no-uvicorn.zip` to get that.
I'm going to try out your `datasette-lambda` project next - really excited to see how far you've got with it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",317001500,
https://github.com/simonw/datasette/issues/850#issuecomment-645064332,https://api.github.com/repos/simonw/datasette/issues/850,645064332,MDEyOklzc3VlQ29tbWVudDY0NTA2NDMzMg==,9599,2020-06-16T23:37:34Z,2020-06-16T23:37:34Z,OWNER,Just realized Colin Dellow reported an issue with Datasette and Mangum back in April - #719 - and has in fact been working on https://github.com/code402/datasette-lambda for a while!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645063386,https://api.github.com/repos/simonw/datasette/issues/850,645063386,MDEyOklzc3VlQ29tbWVudDY0NTA2MzM4Ng==,9599,2020-06-16T23:34:07Z,2020-06-16T23:34:07Z,OWNER,"Tried `sam local invoke`:
```
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam local invoke
Invoking app.lambda_handler (python3.8)
Fetching lambci/lambda:python3.8 Docker container image......
Mounting /private/tmp/datasette-proof-of-concept/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
START RequestId: 7c04480b-5d42-168e-dec0-4e8bf34fa596 Version: $LATEST
[INFO] 2020-06-16T23:33:27.24Z 7c04480b-5d42-168e-dec0-4e8bf34fa596 Waiting for application startup.
[INFO] 2020-06-16T23:33:27.24Z 7c04480b-5d42-168e-dec0-4e8bf34fa596 LifespanCycleState.STARTUP: 'lifespan.startup.complete' event received from application.
[INFO] 2020-06-16T23:33:27.24Z 7c04480b-5d42-168e-dec0-4e8bf34fa596 Application startup complete.
[INFO] 2020-06-16T23:33:27.24Z 7c04480b-5d42-168e-dec0-4e8bf34fa596 Waiting for application shutdown.
[INFO] 2020-06-16T23:33:27.24Z 7c04480b-5d42-168e-dec0-4e8bf34fa596 LifespanCycleState.SHUTDOWN: 'lifespan.shutdown.complete' event received from application.
[ERROR] KeyError: 'requestContext'
Traceback (most recent call last):
File ""/var/task/mangum/adapter.py"", line 110, in __call__
return self.handler(event, context)
File ""/var/task/mangum/adapter.py"", line 130, in handler
if ""eventType"" in event[""requestContext""]:
END RequestId: 7c04480b-5d42-168e-dec0-4e8bf34fa596
REPORT RequestId: 7c04480b-5d42-168e-dec0-4e8bf34fa596 Init Duration: 1120.76 ms Duration: 7.08 ms Billed Duration: 100 ms Memory Size: 128 MBMax Memory Used: 47 MB
{""errorType"":""KeyError"",""errorMessage"":""'requestContext'"",""stackTrace"":["" File \""/var/task/mangum/adapter.py\"", line 110, in __call__\n return self.handler(event, context)\n"","" File \""/var/task/mangum/adapter.py\"", line 130, in handler\n if \""eventType\"" in event[\""requestContext\""]:\n""]}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645062266,https://api.github.com/repos/simonw/datasette/issues/850,645062266,MDEyOklzc3VlQ29tbWVudDY0NTA2MjI2Ng==,9599,2020-06-16T23:30:12Z,2020-06-16T23:33:12Z,OWNER,"OK, changed `requirements.txt` to this:
```
https://github.com/simonw/datasette/archive/no-uvicorn.zip
mangum
```
No `sam build --use-container` runs without errors. Ran `sam deploy` too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645063058,https://api.github.com/repos/simonw/datasette/issues/850,645063058,MDEyOklzc3VlQ29tbWVudDY0NTA2MzA1OA==,9599,2020-06-16T23:32:57Z,2020-06-16T23:32:57Z,OWNER,https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/ is now giving me a 500 internal server error.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645061088,https://api.github.com/repos/simonw/datasette/issues/850,645061088,MDEyOklzc3VlQ29tbWVudDY0NTA2MTA4OA==,9599,2020-06-16T23:25:41Z,2020-06-16T23:25:41Z,OWNER,"Someone else ran into this problem: https://github.com/iwpnd/fastapi-aws-lambda-example/issues/1
So I need to be able to pip install MOST of Datasette, but skip `uvicorn`. Tricky. I'll try installing a custom fork?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645060598,https://api.github.com/repos/simonw/datasette/issues/850,645060598,MDEyOklzc3VlQ29tbWVudDY0NTA2MDU5OA==,9599,2020-06-16T23:24:01Z,2020-06-16T23:24:01Z,OWNER,"I changed `requirements.txt` to this:
```
datasette
mangum
```
And `app.py` to this:
```python
from datasette.app import Datasette
from mangum import Mangum
datasette = Datasette([], memory=True)
lambda_handler = Mangum(datasette.app())
```
But then when I ran `sam build --use-container` I got this:
```
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam build --use-container
Starting Build inside a container
Building function 'HelloWorldFunction'
Fetching lambci/lambda:build-python3.8 Docker container image......
Mounting /private/tmp/datasette-proof-of-concept/hello_world as /tmp/samcli/source:ro,delegated inside runtime container
Build Failed
Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - {uvloop==0.14.0(wheel)}
```
`uvloop` isn't actually necessary for this project, since it's used by `uvicorn` which isn't needed if Lambda is serving ASGI traffic directly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645059663,https://api.github.com/repos/simonw/datasette/issues/850,645059663,MDEyOklzc3VlQ29tbWVudDY0NTA1OTY2Mw==,9599,2020-06-16T23:20:46Z,2020-06-16T23:20:46Z,OWNER,"I added an exclamation mark to hello world and ran `sam deploy` again. https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/ still shows the old message.
Running `sam build --use-container` first and then `sam deploy` did the right thing.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645058947,https://api.github.com/repos/simonw/datasette/issues/850,645058947,MDEyOklzc3VlQ29tbWVudDY0NTA1ODk0Nw==,9599,2020-06-16T23:18:18Z,2020-06-16T23:18:18Z,OWNER,"https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/
That's a pretty ugly URL. I'm not sure how to get rid of the `/Prod/` prefix on it. Might have to use the `base_url` setting to get something working: https://datasette.readthedocs.io/en/stable/config.html#base-url ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645058617,https://api.github.com/repos/simonw/datasette/issues/850,645058617,MDEyOklzc3VlQ29tbWVudDY0NTA1ODYxNw==,9599,2020-06-16T23:17:09Z,2020-06-16T23:17:09Z,OWNER,"OK, `sam deploy --guided` now works!
```
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam deploy --guided
Configuring SAM deploy
======================
Looking for samconfig.toml : Not found
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-app]: datasette-proof-of-concept
AWS Region [us-east-1]:
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]:
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]:
HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
Save arguments to samconfig.toml [Y/n]:
Looking for resources needed for deployment: Not found.
Creating the required resources...
Successfully created!
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-1ksajo4h62s07
A different default S3 bucket can be set in samconfig.toml
Saved arguments to config file
Running 'sam deploy' for future deployments will use the parameters saved above.
The above parameters can be changed by modifying samconfig.toml
Learn more about samconfig.toml syntax at
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
Deploying with following values
===============================
Stack name : datasette-proof-of-concept
Region : us-east-1
Confirm changeset : False
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-1ksajo4h62s07
Capabilities : [""CAPABILITY_IAM""]
Parameter overrides : {}
Initiating deployment
=====================
Uploading to datasette-proof-of-concept/0c208b5656a7aeb6186d49bebc595237 535344 / 535344.0 (100.00%)
HelloWorldFunction may not have authorization defined.
Uploading to datasette-proof-of-concept/14bd9ce3e21f9c88634d13c0c9b377e4.template 1147 / 1147.0 (100.00%)
Waiting for changeset to be created..
CloudFormation stack changeset
---------------------------------------------------------------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType
---------------------------------------------------------------------------------------------------------------------------------------------------------
+ Add HelloWorldFunctionHelloWorldPermissionProd AWS::Lambda::Permission
+ Add HelloWorldFunctionRole AWS::IAM::Role
+ Add HelloWorldFunction AWS::Lambda::Function
+ Add ServerlessRestApiDeployment47fc2d5f9d AWS::ApiGateway::Deployment
+ Add ServerlessRestApiProdStage AWS::ApiGateway::Stage
+ Add ServerlessRestApi AWS::ApiGateway::RestApi
---------------------------------------------------------------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:us-east-1:462092780466:changeSet/samcli-deploy1592349262/d685f2de-87c1-4b8e-b13a-67b94f8fc928
2020-06-16 16:14:29 - Waiting for stack create/update to complete
CloudFormation events from changeset
---------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
---------------------------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole -
CREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole Resource creation Initiated
CREATE_COMPLETE AWS::IAM::Role HelloWorldFunctionRole -
CREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction Resource creation Initiated
CREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction -
CREATE_COMPLETE AWS::Lambda::Function HelloWorldFunction -
CREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi Resource creation Initiated
CREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi -
CREATE_COMPLETE AWS::ApiGateway::RestApi ServerlessRestApi -
CREATE_IN_PROGRESS AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissi -
onProd
CREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc2d5f9 -
d
CREATE_COMPLETE AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc2d5f9 -
d
CREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc2d5f9 Resource creation Initiated
d
CREATE_IN_PROGRESS AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissi Resource creation Initiated
onProd
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage -
CREATE_COMPLETE AWS::ApiGateway::Stage ServerlessRestApiProdStage -
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage Resource creation Initiated
CREATE_COMPLETE AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissi -
onProd
CREATE_COMPLETE AWS::CloudFormation::Stack datasette-proof-of-concept -
---------------------------------------------------------------------------------------------------------------------------------------------------------
CloudFormation outputs from deployed stack
---------------------------------------------------------------------------------------------------------------------------------------------------------
Outputs
---------------------------------------------------------------------------------------------------------------------------------------------------------
Key HelloWorldFunctionIamRole
Description Implicit IAM Role created for Hello World function
Value arn:aws:iam::462092780466:role/datasette-proof-of-concept-HelloWorldFunctionRole-8MIDNIV5ECA6
Key HelloWorldApi
Description API Gateway endpoint URL for Prod stage for Hello World function
Value https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/
Key HelloWorldFunction
Description Hello World Lambda Function ARN
Value arn:aws:lambda:us-east-1:462092780466:function:datasette-proof-of-concept-HelloWorldFunction-QTF78ZEUDCB
---------------------------------------------------------------------------------------------------------------------------------------------------------
Successfully created/updated stack - datasette-proof-of-concept in us-east-1
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645056636,https://api.github.com/repos/simonw/datasette/issues/850,645056636,MDEyOklzc3VlQ29tbWVudDY0NTA1NjYzNg==,9599,2020-06-16T23:10:22Z,2020-06-16T23:10:22Z,OWNER,"Clicking that button generated me an access key ID / access key secret pair. Dropping those into `~/.aws/credentials` using this format:
```
[default]
aws_access_key_id = your_access_key_id
aws_secret_access_key = your_secret_access_key
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645055200,https://api.github.com/repos/simonw/datasette/issues/850,645055200,MDEyOklzc3VlQ29tbWVudDY0NTA1NTIwMA==,9599,2020-06-16T23:05:48Z,2020-06-16T23:05:48Z,OWNER,"Logged in as `simon-administrator` I'm using https://console.aws.amazon.com/iam/home?region=us-east-2#/security_credentials to create credentials:
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645054206,https://api.github.com/repos/simonw/datasette/issues/850,645054206,MDEyOklzc3VlQ29tbWVudDY0NTA1NDIwNg==,9599,2020-06-16T23:02:54Z,2020-06-16T23:04:59Z,OWNER,"I think I need to sign in to the AWS console with this new `simon-administrator` account and create IAM credentials for it.
... for which I needed my root ""account ID"" - a 12 digit number - to use on the IAM login form.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645053923,https://api.github.com/repos/simonw/datasette/issues/850,645053923,MDEyOklzc3VlQ29tbWVudDY0NTA1MzkyMw==,9599,2020-06-16T23:01:49Z,2020-06-16T23:01:49Z,OWNER,"I used https://console.aws.amazon.com/billing/home?#/account and activated ""IAM user/role access to billing information"" - what a puzzling first step!
I created a new user with AWS console access (which means access to the web UI) called `simon-administrator` and set a password. I created an `Administrators` group with `AdministratorAccess`.
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645051972,https://api.github.com/repos/simonw/datasette/issues/850,645051972,MDEyOklzc3VlQ29tbWVudDY0NTA1MTk3Mg==,9599,2020-06-16T22:55:04Z,2020-06-16T22:55:04Z,OWNER,"```
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam deploy --guided
Configuring SAM deploy
======================
Looking for samconfig.toml : Not found
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-app]: datasette-proof-of-concept
AWS Region [us-east-1]:
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: y
HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
Save arguments to samconfig.toml [Y/n]: y
Error: Failed to create managed resources: Unable to locate credentials
```
I need to get my AWS credentials sorted. I'm going to follow https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html and https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-set-up-credentials.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645051370,https://api.github.com/repos/simonw/datasette/issues/850,645051370,MDEyOklzc3VlQ29tbWVudDY0NTA1MTM3MA==,9599,2020-06-16T22:53:05Z,2020-06-16T22:53:05Z,OWNER,"```
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam local invoke
Invoking app.lambda_handler (python3.8)
Fetching lambci/lambda:python3.8 Docker container image....................................................................................................................................................................................................................................
Mounting /private/tmp/datasette-proof-of-concept/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
START RequestId: 4616ab43-6882-1627-e5e3-5a29730d52f9 Version: $LATEST
END RequestId: 4616ab43-6882-1627-e5e3-5a29730d52f9
REPORT RequestId: 4616ab43-6882-1627-e5e3-5a29730d52f9 Init Duration: 140.84 ms Duration: 2.49 ms Billed Duration: 100 ms Memory Size: 128 MBMax Memory Used: 25 MB
{""statusCode"":200,""body"":""{\""message\"": \""hello world\""}""}
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam local invoke
Invoking app.lambda_handler (python3.8)
Fetching lambci/lambda:python3.8 Docker container image......
Mounting /private/tmp/datasette-proof-of-concept/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
START RequestId: 3189df2f-e9c0-1be4-b9ac-f329c5fcd067 Version: $LATEST
END RequestId: 3189df2f-e9c0-1be4-b9ac-f329c5fcd067
REPORT RequestId: 3189df2f-e9c0-1be4-b9ac-f329c5fcd067 Init Duration: 87.22 ms Duration: 2.34 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 25 MB
{""statusCode"":200,""body"":""{\""message\"": \""hello world\""}""}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,
https://github.com/simonw/datasette/issues/850#issuecomment-645050948,https://api.github.com/repos/simonw/datasette/issues/850,645050948,MDEyOklzc3VlQ29tbWVudDY0NTA1MDk0OA==,9599,2020-06-16T22:51:30Z,2020-06-16T22:52:30Z,OWNER,"```
simon@Simons-MacBook-Pro datasette-proof-of-concept % sam build --use-container
Starting Build inside a container
Building function 'HelloWorldFunction'
Fetching lambci/lambda:build-python3.8 Docker container image..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Mounting /private/tmp/datasette-proof-of-concept/hello_world as /tmp/samcli/source:ro,delegated inside runtime container
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639993467,