html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/842#issuecomment-650684635,https://api.github.com/repos/simonw/datasette/issues/842,650684635,MDEyOklzc3VlQ29tbWVudDY1MDY4NDYzNQ==,9599,simonw,2020-06-28T03:30:31Z,2020-06-28T03:30:31Z,OWNER,Live demo: https://latest.datasette.io/fixtures/magic_parameters,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650679100,https://api.github.com/repos/simonw/datasette/issues/842,650679100,MDEyOklzc3VlQ29tbWVudDY1MDY3OTEwMA==,9599,simonw,2020-06-28T03:00:44Z,2020-06-28T03:00:44Z,OWNER,I'm going to add some canned queries to the `metadata.json` used by the live demo that illustrate this feature.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650678951,https://api.github.com/repos/simonw/datasette/issues/842,650678951,MDEyOklzc3VlQ29tbWVudDY1MDY3ODk1MQ==,9599,simonw,2020-06-28T02:59:52Z,2020-06-28T02:59:52Z,OWNER,"Documentation: https://datasette.readthedocs.io/en/latest/sql_queries.html#magic-parameters
Plugin hook documentation: https://datasette.readthedocs.io/en/latest/plugin_hooks.html#plugin-hook-register-magic-parameters","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650648434,https://api.github.com/repos/simonw/datasette/issues/842,650648434,MDEyOklzc3VlQ29tbWVudDY1MDY0ODQzNA==,9599,simonw,2020-06-27T23:27:35Z,2020-06-27T23:37:38Z,OWNER,I'm going to rename `_request_X` to `_header_X` as that better reflects what it now does.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650593122,https://api.github.com/repos/simonw/datasette/issues/842,650593122,MDEyOklzc3VlQ29tbWVudDY1MDU5MzEyMg==,9599,simonw,2020-06-27T18:03:02Z,2020-06-27T18:03:10Z,OWNER,"> Security thought: make sure it's not possible to accidentally open up a security hole where an attacker can send a GET request that causes the magic parameter `_cookie_ds_actor` to be resolved and returned as JSON data that the attacker can see.
This is an open security hole in https://github.com/simonw/datasette/commit/94c1315f0030fd58ce46a9294052c5c9d9d181c7 - it's useful for testing, but I need to remove it before I land that branch.
https://github.com/simonw/datasette/blob/94c1315f0030fd58ce46a9294052c5c9d9d181c7/datasette/views/database.py#L231-L237
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650458857,https://api.github.com/repos/simonw/datasette/issues/842,650458857,MDEyOklzc3VlQ29tbWVudDY1MDQ1ODg1Nw==,9599,simonw,2020-06-27T00:11:04Z,2020-06-27T00:11:04Z,OWNER,Security thought: make sure it's not possible to accidentally open up a security hole where an attacker can send a GET request that causes the magic parameter `_cookie_ds_actor` to be resolved and returned as JSON data that the attacker can see.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650455793,https://api.github.com/repos/simonw/datasette/issues/842,650455793,MDEyOklzc3VlQ29tbWVudDY1MDQ1NTc5Mw==,9599,simonw,2020-06-26T23:57:30Z,2020-06-27T00:00:16Z,OWNER,"Maybe I should ship a default `_scope_headers_...` parameter instead, which reads from a dictionary of `scope[""headers""]` - https://asgi-scope.now.sh/ shows what those look like.
```
{'client': ('148.64.98.14', 0),
'headers': [[b'host', b'asgi-scope.now.sh'],
[b'x-forwarded-for', b'148.64.98.14'],
[b'x-vercel-id', b'sw72x-1593215573008-024e4e603806'],
[b'x-forwarded-host', b'asgi-scope.now.sh'],
[b'accept',
b'text/html,application/xhtml+xml,application/xml;q=0.9,image/'
b'webp,*/*;q=0.8'],
[b'x-real-ip', b'148.64.98.14'],
[b'x-vercel-deployment-url', b'asgi-scope-9eyeojbek.now.sh'],
[b'upgrade-insecure-requests', b'1'],
[b'x-vercel-trace', b'sfo1'],
[b'x-forwarded-proto', b'https'],
[b'accept-language', b'en-US,en;q=0.5'],
[b'user-agent',
b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko'
b'/20100101 Firefox/77.0'],
[b'x-vercel-forwarded-for', b'148.64.98.14'],
[b'accept-encoding', b'gzip, deflate, br'],
[b'dnt', b'1'],
[b'te', b'trailers']],
'http_version': '1.1',
'method': 'GET',
'path': '/',
'query_string': b'',
'raw_path': b'/',
'root_path': '',
'scheme': 'https',
'server': ('asgi-scope.now.sh', 80),
'type': 'http'}
```
I'm going to have `_request_X` actually mean ""find the first value for X in `scope[""headers""`]"" - with underscores converted to hyphens.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-650455353,https://api.github.com/repos/simonw/datasette/issues/842,650455353,MDEyOklzc3VlQ29tbWVudDY1MDQ1NTM1Mw==,9599,simonw,2020-06-26T23:55:40Z,2020-06-26T23:55:40Z,OWNER,"`_request_ip` is actually quite hard to implement - should it take into account things like the `x-forwarded-for` header?
It probably should - but that means it now needs a bunch of extra configuration to tell it which of those headers can be trusted in the current environment.
As such I think I'll leave that for a plugin.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-649014757,https://api.github.com/repos/simonw/datasette/issues/842,649014757,MDEyOklzc3VlQ29tbWVudDY0OTAxNDc1Nw==,9599,simonw,2020-06-24T19:15:46Z,2020-06-24T19:31:52Z,OWNER,I'm building this documentation-first - here's the documentation so far: https://github.com/simonw/datasette/blob/6fc8bd9c473f4a25e0a076f24c7e5a9b2f353bb8/docs/sql_queries.rst#magic-parameters,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646271834,https://api.github.com/repos/simonw/datasette/issues/842,646271834,MDEyOklzc3VlQ29tbWVudDY0NjI3MTgzNA==,9599,simonw,2020-06-18T19:49:41Z,2020-06-24T18:49:22Z,OWNER,"But then what kind of magic parameters might plugins want to add?
Here's a crazy idea: `_scrapedcontent_url` - it would look for the `url` column on the data being inserted, scrape the content from it and insert that. This does suggest that the magic resolving function `scrapedcontent()` would need to optionally be sent the full row dictionary being inserted too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646270702,https://api.github.com/repos/simonw/datasette/issues/842,646270702,MDEyOklzc3VlQ29tbWVudDY0NjI3MDcwMg==,9599,simonw,2020-06-18T19:47:19Z,2020-06-24T18:48:48Z,OWNER,"Brainstorming more potential magic parameters:
* `_actor_id`
* `_actor_name`
* `_request_ip`
* `_request_user_agent`
* `_cookie_cookiename`
* `_signedcookie_cookiename` - reading signed cookies would be cool, not sure how to specify namespace though, maybe always use the same one? Or have the namespace come last, `_signedcookie_cookiename_mynamespace`. Might not need special signed cookie support since `actor` is already usually from a signed cookie.
* `_timestamp_unix` (not happy with these names yet)
* `_timestamp_localtime`
* `_timestamp_datetime`
* `_timestamp_utc`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-649000075,https://api.github.com/repos/simonw/datasette/issues/842,649000075,MDEyOklzc3VlQ29tbWVudDY0OTAwMDA3NQ==,9599,simonw,2020-06-24T18:46:36Z,2020-06-24T18:47:37Z,OWNER,"Another magic parameter that would be useful would be `_random`. Consider https://github.com/simonw/datasette-auth-tokens/issues/1 for example - I'd like to be able to provide a writable canned query which can create new authentication tokens in the database, but ideally it would automatically populate a secure random secret for each one.
Maybe `_random_chars_128` to create a 128 character long random string (using `os.urandom(64).hex()`).
This would be the first example of a magic parameter where part of the parameter name is used to configure the resulting value. Maybe neater to separate that with a different character? Unfortunately `_random_chars:128` wouldn't work because these parameters are used in a SQLite query where `:` has special meaning: `insert into blah (secret) values (:_random_chars:128)` wouldn't make sense.
Actually this is already supported by the proposed design - `_random_chars_128` would become `random(""chars_128"")` so the `random()` function could split off the 128 itself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646272627,https://api.github.com/repos/simonw/datasette/issues/842,646272627,MDEyOklzc3VlQ29tbWVudDY0NjI3MjYyNw==,9599,simonw,2020-06-18T19:51:32Z,2020-06-18T19:51:32Z,OWNER,I'd be OK with the first version of this not including a plugin hook.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646264051,https://api.github.com/repos/simonw/datasette/issues/842,646264051,MDEyOklzc3VlQ29tbWVudDY0NjI2NDA1MQ==,9599,simonw,2020-06-18T19:32:13Z,2020-06-18T19:32:37Z,OWNER,"If every magic parameter has a prefix and suffix, like `_request_ip` and `_actor_id`, then plugins could register a function for a prefix. Register a function to `_actor` and `actor(""id"")`will be called for `_actor_id`.
But does it make sense for every magic parameter to be of form `_a_b`? I think so.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646246062,https://api.github.com/repos/simonw/datasette/issues/842,646246062,MDEyOklzc3VlQ29tbWVudDY0NjI0NjA2Mg==,9599,simonw,2020-06-18T18:54:41Z,2020-06-18T18:54:41Z,OWNER,"The `_actor_id` param makes this a bit trickier, because we can't just say ""if you see an unknown parameter called X call this function"" - our magic parameter logic isn't adding single parameters, it might add a whole family of them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646242172,https://api.github.com/repos/simonw/datasette/issues/842,646242172,MDEyOklzc3VlQ29tbWVudDY0NjI0MjE3Mg==,9599,simonw,2020-06-18T18:46:06Z,2020-06-18T18:53:31Z,OWNER,"Yes that can work - and using `__missing__` (new in Python 3) is nicer because then the regular dictionary gets checked first:
```python
import sqlite3
conn = sqlite3.connect("":memory:"")
class Magic(dict):
def __missing__(self, key):
return key.upper()
conn.execute(""select :name"", Magic()).fetchall()
```
Outputs:
```
[('NAME',)]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-646238702,https://api.github.com/repos/simonw/datasette/issues/842,646238702,MDEyOklzc3VlQ29tbWVudDY0NjIzODcwMg==,9599,simonw,2020-06-18T18:39:07Z,2020-06-18T18:39:07Z,OWNER,"It would be nice if Datasette didn't have to do any additional work to find e.g. `_request_ip` if that parameter turned out not to be used by the query.
Could I do this with a custom class that implements `__getitem__()` and then gets passed as SQLite arguments?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,
https://github.com/simonw/datasette/issues/842#issuecomment-643663005,https://api.github.com/repos/simonw/datasette/issues/842,643663005,MDEyOklzc3VlQ29tbWVudDY0MzY2MzAwNQ==,9599,simonw,2020-06-13T18:51:57Z,2020-06-13T18:51:57Z,OWNER,"Two potential designs:
- `_actor_id`, `_request_ip`, `_now_timestamp` - so special reserved parameters
- a SQL function: `update blah set up = special('ip')`
I fee the first would be easier to implement.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638212085,Magic parameters for canned queries,