issue_comments
8,358 rows where author_association = "OWNER" sorted by updated_at descending
This data as json, CSV (advanced)
user 1
- simonw 8,158
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1691767797 | https://github.com/simonw/datasette/pull/2152#issuecomment-1691767797 | https://api.github.com/repos/simonw/datasette/issues/2152 | IC_kwDOBm6k_c5k1lP1 | simonw 9599 | 2023-08-24T14:15:10Z | 2023-08-24T14:15:10Z | OWNER | This is broken because Sphinx no longer supports Python 3.8. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump the python-packages group with 3 updates 1865174661 | |
1691763427 | https://github.com/simonw/datasette/issues/2153#issuecomment-1691763427 | https://api.github.com/repos/simonw/datasette/issues/2153 | IC_kwDOBm6k_c5k1kLj | simonw 9599 | 2023-08-24T14:12:43Z | 2023-08-24T14:12:43Z | OWNER | Annoying that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette --get --actor option 1865232341 | |
1691761685 | https://github.com/simonw/datasette/issues/2153#issuecomment-1691761685 | https://api.github.com/repos/simonw/datasette/issues/2153 | IC_kwDOBm6k_c5k1jwV | simonw 9599 | 2023-08-24T14:11:41Z | 2023-08-24T14:11:41Z | OWNER | Another option: implement this as a plugin, providing a new command like Or implement |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette --get --actor option 1865232341 | |
1691758168 | https://github.com/simonw/datasette/issues/2102#issuecomment-1691758168 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5k1i5Y | simonw 9599 | 2023-08-24T14:09:45Z | 2023-08-24T14:09:45Z | OWNER | I'm going to implement this in a branch to make it easier to test out. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1691753489 | https://github.com/simonw/datasette/issues/2153#issuecomment-1691753489 | https://api.github.com/repos/simonw/datasette/issues/2153 | IC_kwDOBm6k_c5k1hwR | simonw 9599 | 2023-08-24T14:07:25Z | 2023-08-24T14:09:16Z | OWNER | Building that
I feel like the names would have to have a common prefix though. Maybe something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette --get --actor option 1865232341 | |
1691045051 | https://github.com/simonw/datasette/issues/2102#issuecomment-1691045051 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5ky0y7 | simonw 9599 | 2023-08-24T05:51:59Z | 2023-08-24T05:51:59Z | OWNER | With that fix in place, this works:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1691044283 | https://github.com/simonw/datasette/issues/2102#issuecomment-1691044283 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5ky0m7 | simonw 9599 | 2023-08-24T05:51:02Z | 2023-08-24T05:51:02Z | OWNER | Also need to confirm that permissions like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1691043475 | https://github.com/simonw/datasette/issues/2102#issuecomment-1691043475 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5ky0aT | simonw 9599 | 2023-08-24T05:50:04Z | 2023-08-24T05:50:04Z | OWNER | On first test this seems to work! ```diff diff --git a/datasette/default_permissions.py b/datasette/default_permissions.py index 63a66c3c..9303dac8 100644 --- a/datasette/default_permissions.py +++ b/datasette/default_permissions.py @@ -187,6 +187,30 @@ def permission_allowed_actor_restrictions(datasette, actor, action, resource): return None _r = actor.get("_r")
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1691037971 | https://github.com/simonw/datasette/issues/2102#issuecomment-1691037971 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5kyzET | simonw 9599 | 2023-08-24T05:42:47Z | 2023-08-24T05:42:47Z | OWNER | I applied a fun trick to help test this out:
With that in place I can try this, with a token that has view-instance and view-database and view-table:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1691036559 | https://github.com/simonw/datasette/issues/2102#issuecomment-1691036559 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5kyyuP | simonw 9599 | 2023-08-24T05:40:53Z | 2023-08-24T05:40:53Z | OWNER | There might be an easier way to solve this. Here's some permission checks that run when hitting ``` permission_allowed: action=view-table, resource=('fixtures', 'facetable'), actor={'_r': {'a': ['vi'], 'd': {'fixtures': ['vd']}, 'r': {'fixtures': {'facetable': ['vt']}}}, 'a': 'user'} File "/datasette/views/table.py", line 727, in table_view_traced view_data = await table_view_data( File "/datasette/views/table.py", line 875, in table_view_data visible, private = await datasette.check_visibility( File "/datasette/app.py", line 890, in check_visibility await self.ensure_permissions(actor, permissions) permission_allowed: action=view-database, resource=fixtures, actor={'_r': {'a': ['vi'], 'd': {'fixtures': ['vd']}, 'r': {'fixtures': {'facetable': ['vt']}}}, 'a': 'user'} File "/datasette/views/table.py", line 727, in table_view_traced view_data = await table_view_data( File "/datasette/views/table.py", line 875, in table_view_data visible, private = await datasette.check_visibility( File "/datasette/app.py", line 890, in check_visibility await self.ensure_permissions(actor, permissions) permission_allowed: action=view-instance, resource=<None>, actor={'_r': {'a': ['vi'], 'd': {'fixtures': ['vd']}, 'r': {'fixtures': {'facetable': ['vt']}}}, 'a': 'user'} File "/datasette/views/table.py", line 727, in table_view_traced view_data = await table_view_data( File "/datasette/views/table.py", line 875, in table_view_data visible, private = await datasette.check_visibility( File "/datasette/app.py", line 890, in check_visibility await self.ensure_permissions(actor, permissions) ``` That's with a token that has the view instance, view database and view table permissions required. But... what if the restrictions logic said that if you have view-table you automatically also get view-database and view-instance? Would that actually let people do anything they shouldn't be able to do? I don't think it would even let them see a list of tables that they weren't allowed to visit, so it might be OK. I'll try that and see how it works. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1690800119 | https://github.com/simonw/datasette/issues/2143#issuecomment-1690800119 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kx4_3 | simonw 9599 | 2023-08-24T00:10:32Z | 2023-08-24T00:39:00Z | OWNER | Something notable about this design is that, because the values in the key-value pairs are treated as JSON first and then strings only if they don't parse cleanly as JSON, it's possible to represent any structure (including nesting structures) using this syntax. You can do things like this if you need to (settings for an imaginary plugin):
That previous design was meant to support round-trips, so you could take any nested JSON object and turn it into an HTMl form or query string where every value can have its own form field, then turn the result back again. For the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1690800641 | https://github.com/simonw/datasette/issues/2143#issuecomment-1690800641 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kx5IB | simonw 9599 | 2023-08-24T00:11:16Z | 2023-08-24T00:11:16Z | OWNER |
That's a neat example thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1690792514 | https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kx3JC | simonw 9599 | 2023-08-24T00:00:16Z | 2023-08-24T00:02:55Z | OWNER | I've been thinking about what it might look like to allow command-line arguments to be used to define any of the configuration options in Here's what I've come up with:
def _handle_pair(key: str, value: str) -> dict: """ Turn a key-value pair into a nested dictionary. foo, bar => {'foo': 'bar'} foo.bar, baz => {'foo': {'bar': 'baz'}} foo.bar, [1, 2, 3] => {'foo': {'bar': [1, 2, 3]}} foo.bar, "baz" => {'foo': {'bar': 'baz'}} foo.bar, '{"baz": "qux"}' => {'foo': {'bar': "{'baz': 'qux'}"}} """ try: value = json.loads(value) except json.JSONDecodeError: # If it doesn't parse as JSON, treat it as a string pass
def _combine(base: dict, update: dict) -> dict: """ Recursively merge two dictionaries. """ for key, value in update.items(): if isinstance(value, dict) and key in base and isinstance(base[key], dict): base[key] = _combine(base[key], value) else: base[key] = value return base def handle_pairs(pairs: List[Tuple[str, Any]]) -> dict:
"""
Parse a list of key-value pairs into a nested dictionary.
"""
result = {}
for key, value in pairs:
parsed_pair = _handle_pair(key, value)
result = _combine(result, parsed_pair)
return result
Although... we could keep compatibility by saying that if you call |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1690787394 | https://github.com/simonw/datasette/issues/2143#issuecomment-1690787394 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kx15C | simonw 9599 | 2023-08-23T23:52:02Z | 2023-08-23T23:52:02Z | OWNER |
Having multiple configs that combine in that way is a really interesting direction.
I'm very keen on separating out the "metadata" - where metadata is the slimmest possible set of things, effectively the data license and the source and the column and table descriptions - from everything else, mainly because I want metadata to be able to travel with the data. One idea that's been discussed before is having an optional mechanism for storing metadata in the SQLite database file itself - potentially in a That's why I'm so keen on splitting out metadata from all of the other stuff - settings and plugin configuration and authentication rules. So really it becomes "true metadata" v.s. "all of the other junk that's accumulated in metadata and |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1690705243 | https://github.com/simonw/datasette/issues/2102#issuecomment-1690705243 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5kxh1b | simonw 9599 | 2023-08-23T22:03:54Z | 2023-08-23T22:03:54Z | OWNER | Idea: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1690703764 | https://github.com/simonw/datasette/issues/2102#issuecomment-1690703764 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5kxheU | simonw 9599 | 2023-08-23T22:02:14Z | 2023-08-23T22:02:14Z | OWNER | Built this new test:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
1690693830 | https://github.com/simonw/datasette/issues/2102#issuecomment-1690693830 | https://api.github.com/repos/simonw/datasette/issues/2102 | IC_kwDOBm6k_c5kxfDG | simonw 9599 | 2023-08-23T21:51:52Z | 2023-08-23T21:52:58Z | OWNER | This is the hook in question: https://github.com/simonw/datasette/blob/bdf59eb7db42559e538a637bacfe86d39e5d17ca/datasette/hookspecs.py#L108-L110
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
API tokens with view-table but not view-database/view-instance cannot access the table 1805076818 | |
680262437 | https://github.com/simonw/datasette/issues/950#issuecomment-680262437 | https://api.github.com/repos/simonw/datasette/issues/950 | MDEyOklzc3VlQ29tbWVudDY4MDI2MjQzNw== | simonw 9599 | 2020-08-25T20:49:24Z | 2023-08-23T21:34:09Z | OWNER | The alternative to this would be to use regular databases and control access to them using Authentication and permissions. My concern there is that it's just too easy for someone to mess up their configuration, which would be really bad. I like the idea of a much stronger defense mechanism specifically designed for secrets that should not be exposed. Outside of secrets, passwords and tokens this mechanism could also be useful for the use-case of using Datasette to power websites - as seen on https://www.niche-museums.com/ and https://www.rockybeaches.com/ - maybe those sites don't want to expose their data through their API but still want to use |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Private/secret databases: database files that are only visible to plugins 685806511 | |
1690438270 | https://github.com/simonw/datasette/issues/2150#issuecomment-1690438270 | https://api.github.com/repos/simonw/datasette/issues/2150 | IC_kwDOBm6k_c5kwgp- | simonw 9599 | 2023-08-23T18:27:47Z | 2023-08-23T18:27:47Z | OWNER | I added Oh interesting, looks like it's over-ridden here too: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
form label { width: 15% } is a bad default 1863810783 | |
1690435866 | https://github.com/simonw/datasette/issues/2150#issuecomment-1690435866 | https://api.github.com/repos/simonw/datasette/issues/2150 | IC_kwDOBm6k_c5kwgEa | simonw 9599 | 2023-08-23T18:25:51Z | 2023-08-23T18:25:51Z | OWNER | Looks like that affects a few forms: The search form on the table page over-rides it already: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
form label { width: 15% } is a bad default 1863810783 | |
1690432928 | https://github.com/simonw/datasette/issues/2150#issuecomment-1690432928 | https://api.github.com/repos/simonw/datasette/issues/2150 | IC_kwDOBm6k_c5kwfWg | simonw 9599 | 2023-08-23T18:23:26Z | 2023-08-23T18:23:26Z | OWNER | That should be scoped just to the query filters form on the table page. I'll fix it in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
form label { width: 15% } is a bad default 1863810783 | |
1690431509 | https://github.com/simonw/datasette/issues/2150#issuecomment-1690431509 | https://api.github.com/repos/simonw/datasette/issues/2150 | IC_kwDOBm6k_c5kwfAV | simonw 9599 | 2023-08-23T18:22:47Z | 2023-08-23T18:22:47Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
form label { width: 15% } is a bad default 1863810783 | ||
1690423878 | https://github.com/simonw/datasette/issues/2091#issuecomment-1690423878 | https://api.github.com/repos/simonw/datasette/issues/2091 | IC_kwDOBm6k_c5kwdJG | simonw 9599 | 2023-08-23T18:18:18Z | 2023-08-23T18:18:18Z | OWNER | Dupe of: - #2097 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Drop support for Python 3.7 1781022369 | |
1689207309 | https://github.com/simonw/datasette/issues/2123#issuecomment-1689207309 | https://api.github.com/repos/simonw/datasette/issues/2123 | IC_kwDOBm6k_c5kr0IN | simonw 9599 | 2023-08-23T03:07:27Z | 2023-08-23T03:07:27Z | OWNER |
Yes please! What an odd bug. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette serve when invoked with --reload interprets the serve command as a file 1825007061 | |
1689206768 | https://github.com/simonw/datasette/issues/2147#issuecomment-1689206768 | https://api.github.com/repos/simonw/datasette/issues/2147 | IC_kwDOBm6k_c5krz_w | simonw 9599 | 2023-08-23T03:06:32Z | 2023-08-23T03:06:32Z | OWNER | I'm less convinced by the "rewrite the query in some way" optional idea. What kind of use-cases can you imagine for that? My hunch is that it's much more likely to cause weird breakages than it is to allow for useful plugin extensions, but I'm willing to be convinced otherwise. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for database queries that are run 1858228057 | |
1689206170 | https://github.com/simonw/datasette/issues/2147#issuecomment-1689206170 | https://api.github.com/repos/simonw/datasette/issues/2147 | IC_kwDOBm6k_c5krz2a | simonw 9599 | 2023-08-23T03:05:32Z | 2023-08-23T03:05:32Z | OWNER | Interestingly enough there's actually a mechanism that looks like that a bit already: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/views/database.py#L496-L508 That Against these constants: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/init.py#L223-L253 Which isn't a million miles away from your suggestion to have a hook that can say if the query should be executed or not. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for database queries that are run 1858228057 | |
1689198368 | https://github.com/simonw/datasette/pull/2148#issuecomment-1689198368 | https://api.github.com/repos/simonw/datasette/issues/2148 | IC_kwDOBm6k_c5krx8g | simonw 9599 | 2023-08-23T02:57:53Z | 2023-08-23T02:57:53Z | OWNER | @dependabot rebase |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump sphinx, furo, blacken-docs dependencies 1859415334 | |
1689177556 | https://github.com/simonw/datasette/pull/2148#issuecomment-1689177556 | https://api.github.com/repos/simonw/datasette/issues/2148 | IC_kwDOBm6k_c5krs3U | simonw 9599 | 2023-08-23T02:44:29Z | 2023-08-23T02:44:29Z | OWNER | Simplest possible solution is to only run the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump sphinx, furo, blacken-docs dependencies 1859415334 | |
1689175062 | https://github.com/simonw/datasette/pull/2148#issuecomment-1689175062 | https://api.github.com/repos/simonw/datasette/issues/2148 | IC_kwDOBm6k_c5krsQW | simonw 9599 | 2023-08-23T02:40:46Z | 2023-08-23T02:40:46Z | OWNER | Here's why the tests are failing: It looks like those tests don't actually need Sphinx installed - they install https://github.com/simonw/datasette/blob/2ce7872e3ba8d07248c194ef554bbdc1df510f32/setup.py#L70-L80 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump sphinx, furo, blacken-docs dependencies 1859415334 | |
1689173748 | https://github.com/simonw/datasette/pull/2148#issuecomment-1689173748 | https://api.github.com/repos/simonw/datasette/issues/2148 | IC_kwDOBm6k_c5krr70 | simonw 9599 | 2023-08-23T02:38:31Z | 2023-08-23T02:38:31Z | OWNER | Sphinx dropped support for Python 3.8 in their version 7.2.0: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump sphinx, furo, blacken-docs dependencies 1859415334 | |
1689159200 | https://github.com/simonw/datasette/issues/516#issuecomment-1689159200 | https://api.github.com/repos/simonw/datasette/issues/516 | IC_kwDOBm6k_c5kroYg | simonw 9599 | 2023-08-23T02:15:36Z | 2023-08-23T02:15:36Z | OWNER | This could play havoc with unmerged PRs. I should merge any big ones (like the JavaScript plugins work) first. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Enforce import sort order with isort 459509126 | |
1689158712 | https://github.com/simonw/datasette/issues/516#issuecomment-1689158712 | https://api.github.com/repos/simonw/datasette/issues/516 | IC_kwDOBm6k_c5kroQ4 | simonw 9599 | 2023-08-23T02:14:45Z | 2023-08-23T02:14:45Z | OWNER | Thinking about this again today. Posted about it on Discord: https://discord.com/channels/823971286308356157/823971286941302908/1143729300349132933 I won't enforce it in a https://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/Justfile#L19-L23 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Enforce import sort order with isort 459509126 | |
1689154837 | https://github.com/simonw/datasette/issues/516#issuecomment-1689154837 | https://api.github.com/repos/simonw/datasette/issues/516 | IC_kwDOBm6k_c5krnUV | simonw 9599 | 2023-08-23T02:08:33Z | 2023-08-23T02:08:50Z | OWNER | Browse this commit to see the result: https://github.com/simonw/datasette/tree/59a5d336bd4336bc53103922ada4bf726f4336c9 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Enforce import sort order with isort 459509126 | |
1689153446 | https://github.com/simonw/datasette/issues/516#issuecomment-1689153446 | https://api.github.com/repos/simonw/datasette/issues/516 | IC_kwDOBm6k_c5krm-m | simonw 9599 | 2023-08-23T02:06:35Z | 2023-08-23T02:06:35Z | OWNER | I just tried this again today by dropping this into |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Enforce import sort order with isort 459509126 | |
1689130061 | https://github.com/simonw/datasette/pull/524#issuecomment-1689130061 | https://api.github.com/repos/simonw/datasette/issues/524 | IC_kwDOBm6k_c5krhRN | simonw 9599 | 2023-08-23T01:31:08Z | 2023-08-23T01:31:08Z | OWNER | This branch is WAY out of date. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Sort commits using isort, refs #516 459689615 | |
1689128911 | https://github.com/simonw/datasette/issues/493#issuecomment-1689128911 | https://api.github.com/repos/simonw/datasette/issues/493 | IC_kwDOBm6k_c5krg_P | simonw 9599 | 2023-08-23T01:29:20Z | 2023-08-23T01:29:20Z | OWNER | It's going to be called
|
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rename metadata.json to config.json 449886319 | |
1689128553 | https://github.com/simonw/datasette/issues/275#issuecomment-1689128553 | https://api.github.com/repos/simonw/datasette/issues/275 | IC_kwDOBm6k_c5krg5p | simonw 9599 | 2023-08-23T01:28:37Z | 2023-08-23T01:28:37Z | OWNER | This is obsoleted by the work happening here: - #2093 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"config" section in metadata.json (root, database and table level) 324720095 | |
1689127479 | https://github.com/simonw/datasette/pull/2148#issuecomment-1689127479 | https://api.github.com/repos/simonw/datasette/issues/2148 | IC_kwDOBm6k_c5krgo3 | simonw 9599 | 2023-08-23T01:26:53Z | 2023-08-23T01:26:53Z | OWNER | @dependabot recreate |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump sphinx, furo, blacken-docs dependencies 1859415334 | |
1689125244 | https://github.com/simonw/datasette/pull/2149#issuecomment-1689125244 | https://api.github.com/repos/simonw/datasette/issues/2149 | IC_kwDOBm6k_c5krgF8 | simonw 9599 | 2023-08-23T01:23:27Z | 2023-08-23T01:23:27Z | OWNER | This is a really great start - tests pass, implementation looks clean, the new stubbed documentation page makes sense. Let's land it in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Start a new `datasette.yaml` configuration file, with settings support 1861812208 | |
1686747420 | https://github.com/simonw/datasette/issues/2147#issuecomment-1686747420 | https://api.github.com/repos/simonw/datasette/issues/2147 | IC_kwDOBm6k_c5kibkc | simonw 9599 | 2023-08-21T17:31:42Z | 2023-08-21T17:34:19Z | OWNER | Are you talking just about queries submitted to There are a few ways you could solve this at the moment. The easiest would be with a piece of ASGI middleware that looks for URLs matching Then you can load that middleware from a plugin using https://docs.datasette.io/en/stable/plugin_hooks.html#asgi-wrapper-datasette That feels a bit delicate because it's relying on the URL design not changing, but I'm happy to confirm that URL is going to stay the same for Datasette 1.0 and I have no plans to change it ever. There's also a tracing mechanism built into Datasette itself that you could hook into. The internals of that are documented here: https://docs.datasette.io/en/stable/internals.html#datasette-tracer - but I don't yet consider it a 100% stable API. I don't plan to change it but I won't promise not to either. I used that mechanism in this plugin: https://datasette.io/plugins/datasette-pretty-traces - demonstrated here: https://latest-with-plugins.datasette.io/github/commits?_trace=1 The hackiest way to do this would be to patch Datasette itself and try to replace the So my recommendation for the moment is the ASGI middleware option. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for database queries that are run 1858228057 | |
1686749342 | https://github.com/simonw/datasette/issues/2147#issuecomment-1686749342 | https://api.github.com/repos/simonw/datasette/issues/2147 | IC_kwDOBm6k_c5kicCe | simonw 9599 | 2023-08-21T17:33:11Z | 2023-08-21T17:33:11Z | OWNER | I'm definitely open to suggestions for plugin hooks that might make this kind of thing easier. One idea I've been mulling is whether there should be a plugin hook that files on arbitrary views - similar to Django's That would allow people to setup code that runs before or after any of the default views in Datasette. I'm not yet 100% sold on the idea, because I worry about implementing it in a way that guarantees plugins won't break on future releases. But I'm open to considering it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for database queries that are run 1858228057 | |
1686683596 | https://github.com/simonw/datasette/issues/2145#issuecomment-1686683596 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kiL_M | simonw 9599 | 2023-08-21T16:49:12Z | 2023-08-21T16:49:12Z | OWNER | Suggestion from @asg017 is that we say that if your row has a null primary key you don't get a link to a row page for that row. Which has some precedent, because our SQL view display doesn't link to row pages at all (since they don't make sense for views): https://latest.datasette.io/fixtures/simple_view |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1685096381 | https://github.com/simonw/sqlite-utils/issues/587#issuecomment-1685096381 | https://api.github.com/repos/simonw/sqlite-utils/issues/587 | IC_kwDOCGYnMM5kcIe9 | simonw 9599 | 2023-08-19T20:04:32Z | 2023-08-19T20:04:32Z | OWNER | I'm inclined to say this isn't a bug in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference 1857851384 | |
1685096284 | https://github.com/simonw/sqlite-utils/issues/587#issuecomment-1685096284 | https://api.github.com/repos/simonw/sqlite-utils/issues/587 | IC_kwDOCGYnMM5kcIdc | simonw 9599 | 2023-08-19T20:03:59Z | 2023-08-19T20:03:59Z | OWNER | Although this is revealing a problem in the underlying code (that schema is invalid), it also represents a regression: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference 1857851384 | |
1685096129 | https://github.com/simonw/sqlite-utils/issues/587#issuecomment-1685096129 | https://api.github.com/repos/simonw/sqlite-utils/issues/587 | IC_kwDOCGYnMM5kcIbB | simonw 9599 | 2023-08-19T20:03:00Z | 2023-08-19T20:03:00Z | OWNER | Simplest possible recreation of the bug:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New .add_foreign_key() can break if PRAGMA legacy_alter_table=ON and there's an invalid foreign key reference 1857851384 | |
1684530060 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684530060 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ-OM | simonw 9599 | 2023-08-18T23:09:03Z | 2023-08-18T23:09:14Z | OWNER | Ran a quick benchmark on ChatGPT Code Interpreter: https://chat.openai.com/share/8357dc01-a97e-48ae-b35a-f06249935124 Conclusion from there is that this query returns fast no matter how much the table grows:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684526447 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684526447 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ9Vv | simonw 9599 | 2023-08-18T23:05:02Z | 2023-08-18T23:05:02Z | OWNER | How expensive is it to detect if a SQLite table contains at least one |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684525943 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684525943 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ9N3 | simonw 9599 | 2023-08-18T23:04:14Z | 2023-08-18T23:04:14Z | OWNER | This is hard. I tried this: ```python def path_from_row_pks(row, pks, use_rowid, quote=True): """Generate an optionally tilde-encoded unique identifier for a row from its primary keys.""" if use_rowid or any(row[pk] is None for pk in pks): bits = [row["rowid"]] else: bits = [ row[pk]["value"] if isinstance(row[pk], dict) else row[pk] for pk in pks ] if quote: bits = [tilde_encode(str(bit)) for bit in bits] else: bits = [str(bit) for bit in bits]
But I got this error on http://127.0.0.1:8003/nulls/nasty :
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684525054 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684525054 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ8_- | simonw 9599 | 2023-08-18T23:02:26Z | 2023-08-18T23:02:26Z | OWNER | Creating a quick test database:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684523322 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684523322 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ8k6 | simonw 9599 | 2023-08-18T22:59:14Z | 2023-08-18T22:59:14Z | OWNER | Except it looks like the Links from other tables section is broken: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684522567 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684522567 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ8ZH | simonw 9599 | 2023-08-18T22:58:07Z | 2023-08-18T22:58:07Z | OWNER | Here's a prototype of that: ```diff diff --git a/datasette/app.py b/datasette/app.py index b2644ace..acc55249 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -1386,7 +1386,7 @@ class Datasette: ) add_route( RowView.as_view(self), - r"/(?P<database>[^\/.]+)/(?P<table>[^/]+?)/(?P<pks>[^/]+?)(.(?P<format>\w+))?$", + r"/(?P<database>[^\/.]+)/(?P<table>[^/]+?)/(?P<pks>[A-Za-z0-9_-\~]+|.\d+)(.(?P<format>\w+))?$", ) add_route( TableInsertView.as_view(self), @@ -1440,7 +1440,15 @@ class Datasette: async def resolve_row(self, request): db, table_name, _ = await self.resolve_table(request) pk_values = urlsafe_components(request.url_vars["pks"]) - sql, params, pks = await row_sql_params_pks(db, table_name, pk_values) + + if len(pk_values) == 1 and pk_values[0].startswith("."): + # It's a special .rowid value + pk_values = (pk_values[0][1:],) + sql, params, pks = await row_sql_params_pks( + db, table_name, pk_values, rowid=True + ) + else: + sql, params, pks = await row_sql_params_pks(db, table_name, pk_values) results = await db.execute(sql, params, truncate=True) row = results.first() if row is None: diff --git a/datasette/utils/init.py b/datasette/utils/init.py index c388673d..96669281 100644 --- a/datasette/utils/init.py +++ b/datasette/utils/init.py @@ -1206,9 +1206,12 @@ def truncate_url(url, length): return url[: length - 1] + "…" -async def row_sql_params_pks(db, table, pk_values): +async def row_sql_params_pks(db, table, pk_values, rowid=False): pks = await db.primary_keys(table) - use_rowid = not pks + if rowid: + use_rowid = True + else: + use_rowid = not pks select = "" if use_rowid: select = "rowid, " ``` It works: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684505071 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684505071 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ4Hv | simonw 9599 | 2023-08-18T22:44:35Z | 2023-08-18T22:44:35Z | OWNER | Also relevant: https://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/utils/init.py#L1147-L1153 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684504398 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684504398 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ39O | simonw 9599 | 2023-08-18T22:43:31Z | 2023-08-18T22:43:46Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684504051 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684504051 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ33z | simonw 9599 | 2023-08-18T22:43:06Z | 2023-08-18T22:43:06Z | OWNER | Here's the regex in question at the moment: https://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/app.py#L1387-L1390 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684503587 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684503587 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ3wj | simonw 9599 | 2023-08-18T22:42:28Z | 2023-08-18T22:42:39Z | OWNER | I could set a rule that extensions (including custom render extensions set by plugins) must not be valid integers, and teach Datasette that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684503189 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684503189 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ3qV | simonw 9599 | 2023-08-18T22:41:51Z | 2023-08-18T22:41:51Z | OWNER | ```pycon
But... I worry about that colliding with my URL routing code that spots the difference between these:
etc. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684502278 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684502278 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ3cG | simonw 9599 | 2023-08-18T22:40:20Z | 2023-08-18T22:40:20Z | OWNER | From reviewing https://simonwillison.net/2022/Mar/19/weeknotes/
That's how I chose the tilde character - but it also suggests that I could use So maybe No, that doesn't work: ```pycon
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684500540 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684500540 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ3A8 | simonw 9599 | 2023-08-18T22:37:37Z | 2023-08-18T22:37:37Z | OWNER | I just found this and panicked, thinking maybe tilde encoding is a bad idea after all! https://jkorpela.fi/tilde.html But... "Date of last update: 1999-08-27" - I think I'm OK. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684500172 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684500172 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ27M | simonw 9599 | 2023-08-18T22:37:04Z | 2023-08-18T22:37:04Z | OWNER | Looking at the way these URLs work: because the components themselves in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684498947 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684498947 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ2oD | simonw 9599 | 2023-08-18T22:35:04Z | 2023-08-18T22:35:04Z | OWNER | The most interesting row URL in the fixtures database right now is this one: https://latest.datasette.io/fixtures/compound_primary_key/a~2Fb,~2Ec-d |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684497642 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684497642 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ2Tq | simonw 9599 | 2023-08-18T22:32:53Z | 2023-08-18T22:32:53Z | OWNER | Here's a potential solution: make it so ALL Then teach the code that outputs the URL to a row page to spot if there are |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684497000 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684497000 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ2Jo | simonw 9599 | 2023-08-18T22:31:53Z | 2023-08-18T22:31:53Z | OWNER | So it sounds like SQLite does ensure that a So one solution here would be to detect a null primary key and switch that table over to using https://latest.datasette.io/fixtures/infinity/1 But when would we run that check? And does every row in the table get a new |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684495674 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684495674 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ106 | simonw 9599 | 2023-08-18T22:29:47Z | 2023-08-18T22:29:47Z | OWNER | https://www.sqlite.org/lang_createtable.html#the_primary_key says:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684494464 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684494464 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZ1iA | simonw 9599 | 2023-08-18T22:27:51Z | 2023-08-18T22:28:40Z | OWNER | Oh wow, null primary keys are bad news... SQLite lets you insert multiple rows with the same <Table foo (id, name)> >>> db.schema 'CREATE TABLE [foo] (\n [id] TEXT PRIMARY KEY,\n [name] TEXT\n);' >>> db["foo"].insert({"id": None, "name": "No ID"}, pk="id") <Table foo (id, name)> >>> db.schema 'CREATE TABLE [foo] (\n [id] TEXT PRIMARY KEY,\n [name] TEXT\n);' >>> list(db["foo"].rows) [{'id': None, 'name': 'No ID'}, {'id': None, 'name': 'No ID'}] >>> list(db.query('select * from foo where id = null')) [] >>> list(db.query('select * from foo where id is null')) [{'id': None, 'name': 'No ID'}, {'id': None, 'name': 'No ID'}] ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684488526 | https://github.com/simonw/datasette/issues/2143#issuecomment-1684488526 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kZ0FO | simonw 9599 | 2023-08-18T22:18:39Z | 2023-08-18T22:18:39Z | OWNER |
I'm not a fan of that. I feel like software history is full of examples of projects that implemented configuration-as-code and then later regretted it - the most recent example is I don't think having people dynamically generate JSON/YAML for their configuration is a big burden. I'd have to see some very compelling use-cases to convince me otherwise. That said, I do really like a bias towards settings that can be changed at runtime. Datasette has suffered a bit from some settings that can't be easily changed at runtime already - hence my gnarly https://github.com/simonw/datasette-remote-metadata plugin. For things like Datasette Cloud for example the more people can configure without rebooting their container the better! I don't think live reconfiguration at runtime is incompatible with JSON/YAML configuration though. Caddy is one of my favourite examples of software that can be entirely re-configured at runtime by POSTING a big blob of JSON to it: https://caddyserver.com/docs/quick-starts/api |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1684485591 | https://github.com/simonw/datasette/issues/2143#issuecomment-1684485591 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kZzXX | simonw 9599 | 2023-08-18T22:14:35Z | 2023-08-18T22:14:35Z | OWNER | Actually there is one thing that I'm not comfortable about with respect to the existing design: the way the database / tables stuff is nested. They assume that the user will attach the database to Datasette using a fixed name - But what if we want to support users downloading databases from each other and attaching them to Datasette where those DBs might carry some of their own configuration? Moving metadata into the databases makes sense there, but what about database-specific settings like the default sort order for a table, or configured canned queries? Having those tied to the filename of the database itself feels unpleasant to me. But how else could we handle this? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1684484426 | https://github.com/simonw/datasette/issues/2143#issuecomment-1684484426 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kZzFK | simonw 9599 | 2023-08-18T22:12:52Z | 2023-08-18T22:12:52Z | OWNER | Yeah, I'm convinced by that. There's not point in having both I like Here's a thought for how it could look - I'll go with the YAML format because I expect that to be the default most people use, just because it supports multi-line strings better. I based this on the big example at https://docs.datasette.io/en/1.0a3/metadata.html#using-yaml-for-metadata - and combined some bits from https://docs.datasette.io/en/1.0a3/authentication.html as well. ```yaml
title: Demonstrating Metadata from YAML
description_html: |-
This description includes a long HTML string
settings: default_page_size: 10 max_returned_rows: 3000 sql_time_limit_ms": 8000 databases: docs: permissions: create-table: id: editor fixtures: tables: no_primary_key: hidden: true queries: neighborhood_search: sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood; title: Search neighborhoods description_html: |- This demonstrates basic LIKE search permissions: debug-menu: id: '*' plugins:
datasette-ripgrep:
path: /usr/local/lib/python3.11/site-packages
In this example I've mixed in one extra concept: that There are some things in there that look a little bit like metadata - the But are they metadata? The title and description of the overall instance feels like it could be described as general configuration. The stuff for the Note that queries can be defined by a plugin hook too: https://docs.datasette.io/en/1.0a3/plugin_hooks.html#canned-queries-datasette-database-actor What do you think? Is this the right direction, or are you thinking there's a more radical redesign that would make sense here? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1684384750 | https://github.com/simonw/datasette/issues/2145#issuecomment-1684384750 | https://api.github.com/repos/simonw/datasette/issues/2145 | IC_kwDOBm6k_c5kZavu | simonw 9599 | 2023-08-18T20:07:18Z | 2023-08-18T20:07:18Z | OWNER | The big challenge here is what the URL to that row page should look like. How can I encode a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
If a row has a primary key of `null` various things break 1857234285 | |
1684235760 | https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1684235760 | https://api.github.com/repos/simonw/sqlite-utils/issues/577 | IC_kwDOCGYnMM5kY2Xw | simonw 9599 | 2023-08-18T17:43:11Z | 2023-08-18T17:43:11Z | OWNER | Here's a new plugin that brings back the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get `add_foreign_keys()` to work without modifying `sqlite_master` 1817289521 | |
1683429959 | https://github.com/simonw/datasette/issues/2143#issuecomment-1683429959 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kVxpH | simonw 9599 | 2023-08-18T06:43:33Z | 2023-08-18T15:19:07Z | OWNER | The single biggest design challenge I've had with metadata relates to how it should or should not be inherited. If you apply a license to a Datasette instance, it feels like that should flow down to cover all of the databases and all of the tables within those databases. If the license is at the database level, it should cover all tables. But... should source do the same thing? I made it behave the same way as license, but it's presumably common for one database to have a single license but multiple different sources of data. Then there's title - should that inherit? It feels like title should apply to only one level - you may want a title that applies to the instance, then a different custom title for databases and tables. Here's the current state of play for metadata: https://docs.datasette.io/en/1.0a3/metadata.html So there's There's Then there are these six:
I added Tables can also have column descriptions - just a string for each column. There's a demo of those here: https://latest.datasette.io/fixtures/roadside_attractions And then there's all of the other stuff, most of which feels much more like "settings" than "metadata":
And the authentication stuff! And the new I think that might be everything (excluding the And to make things even more confusing... I believe you can add arbitrary key/value pairs to your metadata and then use them in your templates! I think I've heard from at least one person who uses that ability. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1683420879 | https://github.com/simonw/datasette/issues/2143#issuecomment-1683420879 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kVvbP | simonw 9599 | 2023-08-18T06:33:24Z | 2023-08-18T15:15:34Z | OWNER | I completely agree: metadata is a mess, and it deserves our attention.
That's not completely true - there are hacks around that. I have a plugin that applies one set of gnarly hacks for that here: https://github.com/simonw/datasette-remote-metadata - it's pretty grim though!
100% this: it's a complete mess. Datasette used to have a
Yes, they're not pretty at all. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1683443891 | https://github.com/simonw/datasette/issues/2143#issuecomment-1683443891 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kV1Cz | simonw 9599 | 2023-08-18T06:58:15Z | 2023-08-18T06:58:15Z | OWNER | Hah, that Hence the whole
If configuration and metadata were separate we could ditch that whole messy situation - configuration can stay hidden, metadata can stay public. Though I have been thinking that Datasette might benefit from a "secrets" mechanism that's separate from configuration and metadata... kind of like what LLM has: https://llm.datasette.io/en/stable/help.html#llm-keys-help |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1683440597 | https://github.com/simonw/datasette/issues/2143#issuecomment-1683440597 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kV0PV | simonw 9599 | 2023-08-18T06:54:49Z | 2023-08-18T06:54:49Z | OWNER | A related point that I've been considering a lot recently: it turns out that sometimes I really want to define settings on the CLI instead of in a file, purely for convenience. It's pretty annoying when I want to try out a new plugin but I have to create a dedicated
So maybe there's a world in which all of the settings can be applied in a That gets trickier when you need to pass a nested structure or similar, but we could always support those as JSON:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1683435579 | https://github.com/simonw/datasette/issues/2143#issuecomment-1683435579 | https://api.github.com/repos/simonw/datasette/issues/2143 | IC_kwDOBm6k_c5kVzA7 | simonw 9599 | 2023-08-18T06:49:39Z | 2023-08-18T06:49:39Z | OWNER | My ideal situation then would be something like this:
Currently we have three types of things:
Should settings and configuration be separate? I'm not 100% sure that they should - maybe those two concepts should be combined somehow. Configuration directory mode needs to be considered too: https://docs.datasette.io/en/stable/settings.html#configuration-directory-mode - interestingly it already has a thing where it can pick up settings from a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
De-tangling Metadata before Datasette 1.0 1855885427 | |
1683404978 | https://github.com/simonw/sqlite-utils/issues/586#issuecomment-1683404978 | https://api.github.com/repos/simonw/sqlite-utils/issues/586 | IC_kwDOCGYnMM5kVriy | simonw 9599 | 2023-08-18T06:13:46Z | 2023-08-18T06:13:46Z | OWNER | I shipped the view recreating fix in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() fails to drop column if table is part of a view 1856075668 | |
1683398866 | https://github.com/simonw/sqlite-utils/issues/586#issuecomment-1683398866 | https://api.github.com/repos/simonw/sqlite-utils/issues/586 | IC_kwDOCGYnMM5kVqDS | simonw 9599 | 2023-08-18T06:05:50Z | 2023-08-18T06:06:42Z | OWNER | Options:
- Provide a I'm on the fence as to which of these I like the most. I'm tempted to go with the one which just drops VIEWS and recreates them all the time, because it feels simpler. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() fails to drop column if table is part of a view 1856075668 | |
1683396150 | https://github.com/simonw/sqlite-utils/issues/586#issuecomment-1683396150 | https://api.github.com/repos/simonw/sqlite-utils/issues/586 | IC_kwDOCGYnMM5kVpY2 | simonw 9599 | 2023-08-18T06:02:18Z | 2023-08-18T06:06:31Z | OWNER | More notes in here: - https://github.com/simonw/datasette-edit-schema/issues/35#issuecomment-1683392873 Not all Python/SQLite installations exhibit this problem by default! It turns out this is controlled by the If that PRAGMA is turned on (default in newer SQLites) then Here's a one-liner to test if it is on or not:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() fails to drop column if table is part of a view 1856075668 | |
1683217284 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683217284 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU9uE | simonw 9599 | 2023-08-18T01:50:21Z | 2023-08-18T01:50:21Z | OWNER | And a test of the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683212074 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683212074 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU8cq | simonw 9599 | 2023-08-18T01:43:54Z | 2023-08-18T01:43:54Z | OWNER | Some manual testing:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683201239 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683201239 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU5zX | simonw 9599 | 2023-08-18T01:30:46Z | 2023-08-18T01:30:46Z | OWNER | Help can now look like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683200128 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683200128 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU5iA | simonw 9599 | 2023-08-18T01:29:00Z | 2023-08-18T01:29:00Z | OWNER | I'm not going to implement the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683198740 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683198740 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU5MU | simonw 9599 | 2023-08-18T01:26:47Z | 2023-08-18T01:26:47Z | OWNER | The only CLI feature that supports providing just the column name appears to be this:
Add a new foreign key constraint to an existing table Example:
WARNING: Could corrupt your database! Back up your database file first.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683197882 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683197882 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU4-6 | simonw 9599 | 2023-08-18T01:25:53Z | 2023-08-18T01:25:53Z | OWNER | Probably most relevant here is this snippet from:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683195669 | https://github.com/simonw/sqlite-utils/issues/585#issuecomment-1683195669 | https://api.github.com/repos/simonw/sqlite-utils/issues/585 | IC_kwDOCGYnMM5kU4cV | simonw 9599 | 2023-08-18T01:24:57Z | 2023-08-18T01:24:57Z | OWNER | Currently:
Transform a table beyond the capabilities of ALTER TABLE Example:
Options: --type <TEXT CHOICE>... Change column type to INTEGER, TEXT, FLOAT or BLOB --drop TEXT Drop this column --rename <TEXT TEXT>... Rename this column to X -o, --column-order TEXT Reorder columns --not-null TEXT Set this column to NOT NULL --not-null-false TEXT Remove NOT NULL from this column --pk TEXT Make this column the primary key --pk-none Remove primary key (convert to rowid table) --default <TEXT TEXT>... Set default value for this column --default-none TEXT Remove default from this column --drop-foreign-key TEXT Drop foreign key constraint for this column --sql Output SQL without executing it --load-extension TEXT Path to SQLite extension, with optional :entrypoint -h, --help Show this message and exit. ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
CLI equivalents to `transform(add_foreign_keys=)` 1855894222 | |
1683164661 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683164661 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUw31 | simonw 9599 | 2023-08-18T00:45:53Z | 2023-08-18T00:45:53Z | OWNER | More updated documentation: - https://sqlite-utils--584.org.readthedocs.build/en/584/reference.html#sqlite_utils.db.Table.transform - https://sqlite-utils--584.org.readthedocs.build/en/584/python-api.html#python-api-transform-add-foreign-key-constraints |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683145819 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683145819 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUsRb | simonw 9599 | 2023-08-18T00:17:26Z | 2023-08-18T00:17:26Z | OWNER | Updated documentation: https://sqlite-utils--584.org.readthedocs.build/en/584/python-api.html#adding-foreign-key-constraints |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683145110 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683145110 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUsGW | simonw 9599 | 2023-08-18T00:16:28Z | 2023-08-18T00:16:48Z | OWNER | One last piece of documentation: need to document the new option to I should write tests for them too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683143723 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683143723 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUrwr | simonw 9599 | 2023-08-18T00:14:52Z | 2023-08-18T00:14:52Z | OWNER | Another docs update: this bit in here https://sqlite-utils.datasette.io/en/3.34/python-api.html#adding-multiple-foreign-key-constraints-at-once talks about how
That doesn't apply any more - the new mechanism using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683139304 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683139304 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUqro | simonw 9599 | 2023-08-18T00:09:56Z | 2023-08-18T00:09:56Z | OWNER | Upgrading |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683138953 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683138953 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUqmJ | simonw 9599 | 2023-08-18T00:09:20Z | 2023-08-18T00:09:20Z | OWNER | Weird, I'm getting a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683137259 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683137259 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUqLr | simonw 9599 | 2023-08-18T00:06:59Z | 2023-08-18T00:06:59Z | OWNER | The docs still describe the old trick, I need to update that: https://sqlite-utils.datasette.io/en/3.34/python-api.html#adding-foreign-key-constraints |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683122767 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683122767 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUmpP | simonw 9599 | 2023-08-17T23:46:09Z | 2023-08-17T23:46:09Z | OWNER | Oops,
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683118376 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683118376 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUlko | simonw 9599 | 2023-08-17T23:41:10Z | 2023-08-17T23:41:10Z | OWNER | The problem here is that the table created by this line:
On the Here's what the debugger reveals about this code:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683114719 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683114719 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUkrf | simonw 9599 | 2023-08-17T23:36:02Z | 2023-08-17T23:36:02Z | OWNER | Just these three lines recreate the problem: ```python from sqlite_utils import Database fresh_db = Database(memory=True) fresh_db.create_table("dogs", {"name": str})
fresh_db.create_table("breeds", {"name": str})
fresh_db["dogs"].add_column("breed_id", fk="breeds")
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683112857 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683112857 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUkOZ | simonw 9599 | 2023-08-17T23:33:58Z | 2023-08-17T23:33:58Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | ||
1683112298 | https://github.com/simonw/sqlite-utils/pull/584#issuecomment-1683112298 | https://api.github.com/repos/simonw/sqlite-utils/issues/584 | IC_kwDOCGYnMM5kUkFq | simonw 9599 | 2023-08-17T23:33:14Z | 2023-08-17T23:33:14Z | OWNER | Just one failing test left: ``` # Soundness check foreign_keys point to existing tables for fk in foreign_keys: if fk.other_table == name and columns.get(fk.other_column): continue if not any( c for c in self[fk.other_table].columns if c.name == fk.other_column ):
sqlite_utils/db.py:882: AlterError ==== short test summary info ==== FAILED tests/test_create.py::test_add_column_foreign_key - sqlite_utils.db.AlterError: No such column: breeds.rowid ==== 1 failed, 378 deselected in 0.49s ==== ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() instead of modifying sqlite_master for add_foreign_keys 1855838223 | |
1683110636 | https://github.com/simonw/sqlite-utils/issues/583#issuecomment-1683110636 | https://api.github.com/repos/simonw/sqlite-utils/issues/583 | IC_kwDOCGYnMM5kUjrs | simonw 9599 | 2023-08-17T23:31:27Z | 2023-08-17T23:31:27Z | OWNER | Spotted this while working on: - #577 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get rid of test.utils.collapse_whitespace 1855836914 | |
1683098094 | https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683098094 | https://api.github.com/repos/simonw/sqlite-utils/issues/577 | IC_kwDOCGYnMM5kUgnu | simonw 9599 | 2023-08-17T23:15:36Z | 2023-08-17T23:15:36Z | OWNER | An interesting side-effect of this change is that it does result in a slightly different schema - e.g. this test: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_extract.py#L118-L133 Needs updating like so:
I don't think this should count as a breaking change release though, but it's still worth noting. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get `add_foreign_keys()` to work without modifying `sqlite_master` 1817289521 | |
1683076325 | https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683076325 | https://api.github.com/repos/simonw/sqlite-utils/issues/577 | IC_kwDOCGYnMM5kUbTl | simonw 9599 | 2023-08-17T22:48:36Z | 2023-08-17T22:48:36Z | OWNER | I'm inclined to just go with the It would be nice to drop some code complexity, plus I don't yet have a way of running automated tests against Python + SQLite versions that exhibit the problem. |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
Get `add_foreign_keys()` to work without modifying `sqlite_master` 1817289521 | |
1683074009 | https://github.com/simonw/sqlite-utils/issues/577#issuecomment-1683074009 | https://api.github.com/repos/simonw/sqlite-utils/issues/577 | IC_kwDOCGYnMM5kUavZ | simonw 9599 | 2023-08-17T22:45:29Z | 2023-08-17T22:47:08Z | OWNER | Actually I think |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get `add_foreign_keys()` to work without modifying `sqlite_master` 1817289521 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue >1000