{"html_url": "https://github.com/simonw/datasette/pull/2008#issuecomment-1407567753", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2008", "id": 1407567753, "node_id": "IC_kwDOBm6k_c5T5ceJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-29T05:39:54Z", "updated_at": "2023-01-29T05:40:34Z", "author_association": "OWNER", "body": "I absolutely _love_ this performance boost - really nice find.\r\n\r\nOne concern: this will be the first time Datasette ships a core feature that uses window functions.\r\n\r\nWindow functions were added to SQLite in [version 3.25.0](https://www.sqlite.org/releaselog/3_25_0.html) on 2018-09-15 - which means it's still very common for Datasette to run on versions that don't yet support them.\r\n\r\nSo I see two options:\r\n- Detect window function support and switch between the old implementation and this better, new one\r\n- Detect window functions and disable the facet-by-JSON feature entirely if they are missing\r\n\r\nI like the first option a bit better.\r\n\r\nThis also leads to a tricky CI challenge: Datasette needs to be able to run its test suite against more than one SQLite version to confidently test this feature going forward.\r\n\r\nI don't yet have a good GitHub Actions recipe for this, but I _really_ need one - for `sqlite-utils` too.\r\n\r\nMight be able to use this trick for that: https://til.simonwillison.net/sqlite/ld-preload", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1560982210, "label": "array facet: don't materialize unnecessary columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2008#issuecomment-1407471459", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2008", "id": 1407471459, "node_id": "IC_kwDOBm6k_c5T5E9j", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-28T19:40:18Z", "updated_at": "2023-01-29T04:55:39Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.11**% // Head: **91.78**% // Decreases project coverage by **`-0.34%`** :warning:\n> Coverage data is based on head [(`f529a30`)](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`e4ebef0`)](https://codecov.io/gh/simonw/datasette/commit/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch has no changes to coverable lines.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #2008 +/- ##\n==========================================\n- Coverage 92.11% 91.78% -0.34% \n==========================================\n Files 38 39 +1 \n Lines 5555 5599 +44 \n==========================================\n+ Hits 5117 5139 +22 \n- Misses 438 460 +22 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/facets.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2ZhY2V0cy5weQ==) | `91.84% <\u00f8> (\u00f8)` | |\n| [datasette/views/row.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3Jvdy5weQ==) | `87.82% <0.00%> (\u00f8)` | |\n| [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `92.57% <0.00%> (\u00f8)` | |\n| [datasette/views/database.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2RhdGFiYXNlLnB5) | `96.61% <0.00%> (\u00f8)` | |\n| [datasette/utils/shutil\\_backport.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL3NodXRpbF9iYWNrcG9ydC5weQ==) | `9.09% <0.00%> (\u00f8)` | |\n| [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `82.40% <0.00%> (+2.77%)` | :arrow_up: |\n| [datasette/plugins.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3BsdWdpbnMucHk=) | `85.29% <0.00%> (+2.94%)` | :arrow_up: |\n| [datasette/utils/asgi.py](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL2FzZ2kucHk=) | `93.12% <0.00%> (+3.05%)` | :arrow_up: |\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/2008?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1560982210, "label": "array facet: don't materialize unnecessary columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2008#issuecomment-1407561308", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2008", "id": 1407561308, "node_id": "IC_kwDOBm6k_c5T5a5c", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-29T04:50:50Z", "updated_at": "2023-01-29T04:50:50Z", "author_association": "CONTRIBUTOR", "body": "I pushed a revised version which ends up being faster -- the example which currently takes 4 seconds now runs in 500ms.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1560982210, "label": "array facet: don't materialize unnecessary columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2008#issuecomment-1407558284", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2008", "id": 1407558284, "node_id": "IC_kwDOBm6k_c5T5aKM", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-29T04:23:58Z", "updated_at": "2023-01-29T04:24:27Z", "author_association": "CONTRIBUTOR", "body": "Ack, this PR is broken. I see now that the `inner.*` is necessary for ensuring the correct count in the face of rows having duplicate values in views.\r\n\r\nThat fixes the overcounting, but I think can undercount when the rows have the same data, eg a view like:\r\n\r\n```sql\r\nSELECT '[\"bar\"]' tags UNION ALL SELECT '[\"bar\"]'\r\n```\r\n\r\nwill produce a count of `{\"bar\": 1 }`, when it should be `{\"bar\": 2}`. In fact, this could apply in tables without primary keys, too.\r\n\r\nIf `inner` came from a base table that had a primary key or a rowid, we could use those column(s) to solve that case.\r\n\r\nI guess a general solution would be to compute a window function so we have a distinct ID for each row. Will fiddle to see if I can get that working.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1560982210, "label": "array facet: don't materialize unnecessary columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1973#issuecomment-1407523547", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1973", "id": 1407523547, "node_id": "IC_kwDOBm6k_c5T5Rrb", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-29T00:40:31Z", "updated_at": "2023-01-29T00:40:31Z", "author_association": "CONTRIBUTOR", "body": "A +1 for switching to `CustomRow`: I think you currently only get a `CustomRow` if the result set had a column that was an fkey ([this code](https://github.com/simonw/datasette/blob/3c352b7132ef09b829abb69a0da0ad00be5edef9/datasette/views/table.py#L667-L682))\r\n\r\nOtherwise you get vanilla `sqlite3.Row`s, which will fail if you try to access `.columns` or lookup the cell by name, which surprised me recently", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515815014, "label": "render_cell plugin hook's row object is not a sqlite.Row"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2008#issuecomment-1407470429", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2008", "id": 1407470429, "node_id": "IC_kwDOBm6k_c5T5Etd", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-28T19:34:29Z", "updated_at": "2023-01-28T19:34:29Z", "author_association": "CONTRIBUTOR", "body": "I don't know how/if you do automated tests for performance, so I haven't changed any of the tests.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1560982210, "label": "array facet: don't materialize unnecessary columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/523#issuecomment-1407264466", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/523", "id": 1407264466, "node_id": "IC_kwDOCGYnMM5T4SbS", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-28T02:41:14Z", "updated_at": "2023-01-28T02:41:14Z", "author_association": "CONTRIBUTOR", "body": "I also often then run another little script to cast all empty strings to null, but i save that for another issue if this gets accepted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1560651350, "label": "Feature request: trim all leading and trailing white space for all columns for all tables in a database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2006#issuecomment-1405488884", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2006", "id": 1405488884, "node_id": "IC_kwDOBm6k_c5Txg70", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-26T19:20:53Z", "updated_at": "2023-01-26T19:20:53Z", "author_association": "OWNER", "body": "I can run a GitHub code search a week or so before the release to give me time to get in touch with anyone who has actions that look like they might break.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1558644003, "label": "Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2006#issuecomment-1405488523", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2006", "id": 1405488523, "node_id": "IC_kwDOBm6k_c5Txg2L", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-26T19:20:32Z", "updated_at": "2023-01-26T19:20:32Z", "author_association": "OWNER", "body": "This won't actually help that much if the user's GitHub Actions workflow does the equivalent of this:\r\n```\r\npip install datasette\r\ndatasette publish cloudrun mydb.db ...\r\n```\r\nSince they'll get the 1.0 release too.\r\n\r\nSince they'll need to make changes anyway, maybe a better solution is to document this and tell people they should use this instead:\r\n\r\n```\r\ndatasette publish cloudrun mydb.db ... --branch 0.64.1\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1558644003, "label": "Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2005#issuecomment-1404458011", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2005", "id": 1404458011, "node_id": "IC_kwDOBm6k_c5TtlQb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-26T01:41:30Z", "updated_at": "2023-01-26T01:41:50Z", "author_association": "OWNER", "body": "My code looked like this:\r\n```python\r\n@hookimpl\r\ndef extra_template_vars(datasette, view_name, database):\r\n async def inner():\r\n if view_name == \"database\":\r\n return {\"blah\": 1}\r\n\r\n return inner\r\n```\r\nIt returns `None` in the case where the `if` condition does not match.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1557507274, "label": "`extra_template_vars` should be OK to return `None`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1404253358", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1404253358, "node_id": "IC_kwDOBm6k_c5TszSu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-25T21:35:32Z", "updated_at": "2023-01-25T21:35:32Z", "author_association": "OWNER", "body": "This issue here would benefit from some kid of mechanism for returning just the HTML of the table itself, without any of the surrounding material. I'm not sure if that would make sense as an extra or not:\r\n- https://github.com/simonw/datasette-search-all/issues/17", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/203#issuecomment-1404070841", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/203", "id": 1404070841, "node_id": "IC_kwDOCGYnMM5TsGu5", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-25T18:47:18Z", "updated_at": "2023-01-25T18:47:18Z", "author_association": "CONTRIBUTOR", "body": "i'll adopt this PR to make the changes @simonw suggested https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567932", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743384829, "label": "changes to allow for compound foreign keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2003#issuecomment-1404065571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2003", "id": 1404065571, "node_id": "IC_kwDOBm6k_c5TsFcj", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-25T18:44:42Z", "updated_at": "2023-01-25T18:44:42Z", "author_association": "CONTRIBUTOR", "body": "see this related discussion to a change in API in sqlite-utils https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567932", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1555701851, "label": "Show referring tables and rows when the referring foreign key is compound"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2004#issuecomment-1403110269", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2004", "id": 1403110269, "node_id": "IC_kwDOBm6k_c5TocN9", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-25T05:18:54Z", "updated_at": "2023-01-25T05:18:54Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/2004?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.11**% // Head: **92.11**% // No change to project coverage :thumbsup:\n> Coverage data is based on head [(`dca7634`)](https://codecov.io/gh/simonw/datasette/pull/2004?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`e4ebef0`)](https://codecov.io/gh/simonw/datasette/commit/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch has no changes to coverable lines.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #2004 +/- ##\n=======================================\n Coverage 92.11% 92.11% \n=======================================\n Files 38 38 \n Lines 5555 5555 \n=======================================\n Hits 5117 5117 \n Misses 438 438 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/2004?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/utils/\\_\\_init\\_\\_.py](https://codecov.io/gh/simonw/datasette/pull/2004?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.86% <\u00f8> (\u00f8)` | |\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/2004?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1556065335, "label": "use single quotes for string literals, fixes #2001"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2001#issuecomment-1403084856", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2001", "id": 1403084856, "node_id": "IC_kwDOBm6k_c5ToWA4", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-25T04:31:02Z", "updated_at": "2023-01-25T04:31:02Z", "author_association": "CONTRIBUTOR", "body": "Aha, it's user error on my part.\r\n\r\nAdding\r\n\r\n```\r\nsqlite3_db_config.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int, ctypes.c_int]\r\n```\r\n\r\nmakes it work reliably both on the CLI and from datasette, and now I can reproduce the errors you mentioned in the issue description.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1553615704, "label": "Datasette is not compatible with SQLite's strict quoting compilation option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2001#issuecomment-1403078134", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2001", "id": 1403078134, "node_id": "IC_kwDOBm6k_c5ToUX2", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-25T04:20:43Z", "updated_at": "2023-01-25T04:22:28Z", "author_association": "CONTRIBUTOR", "body": "I'm on Ubuntu, unfortunately. :( Would it still be relevant?\r\n\r\nI think I've narrowed things down a bit more.\r\n\r\nEven `sqlite3_free(sqlite3_malloc(128))` segfaults -- this suggests to me that it's something about the sqlite3 library that was loaded, vs, say, getting the wrong db handle when I go spelunking in the Connection object.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1553615704, "label": "Datasette is not compatible with SQLite's strict quoting compilation option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2001#issuecomment-1403071122", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2001", "id": 1403071122, "node_id": "IC_kwDOBm6k_c5ToSqS", "user": {"value": 406380, "label": "gwk"}, "created_at": "2023-01-25T04:12:41Z", "updated_at": "2023-01-25T04:12:41Z", "author_association": "NONE", "body": "@cldellow glad to hear you tried it, as I got grossed out by my own suggestion ;) If you are on macOS I do have one trick for debugging segfaults using lldb.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1553615704, "label": "Datasette is not compatible with SQLite's strict quoting compilation option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2001#issuecomment-1403053144", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2001", "id": 1403053144, "node_id": "IC_kwDOBm6k_c5ToORY", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-25T03:34:53Z", "updated_at": "2023-01-25T03:34:53Z", "author_association": "CONTRIBUTOR", "body": "Your comment introduced me to this issue in sqlite and to the `ctypes` module - thanks!\r\n\r\n> I also hope that the datasette developers will enable this mode in a test environment [...]\r\n> perhaps we could figure out how to invoke it using `ctypes`\r\n\r\nI'm not a Datasette developer, but I _am_ curious to learn more about getting unholy access to the sqlite C APIs inside of Datasette. (Such access could also help #1293, and if done without grovelling inside of pysqlite's Connection object for the db handle, could even be relatively safe.)\r\n\r\nI experimented a bit. I came up with https://gist.github.com/cldellow/85bba507c314b127f85563869cd94820\r\n\r\nIf you run `python3 enable-strict-quoting-sqlite3.py`, it seems to set those flags correctly -- `SELECT \"foo\"` fails where it would normally succeed.\r\n\r\nBut if you put it in a `plugins/` dir and run `datasette --plugins-dir plugins/`, it segfaults when it tries to call `sqlite3_db_config` on the connections created by Datasette.\r\n\r\nI am... confused. I'm _pretty_ sure I'm using the same python and the same libsqlite3 in both scenarios, so I would expect it to work.\r\n\r\n@gwk do you know anything that might help me debug the segfault? I gather that my approach of going grovelling inside of a `PyObject` is particularly dangerous, but I was thinking (a) it's necessary in order to test Datasette's use of the sqlite3 library and (b) even if it's not portable, it'd be good enough for running the tests on a single machine.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1553615704, "label": "Datasette is not compatible with SQLite's strict quoting compilation option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1099#issuecomment-1402900354", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1099", "id": 1402900354, "node_id": "IC_kwDOBm6k_c5Tno-C", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-25T00:58:26Z", "updated_at": "2023-01-25T00:58:26Z", "author_association": "CONTRIBUTOR", "body": "> My original idea for compound foreign keys was to turn both of those columns into links, but that doesn't fit here because `database_name` is already part of a different foreign key.\r\n\r\nit's pretty hard to know what the right thing to do is if a field is part of multiple foreign keys. \r\n\r\nbut, if that's not the case, what about making each of the columns a link. seems like an improvement over the status quo.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743371103, "label": "Support linking to compound foreign keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1099#issuecomment-1402898291", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1099", "id": 1402898291, "node_id": "IC_kwDOBm6k_c5Tnodz", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-25T00:55:06Z", "updated_at": "2023-01-25T00:55:06Z", "author_association": "CONTRIBUTOR", "body": "I went ahead and spiked something together, in #2003 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743371103, "label": "Support linking to compound foreign keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2003#issuecomment-1402898033", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2003", "id": 1402898033, "node_id": "IC_kwDOBm6k_c5TnoZx", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-25T00:54:41Z", "updated_at": "2023-01-25T00:54:41Z", "author_association": "CONTRIBUTOR", "body": "@simonw, let me know what you think about this approach!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1555701851, "label": "Show referring tables and rows when the referring foreign key is compound"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2003#issuecomment-1402894191", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2003", "id": 1402894191, "node_id": "IC_kwDOBm6k_c5Tnndv", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-25T00:49:23Z", "updated_at": "2023-01-25T00:49:23Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.11**% // Head: **92.12**% // Increases project coverage by **`+0.01%`** :tada:\n> Coverage data is based on head [(`1e5b42f`)](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`e4ebef0`)](https://codecov.io/gh/simonw/datasette/commit/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch coverage: 100.00% of modified lines in pull request are covered.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #2003 +/- ##\n==========================================\n+ Coverage 92.11% 92.12% +0.01% \n==========================================\n Files 38 38 \n Lines 5555 5565 +10 \n==========================================\n+ Hits 5117 5127 +10 \n Misses 438 438 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.50% <100.00%> (+0.01%)` | :arrow_up: |\n| [datasette/filters.py](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2ZpbHRlcnMucHk=) | `95.73% <100.00%> (+0.04%)` | :arrow_up: |\n| [datasette/utils/\\_\\_init\\_\\_.py](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.89% <100.00%> (+0.02%)` | :arrow_up: |\n| [datasette/views/row.py](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3Jvdy5weQ==) | `87.82% <100.00%> (\u00f8)` | |\n| [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `92.61% <100.00%> (+0.03%)` | :arrow_up: |\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/2003?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1555701851, "label": "Show referring tables and rows when the referring foreign key is compound"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1099#issuecomment-1402563930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1099", "id": 1402563930, "node_id": "IC_kwDOBm6k_c5TmW1a", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-24T20:11:11Z", "updated_at": "2023-01-24T20:11:11Z", "author_association": "CONTRIBUTOR", "body": "hi @simonw, this bug bit me today.\r\n\r\nthe UX for linking from a table to the foreign key seems tough! \r\n\r\nthe design in the other direction seems a lot easier, for a given primary key detail page, add links back to the tables that refer to the row.\r\n\r\nwould you be open to a PR that solved the second problem but not the first?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743371103, "label": "Support linking to compound foreign keys"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1989#issuecomment-1402347667", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1989", "id": 1402347667, "node_id": "IC_kwDOBm6k_c5TliCT", "user": {"value": 116795, "label": "pax"}, "created_at": "2023-01-24T17:48:59Z", "updated_at": "2023-01-24T17:48:59Z", "author_association": "NONE", "body": "The problem (in my particular use case) with using a VIEW is that I'd need one of the columns to be searchable \u2013 but that ([enable-fts](https://github.com/simonw/datasette-search-all)) doesn't work with views :/\r\n\r\n__\r\nside-suggestion: I don't know how feasible this might be, but when one column (or table) would be marked as hidden, could the _Download SQLite DB_ link take that into account? \ud83e\uddd0", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1531991339, "label": "Suggestion: Hiding columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1994#issuecomment-1399957897", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1994", "id": 1399957897, "node_id": "IC_kwDOBm6k_c5TcamJ", "user": {"value": 201897, "label": "julienma"}, "created_at": "2023-01-23T08:21:08Z", "updated_at": "2023-01-23T08:21:08Z", "author_association": "NONE", "body": "Me too, on a M1. Not sure if it's compatible?", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 1, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1536851861, "label": "Stuck on loading screen"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2000#issuecomment-1399847946", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2000", "id": 1399847946, "node_id": "IC_kwDOBm6k_c5Tb_wK", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-23T06:08:00Z", "updated_at": "2023-01-23T06:08:00Z", "author_association": "CONTRIBUTOR", "body": "Actually, I discovered [your post](https://til.simonwillison.net/datasette/register-new-plugin-hooks) showing how a plugin can add a Datasette hook. That's wild! I've released `datasette-rewrite-sql` that adds this ability, albeit via monkey patching.\r\n\r\nI had hoped to be able to expose `request` to the hook (or, even better `actor`) when the SQL was being run as a result of a user's HTTP request.\r\n\r\nBut some spelunking in the code makes me suspect that would actually require co-operation from Datasette itself. I'd be happy to be wrong and pointed in the right direction, though!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1552368054, "label": "rewrite_sql hook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1159#issuecomment-1399606104", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1159", "id": 1399606104, "node_id": "IC_kwDOBm6k_c5TbEtY", "user": {"value": 552629, "label": "lovasoa"}, "created_at": "2023-01-22T20:58:10Z", "updated_at": "2023-01-22T20:58:10Z", "author_association": "NONE", "body": "great initiative, @cldellow :+1: \r\n\r\n@simonw, if you want to merge this, that would still be welcome :)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 774332247, "label": "Improve the display of facets information"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1159#issuecomment-1399589414", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1159", "id": 1399589414, "node_id": "IC_kwDOBm6k_c5TbAom", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-22T19:48:41Z", "updated_at": "2023-01-22T19:48:41Z", "author_association": "CONTRIBUTOR", "body": "Hey @lovasoa, I hope you don't mind - I pulled this PR into [datasette-ui-extras](https://github.com/cldellow/datasette-ui-extras), a plugin I'm making that collects UI tweaks to Datasette.\r\n\r\nYou can apply it to your own Datasette instance by running `datasette install datasette-ui-extras`", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 774332247, "label": "Improve the display of facets information"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399341658", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399341658, "node_id": "IC_kwDOBm6k_c5TaEJa", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:06:29Z", "updated_at": "2023-01-21T22:07:30Z", "author_association": "OWNER", "body": "Relevant:\r\n- #1101\r\n- #1672\r\n- #1062", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1101#issuecomment-1399341761", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1101", "id": 1399341761, "node_id": "IC_kwDOBm6k_c5TaELB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:07:19Z", "updated_at": "2023-01-21T22:07:19Z", "author_association": "OWNER", "body": "Idea for supporting streaming with the `register_output_renderer` hook:\r\n\r\n```python\r\n@hookimpl\r\ndef register_output_renderer(datasette):\r\n return {\r\n \"extension\": \"test\",\r\n \"render\": render_demo,\r\n \"can_render\": can_render_demo,\r\n \"render_stream\": render_demo_stream, # This is new\r\n }\r\n```\r\nSo there's a new `\"render_stream\"` key which can be returned, which if present means that the output renderer supports streaming.\r\n\r\nI'll play around with the design of that function signature in:\r\n\r\n- #1999\r\n- #1062 ", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 749283032, "label": "register_output_renderer() should support streaming data"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399341151", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399341151, "node_id": "IC_kwDOBm6k_c5TaEBf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:03:20Z", "updated_at": "2023-01-21T22:03:20Z", "author_association": "OWNER", "body": "I think I'm going to have to write a new view function from scratch which completely ignores the existing BaseView/DataView/TableView hierarchy.\r\n\r\nHere's what I get on the incoming request:\r\n```\r\n(Pdb) request.url, request.full_path, request.host, request.url_vars\r\n('http://127.0.0.1:8001/content/repos.json', '/content/repos.json', '127.0.0.1:8001',\r\n {'database': 'content', 'table': 'repos', 'format': 'json'})\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1399184642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1399184642, "node_id": "IC_kwDOBm6k_c5TZd0C", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T05:36:22Z", "updated_at": "2023-01-21T05:41:06Z", "author_association": "OWNER", "body": "Maybe `\"rows\"` should be a default `?_extra=`... but it should be possible to request `\"arrays\"` instead which would be a list of arrays, more suitable perhaps for custom renderers such as the CSV one.\r\n\r\nThis could be quite neat, in that EVERY key in the JSON representation would be defined as an extra - just some would be on by default. There could even be a mechanism for turning them back off again, maybe using `?_extra=-rows`.\r\n\r\nIn which case maybe `?_extra=` isn't actually the right name for this feature. It could be `?_key=` perhaps, or `?_field=`.\r\n\r\nBeing able to pass `?_field=count,-rows` to get back just the count (and skip executing the count entirely) would be pretty neat.\r\n\r\nAlthough `?_only=count` would be tidier. So maybe the pair of `?_only=` and `?_extra=` would make sense.\r\n\r\nWould `?_only=rows` still return the `\"ok\"` field so you can always look at that to confirm an error didn't occur?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1399184540", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1399184540, "node_id": "IC_kwDOBm6k_c5TZdyc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T05:35:32Z", "updated_at": "2023-01-21T05:35:32Z", "author_association": "OWNER", "body": "It's annoying that the https://docs.datasette.io/en/0.64.1/plugin_hooks.html#register-output-renderer-datasette plugin hook passes `rows` as \"list of sqlite3.Row objects\" - I'd prefer it if that plugin hook worked with JSON data, not `sqlite3.Row`.\r\n\r\nhttps://docs.datasette.io/en/0.64.1/plugin_hooks.html#render-cell-row-value-column-table-database-datasette is documented as accepting `Row` but actually gets `CustomRow`, see:\r\n\r\n- #1973", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1399178823", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1399178823, "node_id": "IC_kwDOBm6k_c5TZcZH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T04:54:49Z", "updated_at": "2023-01-21T04:54:49Z", "author_association": "OWNER", "body": "I pushed my prototype so far, going to start a draft PR for it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1399178591", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1399178591, "node_id": "IC_kwDOBm6k_c5TZcVf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T04:53:15Z", "updated_at": "2023-01-21T04:53:15Z", "author_association": "OWNER", "body": "Implementing this to work with the `.json` extension is going to be a lot harder.\r\n\r\nThe challenge here is that we're working with the whole `BaseView()` v.s. `TableView()` abstraction, which I've been wanting to get rid of for a long time.\r\n\r\n`BaseView()` calls `.data()` and expects to get back a `(data, extra_template_data, templates)` tuple - then if a format is in play (`.json` or `.geojson` or similar from a plugin) it hands off `data` to that. If `.csv` is involved it does something special, in order to support streaming responses. And if it's regular HTML it calls `await extra_template_data()` and combines that with `data` and passes it to the template.\r\n\r\nI want this to work completely differently: I want the formats (including HTML) to have the option of adding some extra `?_extra=` extras, then I want HTML to be able to render the page entirely from the JSON if necessary.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1399145981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1399145981, "node_id": "IC_kwDOBm6k_c5TZUX9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T01:56:52Z", "updated_at": "2023-01-21T01:56:52Z", "author_association": "OWNER", "body": "Got first prototype working using `asyncinject` and it's pretty nice:\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex ad45ecd3..c8690b22 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -2,6 +2,7 @@ import asyncio\r\n import itertools\r\n import json\r\n \r\n+from asyncinject import Registry\r\n import markupsafe\r\n \r\n from datasette.plugins import pm\r\n@@ -538,57 +539,60 @@ class TableView(DataView):\r\n # Execute the main query!\r\n results = await db.execute(sql, params, truncate=True, **extra_args)\r\n \r\n- # Calculate the total count for this query\r\n- count = None\r\n- if (\r\n- not db.is_mutable\r\n- and self.ds.inspect_data\r\n- and count_sql == f\"select count(*) from {table_name} \"\r\n- ):\r\n- # We can use a previously cached table row count\r\n- try:\r\n- count = self.ds.inspect_data[database_name][\"tables\"][table_name][\r\n- \"count\"\r\n- ]\r\n- except KeyError:\r\n- pass\r\n-\r\n- # Otherwise run a select count(*) ...\r\n- if count_sql and count is None and not nocount:\r\n- try:\r\n- count_rows = list(await db.execute(count_sql, from_sql_params))\r\n- count = count_rows[0][0]\r\n- except QueryInterrupted:\r\n- pass\r\n-\r\n- # Faceting\r\n- if not self.ds.setting(\"allow_facet\") and any(\r\n- arg.startswith(\"_facet\") for arg in request.args\r\n- ):\r\n- raise BadRequest(\"_facet= is not allowed\")\r\n+ # Resolve extras\r\n+ extras = _get_extras(request)\r\n+ if request.args.getlist(\"_facet\"):\r\n+ extras.add(\"facet_results\")\r\n \r\n- # pylint: disable=no-member\r\n- facet_classes = list(\r\n- itertools.chain.from_iterable(pm.hook.register_facet_classes())\r\n- )\r\n- facet_results = {}\r\n- facets_timed_out = []\r\n- facet_instances = []\r\n- for klass in facet_classes:\r\n- facet_instances.append(\r\n- klass(\r\n- self.ds,\r\n- request,\r\n- database_name,\r\n- sql=sql_no_order_no_limit,\r\n- params=params,\r\n- table=table_name,\r\n- metadata=table_metadata,\r\n- row_count=count,\r\n- )\r\n+ async def extra_count():\r\n+ # Calculate the total count for this query\r\n+ count = None\r\n+ if (\r\n+ not db.is_mutable\r\n+ and self.ds.inspect_data\r\n+ and count_sql == f\"select count(*) from {table_name} \"\r\n+ ):\r\n+ # We can use a previously cached table row count\r\n+ try:\r\n+ count = self.ds.inspect_data[database_name][\"tables\"][table_name][\r\n+ \"count\"\r\n+ ]\r\n+ except KeyError:\r\n+ pass\r\n+\r\n+ # Otherwise run a select count(*) ...\r\n+ if count_sql and count is None and not nocount:\r\n+ try:\r\n+ count_rows = list(await db.execute(count_sql, from_sql_params))\r\n+ count = count_rows[0][0]\r\n+ except QueryInterrupted:\r\n+ pass\r\n+ return count\r\n+\r\n+ async def facet_instances(extra_count):\r\n+ facet_instances = []\r\n+ facet_classes = list(\r\n+ itertools.chain.from_iterable(pm.hook.register_facet_classes())\r\n )\r\n+ for facet_class in facet_classes:\r\n+ facet_instances.append(\r\n+ facet_class(\r\n+ self.ds,\r\n+ request,\r\n+ database_name,\r\n+ sql=sql_no_order_no_limit,\r\n+ params=params,\r\n+ table=table_name,\r\n+ metadata=table_metadata,\r\n+ row_count=extra_count,\r\n+ )\r\n+ )\r\n+ return facet_instances\r\n+\r\n+ async def extra_facet_results(facet_instances):\r\n+ facet_results = {}\r\n+ facets_timed_out = []\r\n \r\n- async def execute_facets():\r\n if not nofacet:\r\n # Run them in parallel\r\n facet_awaitables = [facet.facet_results() for facet in facet_instances]\r\n@@ -607,9 +611,13 @@ class TableView(DataView):\r\n facet_results[key] = facet_info\r\n facets_timed_out.extend(instance_facets_timed_out)\r\n \r\n- suggested_facets = []\r\n+ return {\r\n+ \"results\": facet_results,\r\n+ \"timed_out\": facets_timed_out,\r\n+ }\r\n \r\n- async def execute_suggested_facets():\r\n+ async def extra_suggested_facets(facet_instances):\r\n+ suggested_facets = []\r\n # Calculate suggested facets\r\n if (\r\n self.ds.setting(\"suggest_facets\")\r\n@@ -624,8 +632,15 @@ class TableView(DataView):\r\n ]\r\n for suggest_result in await gather(*facet_suggest_awaitables):\r\n suggested_facets.extend(suggest_result)\r\n+ return suggested_facets\r\n+\r\n+ # Faceting\r\n+ if not self.ds.setting(\"allow_facet\") and any(\r\n+ arg.startswith(\"_facet\") for arg in request.args\r\n+ ):\r\n+ raise BadRequest(\"_facet= is not allowed\")\r\n \r\n- await gather(execute_facets(), execute_suggested_facets())\r\n+ # pylint: disable=no-member\r\n \r\n # Figure out columns and rows for the query\r\n columns = [r[0] for r in results.description]\r\n@@ -732,17 +747,56 @@ class TableView(DataView):\r\n rows = rows[:page_size]\r\n \r\n # human_description_en combines filters AND search, if provided\r\n- human_description_en = filters.human_description_en(\r\n- extra=extra_human_descriptions\r\n- )\r\n+ async def extra_human_description_en():\r\n+ human_description_en = filters.human_description_en(\r\n+ extra=extra_human_descriptions\r\n+ )\r\n+ if sort or sort_desc:\r\n+ human_description_en = \" \".join(\r\n+ [b for b in [human_description_en, sorted_by] if b]\r\n+ )\r\n+ return human_description_en\r\n \r\n if sort or sort_desc:\r\n sorted_by = \"sorted by {}{}\".format(\r\n (sort or sort_desc), \" descending\" if sort_desc else \"\"\r\n )\r\n- human_description_en = \" \".join(\r\n- [b for b in [human_description_en, sorted_by] if b]\r\n- )\r\n+\r\n+ async def extra_next_url():\r\n+ return next_url\r\n+\r\n+ async def extra_columns():\r\n+ return columns\r\n+\r\n+ async def extra_primary_keys():\r\n+ return pks\r\n+\r\n+ registry = Registry(\r\n+ extra_count,\r\n+ extra_facet_results,\r\n+ extra_suggested_facets,\r\n+ facet_instances,\r\n+ extra_human_description_en,\r\n+ extra_next_url,\r\n+ extra_columns,\r\n+ extra_primary_keys,\r\n+ )\r\n+\r\n+ results = await registry.resolve_multi(\r\n+ [\"extra_{}\".format(extra) for extra in extras]\r\n+ )\r\n+ data = {\r\n+ \"ok\": True,\r\n+ \"rows\": rows[:page_size],\r\n+ \"next\": next_value and str(next_value) or None,\r\n+ }\r\n+ data.update({\r\n+ key.replace(\"extra_\", \"\"): value\r\n+ for key, value in results.items()\r\n+ if key.startswith(\"extra_\")\r\n+ and key.replace(\"extra_\", \"\") in extras\r\n+ })\r\n+ return Response.json(data, default=repr)\r\n \r\n async def extra_template():\r\n nonlocal sort\r\n@@ -1334,3 +1388,11 @@ class TableDropView(BaseView):\r\n \r\n await db.execute_write_fn(drop_table)\r\n return Response.json({\"ok\": True}, status=200)\r\n+\r\n+\r\n+def _get_extras(request):\r\n+ extra_bits = request.args.getlist(\"_extra\")\r\n+ extras = set()\r\n+ for bit in extra_bits:\r\n+ extras.update(bit.split(\",\"))\r\n+ return extras\r\n```\r\nWith that in place, `http://127.0.0.1:8001/content/releases?author=25778&_size=1&_extra=count,primary_keys,columns&_facet=author` returns:\r\n```json\r\n{\r\n \"ok\": true,\r\n \"rows\": [\r\n {\r\n \"html_url\": \"https://github.com/eyeseast/geocode-sqlite/releases/tag/0.1.2\",\r\n \"id\": 30926270,\r\n \"author\": {\r\n \"value\": 25778,\r\n \"label\": \"eyeseast\"\r\n },\r\n \"node_id\": \"MDc6UmVsZWFzZTMwOTI2Mjcw\",\r\n \"tag_name\": \"0.1.2\",\r\n \"target_commitish\": \"master\",\r\n \"name\": \"v0.1.2\",\r\n \"draft\": 0,\r\n \"prerelease\": 1,\r\n \"created_at\": \"2020-09-08T17:48:24Z\",\r\n \"published_at\": \"2020-09-08T17:50:15Z\",\r\n \"body\": \"Basic API is in place, with CLI support for Google, Bing, MapQuest and Nominatum (OSM) geocoders.\",\r\n \"repo\": {\r\n \"value\": 293361514,\r\n \"label\": \"geocode-sqlite\"\r\n },\r\n \"reactions\": null,\r\n \"mentions_count\": null\r\n }\r\n ],\r\n \"next\": \"30926270\",\r\n \"primary_keys\": [\r\n \"id\"\r\n ],\r\n \"columns\": [\r\n \"html_url\",\r\n \"id\",\r\n \"author\",\r\n \"node_id\",\r\n \"tag_name\",\r\n \"target_commitish\",\r\n \"name\",\r\n \"draft\",\r\n \"prerelease\",\r\n \"created_at\",\r\n \"published_at\",\r\n \"body\",\r\n \"repo\",\r\n \"reactions\",\r\n \"mentions_count\"\r\n ],\r\n \"count\": 25,\r\n \"facet_results\": {\r\n \"results\": {\r\n \"author\": {\r\n \"name\": \"author\",\r\n \"type\": \"column\",\r\n \"hideable\": true,\r\n \"toggle_url\": \"/content/releases?author=25778&_size=1&_extra=count%2Cprimary_keys%2Ccolumns\",\r\n \"results\": [\r\n {\r\n \"value\": 25778,\r\n \"label\": \"eyeseast\",\r\n \"count\": 25,\r\n \"toggle_url\": \"http://127.0.0.1:8001/content/releases?_size=1&_extra=count%2Cprimary_keys%2Ccolumns&_facet=author\",\r\n \"selected\": true\r\n }\r\n ],\r\n \"truncated\": false\r\n }\r\n },\r\n \"timed_out\": []\r\n }\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1998#issuecomment-1398768399", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1998", "id": 1398768399, "node_id": "IC_kwDOBm6k_c5TX4MP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-20T18:19:06Z", "updated_at": "2023-01-20T18:19:06Z", "author_association": "OWNER", "body": "Simplest solution would be to ditch the `version_option` decorator and roll a custom option based on it instead, imitating what this code does:\r\n\r\nhttps://github.com/pallets/click/blob/7586834cab38c5592d9d6de3ee0ebe75d4353bfb/src/click/decorators.py#L413-L524", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551113681, "label": "`datasette --version` should also show the SQLite version"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1998#issuecomment-1398767813", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1998", "id": 1398767813, "node_id": "IC_kwDOBm6k_c5TX4DF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-20T18:18:27Z", "updated_at": "2023-01-20T18:18:27Z", "author_association": "OWNER", "body": "Fell down a bit of a rabbit hole trying to figure out how to get Click's `version_option()` to evaluate a custom message. Got this far:\r\n\r\n```python\r\nclass _VersionMessage(UserString):\r\n @property\r\n def data(self):\r\n return \"%(prog)s, version %(version)s (SQLite {})\".format(\r\n sqlite3.connect(\":memory:\").execute(\"select sqlite_version()\").fetchone()[0]\r\n )\r\n\r\n @data.setter\r\n def data(self, value):\r\n pass\r\n\r\n\r\n@click.group(cls=DefaultGroup, default=\"serve\", default_if_no_args=True)\r\n@click.version_option(version=__version__, message=_VersionMessage(\"\"))\r\ndef cli():\r\n \"\"\"\r\n Datasette is an open source multi-tool for exploring and publishing data\r\n\r\n \\b\r\n About Datasette: https://datasette.io/\r\n Full documentation: https://docs.datasette.io/\r\n \"\"\"\r\n```\r\nBut now:\r\n\r\n```\r\n% datasette --version\r\n%(prog)s, version %(version)s (SQLite 3.40.1)\r\n```\r\n\r\nI was trying to avoid running that `select sqlite_version()` thing unless the `--version` option was used.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551113681, "label": "`datasette --version` should also show the SQLite version"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1397942113", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1397942113, "node_id": "IC_kwDOBm6k_c5TUudh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-20T05:33:00Z", "updated_at": "2023-01-20T05:33:00Z", "author_association": "OWNER", "body": "I'm going to write code which parses `?_extra=` in the comma separated or multiple parameter format and then looks up functions in a dictionary.\r\n\r\nIt will return an error if you ask for an extra that does not exist.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1385807684", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1385807684, "node_id": "IC_kwDOBm6k_c5Smb9E", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-17T17:51:54Z", "updated_at": "2023-01-19T23:20:59Z", "author_association": "OWNER", "body": "In most cases, the `?_extra=xxx` name exactly corresponds to the additional key that is added to the JSON.\r\n\r\n`?_facet=...` is one example of a query string argument that causes an extra key - `\"facet_results\"` - to be added to the JSON even though it wasn't requested by name in a `?_extra=`.\r\n\r\nAm I OK with that? I think so.\r\n\r\nRelated issue:\r\n- #1558 \r\n\r\nActually there's an edge-case here that's worth considering: it's possible to use metadata to set default facets for a table. If you do this for a table, then `.json` for that table will always calculate and return those facets - which may be an expensive and unnecessary operation.\r\n\r\nSo maybe we don't include `facet_results` in the JSON unless explicitly asked for in that case, but have a rule that `?_facet` implies `?_extra=facet_results`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1989#issuecomment-1397734424", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1989", "id": 1397734424, "node_id": "IC_kwDOBm6k_c5TT7wY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-19T23:18:45Z", "updated_at": "2023-01-19T23:18:45Z", "author_association": "OWNER", "body": "There's a `?_nocol=x` argument you can use to hide a column when you link to a table, but that won't help you if you need to hide the column for privacy reasons: https://docs.datasette.io/en/latest/json_api.html#special-table-arguments\r\n\r\nOne solution right now is to define a SQL view for the things that you DO want people to be able to see, and then use Datasette's permission system to hide the tables (`'\"allow\": false` in metadata for each table) but show the views.\r\n\r\nIf you want to redact specific columns there's a plugin for doing that: https://datasette.io/plugins/datasette-mask-columns\r\n\r\nThis does make sense as more of a core Datasette feature though - tagging it as a feature suggestion.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1531991339, "label": "Suggestion: Hiding columns"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1996#issuecomment-1396933348", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1996", "id": 1396933348, "node_id": "IC_kwDOBm6k_c5TQ4Lk", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-19T12:55:57Z", "updated_at": "2023-01-19T12:55:57Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.11**% // Head: **92.11**% // Increases project coverage by **`+0.00%`** :tada:\n> Coverage data is based on head [(`96b8b7f`)](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`6a352e9`)](https://codecov.io/gh/simonw/datasette/commit/6a352e99ab988dbf8fd22a100049caa6ad33f1ec?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch coverage: 100.00% of modified lines in pull request are covered.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #1996 +/- ##\n=======================================\n Coverage 92.11% 92.11% \n=======================================\n Files 38 38 \n Lines 5555 5556 +1 \n=======================================\n+ Hits 5117 5118 +1 \n Misses 438 438 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/utils/\\_\\_init\\_\\_.py](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.87% <100.00%> (+<0.01%)` | :arrow_up: |\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1538342965, "label": "Document custom json encoder"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1497#issuecomment-1387433455", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1497", "id": 1387433455, "node_id": "IC_kwDOBm6k_c5Sso3v", "user": {"value": 270255, "label": "adamalton"}, "created_at": "2023-01-18T17:13:45Z", "updated_at": "2023-01-18T17:13:45Z", "author_association": "NONE", "body": "You may have just been talking to yourself here, but I found your documentation of the process incredibly useful when I hit the same error myself. Thanks! \ud83c\udf08 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1034535001, "label": "Publish to Docker Hub failing with \"libcrypt.so.1: cannot open shared object file\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1385805702", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1385805702, "node_id": "IC_kwDOBm6k_c5SmbeG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-17T17:50:17Z", "updated_at": "2023-01-17T17:50:17Z", "author_association": "OWNER", "body": "Or maybe have a `permissions` extra which includes `allow_execute_sql` and `private`? Could anything else go in there?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1384752452", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1384752452, "node_id": "IC_kwDOBm6k_c5SiaVE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-17T02:14:41Z", "updated_at": "2023-01-17T02:15:58Z", "author_association": "OWNER", "body": "Thinking about `?_extra=` values just for the table JSON. The default shape will look like this:\r\n\r\n```json\r\n{\r\n \"ok\": true,\r\n \"rows\": [{\"id\": 1, \"name\": \"Name\"}],\r\n \"next\": null,\r\n}\r\n```\r\nThe table extras could be:\r\n\r\n- `count` - adds a `\"count\"` field with a full `count(*)` for that filtered table\r\n- `next_url` - the full URL to the next page\r\n- `columns` - adds `\"columns\": [\"id\", \"name\"]`\r\n- `expandable_columns` - a list of columns that can be expanded (note that `\"expanded_columns\": [...]` shows up automatically if the user passes any `?_label=` options, like on https://latest.datasette.io/fixtures/facetable.json?_label=_city_id ) - I'm tempted to rename this to `label_columns` and have it add both `label_columns` and `label_columns_selected` or similar.\r\n- `primary_keys` - a list of primary keys e.g. `[\"id\"]` - not sure what to do about `rowid` columns here\r\n- `query` - a `{\"sql\": \"select ...\", \"params\": {\"p0\": \"1\"}}` object\r\n- `units` - the units feature\r\n- `suggested_facets` - suggested facets\r\n- `metadata` - a `{\"metadata\": {\"source_url\": \"...\"}}` etc block - differs from current in that it would be nested in `\"metadata\": {...}`.\r\n\r\nStuff currently in https://latest.datasette.io/fixtures/facetable.json that is not yet covered by the above:\r\n```\r\n \"database\": \"fixtures\",\r\n \"table\": \"facetable\",\r\n \"is_view\": false,\r\n \"human_description_en\": \"where id = 1\",\r\n \"private\": false,\r\n \"allow_execute_sql\": true,\r\n \"query_ms\": 16.749476999393664,\r\n```\r\nI'm tempted to bundle `database`, `table`, `is_view` and `human_description_en` into one (not sure what to call it though, perhaps `display_details`?) - and then drop `allow_execute_sql` entirely and have `private` and `query_ms` as their own named extras.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-1368269732", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 1368269732, "node_id": "IC_kwDOBm6k_c5RjiOk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:32:33Z", "updated_at": "2023-01-17T02:05:45Z", "author_association": "OWNER", "body": "New thinking on the trimmed-down default. Previously I was going to use `\"row\"` and `\"next_url\"` - I now want to do this instead:\r\n```json\r\n{\r\n \"ok\": true,\r\n \"rows\": [\r\n {\r\n \"pk1\": \"a\",\r\n \"pk2\": \"a\",\r\n \"pk3\": \"a\",\r\n \"content\": \"a-a-a\"\r\n },\r\n {\r\n \"pk1\": \"a\",\r\n \"pk2\": \"a\",\r\n \"pk3\": \"b\",\r\n \"content\": \"a-a-b\"\r\n }\r\n ],\r\n \"next\": \"a,a,b\"\r\n}\r\n```\r\nIf there isn't a next page it will return `\"next\": null`.\r\n\r\nThis is even more succinct. I'm OK with people having to request `next_url` if they don't want to construct the new URL themselves.\r\n\r\nThe `\"ok\": true` is there so it can be `false` for errors, consistently.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1384743243", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1384743243, "node_id": "IC_kwDOBm6k_c5SiYFL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-17T02:01:26Z", "updated_at": "2023-01-17T02:01:26Z", "author_association": "OWNER", "body": "I'm tempted NOT to document the JSON for the `/.json` page, simply because I'm not at all convinced that the current homepage design is the best possible use of that space - and I'd like to reserve the opportunity to redesign that in e.g. Datasette 1.1 without it being a breaking change to the documented JSON API.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1384742385", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1384742385, "node_id": "IC_kwDOBm6k_c5SiX3x", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-17T02:00:23Z", "updated_at": "2023-01-17T02:00:38Z", "author_association": "OWNER", "body": "I'm not actually too happy about how `/fixtures.json` currently entirely changes shape based on whether or not you pass a `?sql=` argument to it.\r\n\r\nMaybe I can fix that disparity with extras too?\r\n\r\nThe list of tables you see on `/fixtures.json` without the `?sql=` could become another extra. The HTML version of that page could know to request that extra by default.\r\n\r\nThis would also support running a SQL query but also returning a list of tables - which can be useful for building a SQL editor interface which hints at the tables that are available to the user - or even for generating the configuration needed by the CodeMirror editor's SQL completion, added in:\r\n- #1893", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1384741055", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1384741055, "node_id": "IC_kwDOBm6k_c5SiXi_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-17T01:58:24Z", "updated_at": "2023-01-17T01:58:24Z", "author_association": "OWNER", "body": "As suggested in this issue:\r\n- #1721 \r\n\r\nThere are three parts of the Datasette API that need to support extras:\r\n\r\n- Table, e.g. https://latest.datasette.io/fixtures/facetable.json\r\n- Row, e.g. https://latest.datasette.io/fixtures/facetable/1.json\r\n- Query, e.g. https://latest.datasette.io/fixtures/neighborhood_search.json or https://latest.datasette.io/fixtures.json?sql=%0Aselect+_neighborhood%2C+facet_cities.name%2C+state%0Afrom+facetable%0A++++join+facet_cities%0A++++++++on+facetable._city_id+%3D+facet_cities.id%0Awhere+_neighborhood+like+%27%25%27+||+%3Atext+||+%27%25%27%0Aorder+by+_neighborhood%3B%0A&text=\r\n\r\nThere are two other pages I should consider though:\r\n\r\n- https://latest.datasette.io/.json - the JSON version of the https://latest.datasette.io/ homepage\r\n- https://latest.datasette.io/fixtures.json - note that this is different from the same URL with `?sql=...` appended to it. This is the index of tables in a specific database", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-712988146", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 712988146, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjk4ODE0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-20T16:32:02Z", "updated_at": "2023-01-17T01:54:13Z", "author_association": "OWNER", "body": "Just realized I added an undocumented `?_extras=` option to the row view years ago and forgot about it - it's not even documented. Added in a30c5b220c15360d575e94b0e67f3255e120b916 - https://latest.datasette.io/fixtures/attraction_characteristic/2.json?_extras=foreign_key_tables\r\n\r\nThat will need to be made consistent with the new mechanism. I think `?_extra=a&_extra=b` is more consistent with other Datasette features (like `?_facet=col1&_facet=col2`) but potentially quite verbose.\r\n\r\nSo I could support `?_extra=a,b,c` as an alternative allowed syntax, or I could allow `?_extra=single` and `?_extras=comma,separated`.\r\n\r\nI think I prefer allowing commas in `?_extra=`.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1382655354", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31", "id": 1382655354, "node_id": "IC_kwDOD079W85SaaV6", "user": {"value": 2704860, "label": "fidiego"}, "created_at": "2023-01-14T04:08:36Z", "updated_at": "2023-01-14T04:08:36Z", "author_association": "NONE", "body": "I just tried this branch and saw some errors. \r\n\r\nI installed this PR locally with:\r\n\r\n```bash\r\npip install https://github.com/RhetTbull/dogsheep-photos/archive/update_for_bigsur.zip\r\n```\r\n\r\n
\r\nSystem Details\r\n\r\n**OS:** MacOS Monterey\r\n**Python Version:** Python 3.10.8\r\n\r\n
\r\n\r\n
\r\nStacktrace\r\n\r\n```python\r\nTraceback (most recent call last):\r\n File \"/Users/df/.venvs/photo-experiments/bin/dogsheep-photos\", line 8, in \r\n sys.exit(cli())\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1130, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1055, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1657, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 1404, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py\", line 760, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/dogsheep_photos/cli.py\", line 254, in apple_photos\r\n sha256 = calculate_hash(pathlib.Path(photo.path))\r\n File \"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py\", line 960, in __new__\r\n self = cls._from_parts(args)\r\n File \"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py\", line 594, in _from_parts\r\n drv, root, parts = self._parse_args(args)\r\n File \"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py\", line 578, in _parse_args\r\n a = os.fspath(a)\r\nTypeError: expected str, bytes or os.PathLike object, not NoneType\r\n```\r\n
", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 771511344, "label": "Update for Big Sur"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1988#issuecomment-1379502161", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1988", "id": 1379502161, "node_id": "IC_kwDOBm6k_c5SOYhR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T21:24:24Z", "updated_at": "2023-01-11T21:25:05Z", "author_association": "OWNER", "body": "Renaming `sql()` to `template_sql()` in https://github.com/simonw/datasette-template-sql and shipping a 2.0 release of that plugin (it is currently at 1.0.2) would be a sensible short-term fix too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529707837, "label": "Reconsider pattern where plugins could break existing template context"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1988#issuecomment-1379500131", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1988", "id": 1379500131, "node_id": "IC_kwDOBm6k_c5SOYBj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T21:22:31Z", "updated_at": "2023-01-11T21:22:31Z", "author_association": "OWNER", "body": "Relevant code:\r\n\r\nhttps://github.com/simonw/datasette/blob/6a352e99ab988dbf8fd22a100049caa6ad33f1ec/datasette/app.py#L1138-L1149\r\n\r\nhttps://github.com/simonw/datasette/blob/6a352e99ab988dbf8fd22a100049caa6ad33f1ec/datasette/app.py#L1193-L1195", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529707837, "label": "Reconsider pattern where plugins could break existing template context"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1988#issuecomment-1379492499", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1988", "id": 1379492499, "node_id": "IC_kwDOBm6k_c5SOWKT", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T21:15:01Z", "updated_at": "2023-01-11T21:20:14Z", "author_association": "OWNER", "body": "This is tricky. `datasette-template-sql` registers a custom `sql()` function, but @cldellow spotted a situation where this would interfere with an existing `{{ sql }}` variable in a template context!\r\n\r\nhttps://github.com/simonw/datasette-template-sql/blob/70483fd94bb0349761830e37c9ab928e8966841a/datasette_template_sql/__init__.py\r\n\r\n```python\r\n@hookimpl\r\ndef extra_template_vars(datasette):\r\n async def execute_sql(sql, args=None, database=None):\r\n db = datasette.get_database(database)\r\n return (await db.execute(sql, args)).rows\r\n\r\n return {\"sql\": execute_sql}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529707837, "label": "Reconsider pattern where plugins could break existing template context"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1988#issuecomment-1379495504", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1988", "id": 1379495504, "node_id": "IC_kwDOBm6k_c5SOW5Q", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T21:18:00Z", "updated_at": "2023-01-11T21:18:25Z", "author_association": "OWNER", "body": "A few options:\r\n- Ensure that the explicit template context overrides anything that plugins might do - so in this case the `sql(...)` function would not be available on that page. This would break people who have custom templates that use that function though.\r\n- Encourage naming conventions where functions and variables from plugins are less likely to interfere with existing functionality.\r\n- Completely change how templates work, so you never have a variable called `{{ sql }}` - it is always accessed via some parent object instead, such as `{{ page.sql }}`.\r\n\r\nThat last option actually fits quite well with my efforts to unify template rendering with JSON (and JSON extras) so it might be the best way to address this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529707837, "label": "Reconsider pattern where plugins could break existing template context"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1987#issuecomment-1379313564", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1987", "id": 1379313564, "node_id": "IC_kwDOBm6k_c5SNqec", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T18:29:25Z", "updated_at": "2023-01-11T18:29:25Z", "author_association": "OWNER", "body": "Fixed:\r\n- https://docs.datasette.io/en/latest/installation.html#using-pip\r\n- https://docs.datasette.io/en/stable/installation.html#using-pip\r\n- https://docs.datasette.io/en/0.64.1/installation.html#using-pip", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529452371, "label": "installpython3.com is now a spam website"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1987#issuecomment-1379311498", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1987", "id": 1379311498, "node_id": "IC_kwDOBm6k_c5SNp-K", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T18:27:30Z", "updated_at": "2023-01-11T18:27:46Z", "author_association": "OWNER", "body": "I'm going to leave the bad link in the old versions, because I can't figure out a way to update them using ReadTheDocs - shipping 0.64.1 will at least mean that both `/latest/` and `/stable/` have a good link.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529452371, "label": "installpython3.com is now a spam website"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1987#issuecomment-1379282756", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1987", "id": 1379282756, "node_id": "IC_kwDOBm6k_c5SNi9E", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T18:04:22Z", "updated_at": "2023-01-11T18:04:22Z", "author_association": "OWNER", "body": "That change is now live on https://docs.datasette.io/en/latest/installation.html#using-pip\r\n\r\nI may need to ship a new stable dot-release to update the `/en/stable/` page though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529452371, "label": "installpython3.com is now a spam website"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1987#issuecomment-1379276484", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1987", "id": 1379276484, "node_id": "IC_kwDOBm6k_c5SNhbE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T18:00:35Z", "updated_at": "2023-01-11T18:00:35Z", "author_association": "OWNER", "body": "Next challenge: how to best deploy this.\r\n\r\nIdeally I'd like to fix all of the older versions of the docs too - pages like https://docs.datasette.io/en/0.59.4/installation.html#using-pip\r\n\r\nNot sure if ReadTheDocs has a mechanism for that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1529452371, "label": "installpython3.com is now a spam website"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1985#issuecomment-1378253346", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1985", "id": 1378253346, "node_id": "IC_kwDOBm6k_c5SJnoi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-11T05:18:05Z", "updated_at": "2023-01-11T05:18:05Z", "author_association": "OWNER", "body": "`Datasette(path)` should either work or it should return an easily debugged error message.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1528448642, "label": "Don't let Datasette(path) without a list cause weird errors"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1982#issuecomment-1376620851", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1982", "id": 1376620851, "node_id": "IC_kwDOBm6k_c5SDZEz", "user": {"value": 49699333, "label": "dependabot[bot]"}, "created_at": "2023-01-10T02:03:18Z", "updated_at": "2023-01-10T02:03:18Z", "author_association": "CONTRIBUTOR", "body": "Looks like sphinx is up-to-date now, so this is no longer needed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1525560504, "label": "Bump sphinx from 5.3.0 to 6.1.2"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1984#issuecomment-1376619691", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1984", "id": 1376619691, "node_id": "IC_kwDOBm6k_c5SDYyr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-10T02:02:16Z", "updated_at": "2023-01-10T02:02:16Z", "author_association": "OWNER", "body": "Preview at https://datasette--1984.org.readthedocs.build/en/1984/ looks good to me.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1526635374, "label": "Upgrade Sphinx"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1983#issuecomment-1375810027", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1983", "id": 1375810027, "node_id": "IC_kwDOBm6k_c5SATHr", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2023-01-09T15:35:58Z", "updated_at": "2023-01-09T15:35:58Z", "author_association": "CONTRIBUTOR", "body": "Yes please, and thank you. I realized I was maybe getting myself in trouble using that, but I think it's a good way to standardize JSON handling.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1525815985, "label": "Make CustomJSONEncoder a documented public API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/979#issuecomment-1375801928", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/979", "id": 1375801928, "node_id": "IC_kwDOBm6k_c5SARJI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-09T15:32:22Z", "updated_at": "2023-01-09T15:32:22Z", "author_association": "OWNER", "body": "This will not be part of the new default JSON design. It will be handled by an ?_extra instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 710650633, "label": "Default table view JSON should include CREATE TABLE"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1558#issuecomment-1375800361", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1558", "id": 1375800361, "node_id": "IC_kwDOBm6k_c5SAQwp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-09T15:31:17Z", "updated_at": "2023-01-09T15:31:17Z", "author_association": "OWNER", "body": "Related:\r\n- #782 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1082584499, "label": "Redesign `facet_results` JSON structure prior to Datasette 1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1983#issuecomment-1375797856", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1983", "id": 1375797856, "node_id": "IC_kwDOBm6k_c5SAQJg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-09T15:29:34Z", "updated_at": "2023-01-09T15:29:34Z", "author_association": "OWNER", "body": "I had forgotten this had special handling for cursors!\r\n\r\nThe default representation of a row as a tuple rather than a dictionary might not be the right thing to do based on:\r\n- #782 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1525815985, "label": "Make CustomJSONEncoder a documented public API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1983#issuecomment-1375795441", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1983", "id": 1375795441, "node_id": "IC_kwDOBm6k_c5SAPjx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-09T15:27:53Z", "updated_at": "2023-01-09T15:27:53Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/blob/7b48664d751e59222c01c97a2739d53d6363bdda/datasette/utils/__init__.py#L163-L178", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1525815985, "label": "Make CustomJSONEncoder a documented public API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1978#issuecomment-1375708725", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1978", "id": 1375708725, "node_id": "IC_kwDOBm6k_c5R_6Y1", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2023-01-09T14:30:00Z", "updated_at": "2023-01-09T14:30:00Z", "author_association": "CONTRIBUTOR", "body": "Totally missed that issue. I can close this as a duplicate.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1522778923, "label": "Document datasette.urls.row and row_blob"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1977#issuecomment-1375596856", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1977", "id": 1375596856, "node_id": "IC_kwDOBm6k_c5R_fE4", "user": {"value": 49699333, "label": "dependabot[bot]"}, "created_at": "2023-01-09T13:06:14Z", "updated_at": "2023-01-09T13:06:14Z", "author_association": "CONTRIBUTOR", "body": "Superseded by #1982.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1522552817, "label": "Bump sphinx from 5.3.0 to 6.1.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1981#issuecomment-1375142205", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1981", "id": 1375142205, "node_id": "IC_kwDOBm6k_c5R9wE9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-09T06:05:44Z", "updated_at": "2023-01-09T06:05:44Z", "author_association": "OWNER", "body": "I want the labels to take as much space as they need, but line up with each other.\r\n\r\nIf they are more than half the width of the page then the form fields should break to another line.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524983536, "label": "Canned query field labels truncated"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1980#issuecomment-1375062405", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1980", "id": 1375062405, "node_id": "IC_kwDOBm6k_c5R9cmF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-09T03:23:53Z", "updated_at": "2023-01-09T03:23:53Z", "author_association": "OWNER", "body": "Some potential solutions:\r\n- Make it so the primary key is _always_ sortable - not a bad idea, the `sortable_columns` thing was designed to protect against sorting GIANT tables by a non-indexed column, and the primary key always has an index\r\n- Make sure that `\r\n \r\n \r\n```\r\nBut `id` is not a valid sort option any more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524867951, "label": "\"Cannot sort table by id\" when sortable_columns is used"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1978#issuecomment-1374916593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1978", "id": 1374916593, "node_id": "IC_kwDOBm6k_c5R84_x", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-08T20:06:16Z", "updated_at": "2023-01-08T20:06:16Z", "author_association": "OWNER", "body": "Related issue - I'm not 100% settled on the design for these yet:\r\n- https://github.com/simonw/datasette/issues/1048#issuecomment-1179757912", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1522778923, "label": "Document datasette.urls.row and row_blob"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1979#issuecomment-1374659874", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1979", "id": 1374659874, "node_id": "IC_kwDOBm6k_c5R76Ui", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-07T23:59:11Z", "updated_at": "2023-01-07T23:59:11Z", "author_association": "OWNER", "body": "I back-ported this fix to `0.63.x` as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524076587, "label": "More useful error message if enable_load_extension is not available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1979#issuecomment-1374659233", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1979", "id": 1374659233, "node_id": "IC_kwDOBm6k_c5R76Kh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-07T23:56:25Z", "updated_at": "2023-01-07T23:56:25Z", "author_association": "OWNER", "body": "```\r\n% datasette --load-extension foo \r\nError: Your Python installation does not have the ability to load SQLite extensions.\r\n\r\nMore information: https://datasette.io/help/extensions\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524076587, "label": "More useful error message if enable_load_extension is not available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1979#issuecomment-1374658468", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1979", "id": 1374658468, "node_id": "IC_kwDOBm6k_c5R75-k", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-07T23:53:41Z", "updated_at": "2023-01-07T23:53:41Z", "author_association": "OWNER", "body": "https://datasette.io/help/extensions now redirects to this new section of documentation: https://docs.datasette.io/en/latest/installation.html#a-note-about-extensions", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524076587, "label": "More useful error message if enable_load_extension is not available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1979#issuecomment-1374657204", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1979", "id": 1374657204, "node_id": "IC_kwDOBm6k_c5R75q0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-07T23:48:42Z", "updated_at": "2023-01-07T23:48:42Z", "author_association": "OWNER", "body": "Once I ship the next release I should change that `/help/extensions` link to go to `/stable/` and not `/latest/`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524076587, "label": "More useful error message if enable_load_extension is not available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1979#issuecomment-1374582375", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1979", "id": 1374582375, "node_id": "IC_kwDOBm6k_c5R7nZn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-07T19:22:39Z", "updated_at": "2023-01-07T19:22:39Z", "author_association": "OWNER", "body": "This helps:\r\n```diff\r\ndiff --git a/datasette/cli.py b/datasette/cli.py\r\nindex 2b61292b..ea98879c 100644\r\n--- a/datasette/cli.py\r\n+++ b/datasette/cli.py\r\n@@ -4,13 +4,15 @@ import click\r\n from click import formatting\r\n from click.types import CompositeParamType\r\n from click_default_group import DefaultGroup\r\n+import functools\r\n import json\r\n import os\r\n import pathlib\r\n import shutil\r\n from subprocess import call\r\n-import sys\r\n from runpy import run_module\r\n+import sys\r\n+import textwrap\r\n import webbrowser\r\n from .app import (\r\n OBSOLETE_SETTINGS,\r\n@@ -126,7 +128,7 @@ class Setting(CompositeParamType):\r\n \r\n \r\n def sqlite_extensions(fn):\r\n- return click.option(\r\n+ fn = click.option(\r\n \"sqlite_extensions\",\r\n \"--load-extension\",\r\n type=LoadExtension(),\r\n@@ -134,6 +136,25 @@ def sqlite_extensions(fn):\r\n multiple=True,\r\n help=\"Path to a SQLite extension to load, and optional entrypoint\",\r\n )(fn)\r\n+ # Wrap it in a custom error handler\r\n+ @functools.wraps(fn)\r\n+ def wrapped(*args, **kwargs):\r\n+ try:\r\n+ return fn(*args, **kwargs)\r\n+ except AttributeError as e:\r\n+ if \"enable_load_extension\" in str(e):\r\n+ raise click.ClickException(\r\n+ textwrap.dedent(\r\n+ \"\"\"\r\n+ Your Python installation does not have the ability to load SQLite extensions.\r\n+\r\n+ More information: https://docs.datasette.io/en/stable/installation.html#extensions\r\n+ \"\"\"\r\n+ ).strip()\r\n+ )\r\n+ raise\r\n+\r\n+ return wrapped\r\n \r\n \r\n @click.group(cls=DefaultGroup, default=\"serve\", default_if_no_args=True)\r\n```\r\nNeed to write help for that to link to.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1524076587, "label": "More useful error message if enable_load_extension is not available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1409#issuecomment-1374555507", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1409", "id": 1374555507, "node_id": "IC_kwDOBm6k_c5R7g1z", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-07T18:06:01Z", "updated_at": "2023-01-07T18:06:01Z", "author_association": "OWNER", "body": "Documentation:\r\n- https://docs.datasette.io/en/latest/authentication.html#controlling-the-ability-to-execute-arbitrary-sql\r\n- https://docs.datasette.io/en/latest/settings.html#setting-default-allow-sql\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 957310278, "label": "`default_allow_sql` setting (a re-imagining of the old `allow_sql` setting)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1977#issuecomment-1373600402", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1977", "id": 1373600402, "node_id": "IC_kwDOBm6k_c5R33qS", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-06T13:09:51Z", "updated_at": "2023-01-06T13:09:51Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1977?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.15**% // Head: **92.15**% // No change to project coverage :thumbsup:\n> Coverage data is based on head [(`a4eebee`)](https://codecov.io/gh/simonw/datasette/pull/1977?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`c41278b`)](https://codecov.io/gh/simonw/datasette/commit/c41278b46f7936b4b1a8a14bf285bed82c81c609?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch has no changes to coverable lines.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #1977 +/- ##\n=======================================\n Coverage 92.15% 92.15% \n=======================================\n Files 38 38 \n Lines 5542 5542 \n=======================================\n Hits 5107 5107 \n Misses 435 435 \n```\n\n\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1977?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1522552817, "label": "Bump sphinx from 5.3.0 to 6.1.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1976#issuecomment-1373592231", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1976", "id": 1373592231, "node_id": "IC_kwDOBm6k_c5R31qn", "user": {"value": 49699333, "label": "dependabot[bot]"}, "created_at": "2023-01-06T13:02:15Z", "updated_at": "2023-01-06T13:02:15Z", "author_association": "CONTRIBUTOR", "body": "Superseded by #1977.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1520712722, "label": "Bump sphinx from 5.3.0 to 6.1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1976#issuecomment-1372203113", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1976", "id": 1372203113, "node_id": "IC_kwDOBm6k_c5Ryihp", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-05T13:15:04Z", "updated_at": "2023-01-05T13:15:04Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1976?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.15**% // Head: **92.15**% // No change to project coverage :thumbsup:\n> Coverage data is based on head [(`af76abc`)](https://codecov.io/gh/simonw/datasette/pull/1976?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`c41278b`)](https://codecov.io/gh/simonw/datasette/commit/c41278b46f7936b4b1a8a14bf285bed82c81c609?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch has no changes to coverable lines.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #1976 +/- ##\n=======================================\n Coverage 92.15% 92.15% \n=======================================\n Files 38 38 \n Lines 5542 5542 \n=======================================\n Hits 5107 5107 \n Misses 435 435 \n```\n\n\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1976?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1520712722, "label": "Bump sphinx from 5.3.0 to 6.1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1974#issuecomment-1372188571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1974", "id": 1372188571, "node_id": "IC_kwDOBm6k_c5Rye-b", "user": {"value": 49699333, "label": "dependabot[bot]"}, "created_at": "2023-01-05T13:02:40Z", "updated_at": "2023-01-05T13:02:40Z", "author_association": "CONTRIBUTOR", "body": "Superseded by #1976.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1516376583, "label": "Bump sphinx from 5.3.0 to 6.0.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-1370786026", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54", "id": 1370786026, "node_id": "IC_kwDODEm0Qs5RtIjq", "user": {"value": 6764957, "label": "swyxio"}, "created_at": "2023-01-04T11:06:44Z", "updated_at": "2023-01-04T11:06:44Z", "author_association": "NONE", "body": "as of 2023 it appears that `tweets: not yet implemented` is happening.. pretty important for a twitter export functionality!\r\n\r\nthis seems an impossible task with twitter liable to switch this around every other day", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779088071, "label": "Archive import appears to be broken on recent exports"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/520#issuecomment-1369333759", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/520", "id": 1369333759, "node_id": "IC_kwDOCGYnMM5Rnl__", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-03T02:23:43Z", "updated_at": "2023-01-03T02:23:43Z", "author_association": "OWNER", "body": "The documentation here does at least say the following: https://sqlite-utils.datasette.io/en/3.30/python-api.html#reading-rows-from-a-file\r\n\r\n> - **fp** (*BinaryIO*) -- a file-like object containing binary data", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1516644980, "label": "rows_from_file() raises confusing error if file-like object is not in binary mode"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1973#issuecomment-1369044959", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1973", "id": 1369044959, "node_id": "IC_kwDOBm6k_c5Rmfff", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-01-02T15:41:40Z", "updated_at": "2023-01-02T15:41:40Z", "author_association": "CONTRIBUTOR", "body": "Thanks for the response!\r\n\r\nYes, it does seem like a pretty nice developer experience--both the automagical labelling of fkeys, and the ability to index the row by column name in addition to column index.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515815014, "label": "render_cell plugin hook's row object is not a sqlite.Row"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1973#issuecomment-1368936146", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1973", "id": 1368936146, "node_id": "IC_kwDOBm6k_c5RmE7S", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-02T13:13:03Z", "updated_at": "2023-01-02T13:13:03Z", "author_association": "OWNER", "body": "It's quite handy that the label for foreign keys is made available to the custom render function here - if that was properly documented (and maybe had a slightly more intuitive API) I think it might be a good solution here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515815014, "label": "render_cell plugin hook's row object is not a sqlite.Row"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1974#issuecomment-1368935738", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1974", "id": 1368935738, "node_id": "IC_kwDOBm6k_c5RmE06", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-01-02T13:12:30Z", "updated_at": "2023-01-02T13:12:30Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1974?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nBase: **92.10**% // Head: **92.10**% // No change to project coverage :thumbsup:\n> Coverage data is based on head [(`1e213a3`)](https://codecov.io/gh/simonw/datasette/pull/1974?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`572bdb5`)](https://codecov.io/gh/simonw/datasette/commit/572bdb5b8095bbdc5ec2aa95400055e49ea028cb?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> Patch has no changes to coverable lines.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #1974 +/- ##\n=======================================\n Coverage 92.10% 92.10% \n=======================================\n Files 38 38 \n Lines 5538 5538 \n=======================================\n Hits 5101 5101 \n Misses 437 437 \n```\n\n\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1974?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1516376583, "label": "Bump sphinx from 5.3.0 to 6.0.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1973#issuecomment-1368935320", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1973", "id": 1368935320, "node_id": "IC_kwDOBm6k_c5RmEuY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-02T13:12:00Z", "updated_at": "2023-01-02T13:12:00Z", "author_association": "OWNER", "body": "Thanks for catching this!\r\n\r\nDefinitely a bug. This might be a good opportunity to reconsider how his should work though.\r\n\r\nIt's possible that Datasette will eventually support alternative database backends - PostgreSQL, DuckDB and ClickHouse are all on my radar here.\r\n\r\nIf that happens, this plugin hook returning a SQLite Row wouldn't make sense.\r\n\r\nSome kind of thing abstraction layer - maybe CustomRow, maybe something a little bit better - might be a good idea.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515815014, "label": "render_cell plugin hook's row object is not a sqlite.Row"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-1368285442", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 1368285442, "node_id": "IC_kwDOBm6k_c5RjmEC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T22:02:16Z", "updated_at": "2022-12-31T22:02:16Z", "author_association": "OWNER", "body": "https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_size=2 now returns `count`:\r\n\r\n```json\r\n{\r\n \"database\": \"fixtures\",\r\n \"table\": \"compound_three_primary_keys\",\r\n \"is_view\": false,\r\n \"human_description_en\": \"\",\r\n \"rows\": [\r\n {\r\n \"pk1\": \"a\",\r\n \"pk2\": \"a\",\r\n \"pk3\": \"a\",\r\n \"content\": \"a-a-a\"\r\n },\r\n {\r\n \"pk1\": \"a\",\r\n \"pk2\": \"a\",\r\n \"pk3\": \"b\",\r\n \"content\": \"a-a-b\"\r\n }\r\n ],\r\n \"truncated\": false,\r\n \"count\": 1001,\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-1368278278", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 1368278278, "node_id": "IC_kwDOBm6k_c5RjkUG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T20:49:38Z", "updated_at": "2022-12-31T20:49:38Z", "author_association": "OWNER", "body": "I'm going to rename `filtered_table_rows_count` to `count` - to match the SQL `count(*)` function.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-1368269811", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 1368269811, "node_id": "IC_kwDOBm6k_c5RjiPz", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:33:09Z", "updated_at": "2022-12-31T19:33:09Z", "author_association": "OWNER", "body": "Here's the so-far updated documentation for this change: https://github.com/simonw/datasette/blob/a2dca62360ad4a961d4c46f68eae41b7d5c7b2c9/docs/json_api.rst#different-shapes", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-1368269283", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 1368269283, "node_id": "IC_kwDOBm6k_c5RjiHj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:29:45Z", "updated_at": "2022-12-31T19:29:45Z", "author_association": "OWNER", "body": "https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_size=2 now shows the new default:\r\n```json\r\n{\r\n \"database\": \"fixtures\",\r\n \"table\": \"compound_three_primary_keys\",\r\n \"is_view\": false,\r\n \"human_description_en\": \"\",\r\n \"rows\": [\r\n {\r\n \"pk1\": \"a\",\r\n \"pk2\": \"a\",\r\n \"pk3\": \"a\",\r\n \"content\": \"a-a-a\"\r\n },\r\n {\r\n \"pk1\": \"a\",\r\n \"pk2\": \"a\",\r\n \"pk3\": \"b\",\r\n \"content\": \"a-a-b\"\r\n }\r\n ],\r\n```\r\nThe old format can be had like this: https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_size=2&_shape=arrays", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1970#issuecomment-1368268233", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1970", "id": 1368268233, "node_id": "IC_kwDOBm6k_c5Rjh3J", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:22:58Z", "updated_at": "2022-12-31T19:22:58Z", "author_association": "OWNER", "body": "https://latest.datasette.io/_internal/databases shows the fix now:\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515182998, "label": "Path \"None\" in _internal database table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-1368268148", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 1368268148, "node_id": "IC_kwDOBm6k_c5Rjh10", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:22:07Z", "updated_at": "2022-12-31T19:22:07Z", "author_association": "OWNER", "body": "It turned out the most significant part of this change - switching from an array of arrays to an array of objects for the `\"rows\"` key - was really easy: Datasette already had a `?_shape=arrays` v.s. `?_shape=objects` mechanism, so I switched which one was the default in https://github.com/simonw/datasette/commit/234230e59574ccb8d8a24c45ccd325f725812377", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1967#issuecomment-1368267484", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1967", "id": 1368267484, "node_id": "IC_kwDOBm6k_c5Rjhrc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:15:50Z", "updated_at": "2022-12-31T19:15:50Z", "author_association": "OWNER", "body": "My Firefox tab before:\r\n\r\n\"image\"\r\n\r\nAnd after:\r\n\r\n\"image\"\r\n", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1503010009, "label": "Add favicon to documentation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1966#issuecomment-1368267406", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1966", "id": 1368267406, "node_id": "IC_kwDOBm6k_c5RjhqO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:15:10Z", "updated_at": "2022-12-31T19:15:10Z", "author_association": "OWNER", "body": "It's working again now - not sure what happened there, I was paying for Heroku for that already.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1501900064, "label": "Broken link to live demo in Getting started docs"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1971#issuecomment-1368266904", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1971", "id": 1368266904, "node_id": "IC_kwDOBm6k_c5RjhiY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:09:48Z", "updated_at": "2022-12-31T19:09:48Z", "author_association": "OWNER", "body": "For the moment I'm going to pin to Sphinx 5.3.0\r\n\r\nNote that I got this warning there:\r\n\r\n```\r\n[sphinx-autobuild] > sphinx-build -b html /Users/simon/Dropbox/Development/datasette/docs /Users/simon/Dropbox/Development/datasette/docs/_build\r\nRunning Sphinx v5.3.0\r\nloading pickled environment... done\r\nWARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'.\r\n```\r\nSo I should probably fix that in advance of the 6.0 upgrade.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515185383, "label": "Upgrade for Sphinx 6.0 (once Furo has support for it)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1971#issuecomment-1368266658", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1971", "id": 1368266658, "node_id": "IC_kwDOBm6k_c5Rjhei", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:07:19Z", "updated_at": "2022-12-31T19:07:19Z", "author_association": "OWNER", "body": "So it looks like Furo doesn't work with Sphinx 6.0 yet - there' s a fix on `main` but it hasn't been released:\r\n\r\n- https://github.com/pradyunsg/furo/discussions/569", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515185383, "label": "Upgrade for Sphinx 6.0 (once Furo has support for it)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1971#issuecomment-1368266457", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1971", "id": 1368266457, "node_id": "IC_kwDOBm6k_c5RjhbZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-12-31T19:05:26Z", "updated_at": "2022-12-31T19:05:26Z", "author_association": "OWNER", "body": "Ran this locally:\r\n\r\n pip install -U Sphinx\r\n\r\nGot these warnings:\r\n\r\n```\r\nInstalling collected packages: Sphinx\r\n Attempting uninstall: Sphinx\r\n Found existing installation: Sphinx 5.1.1\r\n Uninstalling Sphinx-5.1.1:\r\n Successfully uninstalled Sphinx-5.1.1\r\nERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\nsphinx-basic-ng 0.0.1a12 requires sphinx<6.0,>=4.0, but you have sphinx 6.0.0 which is incompatible.\r\nfuro 2022.6.21 requires sphinx<6.0,>=4.0, but you have sphinx 6.0.0 which is incompatible.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1515185383, "label": "Upgrade for Sphinx 6.0 (once Furo has support for it)"}, "performed_via_github_app": null}