{"html_url": "https://github.com/simonw/datasette/issues/2033#issuecomment-1457117383", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2033", "id": 1457117383, "node_id": "IC_kwDOBm6k_c5W2djH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-06T22:28:55Z", "updated_at": "2023-03-06T22:28:55Z", "author_association": "OWNER", "body": "Documentation: https://docs.datasette.io/en/latest/plugins.html#installing-plugins", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1612296210, "label": "`datasette install -r requirements.txt`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2031#issuecomment-1456997425", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2031", "id": 1456997425, "node_id": "IC_kwDOBm6k_c5W2AQx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-06T21:04:27Z", "updated_at": "2023-03-06T21:06:34Z", "author_association": "OWNER", "body": "This is a very neat fix, for something I've been wanting for a while.\r\n\r\nAdd a unit test for the row HTML page - I suggest against this page: https://latest.datasette.io/fixtures/foreign_key_references/1 - and I'll land this PR.\r\n\r\nYou can model it on this test here: https://github.com/simonw/datasette/blob/a53b893c46453f35decc8c145c138671cee6140c/tests/test_table_html.py#L609-L632\r\n\r\nI think adding it to `test_table_html.py` is OK, even though it's technically for the row page and not the table page.\r\n\r\nThanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1605481359, "label": "Expand foreign key references in row view as well"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2028#issuecomment-1456925875", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2028", "id": 1456925875, "node_id": "IC_kwDOBm6k_c5W1uyz", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2023-03-06T20:26:53Z", "updated_at": "2023-03-06T20:26:53Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/2028?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\nPatch and project coverage have no change.\n> Comparison is base [(`0b4a286`)](https://codecov.io/gh/simonw/datasette/commit/0b4a28691468b5c758df74fa1d72a823813c96bf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.11% compared to head [(`a8dde13`)](https://codecov.io/gh/simonw/datasette/pull/2028?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.11%.\n\n
Additional details and impacted files\n\n\n```diff\n@@ Coverage Diff @@\n## main #2028 +/- ##\n=======================================\n Coverage 92.11% 92.11% \n=======================================\n Files 38 38 \n Lines 5555 5555 \n=======================================\n Hits 5117 5117 \n Misses 438 438 \n```\n\n\n\nHelp us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n
\n\n[:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/2028?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). \n:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1590839187, "label": "add Python 3.11 classifier"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2028#issuecomment-1456914694", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2028", "id": 1456914694, "node_id": "IC_kwDOBm6k_c5W1sEG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-06T20:19:37Z", "updated_at": "2023-03-06T20:19:37Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1590839187, "label": "add Python 3.11 classifier"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1619#issuecomment-1455196849", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1619", "id": 1455196849, "node_id": "IC_kwDOBm6k_c5WvIqx", "user": {"value": 969875, "label": "BryantD"}, "created_at": "2023-03-05T20:29:55Z", "updated_at": "2023-03-05T20:30:14Z", "author_association": "NONE", "body": "I have this same issue, which is happening with both json links and facets. It is not happening with column sort links in the gear popup menus, but it is happening with the sort arrow that results after you use one of those links. I'm using Apache as a proxy to Datasette; the relevant configs are:\r\n\r\n```\r\n ProxyPass /datasette/ http://127.0.0.1:8000/datasette/ nocanon\r\n ProxyPreserveHost on\r\n```\r\n\r\n```\r\n{\r\n\t\"base_url\": \"/datasette/\"\r\n}\r\n```\r\n\r\nIf it would be useful to get a look at the running installation via the Web, Simon, let me know.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1121583414, "label": "JSON link on row page is 404 if base_url setting is used"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1444474487", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433", "id": 1444474487, "node_id": "IC_kwDOCGYnMM5WGO53", "user": {"value": 167893, "label": "mcarpenter"}, "created_at": "2023-02-24T20:57:43Z", "updated_at": "2023-02-24T22:22:18Z", "author_association": "CONTRIBUTOR", "body": "I think I see what is happening here, although I haven't quite work out a fix yet. Usually:\r\n\r\n* `click.progressbar.render_progress()` renders the cursor invisible on each invocation (update of the bar)\r\n* When the progress bar goes out of scope, the `__exit()__` method is invoked, which calls `render_finish()` to make the cursor re-appear.\r\n\r\n(See terminal escape sequences `BEFORE_BAR` and `AFTER_BAR` in click).\r\n\r\nHowever the sqlite-utils `utils.file_progress` context manager wraps `click.progressbar` and yields an instance of a helper class:\r\n\r\n``` python\r\n@contextlib.contextmanager \r\ndef file_progress(file, silent=False, **kwargs):\r\n ...\r\n with click.progressbar(length=file_length, **kwargs) as bar:\r\n yield UpdateWrapper(file, bar.update) \r\n```\r\n\r\nThe yielded `UpdateWrapper` goes out of scope quickly and `click.progressbar.__exit__()` is called. The cursor is made un-invisible. Hoewever `bar` is still live and so when the caller iterates on the yielded wrapper this invokes the bar's update method, calling `render_progress()`, each time printing the \"make cursor invisible\" escape code. The `progressbar.__exit__` function is not called again, so the cursor doesn't re-appear.\r\n\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239034903, "label": "CLI eats my cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2030#issuecomment-1440854834", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2030", "id": 1440854834, "node_id": "IC_kwDOBm6k_c5V4bMy", "user": {"value": 19700859, "label": "gk7279"}, "created_at": "2023-02-22T21:54:39Z", "updated_at": "2023-02-22T21:54:39Z", "author_association": "NONE", "body": "Thanks @dmick . I chose to create a firewall rule under my GCP to open the port of interest and datasette works. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1594383280, "label": "How to use Datasette with apache webserver on GCP?"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2030#issuecomment-1440814680", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2030", "id": 1440814680, "node_id": "IC_kwDOBm6k_c5V4RZY", "user": {"value": 1350673, "label": "dmick"}, "created_at": "2023-02-22T21:22:42Z", "updated_at": "2023-02-22T21:22:42Z", "author_association": "NONE", "body": "@gk7279, you had asked in a separate bug about how to redirect web servers in general. The datasette docs actually have pretty good information on this for both nginx and apache2: https://docs.datasette.io/en/stable/deploying.html#running-datasette-behind-a-proxy\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1594383280, "label": "How to use Datasette with apache webserver on GCP?"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2027#issuecomment-1440811364", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2027", "id": 1440811364, "node_id": "IC_kwDOBm6k_c5V4Qlk", "user": {"value": 19700859, "label": "gk7279"}, "created_at": "2023-02-22T21:19:47Z", "updated_at": "2023-02-22T21:19:47Z", "author_association": "NONE", "body": "yes @dmick . How did you make your public IP redirect to your uvicorn server?\r\n\r\nInstead of nginx, I have apache2 on my GCP VM. Any pointers here are helpful too.\r\n\r\nThanks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1590183272, "label": "How to redirect from \"/\" to a specific db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2027#issuecomment-1440762383", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2027", "id": 1440762383, "node_id": "IC_kwDOBm6k_c5V4EoP", "user": {"value": 1350673, "label": "dmick"}, "created_at": "2023-02-22T20:35:16Z", "updated_at": "2023-02-22T20:35:16Z", "author_association": "NONE", "body": "Was that query to me, @gk7279?\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1590183272, "label": "How to redirect from \"/\" to a specific db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2027#issuecomment-1440355080", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2027", "id": 1440355080, "node_id": "IC_kwDOBm6k_c5V2hMI", "user": {"value": 19700859, "label": "gk7279"}, "created_at": "2023-02-22T16:26:41Z", "updated_at": "2023-02-22T16:26:41Z", "author_association": "NONE", "body": "Can you please help or share your expertise with #2030 ?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1590183272, "label": "How to redirect from \"/\" to a specific db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1258#issuecomment-1437671409", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1258", "id": 1437671409, "node_id": "IC_kwDOBm6k_c5VsR_x", "user": {"value": 2670795, "label": "brandonrobertz"}, "created_at": "2023-02-20T23:39:58Z", "updated_at": "2023-02-20T23:39:58Z", "author_association": "CONTRIBUTOR", "body": "This is pretty annoying for FTS because sqlite throws an error instead of just doing something like returning all or no results. This makes users who are unfamiliar with SQL and Datasette think the canned query page is broken and is a frequent source of confusion.\r\n\r\nTo anyone dealing with this: My solution is to modify the canned query so that it returns no results which cues people to fill in the blank parameters.\r\n\r\nSo instead of `emails_fts match escape_fts(:search))`\r\n\r\nMy canned queries now look like this:\r\n\r\n`emails_fts match escape_fts(iif(:search==\"\", \"*\", :search))`\r\n\r\nThere are no asterisks in my data so the result is always blank.\r\n\r\nUltimately it would be nice to be able to handle this in the metadata. Either making some named parameters required or setting some default values.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 828858421, "label": "Allow canned query params to specify default values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1435318713", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/525", "id": 1435318713, "node_id": "IC_kwDOCGYnMM5VjTm5", "user": {"value": 167893, "label": "mcarpenter"}, "created_at": "2023-02-17T21:55:01Z", "updated_at": "2023-02-17T21:55:01Z", "author_association": "CONTRIBUTOR", "body": "Meanwhile, a cheap workaround is to invalidate the registered function cache:\r\n``` python\r\ntable.convert(...)\r\ndb._registered_functions = set()\r\ntable.convert(...)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1575131737, "label": "Repeated calls to `Table.convert()` fail"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1775#issuecomment-1426158181", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1775", "id": 1426158181, "node_id": "IC_kwDOBm6k_c5VAXJl", "user": {"value": 805751, "label": "metamoof"}, "created_at": "2023-02-10T18:04:40Z", "updated_at": "2023-02-10T18:04:40Z", "author_association": "NONE", "body": "Is this where we talk about i18n of results? Or is that a separate thread.\r\n\r\ne.g. Having `country_long` show `Espa\u00f1a` in the Spanish version of the [global power plants demo site](https://global-power-plants.datasettes.com/) instead of `Spain`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1323346408, "label": "i18n support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2024#issuecomment-1426031395", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2024", "id": 1426031395, "node_id": "IC_kwDOBm6k_c5U_4Mj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-10T16:11:53Z", "updated_at": "2023-02-10T16:11:53Z", "author_association": "OWNER", "body": "Relevant: https://til.simonwillison.net/sqlite/enabling-wal-mode", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1579973223, "label": "Mention WAL mode in documentation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2023#issuecomment-1425988018", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2023", "id": 1425988018, "node_id": "IC_kwDOBm6k_c5U_tmy", "user": {"value": 80409402, "label": "mlaparie"}, "created_at": "2023-02-10T15:39:59Z", "updated_at": "2023-02-10T15:39:59Z", "author_association": "NONE", "body": "Thanks for confirming my doubts! I removed it after opening this issue, yup, then had another issue with `default_cache_ttl_hashed` which I assume was removed at the same time. Sorry for not trying that before opening the issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1579695809, "label": "Error: Invalid setting 'hash_urls' in settings.json in 0.64.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2023#issuecomment-1425974877", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2023", "id": 1425974877, "node_id": "IC_kwDOBm6k_c5U_qZd", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-10T15:32:41Z", "updated_at": "2023-02-10T15:32:41Z", "author_association": "CONTRIBUTOR", "body": "I think this feature was removed in Datasette 0.61 and moved to a plugin. People who want hashed URLs can use the [datasette-hashed-urls](https://docs.datasette.io/en/stable/performance.html#performance-hashed-urls) plugin to achieve the same affect.\r\n\r\nIt looks like you're trying to disable hashed urls, so I think you can just remove that config setting and things will work.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1579695809, "label": "Error: Invalid setting 'hash_urls' in settings.json in 0.64.1"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2022#issuecomment-1424848569", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2022", "id": 1424848569, "node_id": "IC_kwDOBm6k_c5U7Xa5", "user": {"value": 1667631, "label": "DavidPratten"}, "created_at": "2023-02-09T21:13:50Z", "updated_at": "2023-02-09T21:13:50Z", "author_association": "NONE", "body": "Nulls in primary keys, does it every time.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1578609658, "label": "Error 500 - not clear the cause"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1423387341", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/525", "id": 1423387341, "node_id": "IC_kwDOCGYnMM5U1yrN", "user": {"value": 167893, "label": "mcarpenter"}, "created_at": "2023-02-08T23:48:52Z", "updated_at": "2023-02-09T00:17:30Z", "author_association": "CONTRIBUTOR", "body": "PR below", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1575131737, "label": "Repeated calls to `Table.convert()` fail"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1423067724", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1423067724, "node_id": "IC_kwDOBm6k_c5U0kpM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T18:33:32Z", "updated_at": "2023-02-08T18:36:48Z", "author_association": "OWNER", "body": "Just realized that it's useful to be able to tell what parameters were used to generate a page... but reflecting things like `_next` back in the JSON is confusing in the presence of `next`.\r\n\r\nSo I'm going to add an extra for that information too.\r\n\r\nNot sure what to call it though:\r\n\r\n- `params` - confusing because in the code that's usually used for params passed to SQL queries\r\n- `query_string` - wouldn't that be a string, not params as a dictionary?\r\n\r\nI'm going to experiment with a `request` extra that returns some bits of information about the request.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1422681850", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1422681850, "node_id": "IC_kwDOCGYnMM5UzGb6", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-08T14:25:50Z", "updated_at": "2023-02-08T14:29:09Z", "author_association": "NONE", "body": "I live the patch here for others:\r\n\r\n_original code_\r\n```shell\r\n$ which sqlite-utils | xargs cat\r\n```\r\n```python\r\n#!/usr/bin/python3\r\n# -*- coding: utf-8 -*-\r\nimport re\r\nimport sys\r\nfrom sqlite_utils.cli import cli\r\n\r\n\r\nif __name__ == '__main__':\r\n sys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\r\n sys.exit(cli())\r\n```\r\n\r\n_patched/sqlite-utils.py_\r\n```python\r\n#!/usr/bin/python3\r\n# -*- coding: utf-8 -*-\r\nimport re\r\nimport sys\r\nfrom sqlite_utils.cli import cli\r\n\r\n# New imports\r\nfrom unittest.mock import patch\r\nfrom sqlite_utils.cli import VALID_COLUMN_TYPES\r\n\r\n\r\nif __name__ == '__main__':\r\n # Choices of the option `--type`\r\n cli.commands['transform'].params[2].type.types[1].choices.append('DATETIME')\r\n\r\n # The dicts has to be extended with a new type\r\n with patch.dict('sqlite_utils.db.COLUMN_TYPE_MAPPING', {'DATETIME': 'DATETIME'}),\\\r\n patch('sqlite_utils.cli.VALID_COLUMN_TYPES', VALID_COLUMN_TYPES + (\"DATETIME\", )):\r\n\r\n # Command is unchanged\r\n sys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\r\n sys.exit(cli())\r\n```\r\n\r\nAnd now it's working\r\n```bash\r\n$ sqlite-utils schema events.sqlite cards.chunk.get\r\nCREATE TABLE \"cards.chunk.get\" (\r\n [id] INTEGER PRIMARY KEY NOT NULL,\r\n [timestamp] TEXT,\r\n)\r\n\r\n$ python patched/sqlite-utils.py transform events.sqlite cards.chunk.get --type timestamp DATETIME\r\n\r\n$ sqlite-utils schema events.sqlite cards.chunk.get\r\nCREATE TABLE \"cards.chunk.get\" (\r\n [id] INTEGER PRIMARY KEY NOT NULL,\r\n [timestamp] DATETIME,\r\n)\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1421988953", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1421988953, "node_id": "IC_kwDOBm6k_c5UwdRZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T04:35:44Z", "updated_at": "2023-02-08T05:27:48Z", "author_association": "OWNER", "body": "Next step: get `?_next=...` working (it is ignored at the moment, even though the returned JSON includes the `\"next\"` key).\r\n\r\nThen... figure out how to render HTML and other requested formats.\r\n\r\nThen get the tests to pass!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1421784930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1421784930, "node_id": "IC_kwDOBm6k_c5Uvrdi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T01:28:25Z", "updated_at": "2023-02-08T01:40:46Z", "author_association": "OWNER", "body": "Rather than duplicate this rather awful hack:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L694-L714\r\n\r\nI'm tempted to say that the code that calls the new pagination helper needs to ensure that the `sort` or `sort_desc` columns are selected. If it wants to ditch them later (e.g. because they were not included in `?_col=`) it can do that later once the results have come back.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1421600789", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1421600789, "node_id": "IC_kwDOBm6k_c5Uu-gV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T23:12:40Z", "updated_at": "2023-02-07T23:16:20Z", "author_association": "OWNER", "body": "Most complicated example of a paginated query: https://latest.datasette.io/fixtures?sql=select%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++content%2C%0D%0A++sortable%2C%0D%0A++sortable_with_nulls%2C%0D%0A++sortable_with_nulls_2%2C%0D%0A++text%0D%0Afrom%0D%0A++sortable%0D%0Awhere%0D%0A++(%0D%0A++++sortable_with_nulls+is+null%0D%0A++++and+(%0D%0A++++++(pk1+%3E+%3Ap0)%0D%0A++++++or+(%0D%0A++++++++pk1+%3D+%3Ap0%0D%0A++++++++and+pk2+%3E+%3Ap1%0D%0A++++++)%0D%0A++++)%0D%0A++)%0D%0Aorder+by%0D%0A++sortable_with_nulls+desc%2C%0D%0A++pk1%2C%0D%0A++pk2%0D%0Alimit%0D%0A++101&p0=h&p1=r\r\n\r\n```sql\r\nselect\r\n pk1,\r\n pk2,\r\n content,\r\n sortable,\r\n sortable_with_nulls,\r\n sortable_with_nulls_2,\r\n text\r\nfrom\r\n sortable\r\nwhere\r\n (\r\n sortable_with_nulls is null\r\n and (\r\n (pk1 > :p0)\r\n or (\r\n pk1 = :p0\r\n and pk2 > :p1\r\n )\r\n )\r\n )\r\norder by\r\n sortable_with_nulls desc,\r\n pk1,\r\n pk2\r\nlimit\r\n 101\r\n```\r\nGenerated by this page: https://latest.datasette.io/fixtures/sortable?_next=%24null%2Ch%2Cr&_sort_desc=sortable_with_nulls\r\n\r\nThe `_next=` parameter there decodes as `$null,h,r` - and those components are tilde-encoded, so this can be distinguished from an actual `$null` value which would be represented as `~24null`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/520#issuecomment-1421571810", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/520", "id": 1421571810, "node_id": "IC_kwDOCGYnMM5Uu3bi", "user": {"value": 167893, "label": "mcarpenter"}, "created_at": "2023-02-07T22:43:09Z", "updated_at": "2023-02-07T22:43:09Z", "author_association": "CONTRIBUTOR", "body": "Hey, isn't this essentially the same issue as #448 ?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1516644980, "label": "rows_from_file() raises confusing error if file-like object is not in binary mode"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1421274434", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1421274434, "node_id": "IC_kwDOBm6k_c5Utu1C", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T18:42:42Z", "updated_at": "2023-02-07T18:42:42Z", "author_association": "OWNER", "body": "I'm going to build completely separate tests for this in `test_pagination.py`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421177666", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1421177666, "node_id": "IC_kwDOCGYnMM5UtXNC", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-07T17:39:00Z", "updated_at": "2023-02-07T17:39:00Z", "author_association": "NONE", "body": "> lets users make schema changes, so it's important to me that the tool work in a non-surprising way -- if you ask for a column of type X, you should get type X. If the column or table previously had CHECK constraints, they shouldn't be silently removed\r\n\r\nI've got your concern. Let's see if we will be replied on it and i'll close the issue some later.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421081939", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1421081939, "node_id": "IC_kwDOCGYnMM5Us_1T", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-07T16:42:25Z", "updated_at": "2023-02-07T16:43:42Z", "author_association": "NONE", "body": "Ha, yes, I might end up making something very niche. That's OK.\r\n\r\nI'm building a UI for [Datasette](https://datasette.io/) that lets users make schema changes, so it's important to me that the tool work in a non-surprising way -- if you ask for a column of type X, you should get type X. If the column or table previously had CHECK constraints, they shouldn't be silently removed. And so on. I had hoped that I could just lean on sqlite-utils, but I think it's a little too surprising.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421055590", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1421055590, "node_id": "IC_kwDOCGYnMM5Us5Zm", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-07T16:25:31Z", "updated_at": "2023-02-07T16:25:31Z", "author_association": "NONE", "body": "> Ah, it looks like that is controlled by this dict: https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L178\r\n> \r\n> I suspect you could overwrite the datetime entry to achieve what you want\r\n\r\nAnd thank you for pointing me to it. At least, i can make a monkey patch for my need...", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421052195", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1421052195, "node_id": "IC_kwDOCGYnMM5Us4kj", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-07T16:23:17Z", "updated_at": "2023-02-07T16:23:57Z", "author_association": "NONE", "body": "Isn't your suggestion too fundamental for the utility?\r\n\r\nThe bigger flexibility, the bigger complexity. Your idea make sense defenitely, but how often do you make schema changes? And how many people could benefit from it, what do you think?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421033725", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1421033725, "node_id": "IC_kwDOCGYnMM5Us0D9", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-07T16:12:13Z", "updated_at": "2023-02-07T16:12:13Z", "author_association": "NONE", "body": "I think the bigger issue is that `sqlite-utils` mixes mechanism (it implements the [12-step way to alter SQLite tables](https://www.sqlite.org/lang_altertable.html#otheralter)) and policy (it has an opinionated stance on what column types should be used).\r\n\r\nThat might be a design choice to make it accessible to users by providing a reasonable set of defaults, but it doesn't quite fit my use case.\r\n\r\nIt might make sense to extract a separate library that provides just the mechanisms, and then `sqlite-utils` would sit on top of that library with its opinionated set of policies.\r\n\r\nThat would be a very big change, though.\r\n\r\nI might take a stab at extracting the library, but just for the table schema migration piece, not all the other features that `sqlite-utils` supports. I wouldn't expect `sqlite-utils` to depend on it.\r\n\r\nPart of my motivation is that I want to provide some other abilities, too, like support for CHECK constraints. I see that the issue in this repo (https://github.com/simonw/sqlite-utils/issues/358) proposes a bunch of short-hand constraints, which I wouldn't want to accidentally expose to people -- I want a layer that is a 1:1 mapping to SQLite.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1421022917", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1421022917, "node_id": "IC_kwDOCGYnMM5UsxbF", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-07T16:06:03Z", "updated_at": "2023-02-07T16:08:58Z", "author_association": "NONE", "body": "> Do you see a way to enable it without affecting existing users or bumping the major version number?\r\n\r\nI don't see a clean solution, only extending code with a side variable that tells us we want to apply advanced types instead of basic.\r\n\r\nit could be a similiar command like `tranform-v2 --type column DATETIME` or a cli option `transform --adv-type column DATETIME` along with a dict that contains the advanced types. Then with knowledge that we run an advanced command we take that dictionary somehow, we can wrap the current and new dictionaries by a superdict and work with it everywhere according to the knowledge. This way shouldn't affect users who are using the previous lib versions and it have to be merged in the next major one.\r\n\r\nBut this way looks a bad design, too messy.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420992261", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1420992261, "node_id": "IC_kwDOCGYnMM5Usp8F", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-07T15:45:58Z", "updated_at": "2023-02-07T15:45:58Z", "author_association": "NONE", "body": "I'd support that, but I'm not the author of this library.\r\n\r\nOne challenge is that would be a breaking change. Do you see a way to enable it without affecting existing users or bumping the major version number?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420966995", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1420966995, "node_id": "IC_kwDOCGYnMM5UsjxT", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-07T15:29:28Z", "updated_at": "2023-02-07T15:29:28Z", "author_association": "NONE", "body": "I could, of course.\r\n\r\nDoest it worth bringing such the improvement to the library?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/564#issuecomment-1420941334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/564", "id": 1420941334, "node_id": "IC_kwDOBm6k_c5UsdgW", "user": {"value": 82988, "label": "psychemedia"}, "created_at": "2023-02-07T15:14:10Z", "updated_at": "2023-02-07T15:14:10Z", "author_association": "CONTRIBUTOR", "body": "Is this feature covered by any more recent updates to `datasette`, or via any plugins that you're aware of?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 473288428, "label": "First proof-of-concept of Datasette Library"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420809773", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1420809773, "node_id": "IC_kwDOCGYnMM5Ur9Yt", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-07T13:53:01Z", "updated_at": "2023-02-07T13:53:01Z", "author_association": "NONE", "body": "Ah, it looks like that is controlled by this dict: https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L178\r\n\r\nI suspect you could overwrite the datetime entry to achieve what you want", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1420496447", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1420496447, "node_id": "IC_kwDOCGYnMM5Uqw4_", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-07T09:57:38Z", "updated_at": "2023-02-07T09:57:38Z", "author_association": "NONE", "body": "> That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work.\r\n\r\nIt works, but a column becomes `TEXT`\r\n\r\n```python\r\nIn [1]: import sqlite_utils\r\nIn [2]: db = sqlite_utils.Database('events.sqlite')\r\nIn [3]: table = db['cards.chunk.get']\r\nIn [4]: table.columns_dict\r\nOut[4]:\r\n{'id': int,\r\n 'timestamp': float,\r\n 'data_chunk_number': int,\r\n 'user_id': str,\r\n 'meta_duplication_source_id': int,\r\n 'context_sort_attribute': str,\r\n 'context_sort_order': str}\r\n\r\nIn [5]: from datetime import datetime\r\nIn [7]: table.transform(types={'timestamp': datetime})\r\nIn [8]: table.columns_dict\r\nOut[8]:\r\n{'id': int,\r\n 'timestamp': str,\r\n 'data_chunk_number': int,\r\n 'user_id': str,\r\n 'meta_duplication_source_id': int,\r\n 'context_sort_attribute': str,\r\n 'context_sort_order': str}\r\n```\r\n\r\n```bash\r\n\u276f sqlite-utils schema events.sqlite cards.chunk.get\r\nCREATE TABLE \"cards.chunk.get\" (\r\n [id] INTEGER PRIMARY KEY NOT NULL,\r\n [timestamp] TEXT,\r\n ...\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420109153", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420109153, "node_id": "IC_kwDOBm6k_c5UpSVh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:32:36Z", "updated_at": "2023-02-07T02:32:36Z", "author_association": "OWNER", "body": "Doing this as a class makes sense to me. There are a few steps:\r\n\r\n- Instantiate the class with the information it needs, which includes sort order, page size, tiebreaker columns and SQL query and parameters\r\n- Generate the new SQL query that will actually be executed - maybe this takes the optional `_next` parameter? This returns the SQL and params that should be executed, where the SQL now includes pagination logic plus order by and limit\r\n- The calling code then gets to execute the SQL query to fetch the rows\r\n- Last step: those rows are passed to a paginator method which returns `(rows, next)` - where `rows` is the rows truncated to the correct length (really just with the last one cut off if it's too long for the length) and `next` is either `None` or a token, depending on if there should be a next page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420106315", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420106315, "node_id": "IC_kwDOBm6k_c5UpRpL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:28:03Z", "updated_at": "2023-02-07T02:28:36Z", "author_association": "OWNER", "body": "So I think I can write an abstraction that applies keyset pagination to ANY arbitrary SQL query provided it is given the query, the existing params (so it can pick names for the new params that won't overlap with them), the desired sort order, any existing `_next` token AND the columns that should be used to tie-break any duplicates.\r\n\r\nThose tie breakers will be either the primary key(s) or `rowid` if none are provided.\r\n\r\nWhat about the case of SQL views, where offset/limit should be used instead? I'm inclined to have that as a separate pagination abstraction entirely, with the calling code deciding which pagination helper to use based on if keyset pagination makes sense or not.\r\n\r\nMight be easier to design a class structure for this starting with `OffsetPaginator`, then using that to inform the design of `KeysetPaginator`.\r\n\r\nMight put these in `datasette.utils.pagination` to start off with, then maybe extract them out to `sqlite-utils` later once they've proven themselves.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420104254", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420104254, "node_id": "IC_kwDOBm6k_c5UpRI-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:24:46Z", "updated_at": "2023-02-07T02:24:46Z", "author_association": "OWNER", "body": "Even more complicated: https://latest.datasette.io/fixtures/sortable?sortable_with_nulls__notnull=1&_next=0~2E692704598586882%2Ce%2Cr&_sort=sortable_with_nulls_2\r\n\r\nThe rewritten SQL for that is:\r\n\r\n```sql\r\nselect * from (select pk1, pk2, content, sortable, sortable_with_nulls, sortable_with_nulls_2, text from sortable where \"sortable_with_nulls\" is not null)\r\n where (sortable_with_nulls_2 > :p2 or (sortable_with_nulls_2 = :p2 and ((pk1 > :p0)\r\n or\r\n(pk1 = :p0 and pk2 > :p1)))) order by sortable_with_nulls_2, pk1, pk2 limit 101\r\n```\r\nAnd it still has the same number of explain steps as the current SQL witohut the subselect.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420101175", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420101175, "node_id": "IC_kwDOBm6k_c5UpQY3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:22:11Z", "updated_at": "2023-02-07T02:22:11Z", "author_association": "OWNER", "body": "A more complex example: https://latest.datasette.io/fixtures/sortable?_next=0~2E2650566289400591%2Ca%2Cu&_sort=sortable_with_nulls_2\r\n\r\nSQL:\r\n\r\n```sql\r\nselect pk1, pk2, content, sortable, sortable_with_nulls, sortable_with_nulls_2, text from sortable where (sortable_with_nulls_2 > :p2 or (sortable_with_nulls_2 = :p2 and ((pk1 > :p0)\r\n or\r\n(pk1 = :p0 and pk2 > :p1)))) order by sortable_with_nulls_2, pk1, pk2 limit 101\r\n```\r\n\r\nhttps://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable+where+%28sortable_with_nulls_2+%3E+%3Ap2+or+%28sortable_with_nulls_2+%3D+%3Ap2+and+%28%28pk1+%3E+%3Ap0%29%0A++or%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29%29+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p0=a&p1=u&p2=0.2650566289400591\r\n\r\nHere's the explain: 49 steps long https://latest.datasette.io/fixtures?sql=explain+select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable+where+%28sortable_with_nulls_2+%3E+%3Ap2+or+%28sortable_with_nulls_2+%3D+%3Ap2+and+%28%28pk1+%3E+%3Ap0%29%0D%0A++or%0D%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29%29+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p2=0.2650566289400591&p0=a&p1=u\r\n\r\nRewritten with a subselect:\r\n\r\n```sql\r\nselect * from (\r\n select pk1, pk2, content, sortable, sortable_with_nulls, sortable_with_nulls_2, text from sortable\r\n)\r\nwhere (sortable_with_nulls_2 > :p2 or (sortable_with_nulls_2 = :p2 and ((pk1 > :p0)\r\n or\r\n(pk1 = :p0 and pk2 > :p1)))) order by sortable_with_nulls_2, pk1, pk2 limit 101\r\n```\r\nhttps://latest.datasette.io/fixtures?sql=select+*+from+(%0D%0A++select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable%0D%0A)%0D%0Awhere+(sortable_with_nulls_2+%3E+%3Ap2+or+(sortable_with_nulls_2+%3D+%3Ap2+and+((pk1+%3E+%3Ap0)%0D%0A++or%0D%0A(pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1))))+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p2=0.2650566289400591&p0=a&p1=u\r\n\r\nAnd here's the explain for that - also 49 steps: https://latest.datasette.io/fixtures?sql=explain+select+*+from+%28%0D%0A++select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable%0D%0A%29%0D%0Awhere+%28sortable_with_nulls_2+%3E+%3Ap2+or+%28sortable_with_nulls_2+%3D+%3Ap2+and+%28%28pk1+%3E+%3Ap0%29%0D%0A++or%0D%0A%28pk1+%3D+%3Ap0+and+pk2+%3E+%3Ap1%29%29%29%29+order+by+sortable_with_nulls_2%2C+pk1%2C+pk2+limit+101&p2=0.2650566289400591&p0=a&p1=u", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420094396", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420094396, "node_id": "IC_kwDOBm6k_c5UpOu8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:18:11Z", "updated_at": "2023-02-07T02:19:16Z", "author_association": "OWNER", "body": "For the SQL underlying this page (the second page in that compound primary key paginated sequence): https://latest.datasette.io/fixtures/compound_three_primary_keys?_next=a%2Cd%2Cv\r\n\r\nThe explain for the default query: https://latest.datasette.io/fixtures?sql=explain+select%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%2C%0D%0A++content%0D%0Afrom%0D%0A++compound_three_primary_keys%0D%0Awhere%0D%0A++%28%0D%0A++++%28pk1+%3E+%3Ap0%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3E+%3Ap1%0D%0A++++%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3D+%3Ap1%0D%0A++++++and+pk3+%3E+%3Ap2%0D%0A++++%29%0D%0A++%29%0D%0Aorder+by%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%0D%0Alimit%0D%0A++101&p0=a&p1=d&p2=v\r\n\r\nThe explain for that query rewritten as this:\r\n\r\n```sql\r\nexplain\r\nselect\r\n *\r\nfrom\r\n (\r\n select\r\n pk1,\r\n pk2,\r\n pk3,\r\n content\r\n from\r\n compound_three_primary_keys\r\n )\r\nwhere\r\n (\r\n (pk1 > :p0)\r\n or (\r\n pk1 = :p0\r\n and pk2 > :p1\r\n )\r\n or (\r\n pk1 = :p0\r\n and pk2 = :p1\r\n and pk3 > :p2\r\n )\r\n )\r\norder by\r\n pk1,\r\n pk2,\r\n pk3\r\nlimit\r\n 101\r\n```\r\nhttps://latest.datasette.io/fixtures?sql=explain+select+*+from+%28select+%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%2C%0D%0A++content%0D%0Afrom%0D%0A++compound_three_primary_keys%0D%0A%29%0D%0A++where%0D%0A++%28%0D%0A++++%28pk1+%3E+%3Ap0%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3E+%3Ap1%0D%0A++++%29%0D%0A++++or+%28%0D%0A++++++pk1+%3D+%3Ap0%0D%0A++++++and+pk2+%3D+%3Ap1%0D%0A++++++and+pk3+%3E+%3Ap2%0D%0A++++%29%0D%0A++%29%0D%0Aorder+by%0D%0A++pk1%2C%0D%0A++pk2%2C%0D%0A++pk3%0D%0Alimit%0D%0A++101&p0=a&p1=d&p2=v\r\n\r\nBoth explains have 31 steps and look pretty much identical.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1420088670", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1420088670, "node_id": "IC_kwDOBm6k_c5UpNVe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-07T02:14:35Z", "updated_at": "2023-02-07T02:14:35Z", "author_association": "OWNER", "body": "Maybe the correct level of abstraction here is that pagination is something that happens to a SQL query that is defined as SQL and params, without an order by or limit. That's then wrapped in a sub-select and those things are added to it, plus the necessary `where` clauses depending on the page.\r\n\r\nNeed to check that the query plan for pagination of a subquery isn't slower than the plan for pagination as it works today.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419953256", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419953256, "node_id": "IC_kwDOBm6k_c5UosRo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:42:56Z", "updated_at": "2023-02-06T23:43:10Z", "author_association": "OWNER", "body": "Relevant issue:\r\n- https://github.com/simonw/datasette/issues/1773\r\n\r\nExplains this comment: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L697\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419928455", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419928455, "node_id": "IC_kwDOBm6k_c5UomOH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:21:50Z", "updated_at": "2023-02-06T23:21:50Z", "author_association": "OWNER", "body": "Found more logic relating to this:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L684-L732", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419921228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419921228, "node_id": "IC_kwDOBm6k_c5UokdM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:14:15Z", "updated_at": "2023-02-06T23:14:15Z", "author_association": "OWNER", "body": "Crucial utility function: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/__init__.py#L137-L160", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419917661", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419917661, "node_id": "IC_kwDOBm6k_c5Uojld", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:10:51Z", "updated_at": "2023-02-06T23:10:51Z", "author_association": "OWNER", "body": "I should turn `sort` and `sort_desc` into an object representing the sort order earlier in the code.\r\n\r\nI should also create something that bundles together `pks` and `use_rowid` and maybe `is_view` as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2019#issuecomment-1419916684", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2019", "id": 1419916684, "node_id": "IC_kwDOBm6k_c5UojWM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-06T23:09:51Z", "updated_at": "2023-02-06T23:10:13Z", "author_association": "OWNER", "body": "The inputs and outputs for this are pretty complex.\r\n\r\nInputs:\r\n\r\n- `?_next=` from the query string\r\n- `is_view` - is this for a table or view? If it's a view it uses offset/limit pagination - which could actually work for arbitrary queries too. Also views could have keyset pagination if they are known to be sorted by a particular column.\r\n- `sort` and `sort_desc` reflecting the current sort order\r\n- `use_rowid` for if the table is a rowid table with no primary key of its own\r\n- `pks` - the primary keys for the table\r\n- `params` - the current set of parameters, I think used just to count their length so new params can be added as `p5` etc without collisions. This could be handled with a `s0`, `s1` etc naming convention instead.\r\n\r\nOutputs:\r\n\r\n- `where_clauses` - a list of where clauses to add to the query\r\n- `params` - additional parameters to use with the query due to the new where clauses\r\n- `order_by` - the order by clause", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1573424830, "label": "Refactor out the keyset pagination code"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399343659", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399343659, "node_id": "IC_kwDOBm6k_c5TaEor", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:19:20Z", "updated_at": "2023-02-06T23:02:12Z", "author_association": "OWNER", "body": "HTML mode needs a list of renderers so it can show links to `.geojson` etc - can do that as a hidden extra (maybe called `renderers`), repeating this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/views/base.py#L477-L497", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/262#issuecomment-1418288327", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/262", "id": 1418288327, "node_id": "IC_kwDOBm6k_c5UiVzH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-05T22:57:58Z", "updated_at": "2023-02-06T23:01:15Z", "author_association": "OWNER", "body": "I think that does make sense: `?_extra=table` perhaps, which would add `{\"table\": \"...\"}`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 323658641, "label": "Add ?_extra= mechanism for requesting extra properties in JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419734229", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419734229, "node_id": "IC_kwDOCGYnMM5Un2zV", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-06T20:53:28Z", "updated_at": "2023-02-06T21:16:29Z", "author_association": "NONE", "body": "I think it's not currently possible: sqlite-utils requires that it be one of `integer`, `text`, `float`, `blob` ([see code](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/cli.py#L2266))\r\n\r\nIMO, this is a bit of friction and it would be nice if it was more permissive. SQLite permits developers to use any data type when creating a table. For example, this is a perfectly cromulent sqlite session that creates a table with columns of type `baz` and `bar`:\r\n\r\n```\r\nsqlite> create table foo(column1 baz, column2 bar);\r\nsqlite> .schema foo\r\nCREATE TABLE foo(column1 baz, column2 bar);\r\nsqlite> select * from pragma_table_info('foo');\r\ncid name type notnull dflt_value pk \r\n---------- ---------- ---------- ---------- ---------- ----------\r\n0 column1 baz 0 0 \r\n1 column2 bar 0 0 \r\n```\r\n\r\nThe idea is that the application developer will know what meaning to ascribe to those types. For example, I'm working on a plugin to Datasette. Dates are tricky to handle. If you have some existing rows, you can look at the values in them to know how a user is serializing the dates -- as an ISO 8601 string? An RFC 3339 string? With millisecond precision? With timezone offset? But if you don't yet have any rows, you have to guess. If the column is of type `TEXT`, you don't even know that it's meant to hold a date! In this case, my plugin will look to see if the column is of type `DATE` or `DATETIME`, and assume a certain representation when writing.\r\n\r\nPerhaps there is an argument that sqlite-utils is trying to conform to SQLite's strict mode, and that is why it limits the choices. In strict mode, SQLite requires that the data type be one of `INT`, `INTEGER`, `REAL`, `TEXT`, `BLOB`, `ANY`. But that can't be the case -- sqlite-utils supports `FLOAT`, which is not one of the valid types in strict mode, and it rejects `INT`, `REAL` and `ANY`, which _are_ valid.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419740776", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419740776, "node_id": "IC_kwDOCGYnMM5Un4Zo", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-06T20:59:01Z", "updated_at": "2023-02-06T20:59:01Z", "author_association": "NONE", "body": "That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419390560", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419390560, "node_id": "IC_kwDOCGYnMM5Umi5g", "user": {"value": 21095447, "label": "4l1fe"}, "created_at": "2023-02-06T16:43:47Z", "updated_at": "2023-02-06T16:43:47Z", "author_association": "NONE", "body": "> SQLite doesn't have a native `DATETIME` type. It stores dates internally as strings and then has [functions](https://www.sqlite.org/lang_datefunc.html) to work with date-like strings. Yes it's weird.\r\n\r\nThat's correct. But my issue is about the application level libraries that, i suppose, have better data understanding if see a specific type such as `DATETIME`. \r\n\r\nI'm writing data with **dataset** i've mentioned. The lib changes its behavior depending on a type. I saw different behavior with types `DATETIME, FLOAT, TEXT`. Dataset, for their part, is built upon Sqlalchemy, you know what it is.\r\n\r\nTo be honest, i didn't dive into the details of why the behavior changes, but when i altered manually by other util a type of column to `DATETIME` things got back to normal.\r\n\r\nOn the matter, can i achieve it with Sqlite Utils at the moment?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419357290", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419357290, "node_id": "IC_kwDOCGYnMM5Umaxq", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2023-02-06T16:21:44Z", "updated_at": "2023-02-06T16:21:44Z", "author_association": "CONTRIBUTOR", "body": "SQLite doesn't have a native `DATETIME` type. It stores dates internally as strings and then has [functions](https://www.sqlite.org/lang_datefunc.html) to work with date-like strings. Yes it's weird.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2016#issuecomment-1418288077", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2016", "id": 1418288077, "node_id": "IC_kwDOBm6k_c5UiVvN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-05T22:56:43Z", "updated_at": "2023-02-05T22:56:43Z", "author_association": "OWNER", "body": "This absolutely makes sense. One of the biggest goals for Datasette 1.0 is \"documented template contexts\" - for any default template in Datasette that people might want to over-ride there should be documentation that describes the available context variables, plus tests that ensure they don't accidentally get broken by future changes.\r\n\r\nEnsuring description/title/etc are available on the index page feels like it fits well into that bucket.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1571207083, "label": "Database metadata fields like description are not available in the index page template's context"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1416486796", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433", "id": 1416486796, "node_id": "IC_kwDOCGYnMM5Ubd-M", "user": {"value": 16236421, "label": "alecstein"}, "created_at": "2023-02-03T22:32:10Z", "updated_at": "2023-02-03T22:32:10Z", "author_association": "NONE", "body": "Came here to say that I also have this issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239034903, "label": "CLI eats my cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2011#issuecomment-1410827249", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2011", "id": 1410827249, "node_id": "IC_kwDOBm6k_c5UF4Px", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-31T17:58:54Z", "updated_at": "2023-01-31T17:58:54Z", "author_association": "OWNER", "body": "I think this is the relevant code:\r\n\r\nhttps://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/facets.py#L260-L268", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1564769997, "label": "Applied facet did not result in an \"x\" icon to dismiss it"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2010#issuecomment-1409406327", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2010", "id": 1409406327, "node_id": "IC_kwDOBm6k_c5UAdV3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-30T21:51:58Z", "updated_at": "2023-01-30T21:51:58Z", "author_association": "OWNER", "body": "Here's a quick prototype I knocked up for this:\r\n```diff\r\ndiff --git a/datasette/static/app.css b/datasette/static/app.css\r\nindex 71437bd4..d763bcff 100644\r\n--- a/datasette/static/app.css\r\n+++ b/datasette/static/app.css\r\n@@ -695,7 +695,48 @@ p.zero-results {\r\n \r\n \r\n \r\n+/* Force table to not be like tables anymore */\r\n+body.row table.rows-and-columns,\r\n+body.row .rows-and-columns thead,\r\n+body.row .rows-and-columns tbody,\r\n+body.row .rows-and-columns th,\r\n+body.row .rows-and-columns td,\r\n+body.row .rows-and-columns tr {\r\n+ display: block;\r\n+}\r\n+\r\n+/* Hide table headers (but not display: none;, for accessibility) */\r\n+body.row .rows-and-columns thead tr {\r\n+ position: absolute;\r\n+ top: -9999px;\r\n+ left: -9999px;\r\n+}\r\n+\r\n+body.row .rows-and-columns tr {\r\n+ border: 1px solid #ccc;\r\n+ margin-bottom: 1em;\r\n+ border-radius: 10px;\r\n+ background-color: white;\r\n+ padding: 0.2rem;\r\n+}\r\n \r\n+body.row .rows-and-columns td {\r\n+ /* Behave like a \"row\" */\r\n+ border: none;\r\n+ border-bottom: 1px solid #eee;\r\n+ padding: 0;\r\n+ padding-left: 10%;\r\n+ padding-bottom: 0.3em;\r\n+}\r\n+\r\n+body.row .rows-and-columns td:before {\r\n+ display: block;\r\n+ color: black;\r\n+ padding-bottom: 0.2em;\r\n+ font-size: 0.8em;\r\n+ font-weight: bold;\r\n+ background-color: #f5f5f5;\r\n+}\r\n \r\n \r\n /* Overrides ===============================================================*/\r\ndiff --git a/datasette/templates/row.html b/datasette/templates/row.html\r\nindex 1d1b0bfd..339eb643 100644\r\n--- a/datasette/templates/row.html\r\n+++ b/datasette/templates/row.html\r\n@@ -5,6 +5,9 @@\r\n {% block extra_head %}\r\n {{- super() -}}\r\n