{"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898063815", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898063815, "node_id": "IC_kwDOBm6k_c41h13H", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-13T00:33:17Z", "updated_at": "2021-08-13T00:33:17Z", "author_association": "OWNER", "body": "Improved version of that function:\r\n```python\r\ndef columns_for_query(conn, sql):\r\n \"\"\"\r\n Given a SQLite connection ``conn`` and a SQL query ``sql``,\r\n returns a list of ``(table_name, column_name)`` pairs, one\r\n per returned column. ``(None, None)`` if no table and column\r\n could be derived.\r\n \"\"\"\r\n rows = conn.execute('explain ' + sql).fetchall()\r\n table_rootpage_by_register = {r['p1']: r['p2'] for r in rows if r['opcode'] == 'OpenRead'}\r\n names_by_rootpage = dict(\r\n conn.execute(\r\n 'select rootpage, name from sqlite_master where rootpage in ({})'.format(\r\n ', '.join(map(str, table_rootpage_by_register.values()))\r\n )\r\n )\r\n )\r\n columns_by_column_register = {}\r\n for row in rows:\r\n if row['opcode'] in ('Rowid', 'Column'):\r\n addr, opcode, table_id, cid, column_register, p4, p5, comment = row\r\n table = names_by_rootpage[table_rootpage_by_register[table_id]]\r\n columns_by_column_register[column_register] = (table, cid)\r\n result_row = [dict(r) for r in rows if r['opcode'] == 'ResultRow'][0]\r\n registers = list(range(result_row[\"p1\"], result_row[\"p1\"] + result_row[\"p2\"]))\r\n all_column_names = {}\r\n for table in names_by_rootpage.values():\r\n table_xinfo = conn.execute('pragma table_xinfo({})'.format(table)).fetchall()\r\n for row in table_xinfo:\r\n all_column_names[(table, row[\"cid\"])] = row[\"name\"]\r\n final_output = []\r\n for r in registers:\r\n try:\r\n table, cid = columns_by_column_register[r]\r\n final_output.append((table, all_column_names[table, cid]))\r\n except KeyError:\r\n final_output.append((None, None))\r\n return final_output\r\n```\r\nIt works!\r\n\r\n\"Banners_and_Alerts_and_fixtures__select_attraction_id__roadside_attractions_name__characteristic_id__attraction_characteristic_name_as_characteristic_from_roadside_attraction_characteristics_join_roadside_attractions_on_roadside_attractions\"\r\n\r\n```diff\r\ndiff --git a/datasette/templates/query.html b/datasette/templates/query.html\r\nindex 75f7f1b..9fe1d4f 100644\r\n--- a/datasette/templates/query.html\r\n+++ b/datasette/templates/query.html\r\n@@ -67,6 +67,8 @@\r\n

\r\n \r\n \r\n+extra_column_info: {{ extra_column_info }}\r\n+\r\n {% if display_rows %}\r\n

This data as {% for name, url in renderers.items() %}{{ name }}{{ \", \" if not loop.last }}{% endfor %}, CSV

\r\n
\r\ndiff --git a/datasette/views/database.py b/datasette/views/database.py\r\nindex 7c36034..02f8039 100644\r\n--- a/datasette/views/database.py\r\n+++ b/datasette/views/database.py\r\n@@ -10,6 +10,7 @@ import markupsafe\r\n from datasette.utils import (\r\n await_me_maybe,\r\n check_visibility,\r\n+ columns_for_query,\r\n derive_named_parameters,\r\n to_css_class,\r\n validate_sql_select,\r\n@@ -248,6 +249,8 @@ class QueryView(DataView):\r\n \r\n query_error = None\r\n \r\n+ extra_column_info = None\r\n+\r\n # Execute query - as write or as read\r\n if write:\r\n if request.method == \"POST\":\r\n@@ -334,6 +337,10 @@ class QueryView(DataView):\r\n database, sql, params_for_query, truncate=True, **extra_args\r\n )\r\n columns = [r[0] for r in results.description]\r\n+\r\n+ # Try to figure out extra column information\r\n+ db = self.ds.get_database(database)\r\n+ extra_column_info = await db.execute_fn(lambda conn: columns_for_query(conn, sql))\r\n except sqlite3.DatabaseError as e:\r\n query_error = e\r\n results = None\r\n@@ -462,6 +469,7 @@ class QueryView(DataView):\r\n \"show_hide_text\": show_hide_text,\r\n \"show_hide_hidden\": markupsafe.Markup(show_hide_hidden),\r\n \"hide_sql\": hide_sql,\r\n+ \"extra_column_info\": extra_column_info,\r\n }\r\n \r\n return (\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1293#issuecomment-898056013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1293", "id": 898056013, "node_id": "IC_kwDOBm6k_c41hz9N", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-13T00:12:09Z", "updated_at": "2021-08-13T00:12:09Z", "author_association": "OWNER", "body": "Having added column metadata in #1430 (ref #942) I could also include a definition list at the top of the query results page exposing the column descriptions for any columns, using the same EXPLAIN mechanism.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 849978964, "label": "Show column metadata plus links for foreign keys on arbitrary query results"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898051645", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898051645, "node_id": "IC_kwDOBm6k_c41hy49", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-13T00:02:25Z", "updated_at": "2021-08-13T00:02:25Z", "author_association": "OWNER", "body": "And on mobile:\r\n\r\n![5FAF8D73-7199-4BB7-A5B8-9E46DCB4A985](https://user-images.githubusercontent.com/9599/129284817-dc13cbf4-144e-4f4c-8fb7-470602e2eea0.jpeg)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898050457", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898050457, "node_id": "IC_kwDOBm6k_c41hymZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T23:59:53Z", "updated_at": "2021-08-12T23:59:53Z", "author_association": "OWNER", "body": "Documentation: https://docs.datasette.io/en/latest/metadata.html#column-descriptions\r\n\r\nLive demo: https://latest.datasette.io/fixtures/roadside_attractions\r\n\r\n\"fixtures__roadside_attractions__4_rows\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1430#issuecomment-898043575", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1430", "id": 898043575, "node_id": "IC_kwDOBm6k_c41hw63", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-08-12T23:39:36Z", "updated_at": "2021-08-12T23:49:51Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1430](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (9419947) into [main](https://codecov.io/gh/simonw/datasette/commit/b1fed48a95516ae84c0f020582303ab50ab817e2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (b1fed48) will **increase** coverage by `0.00%`.\n> The diff coverage is `100.00%`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1430/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@ Coverage Diff @@\n## main #1430 +/- ##\n=======================================\n Coverage 91.71% 91.71% \n=======================================\n Files 34 34 \n Lines 4417 4418 +1 \n=======================================\n+ Hits 4051 4052 +1 \n Misses 366 366 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [datasette/views/table.py](https://codecov.io/gh/simonw/datasette/pull/1430/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `96.00% <100.00%> (+<0.01%)` | :arrow_up: |\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [b1fed48...9419947](https://codecov.io/gh/simonw/datasette/pull/1430?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 969758038, "label": "Column metadata"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898037650", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898037650, "node_id": "IC_kwDOBm6k_c41hveS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T23:23:54Z", "updated_at": "2021-08-12T23:23:54Z", "author_association": "OWNER", "body": "I like this enough that I'm going to ship it as an alpha and try it out on a couple of live projects.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898037456", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898037456, "node_id": "IC_kwDOBm6k_c41hvbQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T23:23:34Z", "updated_at": "2021-08-12T23:23:34Z", "author_association": "OWNER", "body": "Prototype with a `
`:\r\n\r\n\"fixtures__sortable__201_rows\"\r\n\r\n```diff\r\ndiff --git a/datasette/static/app.css b/datasette/static/app.css\r\nindex c6be1e9..bf068fd 100644\r\n--- a/datasette/static/app.css\r\n+++ b/datasette/static/app.css\r\n@@ -836,6 +841,16 @@ svg.dropdown-menu-icon {\r\n background-repeat: no-repeat;\r\n }\r\n \r\n+dl.column-descriptions dt {\r\n+ font-weight: bold;\r\n+}\r\n+dl.column-descriptions dd {\r\n+ padding-left: 1.5em;\r\n+ white-space: pre-wrap;\r\n+ line-height: 1.1em;\r\n+ color: #666;\r\n+}\r\n+\r\n .anim-scale-in {\r\n animation-name: scale-in;\r\n animation-duration: 0.15s;\r\ndiff --git a/datasette/templates/table.html b/datasette/templates/table.html\r\nindex 211352b..466e8a4 100644\r\n--- a/datasette/templates/table.html\r\n+++ b/datasette/templates/table.html\r\n@@ -51,6 +51,14 @@\r\n \r\n {% block description_source_license %}{% include \"_description_source_license.html\" %}{% endblock %}\r\n \r\n+{% if metadata.columns %}\r\n+
\r\n+ {% for column_name, column_description in metadata.columns.items() %}\r\n+
{{ column_name }}
{{ column_description }}
\r\n+ {% endfor %}\r\n+
\r\n+{% endif %}\r\n+\r\n {% if filtered_table_rows_count or human_description_en %}\r\n

{% if filtered_table_rows_count or filtered_table_rows_count == 0 %}{{ \"{:,}\".format(filtered_table_rows_count) }} row{% if filtered_table_rows_count == 1 %}{% else %}s{% endif %}{% endif %}\r\n {% if human_description_en %}{{ human_description_en }}{% endif %}\r\n ```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898032118", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898032118, "node_id": "IC_kwDOBm6k_c41huH2", "user": {"value": 596279, "label": "zaneselvans"}, "created_at": "2021-08-12T23:12:00Z", "updated_at": "2021-08-12T23:12:00Z", "author_association": "NONE", "body": "This looks awesome. We'll definitely make extensive use of this feature!\n\nOn Thu, Aug 12, 2021 at 5:52 PM Simon Willison ***@***.***>\nwrote:\n\n> I like this. Need to solve for mobile though where the cog menu isn't\n> visible - I think I'll do that with a definition list at the top of the\n> page.\n>\n> \u2014\n> You are receiving this because you are subscribed to this thread.\n> Reply to this email directly, view it on GitHub\n> ,\n> or unsubscribe\n> \n> .\n> Triage notifications on the go with GitHub Mobile for iOS\n> \n> or Android\n> \n> .\n>\n\n\n-- \nZane A. Selvans, PhD\nChief Data Wrangler\nCatalyst Cooperative\nhttps://catalyst.coop\n***@***.***\nSignal/WhatsApp/SMS: +1 720 443 1363\nTwitter: @ZaneSelvans \nPGP : 0x64F7B56F3A127B04\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898022235", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898022235, "node_id": "IC_kwDOBm6k_c41hrtb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T22:52:23Z", "updated_at": "2021-08-12T22:52:23Z", "author_association": "OWNER", "body": "I like this. Need to solve for mobile though where the cog menu isn't visible - I think I'll do that with a definition list at the top of the page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-898021895", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 898021895, "node_id": "IC_kwDOBm6k_c41hroH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T22:51:36Z", "updated_at": "2021-08-12T22:51:36Z", "author_association": "OWNER", "body": "Prototype:\r\n\r\n\"fixtures__sortable__201_rows\"\r\n\r\n```diff\r\ndiff --git a/datasette/static/app.css b/datasette/static/app.css\r\nindex c6be1e9..5ca64cb 100644\r\n--- a/datasette/static/app.css\r\n+++ b/datasette/static/app.css\r\n@@ -784,9 +784,14 @@ svg.dropdown-menu-icon {\r\n font-size: 0.7em;\r\n color: #666;\r\n margin: 0;\r\n- padding: 0;\r\n padding: 4px 8px 4px 8px;\r\n }\r\n+.dropdown-menu .dropdown-column-description {\r\n+ margin: 0;\r\n+ color: #666;\r\n+ padding: 4px 8px 4px 8px;\r\n+ max-width: 20em;\r\n+}\r\n .dropdown-menu li {\r\n border-bottom: 1px solid #ccc;\r\n }\r\ndiff --git a/datasette/static/table.js b/datasette/static/table.js\r\nindex 991346d..a903112 100644\r\n--- a/datasette/static/table.js\r\n+++ b/datasette/static/table.js\r\n@@ -9,6 +9,7 @@ var DROPDOWN_HTML = `
\r\n
  • Show not-blank rows
    • \r\n \r\n

    \r\n+

    \r\n
    `;\r\n \r\n var DROPDOWN_ICON_SVG = `\r\n@@ -166,6 +167,14 @@ var DROPDOWN_ICON_SVG = `\r\n
    \r\n {% for column in display_columns %}\r\n- \r\n+ \r\n {% if not column.sortable %}\r\n {{ column.name }}\r\n {% else %}\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 456d806..486a613 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -125,6 +125,7 @@ class RowTableShared(DataView):\r\n \"\"\"Returns columns, rows for specified table - including fancy foreign key treatment\"\"\"\r\n db = self.ds.databases[database]\r\n table_metadata = self.ds.table_metadata(database, table)\r\n+ column_descriptions = table_metadata.get(\"columns\") or {}\r\n column_details = {col.name: col for col in await db.table_column_details(table)}\r\n sortable_columns = await self.sortable_columns_for_table(database, table, True)\r\n pks = await db.primary_keys(table)\r\n@@ -147,6 +148,7 @@ class RowTableShared(DataView):\r\n \"is_pk\": r[0] in pks_for_display,\r\n \"type\": type_,\r\n \"notnull\": notnull,\r\n+ \"description\": column_descriptions.get(r[0]),\r\n }\r\n )\r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-897996296", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 897996296, "node_id": "IC_kwDOBm6k_c41hlYI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T22:01:36Z", "updated_at": "2021-08-12T22:01:36Z", "author_association": "OWNER", "body": "I'm going with `\"columns\": {\"name-of-column\": \"description-of-column\"}`.\r\n\r\nIf I decide to make `\"col\"` and `\"nocol\"` available in metadata I'll use those as the keys in the metadata, for consistency with the existing query string parameters.\r\n\r\nI'm OK with having both `\"columns\": ...` and `\"col\": ...` keys in the metadata, even though they could be a tiny bit confusing without the documentation.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1429#issuecomment-897960049", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1429", "id": 897960049, "node_id": "IC_kwDOBm6k_c41hchx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T20:53:04Z", "updated_at": "2021-08-12T20:53:04Z", "author_association": "OWNER", "body": "Maybe something like this:\r\n\r\n> [Next page](#) - 100 per page ([show 1,000 per page](#))", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 969548935, "label": "UI for setting `?_size=max` on table page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-897600677", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 897600677, "node_id": "IC_kwDOCGYnMM41gEyl", "user": {"value": 9308268, "label": "rayvoelker"}, "created_at": "2021-08-12T12:32:14Z", "updated_at": "2021-08-12T12:32:14Z", "author_association": "NONE", "body": "Actually, I forgot to include the `bib_pub_year` in the extract ... \r\n\r\nBut also, I tried again with empty string values instead of `NULL` values and it seems to place the foreign key properly / correctly... \r\n\r\n```python3\r\nsql = \"\"\"\\\r\nINSERT INTO \"circulation_info\" (\"item_id\", \"bib_title\", \"bib_creator\", \"bib_format\", \"bib_pub_year\", \"checkout_date\")\r\nVALUES\r\n(1, \"title one\", \"creator one\", \"Book\", 2018, \"2021-08-12 00:01\"),\r\n(2, \"title two\", \"creator one\", \"Book\", 2019, \"2021-08-12 00:02\"),\r\n(3, \"title three\", \"\", \"DVD\", 2020, \"2021-08-12 00:03\"),\r\n(4, \"title four\", \"\", \"DVD\", \"\", \"2021-08-12 00:04\"),\r\n(5, \"title five\", \"\", \"DVD\", \"\", \"2021-08-12 00:05\")\r\n\"\"\"\r\n\r\nwith sqlite3.connect('test_bib_2.db') as con:\r\n con.execute(sql)\r\n```\r\n\r\n```python3\r\ndb[\"circulation_info\"].extract(\r\n [\r\n \"bib_title\",\r\n \"bib_creator\",\r\n \"bib_format\",\r\n \"bib_pub_year\"\r\n ],\r\n table=\"bib_info\", \r\n fk_column=\"bib_info_id\"\r\n)\r\n```\r\n\r\n```\r\n{'id': 1, 'item_id': 1, 'bib_info_id': 1, 'bib_pub_year': 2018, 'checkout_date': '2021-08-12 00:01'}\r\n{'id': 2, 'item_id': 2, 'bib_info_id': 2, 'bib_pub_year': 2019, 'checkout_date': '2021-08-12 00:02'}\r\n{'id': 3, 'item_id': 3, 'bib_info_id': 3, 'bib_pub_year': 2020, 'checkout_date': '2021-08-12 00:03'}\r\n{'id': 4, 'item_id': 4, 'bib_info_id': 4, 'bib_pub_year': '', 'checkout_date': '2021-08-12 00:04'}\r\n{'id': 5, 'item_id': 5, 'bib_info_id': 5, 'bib_pub_year': '', 'checkout_date': '2021-08-12 00:05'}\r\n\r\n---\r\n\r\n{'id': 1, 'bib_title': 'title one', 'bib_creator': 'creator one', 'bib_format': 'Book'}\r\n{'id': 2, 'bib_title': 'title two', 'bib_creator': 'creator one', 'bib_format': 'Book'}\r\n{'id': 3, 'bib_title': 'title three', 'bib_creator': '', 'bib_format': 'DVD'}\r\n{'id': 4, 'bib_title': 'title four', 'bib_creator': '', 'bib_format': 'DVD'}\r\n{'id': 5, 'bib_title': 'title five', 'bib_creator': '', 'bib_format': 'DVD'}\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-897588624", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 897588624, "node_id": "IC_kwDOCGYnMM41gB2Q", "user": {"value": 9308268, "label": "rayvoelker"}, "created_at": "2021-08-12T12:13:25Z", "updated_at": "2021-08-12T12:13:25Z", "author_association": "NONE", "body": "I think I ran into an issue that's perhaps related with `extract()`\r\n\r\nI have a case where I want to create a lookup table for all the related title data where there are possibly multiple null values in the related columns ....\r\n```python3\r\nsql = \"\"\"\\\r\nINSERT INTO \"circulation_info\" (\"item_id\", \"bib_title\", \"bib_creator\", \"bib_format\", \"bib_pub_year\", \"checkout_date\")\r\nVALUES\r\n(1, \"title one\", \"creator one\", \"Book\", 2018, \"2021-08-12 00:01\"),\r\n(2, \"title two\", \"creator one\", \"Book\", 2019, \"2021-08-12 00:02\"),\r\n(3, \"title three\", NULL, \"DVD\", 2020, \"2021-08-12 00:03\"),\r\n(4, \"title four\", NULL, \"DVD\", NULL, \"2021-08-12 00:04\"),\r\n(5, \"title five\", NULL, \"DVD\", NULL, \"2021-08-12 00:05\")\r\n\"\"\"\r\n\r\nwith sqlite3.connect('test_bib.db') as con:\r\n con.execute(sql)\r\n```\r\n\r\nwhen I run the `extract()` method ... \r\n\r\n```python3\r\ndb[\"circulation_info\"].extract(\r\n [\r\n \"bib_title\",\r\n \"bib_creator\",\r\n \"bib_format\" \r\n ],\r\n table=\"bib_info\", \r\n fk_column=\"bib_info_id\"\r\n)\r\n\r\ndb = sqlite_utils.Database(\"test_bib.db\")\r\n\r\nfor row in db[\"circulation_info\"].rows:\r\n print(row)\r\n\r\nprint(\"\\n---\\n\")\r\n\r\nfor row in db[\"bib_info\"].rows:\r\n print(row)\r\n```\r\n\r\nresults in this .. \r\n```\r\n{'id': 1, 'item_id': 1, 'bib_info_id': 1, 'bib_pub_year': 2018, 'checkout_date': '2021-08-12 00:01'}\r\n{'id': 2, 'item_id': 2, 'bib_info_id': 2, 'bib_pub_year': 2019, 'checkout_date': '2021-08-12 00:02'}\r\n{'id': 3, 'item_id': 3, 'bib_info_id': None, 'bib_pub_year': 2020, 'checkout_date': '2021-08-12 00:03'}\r\n{'id': 4, 'item_id': 4, 'bib_info_id': None, 'bib_pub_year': None, 'checkout_date': '2021-08-12 00:04'}\r\n{'id': 5, 'item_id': 5, 'bib_info_id': None, 'bib_pub_year': None, 'checkout_date': '2021-08-12 00:05'}\r\n\r\n---\r\n\r\n{'id': 1, 'bib_title': 'title one', 'bib_creator': 'creator one', 'bib_format': 'Book'}\r\n{'id': 2, 'bib_title': 'title two', 'bib_creator': 'creator one', 'bib_format': 'Book'}\r\n{'id': 3, 'bib_title': 'title three', 'bib_creator': None, 'bib_format': 'DVD'}\r\n{'id': 4, 'bib_title': 'title four', 'bib_creator': None, 'bib_format': 'DVD'}\r\n{'id': 5, 'bib_title': 'title five', 'bib_creator': None, 'bib_format': 'DVD'}\r\n```\r\n\r\nSeems like it's correctly generating the row data for those lookups, but it's not correctly updating the foreign key back to the primary table? Looks like it just results in a `NULL` value in that original table.\r\n\r\nAny ideas on why? Thanks again!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/311#issuecomment-896381184", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/311", "id": 896381184, "node_id": "IC_kwDOCGYnMM41bbEA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T23:33:33Z", "updated_at": "2021-08-10T23:33:33Z", "author_association": "OWNER", "body": "Now live at https://sqlite-utils.datasette.io/en/latest/reference.html\r\n\r\nTIL from what I learned today here: https://til.simonwillison.net/sphinx/sphinx-autodoc", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965102534, "label": "Add reference documentation generated from docstrings"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-896378525", "issue_url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8", "id": 896378525, "node_id": "IC_kwDODFE5qs41baad", "user": {"value": 28565, "label": "maxhawkins"}, "created_at": "2021-08-10T23:28:45Z", "updated_at": "2021-08-10T23:28:45Z", "author_association": "NONE", "body": "I added parsing of text/html emails using BeautifulSoup.\r\n\r\nAround half of the emails in my archive don't include a text/plain payload so adding html parsing makes a good chunk of them searchable.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 954546309, "label": "Add Gmail takeout mbox import (v2)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896162082", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896162082, "node_id": "IC_kwDOCGYnMM41alki", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-08-10T17:10:39Z", "updated_at": "2021-08-10T23:07:35Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#312](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (43bc064) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/ee469e3122d6f5973ec2584c1580d930daca2e7c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (ee469e3) will **decrease** coverage by `0.02%`.\n> The diff coverage is `96.84%`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/312/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@ Coverage Diff @@\n## main #312 +/- ##\n==========================================\n- Coverage 96.30% 96.28% -0.03% \n==========================================\n Files 5 5 \n Lines 2168 2179 +11 \n==========================================\n+ Hits 2088 2098 +10 \n- Misses 80 81 +1 \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [sqlite\\_utils/db.py](https://codecov.io/gh/simonw/sqlite-utils/pull/312/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.91% <96.84%> (-0.08%)` | :arrow_down: |\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [ee469e3...43bc064](https://codecov.io/gh/simonw/sqlite-utils/pull/312?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/314#issuecomment-896369551", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/314", "id": 896369551, "node_id": "IC_kwDOCGYnMM41bYOP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T23:06:41Z", "updated_at": "2021-08-10T23:06:41Z", "author_association": "OWNER", "body": "I took a big bite out of this when I annotated the ``.insert()`` method - but there are a bunch of other places that still need doing: https://github.com/simonw/sqlite-utils/blob/43bc06481783c3cfcee70c0cb541a686e8894adb/sqlite_utils/db.py#L2382-L2397\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965210966, "label": "Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/314#issuecomment-896344833", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/314", "id": 896344833, "node_id": "IC_kwDOCGYnMM41bSMB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T22:07:34Z", "updated_at": "2021-08-10T22:07:34Z", "author_association": "OWNER", "body": "Also the `.insert()` family of methods - they look pretty ugly in Sphinx right now:\r\n\r\n
    \"API_Reference_\u2014_sqlite-utils_3_15-16-gf51b712_documentation\"\r\n\r\nI should probably define reusable types for things like `pk=`, which have complex type signatures (a string or a list/tuple of strings) and show up in multiple places.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965210966, "label": "Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/315#issuecomment-896339144", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/315", "id": 896339144, "node_id": "IC_kwDOCGYnMM41bQzI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T21:55:41Z", "updated_at": "2021-08-10T21:55:41Z", "author_association": "OWNER", "body": "Or should we raise an error if you attempt to call `.delete_where()` on a table that doesn't exist?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965440017, "label": "`.delete_where()` returns `[]` when it should return self"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896284722", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896284722, "node_id": "IC_kwDOCGYnMM41bDgy", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T20:08:03Z", "updated_at": "2021-08-10T20:08:21Z", "author_association": "OWNER", "body": "Spotted a rogue backtick:\r\n\r\n![A0147E27-7506-49B0-BEFB-20D99BBFEBAD](https://user-images.githubusercontent.com/9599/128927930-b3333dee-a385-409b-a945-f108e6ea40df.jpeg)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896200682", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896200682, "node_id": "IC_kwDOCGYnMM41au_q", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T18:03:40Z", "updated_at": "2021-08-10T18:03:40Z", "author_association": "OWNER", "body": "Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896186025", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896186025, "node_id": "IC_kwDOCGYnMM41arap", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:42:51Z", "updated_at": "2021-08-10T17:42:51Z", "author_association": "OWNER", "body": "That worked! https://sqlite-utils.datasette.io/en/autodoc/reference.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896182934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896182934, "node_id": "IC_kwDOCGYnMM41aqqW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:38:44Z", "updated_at": "2021-08-10T17:38:44Z", "author_association": "OWNER", "body": "From https://docs.readthedocs.io/en/stable/config-file/v2.html#packages it looks like I can tell Read The Docs to run `pip install -e .` using a `.readthedocs.yaml` configuration:\r\n\r\n```yaml\r\nversion: 2\r\n\r\nsphinx:\r\n configuration: docs/conf.py\r\n\r\npython:\r\n version: \"3.9\"\r\n install:\r\n - method: pip\r\n path: .\r\n extra_requirements:\r\n - docs\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896180956", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896180956, "node_id": "IC_kwDOCGYnMM41aqLc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:35:51Z", "updated_at": "2021-08-10T17:35:51Z", "author_association": "OWNER", "body": "Reading the rest of https://sphinx-rtd-tutorial.readthedocs.io/en/latest/sphinx-config.html#autodoc-configuration it suggests using a `requirements.txt` file to install dependencies - but I use `setup.py` for that so I need to figure out a different pattern here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896175438", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896175438, "node_id": "IC_kwDOCGYnMM41ao1O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:28:19Z", "updated_at": "2021-08-10T17:28:19Z", "author_association": "OWNER", "body": "https://sphinx-rtd-tutorial.readthedocs.io/en/latest/sphinx-config.html#autodoc-configuration says do something like this at the top of `conf.py`:\r\n\r\n```python\r\nimport os\r\nimport sys\r\nsys.path.insert(0, os.path.abspath('../../simpleble/'))\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896174456", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896174456, "node_id": "IC_kwDOCGYnMM41aol4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:27:01Z", "updated_at": "2021-08-10T17:27:01Z", "author_association": "OWNER", "body": "Docs are now building at https://sqlite-utils.datasette.io/en/autodoc/reference.html\r\n\r\nBut there's a problem! The page is semi-blank:\r\n\r\n\"Reference_\u2014_sqlite-utils_3_15-6-gc11ff89_documentation\"\r\n\r\nI need to teach Read The Docs how to ensure `sqlite_utils` is available for introspection.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896156971", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896156971, "node_id": "IC_kwDOCGYnMM41akUr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:04:22Z", "updated_at": "2021-08-10T17:05:59Z", "author_association": "OWNER", "body": "I'm going to get Read The Docs to build the docs for this branch too - on https://readthedocs.org/projects/sqlite-utils/versions/ I am clicking this button:\r\n\r\n\"Versions___Read_the_Docs\"\r\n\r\nI then set it to \"active\" (so pushes to the branch will build it) and \"hidden\" (so it wouldn't show up in search or in the navigation menu). https://docs.readthedocs.io/en/stable/versions.html#version-states\r\n\r\n\"autodoc_-_sqlite-utils___Read_the_Docs\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/312#issuecomment-896154028", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312", "id": 896154028, "node_id": "IC_kwDOCGYnMM41ajms", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T17:01:06Z", "updated_at": "2021-08-10T17:01:06Z", "author_association": "OWNER", "body": "On Python 3.6:\r\n\r\n```\r\nsqlite_utils/db.py:366: in Database\r\n def tables(self) -> List[Table]:\r\nE NameError: name 'Table' is not defined\r\n```\r\nPython 3.7 can fix this with `from __future__ import annotations` but since we still support 3.6 I'll have to use a string instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965143346, "label": "Add reference page to documentation using Sphinx autodoc"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/311#issuecomment-896152812", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/311", "id": 896152812, "node_id": "IC_kwDOCGYnMM41ajTs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T16:59:34Z", "updated_at": "2021-08-10T16:59:34Z", "author_association": "OWNER", "body": "Work will continue in PR #312.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965102534, "label": "Add reference documentation generated from docstrings"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/311#issuecomment-896149590", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/311", "id": 896149590, "node_id": "IC_kwDOCGYnMM41aihW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T16:55:36Z", "updated_at": "2021-08-10T16:55:36Z", "author_association": "OWNER", "body": "I'm going to use this as an excuse to add a bunch more type signatures too, refs #266.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965102534, "label": "Add reference documentation generated from docstrings"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/311#issuecomment-896131902", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/311", "id": 896131902, "node_id": "IC_kwDOCGYnMM41aeM-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-10T16:31:51Z", "updated_at": "2021-08-10T16:31:51Z", "author_association": "OWNER", "body": "`make livehtml` wasn't picking up changes I made to the docstrings `.py` files.\r\n\r\nFix was to change it to this:\r\n```\r\nsphinx-autobuild -a -b html \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(0) --watch ../sqlite_utils\r\n```\r\nSee https://github.com/executablebooks/sphinx-autobuild#relevant-sphinx-bugs - though that suggested `-a` but didn't suggest `--watch`, which is a tip I got from https://github.com/executablebooks/sphinx-autobuild#working-on-a-sphinx-html-theme\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 965102534, "label": "Add reference documentation generated from docstrings"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895622908", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/309", "id": 895622908, "node_id": "IC_kwDOCGYnMM41Yh78", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T23:40:29Z", "updated_at": "2021-08-09T23:40:29Z", "author_association": "OWNER", "body": "TIL about how the stack inspection works: https://til.simonwillison.net/python/find-local-variables-in-exception-traceback", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963897111, "label": "sqlite-utils insert errors should show SQL and parameters, if possible"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895581038", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/309", "id": 895581038, "node_id": "IC_kwDOCGYnMM41YXtu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T22:03:54Z", "updated_at": "2021-08-09T23:39:53Z", "author_association": "OWNER", "body": "Steps to reproduce:\r\n\r\n echo '{\"v\": 34223049823094832094802398430298048240}' | sqlite-utils insert /tmp/blah.db row -", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963897111, "label": "sqlite-utils insert errors should show SQL and parameters, if possible"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895592507", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/309", "id": 895592507, "node_id": "IC_kwDOCGYnMM41Yag7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T22:26:28Z", "updated_at": "2021-08-09T22:33:48Z", "author_association": "OWNER", "body": "Demo:\r\n```\r\n$ echo '{\"v\": 34223049823094832094802398430298048240}' | sqlite-utils insert /tmp/blah.db row - \r\nError: Python int too large to convert to SQLite INTEGER\r\n\r\nsql = INSERT INTO [row] ([v]) VALUES (?);\r\nparameters = [34223049823094832094802398430298048240]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963897111, "label": "sqlite-utils insert errors should show SQL and parameters, if possible"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895587441", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/309", "id": 895587441, "node_id": "IC_kwDOCGYnMM41YZRx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T22:15:45Z", "updated_at": "2021-08-09T22:15:45Z", "author_association": "OWNER", "body": "```\r\nOverflowError: Python int too large to convert to SQLite INTEGER\r\n>>> import sys\r\n>>> def find_variables(tb, vars):\r\n to_find = list(vars)\r\n found = {}\r\n for var in to_find:\r\n if var in tb.tb_frame.f_locals:\r\n vars.remove(var)\r\n found[var] = tb.tb_frame.f_locals[var]\r\n if vars and tb.tb_next:\r\n found.update(find_variables(tb.tb_next, vars))\r\n return found\r\n... \r\n>>> find_variables(sys.last_traceback, [\"sql\", \"params\"])\r\n{'params': [34223049823094832094802398430298048240], 'sql': 'INSERT INTO [row] ([v]) VALUES (?);'}\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963897111, "label": "sqlite-utils insert errors should show SQL and parameters, if possible"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895587282", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/309", "id": 895587282, "node_id": "IC_kwDOCGYnMM41YZPS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T22:15:25Z", "updated_at": "2021-08-09T22:15:25Z", "author_association": "OWNER", "body": "I'm going to use a bit of a dirty trick for this one: I'm going to recursively inspect the stack on an error and try to find the `sql` and `params` variables.\r\n\r\nThat way I can handle this all at the CLI layer without changing the exceptions that are being raised by the Python library.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963897111, "label": "sqlite-utils insert errors should show SQL and parameters, if possible"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/309#issuecomment-895577012", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/309", "id": 895577012, "node_id": "IC_kwDOCGYnMM41YWu0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T21:55:52Z", "updated_at": "2021-08-09T21:59:03Z", "author_association": "OWNER", "body": "Yeah this error message could certainly be more helpful.\r\n\r\nI thought `OverflowError` might be one of the SQLite exceptions: https://docs.python.org/3/library/sqlite3.html#exceptions - but it turns out it's actually reusing the Python built-in `OverflowError` class:\r\n```python\r\nimport sqlite3\r\ndb = sqlite3.connect(\":memory:\")\r\ncaught = []\r\ntry:\r\n db.execute(\"create table foo (number integer)\")\r\n db.execute(\"insert into foo (number) values (?)\", [34223049823094832094802398430298048240])\r\nexcept Exception as e:\r\n print(e)\r\n caught.append(e)\r\nisinstance(caught[0], OverflowError)\r\n```\r\nHere's where that happens in the Python `sqlite3` module code: https://github.com/python/cpython/blob/058fb35b57ca8c5063d16ec818e668b3babfea65/Modules/_sqlite/util.c#L123-L124", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963897111, "label": "sqlite-utils insert errors should show SQL and parameters, if possible"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/310#issuecomment-895572309", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/310", "id": 895572309, "node_id": "IC_kwDOCGYnMM41YVlV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T21:46:15Z", "updated_at": "2021-08-09T21:46:15Z", "author_association": "OWNER", "body": "Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#flattening-nested-json-objects", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 964400482, "label": "`sqlite-utils insert --flatten` option to flatten nested JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/310#issuecomment-895571420", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/310", "id": 895571420, "node_id": "IC_kwDOCGYnMM41YVXc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T21:44:38Z", "updated_at": "2021-08-09T21:44:38Z", "author_association": "OWNER", "body": "When I ship this I should update the TILs at https://til.simonwillison.net/cloudrun/tailing-cloud-run-request-logs and https://til.simonwillison.net/jq/flatten-nested-json-objects-jq to reference it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 964400482, "label": "`sqlite-utils insert --flatten` option to flatten nested JSON"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1426#issuecomment-895522818", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1426", "id": 895522818, "node_id": "IC_kwDOBm6k_c41YJgC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T20:34:10Z", "updated_at": "2021-08-09T20:34:10Z", "author_association": "OWNER", "body": "At the very least Datasette should serve a blank `/robots.txt` by default - I'm seeing a ton of 404s for it in the logs.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 964322136, "label": "Manage /robots.txt in Datasette core, block robots by default"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1426#issuecomment-895510773", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1426", "id": 895510773, "node_id": "IC_kwDOBm6k_c41YGj1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T20:14:50Z", "updated_at": "2021-08-09T20:19:22Z", "author_association": "OWNER", "body": "https://twitter.com/mal/status/1424825895139876870\r\n\r\n> True pinging google should be part of the build process on a static site :)\r\n\r\nThat's another aspect of this: if you DO want your site crawled, teaching the `datasette publish` command how to ping Google when a deploy has gone out could be a nice improvement.\r\n\r\nAnnoyingly it looks like you need to configure an auth token of some sort in order to use their API though, which is likely too much hassle to be worth building into Datasette itself: https://developers.google.com/search/apis/indexing-api/v3/using-api\r\n\r\n```\r\ncurl -X POST https://indexing.googleapis.com/v3/urlNotifications:publish -d '{\r\n \"url\": \"https://careers.google.com/jobs/google/technical-writer\",\r\n \"type\": \"URL_UPDATED\"\r\n}' -H \"Content-Type: application/json\"\r\n\r\n{\r\n \"error\": {\r\n \"code\": 401,\r\n \"message\": \"Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.\",\r\n \"status\": \"UNAUTHENTICATED\"\r\n }\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 964322136, "label": "Manage /robots.txt in Datasette core, block robots by default"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1426#issuecomment-895509536", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1426", "id": 895509536, "node_id": "IC_kwDOBm6k_c41YGQg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T20:12:57Z", "updated_at": "2021-08-09T20:12:57Z", "author_association": "OWNER", "body": "I could try out the `X-Robots` HTTP header too: https://developers.google.com/search/docs/advanced/robots/robots_meta_tag#xrobotstag", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 964322136, "label": "Manage /robots.txt in Datasette core, block robots by default"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1426#issuecomment-895500565", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1426", "id": 895500565, "node_id": "IC_kwDOBm6k_c41YEEV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T20:00:04Z", "updated_at": "2021-08-09T20:00:04Z", "author_association": "OWNER", "body": "A few options for how this would work:\r\n\r\n- `datasette ... --robots allow`\r\n- `datasette ... --setting robots allow`\r\n\r\nOptions could be:\r\n\r\n- `allow` - allow all crawling\r\n- `deny` - deny all crawling\r\n- `limited` - allow access to the homepage and the index pages for each database and each table, but disallow crawling any further than that\r\n\r\nThe \"limited\" mode is particularly interesting. Could even make it the default, but I think that may be a bit too confusing. Idea would be to get the key pages indexed but use `nofollow` to discourage crawlers from indexing individual row pages or deep pages like `https://datasette.io/content/repos?_facet=owner&_facet=language&_facet_array=topics&topics__arraycontains=sqlite#facet-owner`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 964322136, "label": "Manage /robots.txt in Datasette core, block robots by default"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-895003796", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 895003796, "node_id": "IC_kwDOBm6k_c41WKyU", "user": {"value": 3243482, "label": "abdusco"}, "created_at": "2021-08-09T07:14:35Z", "updated_at": "2021-08-09T07:14:35Z", "author_association": "CONTRIBUTOR", "body": "I believe this also provides a workaround for the problem I face in https://github.com/simonw/datasette/issues/1300. \r\n\r\nNow I should be able to get table PKs and generate a row URL. I'll test this out and report my findings.\r\n\r\n\r\n```py\r\nfrom datasette.utils import path_from_row_pks\r\n\r\npks = await db.primary_keys(table)\r\nurl = self.ds.urls.row_blob(\r\n database,\r\n table,\r\n path_from_row_pks(row, pks, not pks),\r\n column,\r\n)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894930013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894930013, "node_id": "IC_kwDOBm6k_c41V4xd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:38:06Z", "updated_at": "2021-08-09T03:38:06Z", "author_association": "OWNER", "body": "Amusing edge-case: if you run this against a `explain ...` query it falls back to using regular expressions, because `explain explain select ...` is invalid SQL. https://latest.datasette.io/fixtures?sql=explain+select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27&state=&on_earth=", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894929769", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894929769, "node_id": "IC_kwDOBm6k_c41V4tp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:36:49Z", "updated_at": "2021-08-09T03:36:49Z", "author_association": "OWNER", "body": "SQLite carries a warning about using `EXPLAIN` like this: https://www.sqlite.org/lang_explain.html\r\n\r\n> The output from EXPLAIN and EXPLAIN QUERY PLAN is intended for interactive analysis and troubleshooting only. The details of the output format are subject to change from one release of SQLite to the next. Applications should not use EXPLAIN or EXPLAIN QUERY PLAN since their exact behavior is variable and only partially documented.\r\n\r\nI think that's OK here, because of the regular expression fallback. If the format changes in the future in a way that breaks the query the error should be caught and the regex-captured parameters should be returned instead.\r\n\r\nHmmm... actually that's not entirely true:\r\n\r\nhttps://github.com/simonw/datasette/blob/b1fed48a95516ae84c0f020582303ab50ab817e2/datasette/utils/__init__.py#L1084-L1091\r\n\r\nIf the format changes such that the same columns are returned but the `[row[\"p4\"].lstrip(\":\") for row in results if row[\"opcode\"] == \"Variable\"]` list comprehension returns an empty array it will break Datasette!\r\n\r\nI'm going to take that risk for the moment, but I'll actively watch out for problems in the future. If this does turn out to be bad I can always go back to the pure regular expression mechanism.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894929080", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894929080, "node_id": "IC_kwDOBm6k_c41V4i4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:33:02Z", "updated_at": "2021-08-09T03:33:02Z", "author_association": "OWNER", "body": "Fixed! Fantastic, this one has been bothering me for *years*.\r\n\r\nhttps://latest.datasette.io/fixtures?sql=select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27\r\n\r\n\"fixtures__select___from_facetable_where_state____state_and_on_earth____on_earth_and_neighborhood_not_like__00_04__and_pyinfra_pip_py_at_current_\u00b7_Fizzadar_pyinfra\"\r\n\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894927185", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894927185, "node_id": "IC_kwDOBm6k_c41V4FR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:25:01Z", "updated_at": "2021-08-09T03:25:01Z", "author_association": "OWNER", "body": "One catch with this approach: if the SQL query is invalid, the parameters will not be extracted and shown as form fields.\r\n\r\nMaybe that's completely fine? Why display a form if it's going to break when the user actually runs the query?\r\n\r\nBut it does bother me. I worry that someone who is still iterating on and editing their query before actually starting to use it might find the behaviour confusing.\r\n\r\nSo maybe if the query raises an exception it could fall back on the regular expression results?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894925914", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894925914, "node_id": "IC_kwDOBm6k_c41V3xa", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:20:42Z", "updated_at": "2021-08-09T03:20:42Z", "author_association": "OWNER", "body": "I think this works!\r\n\r\n```python\r\n_re_named_parameter = re.compile(\":([a-zA-Z0-9_]+)\")\r\n\r\nasync def derive_named_parameters(db, sql):\r\n explain = 'explain {}'.format(sql.strip().rstrip(\";\"))\r\n possible_params = _re_named_parameter.findall(sql)\r\n try:\r\n results = await db.execute(explain, {p: None for p in possible_params})\r\n return [row[\"p4\"].lstrip(\":\") for row in results if row[\"opcode\"] == \"Variable\"]\r\n except sqlite3.DatabaseError:\r\n return []\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894925437", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894925437, "node_id": "IC_kwDOBm6k_c41V3p9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:19:00Z", "updated_at": "2021-08-09T03:19:00Z", "author_association": "OWNER", "body": "This may not work:\r\n\r\n> `ERROR: sql = 'explain select 1 + :one + :two', params = None: You did not supply a value for binding 1.`\r\n\r\nThe `explain` queries themselves want me to pass them parameters.\r\n\r\nI could try using the regex to pull out candidates and passing `None` for each of those, including incorrect ones like `:31`.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894922703", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894922703, "node_id": "IC_kwDOBm6k_c41V2_P", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:09:29Z", "updated_at": "2021-08-09T03:09:29Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/ad90a72afa21b737b162e2bbdddc301a97d575cd/datasette/views/database.py#L225-L231", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894922145", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894922145, "node_id": "IC_kwDOBm6k_c41V22h", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:07:38Z", "updated_at": "2021-08-09T03:07:38Z", "author_association": "OWNER", "body": "I hoped this would work:\r\n```sql\r\nwith foo as (\r\n explain select * from facetable\r\n where state = :state\r\n and on_earth = :on_earth\r\n and neighborhood not like '00:04'\r\n)\r\nselect p4 from foo where opcode = 'Variable'\r\n```\r\n But sadly [it returns an error](https://latest.datasette.io/fixtures?sql=with+foo+as+%28%0D%0A++explain+select+*+from+facetable%0D%0A++where+state+%3D+%3Astate%0D%0A++and+on_earth+%3D+%3Aon_earth%0D%0A++and+neighborhood+not+like+%2700%3A04%27%0D%0A%29%0D%0Aselect+p4+from+foo+where+opcode+%3D+%27Variable%27&state=&on_earth=&04=):\r\n\r\n> near \"explain\": syntax error", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894921512", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894921512, "node_id": "IC_kwDOBm6k_c41V2so", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T03:05:26Z", "updated_at": "2021-08-09T03:05:26Z", "author_association": "OWNER", "body": "I may have a way to work around this, using `explain`. Consider this query:\r\n\r\n```sql\r\nselect * from facetable\r\nwhere state = :state\r\nand on_earth = :on_earth\r\nand neighborhood not like '00:04'\r\n```\r\nDatasette currently gets confused and shows three form fields: https://latest.datasette.io/fixtures?sql=select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27&state=&on_earth=&04=\r\n\r\n\"fixtures__select___from_facetable_where_state____state_and_on_earth____on_earth_and_neighborhood_not_like__00_04__and_pyinfra_pip_py_at_current_\u00b7_Fizzadar_pyinfra\"\r\n\r\nBut... if I run `explain` [against that](https://latest.datasette.io/fixtures?sql=explain+select+*+from+facetable%0D%0Awhere+state+%3D+%3Astate%0D%0Aand+on_earth+%3D+%3Aon_earth%0D%0Aand+neighborhood+not+like+%2700%3A04%27&state=&on_earth=&04=) I get this (truncated):\r\n\r\naddr | opcode | p1 | p2 | p3 | p4 | p5 | comment\r\n-- | -- | -- | -- | -- | -- | -- | --\r\n20 | ResultRow | 6 | 10 | 0 | \u00a0 | 0 | \u00a0\r\n21 | Next | 0 | 3 | 0 | \u00a0 | 1 | \u00a0\r\n22 | Halt | 0 | 0 | 0 | \u00a0 | 0 | \u00a0\r\n23 | Transaction | 0 | 0 | 35 | 0 | 1 | \u00a0\r\n24 | Variable | 1 | 2 | 0 | :state | 0 | \u00a0\r\n25 | Variable | 2 | 3 | 0 | :on_earth | 0 | \u00a0\r\n26 | String8 | 0 | 4 | 0 | 00:04 | 0 | \u00a0\r\n27 | Goto | 0 | 1 | 0 | \u00a0 | 0 | \u00a0\r\n\r\nCould it be as simple as pulling out those `Variable` rows to figure out the names of the variables in the query?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894900267", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894900267, "node_id": "IC_kwDOBm6k_c41Vxgr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T01:31:22Z", "updated_at": "2021-08-09T01:31:22Z", "author_association": "OWNER", "body": "I used this to build a new plugin: https://github.com/simonw/datasette-query-links\r\n\r\nDemo here: https://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++%27select+*+from+[facetable]%27+as+query%0D%0Aunion%0D%0Aselect%0D%0A++%27select+sqlite_version()%27%0D%0Aunion%0D%0Aselect%0D%0A++%27select+this+is+invalid+SQL+so+will+not+be+linked%27", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894893319", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894893319, "node_id": "IC_kwDOBm6k_c41Vv0H", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T01:08:56Z", "updated_at": "2021-08-09T01:09:12Z", "author_association": "OWNER", "body": "Demo: https://latest.datasette.io/fixtures/simple_primary_key shows `RENDER_CELL_ASYNC_RESULT` where the CSV version shows `RENDER_CELL_ASYNC`: https://latest.datasette.io/fixtures/simple_primary_key.csv - because of this test plugin code: https://github.com/simonw/datasette/blob/a390bdf9cef01d8723d025fc3348e81345ff4856/tests/plugins/my_plugin.py#L98-L122", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894884874", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894884874, "node_id": "IC_kwDOBm6k_c41VtwK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T00:38:20Z", "updated_at": "2021-08-09T00:38:20Z", "author_association": "OWNER", "body": "I'm trying the version where I remove `firstresult=True`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894883664", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894883664, "node_id": "IC_kwDOBm6k_c41VtdQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T00:33:56Z", "updated_at": "2021-08-09T00:33:56Z", "author_association": "OWNER", "body": "I could extract that code out and write my own function which implements the equivalent of calling `pm.hook.render_cell(...)` but runs `await_me_maybe()` before checking if `res is not None`.\r\n\r\nThat's pretty nasty.\r\n\r\nCould I instead call the plugin hook normally, but then have additional logic which says \"if I await it and it returns `None` then try calling the hook again but skip this one\" - not sure if there's a way to do that either.\r\n\r\nI could remove the `firstresult=True` from the hookspec - which would cause it to call and return ALL hooks - but then in my own code use only the first one. This is slightly less efficient (since it calls all the hooks and then discards all-but-one value) but it's the least unpleasant in terms of the code I would have to write - plus I don't think it's going to be THAT common for someone to have multiple expensive `render_cell()` hooks installed at once (they are usually pretty cheap).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894882642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894882642, "node_id": "IC_kwDOBm6k_c41VtNS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T00:29:57Z", "updated_at": "2021-08-09T00:29:57Z", "author_association": "OWNER", "body": "Here's the code in `pluggy` that implements this: https://github.com/pytest-dev/pluggy/blob/0a064fe275060dbdb1fe6e10c888e72bc400fb33/src/pluggy/callers.py#L31-L43\r\n\r\n```python\r\n if hook_impl.hookwrapper:\r\n try:\r\n gen = hook_impl.function(*args)\r\n next(gen) # first yield\r\n teardowns.append(gen)\r\n except StopIteration:\r\n _raise_wrapfail(gen, \"did not yield\")\r\n else:\r\n res = hook_impl.function(*args)\r\n if res is not None:\r\n results.append(res)\r\n if firstresult: # halt further impl calls\r\n break\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894882123", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894882123, "node_id": "IC_kwDOBm6k_c41VtFL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T00:27:43Z", "updated_at": "2021-08-09T00:27:43Z", "author_association": "OWNER", "body": "Good news: `render_cell()` is the only hook to use `firstresult=True`:\r\n\r\nhttps://github.com/simonw/datasette/blob/f3c9edb376a13c09b5ecf97c7390f4e49efaadf2/datasette/hookspecs.py#L62-L64\r\n\r\nhttps://pluggy.readthedocs.io/en/latest/#first-result-only", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894881448", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894881448, "node_id": "IC_kwDOBm6k_c41Vs6o", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T00:24:25Z", "updated_at": "2021-08-09T00:24:39Z", "author_association": "OWNER", "body": "My hunch is that the \"skip this `render_cell()` result if it returns `None`\" logic isn't working correctly, ever since I added the `await_me_maybe` line.\r\n\r\nCould that be because Pluggy handles the \"do the next if `None` is returned\" logic itself, but I'm no-longer returning `None`, I'm returning an awaitable which when awaited returns `None`.\r\n\r\nThis would suggest that all of the `await_me_maybe()` plugin hooks have the same bug. That's definitely possible - it may well be that no-one has yet stumbled across a bug caused by a plugin returning an awaitable and hence not being skipped, because plugin hooks that return awaitable are rare enough that no-one has tried two plugins which both use that trick.\r\n\r\nStill don't see why it would pass on my laptop but fail in CI though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894881016", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894881016, "node_id": "IC_kwDOBm6k_c41Vsz4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-09T00:21:53Z", "updated_at": "2021-08-09T00:21:53Z", "author_association": "OWNER", "body": "Still one test failure:\r\n```\r\n def test_hook_render_cell_link_from_json(app_client):\r\n sql = \"\"\"\r\n select '{\"href\": \"http://example.com/\", \"label\":\"Example\"}'\r\n \"\"\".strip()\r\n path = \"/fixtures?\" + urllib.parse.urlencode({\"sql\": sql})\r\n response = app_client.get(path)\r\n td = Soup(response.body, \"html.parser\").find(\"table\").find(\"tbody\").find(\"td\")\r\n a = td.find(\"a\")\r\n> assert a is not None, str(a)\r\nE AssertionError: None\r\nE assert None is not None\r\n```\r\nThe weird thing about this one is that I can't replicate it on my laptop - but it happens in CI every time, including when I shell in and try to run that single test.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894869692", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894869692, "node_id": "IC_kwDOBm6k_c41VqC8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-08T23:08:29Z", "updated_at": "2021-08-08T23:08:29Z", "author_association": "OWNER", "body": "Updated documentation: https://docs.datasette.io/en/latest/plugin_hooks.html#render-cell-value-column-table-database-datasette", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1425#issuecomment-894865323", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1425", "id": 894865323, "node_id": "IC_kwDOBm6k_c41Vo-r", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-08T22:33:19Z", "updated_at": "2021-08-08T22:33:19Z", "author_association": "OWNER", "body": "I can do this with the `await_me_maybe()` function, as seen here: https://github.com/simonw/datasette/blob/a21853c9dade240734abc6b4f750fae09a3e840a/datasette/app.py#L864-L873", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963528457, "label": "render_cell() hook should support returning an awaitable"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1424#issuecomment-894864744", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1424", "id": 894864744, "node_id": "IC_kwDOBm6k_c41Vo1o", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-08T22:27:31Z", "updated_at": "2021-08-08T22:27:31Z", "author_association": "OWNER", "body": "https://docs.python.org/3/library/sqlite3.html#exceptions is useful - it looks like `sqlite3.DatabaseError` is the super-class of all of the other exceptions that we might see.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963527045, "label": "Document exceptions that can be raised by db.execute() and friends"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1424#issuecomment-894864682", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1424", "id": 894864682, "node_id": "IC_kwDOBm6k_c41Vo0q", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-08T22:26:46Z", "updated_at": "2021-08-08T22:26:46Z", "author_association": "OWNER", "body": "Note that the `sqlite3` exceptions are in `sqlite3` if using the Python standard library but are in `pysqlite3` if that module is being used instead.\r\n\r\nSo maybe encourage people to use them from `datasette.sqlite.sqlite3` instead, which will point to the correct package.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963527045, "label": "Document exceptions that can be raised by db.execute() and friends"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1424#issuecomment-894864616", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1424", "id": 894864616, "node_id": "IC_kwDOBm6k_c41Vozo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-08T22:26:08Z", "updated_at": "2021-08-08T22:26:08Z", "author_association": "OWNER", "body": "- `datasette.database.QueryInterrupted` for queries that were interrupted\r\n- `sqlite3.OperationalError`\r\n- `sqlite3.DatabaseError` and more", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963527045, "label": "Document exceptions that can be raised by db.execute() and friends"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1424#issuecomment-894864404", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1424", "id": 894864404, "node_id": "IC_kwDOBm6k_c41VowU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-08T22:24:06Z", "updated_at": "2021-08-08T22:24:06Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/de5ce2e56339ad8966f417a4758f7c210c017dec/datasette/database.py#L176-L200", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 963527045, "label": "Document exceptions that can be raised by db.execute() and friends"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1422#issuecomment-894607989", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1422", "id": 894607989, "node_id": "IC_kwDOBm6k_c41UqJ1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-07T05:31:57Z", "updated_at": "2021-08-07T05:31:57Z", "author_association": "OWNER", "body": "Demo: https://latest.datasette.io/fixtures/neighborhood_search\r\n\r\nDocumentation: https://docs.datasette.io/en/latest/sql_queries.html#additional-canned-query-options", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 961367843, "label": "Ability to default to hiding the SQL for a canned query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894606843", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894606843, "node_id": "IC_kwDOBm6k_c41Up37", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-07T05:17:12Z", "updated_at": "2021-08-07T05:17:12Z", "author_association": "OWNER", "body": "Marking this blocked because I don't have a way around the needing-a-SQLite-SQL-parser problem at the moment.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1421#issuecomment-894606796", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1421", "id": 894606796, "node_id": "IC_kwDOBm6k_c41Up3M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-07T05:16:39Z", "updated_at": "2021-08-07T05:16:39Z", "author_association": "OWNER", "body": "Urgh, yeah I've seen this one before. Fixing it pretty much requires writing a full SQLite SQL syntax parser in Python, which is frustratingly complicated for solving this issue!\r\n\r\nYou can work around this for a canned query by using the optional `params:` argument documented here: https://docs.datasette.io/en/stable/sql_queries.html#canned-query-parameters", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959999095, "label": "\"Query parameters\" form shows wrong input fields if query contains \"03:31\" style times"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1422#issuecomment-894589140", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1422", "id": 894589140, "node_id": "IC_kwDOBm6k_c41UljU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-07T01:58:16Z", "updated_at": "2021-08-07T01:58:24Z", "author_association": "OWNER", "body": "Also need to consider this hidden field - it should pass the `_hide_sql` or `_show_sql` parameters depending on the same logic: https://github.com/simonw/datasette/blob/acc22436622ff8476c30acf45ed60f54b4aaa5d9/datasette/templates/query.html#L47-L49", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 961367843, "label": "Ability to default to hiding the SQL for a canned query"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-894581223", "issue_url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8", "id": 894581223, "node_id": "IC_kwDODFE5qs41Ujnn", "user": {"value": 28565, "label": "maxhawkins"}, "created_at": "2021-08-07T00:57:48Z", "updated_at": "2021-08-07T00:57:48Z", "author_association": "NONE", "body": "Just added two more fixes:\r\n\r\n* Added parsing for rfc 2047 encoded unicode headers\r\n* Body is now stored as TEXT rather than a BLOB regardless of what order the messages are parsed in.\r\n\r\nI was able to run this on my Takeout export and everything seems to work fine. @simonw let me know if this looks good to merge.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 954546309, "label": "Add Gmail takeout mbox import (v2)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-894454644", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 894454644, "node_id": "IC_kwDOBm6k_c41UEt0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-06T18:52:49Z", "updated_at": "2021-08-06T18:52:49Z", "author_association": "OWNER", "body": "This means that the counts would be unavailable to users who cannot see tooltips (e.g. mobile users) on pages that did not have any facets that broke the 30 limit and hence displayed that \"...\" link.\r\n\r\nI think I'm OK with that, for the moment. May revisit in the future.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-894454087", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 894454087, "node_id": "IC_kwDOBm6k_c41UElH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-06T18:51:42Z", "updated_at": "2021-08-06T18:51:42Z", "author_association": "OWNER", "body": "The invisible tooltip could say \"Showing 30 items, more available\" (helping save you from counting up to 20 if you know about the secret feature). The numbers could then be fully displayed on the \"...\" page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-894453520", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 894453520, "node_id": "IC_kwDOBm6k_c41UEcQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-06T18:50:40Z", "updated_at": "2021-08-06T18:50:40Z", "author_association": "OWNER", "body": "Point of confusion: if only 30 options are shown, but there's a `...` at the end, what would the number be? It can't be the total number of facets because we haven't counted them all - but if it's just the number of displayed facets that's like to be confusing.\r\n\r\nSo the original idea of showing the counts only if `_facet_size=max` is a good one.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-894452990", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 894452990, "node_id": "IC_kwDOBm6k_c41UET-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-06T18:49:37Z", "updated_at": "2021-08-06T18:49:37Z", "author_association": "OWNER", "body": "Could display them always, like this:\r\n\r\n\"fixtures__compound_three_primary_keys__1_001_rows_and_School_enrollment_analysis_Colorado_-_Google_Docs\"\r\n\r\nThat's with `23`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1423#issuecomment-893996604", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1423", "id": 893996604, "node_id": "IC_kwDOBm6k_c41SU48", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-06T04:43:07Z", "updated_at": "2021-08-06T04:43:37Z", "author_association": "OWNER", "body": "Problem: on a page which doesn't have quite enough facet values to trigger the display of the \"...\" link that links to `?_facet_size=max` the user would still have to manually count the values - up to 30 by default.\r\n\r\nSo maybe the count should always be shown, perhaps as a non-bold light colored number?\r\n\r\nI could even hide it in a non-discoverable tooltip.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 962391325, "label": "Show count of facet values if ?_facet_size=max"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1419#issuecomment-893133496", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1419", "id": 893133496, "node_id": "IC_kwDOBm6k_c41PCK4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-05T03:22:44Z", "updated_at": "2021-08-05T03:22:44Z", "author_association": "OWNER", "body": "I ran into this exact same problem today! I only just learned how to use filter on aggregates: https://til.simonwillison.net/sqlite/sqlite-aggregate-filter-clauses\r\n\r\nA workaround I used is to add this to the deploy command:\r\n\r\n datasette publish cloudrun ... --install=pysqlite3-binary\r\n\r\nThis will install the https://pypi.org/project/pysqlite3-binary for package which bundles a more recent SQLite version.", "reactions": "{\"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959710008, "label": "`publish cloudrun` should deploy a more recent SQLite version"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1422#issuecomment-893131703", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1422", "id": 893131703, "node_id": "IC_kwDOBm6k_c41PBu3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-05T03:16:46Z", "updated_at": "2021-08-05T03:16:46Z", "author_association": "OWNER", "body": "The logic for this is a little bit fiddly, due to the need to switch to using `?_show_sql=1` on the link depending on the context.\r\n\r\n- If metadata says hide and there's no query string, hide and link to `?_show_sql=1`\r\n- If metadata says hide but query string says `?_show_sql=1`, show and have hide link linking to URL without `?_show_sql=1`\r\n- Otherwise, show and link to `?_hide_sql=1`\r\n- ... or if that query string is there then hide and link to URL without `?_hide_sql=1`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 961367843, "label": "Ability to default to hiding the SQL for a canned query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1422#issuecomment-893122356", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1422", "id": 893122356, "node_id": "IC_kwDOBm6k_c41O_c0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-05T02:52:31Z", "updated_at": "2021-08-05T02:52:44Z", "author_association": "OWNER", "body": "If you do this it should still be possible to view the SQL - which means we need a new parameter. I propose `?_show_sql=1` to over-ride the hidden default.\r\n\r\nI think the configuration should use `hide_sql: true` - looking like this:\r\n```yaml\r\ndatabases:\r\n fixtures:\r\n queries:\r\n neighborhood_search:\r\n hide_sql: true\r\n sql: |-\r\n select neighborhood, facet_cities.name, state\r\n from facetable\r\n join facet_cities on facetable.city_id = facet_cities.id\r\n where neighborhood like '%' || :text || '%'\r\n order by neighborhood\r\n title: Search neighborhoods\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 961367843, "label": "Ability to default to hiding the SQL for a canned query"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1419#issuecomment-893114612", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1419", "id": 893114612, "node_id": "IC_kwDOBm6k_c41O9j0", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2021-08-05T02:29:06Z", "updated_at": "2021-08-05T02:29:06Z", "author_association": "CONTRIBUTOR", "body": "there's a lot of complexity here, that's probably not worth addressing. i got what i needed by patching the dockerfile that cloudrun uses to install a newer version of sqlite.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959710008, "label": "`publish cloudrun` should deploy a more recent SQLite version"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1420#issuecomment-893079520", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1420", "id": 893079520, "node_id": "IC_kwDOBm6k_c41O0_g", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-05T00:54:59Z", "updated_at": "2021-08-05T00:54:59Z", "author_association": "OWNER", "body": "Just saw this error:\r\n\r\n`ERROR: (gcloud.run.deploy) The `--cpu` flag is not supported on the fully managed version of Cloud Run. Specify `--platform gke` or run `gcloud config set run/platform gke` to work with Cloud Run for Anthos deployed on Google Cloud.`\r\n\r\nWhich is weird because I managed to run this successfully the other day. Maybe a region difference thing?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959898166, "label": "`datasette publish cloudrun --cpu X` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/308#issuecomment-892988609", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/308", "id": 892988609, "node_id": "IC_kwDOCGYnMM41OezB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-04T21:30:59Z", "updated_at": "2021-08-04T21:30:59Z", "author_association": "OWNER", "body": "OK, this works great as a proof-of-concept (though maybe I should remove the output from the cells so people have to run it themselves in the Binder environment?)\r\n\r\nNext challenge: how extensive should it be? What features should I cover? Should I do a basic tutorial and then several advanced tutorials?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 961008507, "label": "Add an interactive tutorial as a Jupyter notebook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/308#issuecomment-892957281", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/308", "id": 892957281, "node_id": "IC_kwDOCGYnMM41OXJh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-04T20:37:00Z", "updated_at": "2021-08-04T20:37:00Z", "author_association": "OWNER", "body": "https://mybinder.org/v2/gh/simonw/sqlite-utils/main?filepath=docs%2Ftutorial.ipynb", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 961008507, "label": "Add an interactive tutorial as a Jupyter notebook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1420#issuecomment-892376353", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1420", "id": 892376353, "node_id": "IC_kwDOBm6k_c41MJUh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-04T05:33:12Z", "updated_at": "2021-08-04T05:33:12Z", "author_association": "OWNER", "body": "In the Cloud Run console (before I deleted these services) when I click \"Edit and deploy new revision\" I see this for the default one:\r\n\r\n\"Deploy_new_revision_\u2013_Cloud_Run_\u2013_datasette_\u2013_Google_Cloud_Platform\"\r\n\r\nAnd this for the big one:\r\n\r\n\"Deploy_new_revision_\u2013_Cloud_Run_\u2013_datasette_\u2013_Google_Cloud_Platform\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959898166, "label": "`datasette publish cloudrun --cpu X` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1420#issuecomment-892374253", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1420", "id": 892374253, "node_id": "IC_kwDOBm6k_c41MIzt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-04T05:27:21Z", "updated_at": "2021-08-04T05:29:59Z", "author_association": "OWNER", "body": "I'll delete these services shortly afterwards. Right now:\r\n\r\nhttps://fixtures-over-provisioned-issue-1420-j7hipcg4aq-uc.a.run.app/-/psutil (deployed with `--memory 8G --cpu 4`) returns:\r\n```\r\nprocess.memory_info()\r\n pmem(rss=60456960, vms=518930432, shared=0, text=0, lib=0, data=0, dirty=0)\r\n\r\n...\r\n\r\npsutil.cpu_times(True)\r\n scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)\r\n scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)\r\n scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)\r\n scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)\r\n\r\npsutil.virtual_memory()\r\n svmem(total=2147483648, available=2092531712, percent=2.6, used=33103872, free=2092531712, active=44130304, inactive=10792960, buffers=0, cached=21848064, shared=262144, slab=0)\r\n```\r\nhttps://fixtures-default-issue-1420-j7hipcg4aq-uc.a.run.app/-/psutil returns:\r\n```\r\nprocess.memory_info()\r\n pmem(rss=49324032, vms=140595200, shared=0, text=0, lib=0, data=0, dirty=0)\r\n\r\n...\r\n\r\npsutil.cpu_times(True)\r\n scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)\r\n scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0)\r\n\r\npsutil.virtual_memory()\r\n svmem(total=2147483648, available=2091188224, percent=2.6, used=40071168, free=2091188224, active=41586688, inactive=7983104, buffers=0, cached=16224256, shared=262144, slab=0)\r\n```\r\nThese numbers are different enough that I assume this works as advertised. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959898166, "label": "`datasette publish cloudrun --cpu X` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1420#issuecomment-892372509", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1420", "id": 892372509, "node_id": "IC_kwDOBm6k_c41MIYd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-04T05:22:29Z", "updated_at": "2021-08-04T05:22:29Z", "author_association": "OWNER", "body": "Testing this manually with:\r\n\r\n datasette publish cloudrun fixtures.db --memory 8G --cpu 4 \\\r\n --service fixtures-over-provisioned-issue-1420 --install datasette-psutil\r\n\r\nAnd for comparison:\r\n\r\n datasette publish cloudrun fixtures.db --service fixtures-default-issue-1420 \\\r\n --install datasette-psutil\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959898166, "label": "`datasette publish cloudrun --cpu X` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1420#issuecomment-892365639", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1420", "id": 892365639, "node_id": "IC_kwDOBm6k_c41MGtH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-04T05:05:07Z", "updated_at": "2021-08-04T05:05:07Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/blob/cd8b7bee8fb5c1cdce7c8dbfeb0166011abc72c6/datasette/publish/cloudrun.py#L153-L158", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959898166, "label": "`datasette publish cloudrun --cpu X` option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1419#issuecomment-892276385", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1419", "id": 892276385, "node_id": "IC_kwDOBm6k_c41Lw6h", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2021-08-04T00:58:49Z", "updated_at": "2021-08-04T00:58:49Z", "author_association": "CONTRIBUTOR", "body": "yes, [filter clause on aggregate queries were added to sqlite3 in 3.30](https://www.sqlite.org/releaselog/3_30_1.html)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959710008, "label": "`publish cloudrun` should deploy a more recent SQLite version"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1418#issuecomment-891987129", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1418", "id": 891987129, "node_id": "IC_kwDOBm6k_c41KqS5", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-08-03T16:26:01Z", "updated_at": "2021-08-03T16:32:20Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1418](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (532170f) into [main](https://codecov.io/gh/simonw/datasette/commit/54b6e96ee8aa553b6671e341a1944f93f3fb89c3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (54b6e96) will **not change** coverage.\n> The diff coverage is `n/a`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1418/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@ Coverage Diff @@\n## main #1418 +/- ##\n=======================================\n Coverage 91.64% 91.64% \n=======================================\n Files 34 34 \n Lines 4382 4382 \n=======================================\n Hits 4016 4016 \n Misses 366 366 \n```\n\n\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [54b6e96...532170f](https://codecov.io/gh/simonw/datasette/pull/1418?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959284434, "label": "Spelling corrections plus CI job for codespell"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1418#issuecomment-891984359", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1418", "id": 891984359, "node_id": "IC_kwDOBm6k_c41Kpnn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-03T16:21:56Z", "updated_at": "2021-08-03T16:21:56Z", "author_association": "OWNER", "body": "Failed with:\r\n```\r\ndocs/authentication.rst:63: perfom ==> perform\r\ndocs/authentication.rst:76: perfom ==> perform\r\ndocs/changelog.rst:429: repsonse ==> response\r\ndocs/changelog.rst:503: permissons ==> permissions\r\ndocs/changelog.rst:717: compatibilty ==> compatibility\r\ndocs/changelog.rst:1172: browseable ==> browsable\r\ndocs/deploying.rst:191: similiar ==> similar\r\ndocs/internals.rst:434: Respons ==> Response, respond\r\ndocs/internals.rst:440: Respons ==> Response, respond\r\ndocs/internals.rst:717: tha ==> than, that, the\r\ndocs/performance.rst:42: databse ==> database\r\ndocs/plugin_hooks.rst:667: utilites ==> utilities\r\ndocs/publish.rst:168: countainer ==> container\r\ndocs/settings.rst:352: inalid ==> invalid\r\ndocs/sql_queries.rst:406: preceeded ==> preceded, proceeded\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959284434, "label": "Spelling corrections plus CI job for codespell"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1417#issuecomment-891979858", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1417", "id": 891979858, "node_id": "IC_kwDOBm6k_c41KohS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-03T16:15:30Z", "updated_at": "2021-08-03T16:15:30Z", "author_association": "OWNER", "body": "Docs: https://pypi.org/project/codespell/\r\n\r\nThere's a `codespell --ignore-words=FILE` option for ignoring words. I don't have any that need ignoring yet but I'm going to add that file anyway, that way I can have codespell be a failing test but still provide a way to work around it if it incorrectly flags a correct word.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 959278472, "label": "Use codespell in CI to spot spelling errors"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/298#issuecomment-891583131", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/298", "id": 891583131, "node_id": "IC_kwDOCGYnMM41JHqb", "user": {"value": 2172260, "label": "qqilihq"}, "created_at": "2021-08-03T06:50:47Z", "updated_at": "2021-08-03T06:50:47Z", "author_association": "NONE", "body": "@simonw Awesome; thanks so much!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 951581763, "label": "Read lines with JSON object"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/251#issuecomment-891380382", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251", "id": 891380382, "node_id": "IC_kwDOCGYnMM41IWKe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T22:39:46Z", "updated_at": "2021-08-02T22:39:46Z", "author_association": "OWNER", "body": "Documentation: https://sqlite-utils.datasette.io/en/stable/cli.html#converting-data-in-columns", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 841377702, "label": "\"sqlite-utils convert\" command to replace the separate \"sqlite-transform\" tool"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/298#issuecomment-891359751", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/298", "id": 891359751, "node_id": "IC_kwDOCGYnMM41IRIH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T21:55:16Z", "updated_at": "2021-08-02T21:55:16Z", "author_association": "OWNER", "body": "This is a feature already! You can do this:\r\n\r\n sqlite-utils insert nl-demo.db mytable data.ndjson --nl\r\n\r\nSee https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-newline-delimited-json\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 951581763, "label": "Read lines with JSON object"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/300#issuecomment-891356906", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/300", "id": 891356906, "node_id": "IC_kwDOCGYnMM41IQbq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T21:49:10Z", "updated_at": "2021-08-02T21:49:10Z", "author_association": "OWNER", "body": "I'm nervous to set this by default in the Python library because it's a global setting, so I can't isolate it to just code that is being called by a `sqlite_utils` method.\r\n\r\nThe new `sqlite-utils convert` command from #251 DOES turn it on - I think having it on by default for a CLI tool that uses user-defined functions makes sense: https://github.com/simonw/sqlite-utils/blob/e83aef951bd3e8c179511faddb607239a5fa8682/sqlite_utils/cli.py#L2006\r\n\r\nI like the suggestion to include this in the documentation. I'll do that!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 956832836, "label": "Returning underlying cause for User Defined Functions "}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/306#issuecomment-891352587", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/306", "id": 891352587, "node_id": "IC_kwDOCGYnMM41IPYL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T21:39:34Z", "updated_at": "2021-08-02T21:39:34Z", "author_association": "OWNER", "body": "This older TIL would have saved me some time! https://til.simonwillison.net/sphinx/sphinx-ext-extlinks", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 958516743, "label": "Configure sphinx.ext.extlinks for issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1227#issuecomment-891352132", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1227", "id": 891352132, "node_id": "IC_kwDOBm6k_c41IPRE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T21:38:39Z", "updated_at": "2021-08-02T21:38:39Z", "author_association": "OWNER", "body": "Relevant TIL: https://til.simonwillison.net/vscode/vs-code-regular-expressions", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 810394616, "label": "Configure sphinx.ext.extlinks for issues"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/306#issuecomment-891352073", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/306", "id": 891352073, "node_id": "IC_kwDOCGYnMM41IPQJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T21:38:32Z", "updated_at": "2021-08-02T21:38:32Z", "author_association": "OWNER", "body": "TIL about this: https://til.simonwillison.net/vscode/vs-code-regular-expressions", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 958516743, "label": "Configure sphinx.ext.extlinks for issues"}, "performed_via_github_app": null}