{"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684384750", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684384750, "node_id": "IC_kwDOBm6k_c5kZavu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T20:07:18Z", "updated_at": "2023-08-18T20:07:18Z", "author_association": "OWNER", "body": "The big challenge here is what the URL to that row page should look like. How can I encode a `None` in a form that can be encoded and decoded without clashing with primary keys that are the string `None` or `null`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684494464", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684494464, "node_id": "IC_kwDOBm6k_c5kZ1iA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:27:51Z", "updated_at": "2023-08-18T22:28:40Z", "author_association": "OWNER", "body": "Oh wow, null primary keys are bad news... SQLite lets you insert multiple rows with the same `null` value!\r\n```pycon\r\n>>> import sqlite_utils\r\n>>> db = sqlite_utils.Database(memory=True)\r\n>>> db[\"foo\"].insert({\"id\": None, \"name\": \"No ID\"}, pk=\"id\")\r\n\r\n>>> db.schema\r\n'CREATE TABLE [foo] (\\n [id] TEXT PRIMARY KEY,\\n [name] TEXT\\n);'\r\n>>> db[\"foo\"].insert({\"id\": None, \"name\": \"No ID\"}, pk=\"id\")\r\n
\r\n>>> db.schema\r\n'CREATE TABLE [foo] (\\n [id] TEXT PRIMARY KEY,\\n [name] TEXT\\n);'\r\n>>> list(db[\"foo\"].rows)\r\n[{'id': None, 'name': 'No ID'}, {'id': None, 'name': 'No ID'}]\r\n>>> list(db.query('select * from foo where id = null'))\r\n[]\r\n>>> list(db.query('select * from foo where id is null'))\r\n[{'id': None, 'name': 'No ID'}, {'id': None, 'name': 'No ID'}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684495674", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684495674, "node_id": "IC_kwDOBm6k_c5kZ106", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:29:47Z", "updated_at": "2023-08-18T22:29:47Z", "author_association": "OWNER", "body": "https://www.sqlite.org/lang_createtable.html#the_primary_key says:\r\n\r\n>According to the SQL standard, PRIMARY KEY should always imply NOT NULL. Unfortunately, due to a bug in some early versions, this is not the case in SQLite. Unless the column is an [INTEGER PRIMARY KEY](https://www.sqlite.org/lang_createtable.html#rowid) or the table is a [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html) table or a [STRICT](https://www.sqlite.org/stricttables.html) table or the column is declared NOT NULL, SQLite allows NULL values in a PRIMARY KEY column. SQLite could be fixed to conform to the standard, but doing so might break legacy applications. Hence, it has been decided to merely document the fact that SQLite allows NULLs in most PRIMARY KEY columns.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684497000", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684497000, "node_id": "IC_kwDOBm6k_c5kZ2Jo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:31:53Z", "updated_at": "2023-08-18T22:31:53Z", "author_association": "OWNER", "body": "So it sounds like SQLite does ensure that a `rowid` before it allows a primary key to be null.\r\n\r\nSo one solution here would be to detect a null primary key and switch that table over to using `rowid` URLs instead. The key problem we're trying to solve here after all is how to link to a row:\r\n\r\nhttps://latest.datasette.io/fixtures/infinity/1\r\n\r\nBut when would we run that check? And does every row in the table get a new `/rowid/` URL just because someone messed up and inserted a `null` by mistake?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684497642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684497642, "node_id": "IC_kwDOBm6k_c5kZ2Tq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:32:53Z", "updated_at": "2023-08-18T22:32:53Z", "author_association": "OWNER", "body": "Here's a potential solution: make it so ALL `rowid` tables in SQLite can be optionally addressed by their `rowid` instead of by their primary key.\r\n\r\nThen teach the code that outputs the URL to a row page to spot if there are `null` primary keys and switch to that alternative addressing mechanism instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684498947", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684498947, "node_id": "IC_kwDOBm6k_c5kZ2oD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:35:04Z", "updated_at": "2023-08-18T22:35:04Z", "author_association": "OWNER", "body": "The most interesting row URL in the fixtures database right now is this one:\r\n\r\nhttps://latest.datasette.io/fixtures/compound_primary_key/a~2Fb,~2Ec-d\r\n\r\n\"CleanShot\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684500172", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684500172, "node_id": "IC_kwDOBm6k_c5kZ27M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:37:04Z", "updated_at": "2023-08-18T22:37:04Z", "author_association": "OWNER", "body": "Looking at the way these URLs work: because the components themselves in `a~2Fb,~2Ec-d` are tilde-encoded, any character that's \"safe\" in tilde-encoding could be used to indicate \"this is actually a rowid\".\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684500540", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684500540, "node_id": "IC_kwDOBm6k_c5kZ3A8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:37:37Z", "updated_at": "2023-08-18T22:37:37Z", "author_association": "OWNER", "body": "I just found this and panicked, thinking maybe tilde encoding is a bad idea after all! https://jkorpela.fi/tilde.html\r\n\r\nBut... \"Date of last update: 1999-08-27\" - I think I'm OK.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684502278", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684502278, "node_id": "IC_kwDOBm6k_c5kZ3cG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:40:20Z", "updated_at": "2023-08-18T22:40:20Z", "author_association": "OWNER", "body": "From reviewing https://simonwillison.net/2022/Mar/19/weeknotes/\r\n\r\n unreserved = ALPHA / DIGIT / \"-\" / \".\" / \"_\" / \"~\"\r\n\r\nThat's how I chose the tilde character - but it also suggests that I could use `-` or `.` or `_` for my new `rowid` encoding.\r\n\r\nSo maybe `/database/table/_4` could indicate \"the row with `rowid` of 4\".\r\n\r\nNo, that doesn't work:\r\n```pycon\r\n>>> from datasette.utils import tilde_encode\r\n>>> tilde_encode(\"_\")\r\n'_'\r\n```\r\nI need a character which tilde-encoding does indeed encode.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684503189", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684503189, "node_id": "IC_kwDOBm6k_c5kZ3qV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:41:51Z", "updated_at": "2023-08-18T22:41:51Z", "author_association": "OWNER", "body": "```pycon\r\n>>> tilde_encode(\"~\")\r\n'~7E'\r\n>>> tilde_encode(\".\")\r\n'~2E'\r\n>>> tilde_encode(\"-\")\r\n'-'\r\n```\r\nI think `.` might be the way to do this:\r\n\r\n /database/table/.4\r\n\r\nBut... I worry about that colliding with my URL routing code that spots the difference between these:\r\n\r\n /database/table/.4\r\n /database/table/.4.json\r\n /database/table/.4.csv\r\n\r\netc.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684503587", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684503587, "node_id": "IC_kwDOBm6k_c5kZ3wj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:42:28Z", "updated_at": "2023-08-18T22:42:39Z", "author_association": "OWNER", "body": "I could set a rule that extensions (including custom render extensions set by plugins) must not be valid integers, and teach Datasette that `/\\.\\d+` is the indication of a `rowid`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684504051", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684504051, "node_id": "IC_kwDOBm6k_c5kZ33z", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:43:06Z", "updated_at": "2023-08-18T22:43:06Z", "author_association": "OWNER", "body": "Here's the regex in question at the moment: https://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/app.py#L1387-L1390", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684504398", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684504398, "node_id": "IC_kwDOBm6k_c5kZ39O", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:43:31Z", "updated_at": "2023-08-18T22:43:46Z", "author_association": "OWNER", "body": "`(?P[^/]+?)` could instead be a regex that is restricted to the tilde-encoded set of characters, or `\\.\\d+`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684505071", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684505071, "node_id": "IC_kwDOBm6k_c5kZ4Hv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:44:35Z", "updated_at": "2023-08-18T22:44:35Z", "author_association": "OWNER", "body": "Also relevant:\r\nhttps://github.com/simonw/datasette/blob/943df09dcca93c3b9861b8c96277a01320db8662/datasette/utils/__init__.py#L1147-L1153", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684522567", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684522567, "node_id": "IC_kwDOBm6k_c5kZ8ZH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:58:07Z", "updated_at": "2023-08-18T22:58:07Z", "author_association": "OWNER", "body": "Here's a prototype of that:\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex b2644ace..acc55249 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -1386,7 +1386,7 @@ class Datasette:\r\n )\r\n add_route(\r\n RowView.as_view(self),\r\n- r\"/(?P[^\\/\\.]+)/(?P
[^/]+?)/(?P[^/]+?)(\\.(?P\\w+))?$\",\r\n+ r\"/(?P[^\\/\\.]+)/(?P
[^/]+?)/(?P[A-Za-z0-9\\_\\-\\~]+|\\.\\d+)(\\.(?P\\w+))?$\",\r\n )\r\n add_route(\r\n TableInsertView.as_view(self),\r\n@@ -1440,7 +1440,15 @@ class Datasette:\r\n async def resolve_row(self, request):\r\n db, table_name, _ = await self.resolve_table(request)\r\n pk_values = urlsafe_components(request.url_vars[\"pks\"])\r\n- sql, params, pks = await row_sql_params_pks(db, table_name, pk_values)\r\n+\r\n+ if len(pk_values) == 1 and pk_values[0].startswith(\".\"):\r\n+ # It's a special .rowid value\r\n+ pk_values = (pk_values[0][1:],)\r\n+ sql, params, pks = await row_sql_params_pks(\r\n+ db, table_name, pk_values, rowid=True\r\n+ )\r\n+ else:\r\n+ sql, params, pks = await row_sql_params_pks(db, table_name, pk_values)\r\n results = await db.execute(sql, params, truncate=True)\r\n row = results.first()\r\n if row is None:\r\ndiff --git a/datasette/utils/__init__.py b/datasette/utils/__init__.py\r\nindex c388673d..96669281 100644\r\n--- a/datasette/utils/__init__.py\r\n+++ b/datasette/utils/__init__.py\r\n@@ -1206,9 +1206,12 @@ def truncate_url(url, length):\r\n return url[: length - 1] + \"\u2026\"\r\n \r\n \r\n-async def row_sql_params_pks(db, table, pk_values):\r\n+async def row_sql_params_pks(db, table, pk_values, rowid=False):\r\n pks = await db.primary_keys(table)\r\n- use_rowid = not pks\r\n+ if rowid:\r\n+ use_rowid = True\r\n+ else:\r\n+ use_rowid = not pks\r\n select = \"*\"\r\n if use_rowid:\r\n select = \"rowid, *\"\r\n```\r\nIt works:\r\n\r\n\"CleanShot\r\n\r\n\"CleanShot\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684523322", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684523322, "node_id": "IC_kwDOBm6k_c5kZ8k6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T22:59:14Z", "updated_at": "2023-08-18T22:59:14Z", "author_association": "OWNER", "body": "Except it looks like the Links from other tables section is broken:\r\n\r\n\"CleanShot\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684525054", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684525054, "node_id": "IC_kwDOBm6k_c5kZ8_-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T23:02:26Z", "updated_at": "2023-08-18T23:02:26Z", "author_association": "OWNER", "body": "Creating a quick test database:\r\n```bash\r\nsqlite-utils create-table nulls.db nasty id text --pk id\r\nsqlite-utils nulls.db 'insert into nasty (id) values (null)'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684525943", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684525943, "node_id": "IC_kwDOBm6k_c5kZ9N3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T23:04:14Z", "updated_at": "2023-08-18T23:04:14Z", "author_association": "OWNER", "body": "This is hard. I tried this:\r\n```python\r\ndef path_from_row_pks(row, pks, use_rowid, quote=True):\r\n \"\"\"Generate an optionally tilde-encoded unique identifier\r\n for a row from its primary keys.\"\"\"\r\n if use_rowid or any(row[pk] is None for pk in pks):\r\n bits = [row[\"rowid\"]]\r\n else:\r\n bits = [\r\n row[pk][\"value\"] if isinstance(row[pk], dict) else row[pk] for pk in pks\r\n ]\r\n if quote:\r\n bits = [tilde_encode(str(bit)) for bit in bits]\r\n else:\r\n bits = [str(bit) for bit in bits]\r\n\r\n return \",\".join(bits)\r\n```\r\nThe ` if use_rowid or any(row[pk] is None for pk in pks)` bit is new.\r\n\r\nBut I got this error on http://127.0.0.1:8003/nulls/nasty :\r\n\r\n```\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/views/table.py\", line 1364, in run_display_columns_and_rows\r\n display_columns, display_rows = await display_columns_and_rows(\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/views/table.py\", line 186, in display_columns_and_rows\r\n pk_path = path_from_row_pks(row, pks, not pks, False)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py\", line 124, in path_from_row_pks\r\n bits = [row[\"rowid\"]]\r\nIndexError: No item with that key\r\n```\r\nBecause the SQL query I ran to populate the page didn't know that it would need to select `rowid` as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684526447", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684526447, "node_id": "IC_kwDOBm6k_c5kZ9Vv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T23:05:02Z", "updated_at": "2023-08-18T23:05:02Z", "author_association": "OWNER", "body": "How expensive is it to detect if a SQLite table contains at least one `null` primary key?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1684530060", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1684530060, "node_id": "IC_kwDOBm6k_c5kZ-OM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-18T23:09:03Z", "updated_at": "2023-08-18T23:09:14Z", "author_association": "OWNER", "body": "Ran a quick benchmark on ChatGPT Code Interpreter: https://chat.openai.com/share/8357dc01-a97e-48ae-b35a-f06249935124\r\n\r\nConclusion from there is that this query returns fast no matter how much the table grows:\r\n\r\n```sql\r\nSELECT EXISTS(SELECT 1 FROM \"nasty\" WHERE \"id\" IS NULL)\r\n```\r\nSo detecting if a table contains any null primary keys is definitely feasible without a performance hit.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1685471752", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1685471752, "node_id": "IC_kwDOBm6k_c5kdkII", "user": {"value": 77071, "label": "pkulchenko"}, "created_at": "2023-08-21T01:07:23Z", "updated_at": "2023-08-21T01:07:23Z", "author_association": "NONE", "body": "@simonw, since you're referencing \"rowid\" column by name, I just want to note that there may be an existing rowid column with completely different semantics (https://www.sqlite.org/lang_createtable.html#rowid), which is likely to break this logic. I don't see a good way to detect a proper \"rowid\" name short of checking if there is a field with that name and using the alternative (`_rowid_` or `oid`), which is not ideal, but may work.\r\n\r\nIn terms of the original issue, maybe a way to deal with it is to use rowid by default and then use primary key for WITHOUT ROWID tables (as they are guaranteed to be not null), but I suspect it may require significant changes to the API (and doesn't fully address the issue of what value to pass to indicate NULL when editing records). Would it make sense to generate a random string to indicate NULL values when editing?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1686683596", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1686683596, "node_id": "IC_kwDOBm6k_c5kiL_M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-21T16:49:12Z", "updated_at": "2023-08-21T16:49:12Z", "author_association": "OWNER", "body": "Suggestion from @asg017 is that we say that if your row has a null primary key you don't get a link to a row page for that row.\r\n\r\nWhich has some precedent, because our SQL view display doesn't link to row pages at all (since they don't make sense for views): https://latest.datasette.io/fixtures/simple_view", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2145#issuecomment-1686745094", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2145", "id": 1686745094, "node_id": "IC_kwDOBm6k_c5kibAG", "user": {"value": 15178711, "label": "asg017"}, "created_at": "2023-08-21T17:30:01Z", "updated_at": "2023-08-21T17:30:01Z", "author_association": "CONTRIBUTOR", "body": "Another point: The new Datasette write API should refuse to insert a row with a NULL primary key. That will likely decrease the likelihood someone find themselves with NULLs in their primary keys, at least with Datasette users. Especially buggy code that uses the write API, like our `datasette-write-ui` bug that led to this issue.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1857234285, "label": "If a row has a primary key of `null` various things break"}, "performed_via_github_app": null}