{"html_url": "https://github.com/simonw/datasette/issues/991#issuecomment-710694711", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/991", "id": 710694711, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDY5NDcxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T23:24:03Z", "updated_at": "2020-10-16T23:24:03Z", "author_association": "OWNER", "body": "I'm really interested in exploring how this page could work with hundreds of database files attached and thousands of total tables.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 714377268, "label": "Redesign application homepage"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/991#issuecomment-710694607", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/991", "id": 710694607, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDY5NDYwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T23:23:34Z", "updated_at": "2020-10-16T23:23:34Z", "author_association": "OWNER", "body": "Thinking more about pagination and search from #461: if the job of the homepage is to showcase the data that is available in the instance - data that is mostly in tables - maybe it's the tables themselves (and the ability to paginate and search through them) that becomes key to the design of the page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 714377268, "label": "Redesign application homepage"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/991#issuecomment-710694144", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/991", "id": 710694144, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDY5NDE0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T23:21:41Z", "updated_at": "2020-10-16T23:21:41Z", "author_association": "OWNER", "body": "Relevant: https://github.com/simonw/datasette/discussions/1021 asks about controlling the order of databases on that page that have been loaded using configuration directory mode.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 714377268, "label": "Redesign application homepage"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/461#issuecomment-710693927", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/461", "id": 710693927, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDY5MzkyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T23:20:43Z", "updated_at": "2020-10-16T23:20:43Z", "author_association": "OWNER", "body": "Related: redesign homepage entirely #991.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 443021509, "label": "Paginate + search for databases/tables on the homepage"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/468#issuecomment-710693818", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/468", "id": 710693818, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDY5MzgxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T23:20:22Z", "updated_at": "2020-10-16T23:20:22Z", "author_association": "OWNER", "body": "This is a duplicate of #461.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 444746021, "label": "Pagination for the database index page"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706834800", "issue_url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5", "id": 706834800, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNjgzNDgwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-12T03:24:57Z", "updated_at": "2020-10-16T20:16:28Z", "author_association": "MEMBER", "body": "Here's my first attempt at a plugin for this:\r\n```python\r\nfrom datasette import hookimpl\r\nimport jinja2\r\n\r\nSTART = \"<en-note\"\r\nEND = \"</en-note>\"\r\nTEMPLATE = \"\"\"\r\n<div style=\"max-width: 500px; white-space: normal; overflow-wrap: break-word;\">{}</div>\r\n\"\"\".strip()\r\n\r\nEN_MEDIA_SCRIPT = \"\"\"\r\nArray.from(document.querySelectorAll('en-media')).forEach(el => {\r\n    let hash = el.getAttribute('hash');\r\n    let type = el.getAttribute('type');\r\n    let path = `/evernote/resources_data/${hash}.json?_shape=array`;\r\n    fetch(path).then(r => r.json()).then(rows => {\r\n        let b64 = rows[0].data.encoded;\r\n        let data = `data:${type};base64,${b64}`;\r\n        el.innerHTML = `<img style=\"max-width: 300px\" src=\"${data}\">`;\r\n    });\r\n});\r\n\"\"\"\r\n\r\n\r\n@hookimpl\r\ndef render_cell(value, table):\r\n    if not table:\r\n        # Don't render content from arbitrary SQL queries, could be XSS hole\r\n        return\r\n    if not value or not isinstance(value, str):\r\n        return\r\n    value = value.strip()\r\n    if value.startswith(START) and value.endswith(END):\r\n        trimmed = value[len(START) : -len(END)]\r\n        trimmed = trimmed.split(\">\", 1)[1]\r\n        # Replace those horrible double newlines\r\n        trimmed = trimmed.replace(\"<div><br /></div>\", \"<br>\")\r\n        return jinja2.Markup(TEMPLATE.format(trimmed))\r\n\r\n\r\n@hookimpl\r\ndef extra_body_script():\r\n    return EN_MEDIA_SCRIPT\r\n```\r\nIt works!\r\n\r\nIt does however demonstrate that Evernote's \"clip this webpage\" feature means there is a LOT of weird HTML that can get into a note. It looks like they've filtered out the scripts but I wouldn't bet on it - they certainly don't filter out many of the inline styles. So running Bleach is almost certainly a good idea.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 718938889, "label": "Figure out how to display images from <en-media> tags inline in Datasette"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710461468", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710461468, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQ2MTQ2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T19:18:19Z", "updated_at": "2020-10-16T19:18:19Z", "author_association": "OWNER", "body": "Reconsidering: #89 was a feature request that relates to this, so maybe this is worth implementing after all.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/89#issuecomment-710460242", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/89", "id": 710460242, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQ2MDI0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T19:17:27Z", "updated_at": "2020-10-16T19:17:50Z", "author_association": "OWNER", "body": "I came up with potential syntax for that here: https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710393550 - based on how `table.extract(...)` works:\r\n```python\r\nfresh_db.table(\"tree\", extracts=[Extract(\r\n    columns=(\"CommonName\", \"LatinName\"),\r\n    table=\"Species\",\r\n    fk_column=\"species_id\",\r\n    rename={\"CommonName\": \"name\", \"LatinName\": \"latin\"}\r\n)])\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 573578548, "label": "Ability to customize columns used by extracts= feature"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/187#issuecomment-710456981", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/187", "id": 710456981, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQ1Njk4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T19:15:13Z", "updated_at": "2020-10-16T19:15:13Z", "author_association": "OWNER", "body": "This is a duplicate of #79.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 723460107, "label": "Maybe: Utility method / CLI tool for initializing SpatiaLite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/187#issuecomment-710440853", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/187", "id": 710440853, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQ0MDg1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T19:04:19Z", "updated_at": "2020-10-16T19:04:19Z", "author_association": "OWNER", "body": "I split this off from #136.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 723460107, "label": "Maybe: Utility method / CLI tool for initializing SpatiaLite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/136#issuecomment-710428802", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/136", "id": 710428802, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQyODgwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:56:47Z", "updated_at": "2020-10-16T18:56:47Z", "author_association": "OWNER", "body": "To keep the code cleaner, I'm tempted to support this instead:\r\n\r\n    --load-extension=spatialite\r\n\r\nWhere `spatialite` is a special shortcut value that triggers a search for that module in known locations.\r\n\r\nUsers could still load a module in a file called `spatialite` in the current directory using:\r\n\r\n    --load-extension=./spatialite\r\n\r\nIn fact, `--load-extension=spatialite` could handle that case too by always checking for a file called `spatialite` before attempting to search for it in known locations.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 683812642, "label": "--load-extension=spatialite shortcut option"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/69#issuecomment-710405658", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/69", "id": 710405658, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQwNTY1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:42:48Z", "updated_at": "2020-10-16T18:42:48Z", "author_association": "OWNER", "body": "Did some work on this for #134 in 7e9aad7e1c09d1cf80d0b4d17d6157212a4b857d\r\n\r\nI still need to add `--load-extension` to other CLI methods, see #137. Closing this issue in favour of that one.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 534507142, "label": "Feature request: enable extensions loading"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/48#issuecomment-710402331", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/48", "id": 710402331, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDQwMjMzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:41:06Z", "updated_at": "2020-10-16T18:41:06Z", "author_association": "OWNER", "body": "I could use this demo from JupyterCon 2020 https://gist.github.com/simonw/656c21b5800d5e4624dec9930f00e093", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 471818939, "label": "Jupyter notebook demo of the library, launchable on Binder"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/58#issuecomment-710399593", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/58", "id": 710399593, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM5OTU5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:39:31Z", "updated_at": "2020-10-16T18:39:31Z", "author_association": "OWNER", "body": "I don't think this is valuable enough to justify adding to the library - especially since you can execute FTS search against views by joining to an FTS table built against an underlying table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 488293926, "label": "Support enabling FTS on views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710397574", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710397574, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM5NzU3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:38:21Z", "updated_at": "2020-10-16T18:38:21Z", "author_association": "OWNER", "body": "I'm not going to implement this. I'll leave `extract=...` as it is right now, suitable for quick simple single-column operations on input, but if users want to do something more complicated involving multiple columns they should use the `table.extract()` method after the initial insert instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710395444", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710395444, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM5NTQ0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:37:10Z", "updated_at": "2020-10-16T18:37:10Z", "author_association": "OWNER", "body": "But this begins to feel too complicated, given that `table.extract()` can already be used to achieve the same thing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710393550", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710393550, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM5MzU1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:35:57Z", "updated_at": "2020-10-16T18:36:39Z", "author_association": "OWNER", "body": "If I want to support that most complicated example, I think the option to pass a `Extracts()` object to `extracts=` is the best way to do it:\r\n```python\r\nfresh_db.table(\"tree\", extracts=[Extract(\r\n    columns=(\"CommonName\", \"LatinName\"),\r\n    table=\"Species\",\r\n    fk_column=\"species_id\",\r\n    rename={\"CommonName\": \"name\", \"LatinName\": \"latin\"}\r\n)])\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710390915", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710390915, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM5MDkxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:34:26Z", "updated_at": "2020-10-16T18:34:50Z", "author_association": "OWNER", "body": "Here's the most complex example of `.extracts()`:\r\n```python\r\ndb[\"Trees\"].extract(\r\n    [\"CommonName\", \"LatinName\"],\r\n    table=\"Species\",\r\n    fk_column=\"species_id\",\r\n    rename={\"CommonName\": \"name\", \"LatinName\": \"latin\"}\r\n)\r\n```\r\nResulting in:\r\n```sql\r\nCREATE TABLE [Species] (\r\n    [id] INTEGER PRIMARY KEY,\r\n    [name] TEXT,\r\n    [latin] TEXT\r\n)\r\n```\r\nFrom https://sqlite-utils.readthedocs.io/en/stable/python-api.html#extracting-columns-into-a-separate-table", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710364942", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710364942, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM2NDk0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:18:48Z", "updated_at": "2020-10-16T18:18:48Z", "author_association": "OWNER", "body": "I think there is. It's a nice existing feature, and I don't think adding tuple support to it would be a huge lift.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710363789", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710363789, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM2Mzc4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:18:05Z", "updated_at": "2020-10-16T18:18:05Z", "author_association": "OWNER", "body": "I wonder if there's value in extending the `extracts=` option at all given the existence of `table.extract()`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710359724", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710359724, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM1OTcyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:15:31Z", "updated_at": "2020-10-16T18:15:31Z", "author_association": "OWNER", "body": "Using a tuple would work:\r\n```python\r\nfresh_db.table(\"tree\", extracts=[(\"common_name\", \"latin_name\")])\r\n```\r\nOr to define a custom name:\r\n```python\r\nfresh_db.table(\"tree\", extracts={(\"common_name\", \"latin_name\"): \"names\"})\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/49#issuecomment-710346830", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/49", "id": 710346830, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDM0NjgzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T18:08:52Z", "updated_at": "2020-10-16T18:09:21Z", "author_association": "OWNER", "body": "The new `.extract()` method can handle multiple columns:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/2c541fac352632e23e40b0d21e3f233f7a744a57/tests/test_extract.py#L70-L87", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 472115381, "label": "extracts= should support multiple-column extracts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/182#issuecomment-710258736", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/182", "id": 710258736, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDI1ODczNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T17:20:41Z", "updated_at": "2020-10-16T17:20:41Z", "author_association": "OWNER", "body": "Documentation: https://sqlite-utils.readthedocs.io/en/latest/cli.html#inserting-csv-or-tsv-data", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 711649325, "label": "Better handling of encodings other than utf-8 for \"sqlite-utils insert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-710198162", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 710198162, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDE5ODE2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T16:41:00Z", "updated_at": "2020-10-16T16:41:00Z", "author_association": "OWNER", "body": "Failing test:\r\n```python\r\ndef test_extract_null_values(fresh_db):\r\n    fresh_db[\"species\"].insert({\"id\": 1, \"species\": \"Wolf\"}, pk=\"id\")\r\n    fresh_db[\"individuals\"].insert_all(\r\n        [\r\n            {\"id\": 10, \"name\": \"Terriana\", \"species\": \"Fox\"},\r\n            {\"id\": 11, \"name\": \"Spenidorm\", \"species\": None},\r\n            {\"id\": 12, \"name\": \"Grantheim\", \"species\": \"Wolf\"},\r\n            {\"id\": 13, \"name\": \"Turnutopia\", \"species\": None},\r\n            {\"id\": 14, \"name\": \"Wargal\", \"species\": \"Wolf\"},\r\n        ],\r\n        pk=\"id\",\r\n    )\r\n    fresh_db[\"individuals\"].extract(\"species\")\r\n    assert fresh_db[\"species\"].schema == (\r\n        \"CREATE TABLE [species] (\\n\"\r\n        \"   [id] INTEGER PRIMARY KEY,\\n\"\r\n        \"   [species] TEXT\\n\"\r\n        \")\"\r\n    )\r\n    assert fresh_db[\"individuals\"].schema == (\r\n        'CREATE TABLE \"individuals\" (\\n'\r\n        \"   [id] INTEGER PRIMARY KEY,\\n\"\r\n        \"   [name] TEXT,\\n\"\r\n        \"   [species_id] INTEGER,\\n\"\r\n        \"   FOREIGN KEY(species_id) REFERENCES species(id)\\n\"\r\n        \")\"\r\n    )\r\n    assert list(fresh_db[\"species\"].rows) == [\r\n        {\"id\": 1, \"species\": \"Wolf\"},\r\n        {\"id\": 2, \"species\": \"Fox\"},\r\n    ]\r\n    assert list(fresh_db[\"individuals\"].rows) == [\r\n        {\"id\": 10, \"name\": \"Terriana\", \"species_id\": 2},\r\n        {\"id\": 11, \"name\": \"Spenidorm\", \"species_id\": None},\r\n        {\"id\": 12, \"name\": \"Grantheim\", \"species_id\": 1},\r\n        {\"id\": 13, \"name\": \"Turnutopia\", \"species_id\": None},\r\n        {\"id\": 14, \"name\": \"Wargal\", \"species_id\": 1},\r\n    ]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/182#issuecomment-710178871", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/182", "id": 710178871, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMDE3ODg3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T16:27:39Z", "updated_at": "2020-10-16T16:28:14Z", "author_association": "OWNER", "body": "The file is opened for me by `click.File()`, which also handles things like `-` for stdin. But i neee to be able to switch the encoding used to read from that based on the `--encoding` option.\r\n\r\nI think the way to do that is to open the file in binary mode and then wrap it in a codec reader:\r\n```python\r\nfp = codecs.getreader(encoding)(fp)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 711649325, "label": "Better handling of encodings other than utf-8 for \"sqlite-utils insert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-709706260", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 709706260, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTcwNjI2MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T03:17:02Z", "updated_at": "2020-10-16T03:17:17Z", "author_association": "OWNER", "body": "Actually I think this should be an option to `.extract()` which controls if nulls are extracted or left alone. Maybe called `extract_nulls=True/False`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-709706065", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 709706065, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTcwNjA2NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T03:16:22Z", "updated_at": "2020-10-16T03:16:22Z", "author_association": "OWNER", "body": "Either way I think I'm going to need to add some SQL which uses `where a = b or (a is null and b is null)`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-709705885", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 709705885, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTcwNTg4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T03:15:39Z", "updated_at": "2020-10-16T03:15:39Z", "author_association": "OWNER", "body": "The alternative solution here would be that a single `null` value DOES get extracted. To implement this I would need to add some logic that uses `is null` instead of `=`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/186#issuecomment-709705624", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/186", "id": 709705624, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTcwNTYyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-16T03:14:39Z", "updated_at": "2020-10-16T03:14:39Z", "author_association": "OWNER", "body": "How should this work with extractions covering multiple columns?\r\n\r\nIf there's a single column then it makes sense that a `null` value would not be extracted into the lookup table, but would instead become stay as `null`.\r\n\r\nFor a multiple column extraction, provided at least one of those columns is not null It should map to a record in the lookup table. Only if ALL of the extracted columns are null should the lookup value stay null.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 722816436, "label": ".extract() shouldn't extract null values"}, "performed_via_github_app": null}