{"html_url": "https://github.com/simonw/sqlite-utils/issues/73#issuecomment-570930239", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/73", "id": 570930239, "node_id": "MDEyOklzc3VlQ29tbWVudDU3MDkzMDIzOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-05T17:15:18Z", "updated_at": "2020-01-05T17:15:18Z", "author_association": "OWNER", "body": "I think this is because you forgot to include a `pk=` argument. I'll change the code to throw a more useful error in this case.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 545407916, "label": "upsert_all() throws issue when upserting to empty table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549435364", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/62", "id": 549435364, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTQzNTM2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-04T16:30:34Z", "updated_at": "2019-11-04T16:30:34Z", "author_association": "OWNER", "body": "Released as 1.12.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 500783373, "label": "[enhancement] Method to delete a row in python"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/605#issuecomment-1846554637", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/605", "id": 1846554637, "node_id": "IC_kwDOCGYnMM5uEDAN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:07:54Z", "updated_at": "2023-12-08T05:07:54Z", "author_association": "OWNER", "body": "Thanks for opening an issue - this should help future Google searchers figure out what's going on here.\r\n\r\nAnother approach here could be to store large integers as `TEXT` in SQLite (or even as `BLOB`).\r\n\r\nBoth storing as `REAL` and storing as `TEXT/BLOB` feel nasty to me, but it looks like SQLite has a hard upper limit of 9223372036854775807 for integers.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2007893839, "label": "Insert fails with `Error: Python int too large to convert to SQLite INTEGER`; can we use `NUMERIC` here?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/537#issuecomment-1539055393", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/537", "id": 1539055393, "node_id": "IC_kwDOCGYnMM5bvB8h", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-05-08T21:10:06Z", "updated_at": "2023-05-08T21:10:06Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1665200812, "label": "Support self-referencing FKs in `Table.create`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/530#issuecomment-1539015064", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/530", "id": 1539015064, "node_id": "IC_kwDOCGYnMM5bu4GY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-05-08T20:35:07Z", "updated_at": "2023-05-08T20:35:07Z", "author_association": "OWNER", "body": "Wow, this is a neat feature I didn't know about. Looks like there are a bunch of options:\r\n\r\n- NO ACTION (default)\r\n- RESTRICT: application is prohibited from deleting a parent key when there exists one or more child keys mapped to it\r\n- SET NULL: when a parent key is deleted the child key columns of all rows in the child table that mapped to the parent key are set to contain SQL NULL values\r\n- SET DEFAULT: set a specific default\r\n- CASCADE: propagates the delete or update operation on the parent key to each dependent child key", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1595340692, "label": "add ability to configure \"on delete\" and \"on update\" attributes of foreign keys:"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/525#issuecomment-1539108140", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/525", "id": 1539108140, "node_id": "IC_kwDOCGYnMM5bvO0s", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-05-08T21:59:41Z", "updated_at": "2023-05-08T21:59:41Z", "author_association": "OWNER", "body": "That original example passes against `main` now.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1575131737, "label": "Repeated calls to `Table.convert()` fail"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/524#issuecomment-1419734229", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/524", "id": 1419734229, "node_id": "IC_kwDOCGYnMM5Un2zV", "user": {"value": 193185, "label": "cldellow"}, "created_at": "2023-02-06T20:53:28Z", "updated_at": "2023-02-06T21:16:29Z", "author_association": "NONE", "body": "I think it's not currently possible: sqlite-utils requires that it be one of `integer`, `text`, `float`, `blob` ([see code](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/cli.py#L2266))\r\n\r\nIMO, this is a bit of friction and it would be nice if it was more permissive. SQLite permits developers to use any data type when creating a table. For example, this is a perfectly cromulent sqlite session that creates a table with columns of type `baz` and `bar`:\r\n\r\n```\r\nsqlite> create table foo(column1 baz, column2 bar);\r\nsqlite> .schema foo\r\nCREATE TABLE foo(column1 baz, column2 bar);\r\nsqlite> select * from pragma_table_info('foo');\r\ncid         name        type        notnull     dflt_value  pk        \r\n----------  ----------  ----------  ----------  ----------  ----------\r\n0           column1     baz         0                       0         \r\n1           column2     bar         0                       0   \r\n```\r\n\r\nThe idea is that the application developer will know what meaning to ascribe to those types. For example, I'm working on a plugin to Datasette. Dates are tricky to handle. If you have some existing rows, you can look at the values in them to know how a user is serializing the dates -- as an ISO 8601 string? An RFC 3339 string? With millisecond precision? With timezone offset? But if you don't yet have any rows, you have to guess. If the column is of type `TEXT`, you don't even know that it's meant to hold a date! In this case, my plugin will look to see if the column is of type `DATE` or `DATETIME`, and assume a certain representation when writing.\r\n\r\nPerhaps there is an argument that sqlite-utils is trying to conform to SQLite's strict mode, and that is why it limits the choices. In strict mode, SQLite requires that the data type be one of `INT`, `INTEGER`, `REAL`, `TEXT`, `BLOB`, `ANY`. But that can't be the case -- sqlite-utils supports `FLOAT`, which is not one of the valid types in strict mode, and it rejects `INT`, `REAL` and `ANY`, which _are_ valid.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1572766460, "label": "Transformation type `--type DATETIME`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/514#issuecomment-1539100300", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/514", "id": 1539100300, "node_id": "IC_kwDOCGYnMM5bvM6M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-05-08T21:50:51Z", "updated_at": "2023-05-08T21:50:51Z", "author_association": "OWNER", "body": "Seeing as `sqlite-utils` doesn't currently provide mechanisms for adding `check` constraints like this I'm going to leave this - I'm happy with the fix I put in for the `not null` constraints.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1465194249, "label": "upsert of new row with check constraints fails"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1320394127", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/510", "id": 1320394127, "node_id": "IC_kwDOCGYnMM5Os52P", "user": {"value": 1176293, "label": "ar-jan"}, "created_at": "2022-11-18T18:37:51Z", "updated_at": "2022-11-18T18:37:51Z", "author_association": "NONE", "body": "I guess it is not incorrect when it says the version is `4`, though it is confusing. Maybe it doesn't even refer to FTS4/FTS5 versions, but something else? In any case, it's not related to sqlite-utils, but SQLite itself.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1434911255, "label": "Cannot enable FTS5 despite it being available"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/463#issuecomment-1218610320", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/463", "id": 1218610320, "node_id": "IC_kwDOCGYnMM5IooSQ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-08-17T23:11:07Z", "updated_at": "2022-08-17T23:11:07Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1334416486, "label": "Use Read the Docs action v1"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/453#issuecomment-1185974145", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/453", "id": 1185974145, "node_id": "IC_kwDOCGYnMM5GsIeB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-07-15T21:52:18Z", "updated_at": "2022-07-15T21:52:18Z", "author_association": "OWNER", "body": "I should warn you that this isn't a supported API - I reserve the right to change how it works between release without a major version bump, because it's not part of the documented API surface.\r\n\r\nYou'll be fine if you pin to exact versions of the library though!\r\n\r\nYou may find this recently-documented function useful though: https://sqlite-utils.datasette.io/en/latest/python-api.html#reading-rows-from-a-file\r\n\r\nSee:\r\n- #443\r\n\r\nI'm going to close this issue for the moment, but if anyone wants to submit a PR that cleans up this I'll happily review it.\r\n\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1303169663, "label": "'unclosed file' warning when using insert_upsert_implementation from Python"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/441#issuecomment-1154373361", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/441", "id": 1154373361, "node_id": "IC_kwDOCGYnMM5Ezlbx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-06-13T20:01:25Z", "updated_at": "2022-06-13T20:01:25Z", "author_association": "OWNER", "body": "Yeah, at the moment the best way to do this is with `search_sql()`, but you're right it really isn't very intuitive.\r\n\r\nHere's how I would do this, using a CTE trick to combine the queries:\r\n```python\r\nsearch_sql = db[\"articles\"].search_sql(columns=[\"title\", \"author\"]))\r\nsql = f\"\"\"\r\nwith search_results as ({search_sql})\r\nselect * from search_results where owner = :owner\r\n\"\"\"\r\nresults = db.query(sql, {\"query\": \"my search query\", \"owner\": \"my owner\"})\r\n```\r\nI'm not sure if `sqlite-utils` should ever evolve to provide a better way of doing this kind of thing to be honest - if it did, it would turn into more of an ORM. Something like [PeeWee](http://docs.peewee-orm.com/en/latest/) may be a better option here.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1257724585, "label": "Combining `rows_where()` and `search()` to limit which rows are searched"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1444474487", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433", "id": 1444474487, "node_id": "IC_kwDOCGYnMM5WGO53", "user": {"value": 167893, "label": "mcarpenter"}, "created_at": "2023-02-24T20:57:43Z", "updated_at": "2023-02-24T22:22:18Z", "author_association": "CONTRIBUTOR", "body": "I think I see what is happening here, although I haven't quite work out a fix yet. Usually:\r\n\r\n* `click.progressbar.render_progress()` renders the cursor invisible on each invocation (update of the bar)\r\n* When the progress bar goes out of scope, the `__exit()__` method is invoked, which calls `render_finish()` to make the cursor re-appear.\r\n\r\n(See terminal escape sequences `BEFORE_BAR` and `AFTER_BAR` in click).\r\n\r\nHowever the sqlite-utils `utils.file_progress` context manager wraps `click.progressbar` and yields an instance of a helper class:\r\n\r\n``` python\r\n@contextlib.contextmanager     \r\ndef file_progress(file, silent=False, **kwargs):\r\n   ...\r\n        with click.progressbar(length=file_length, **kwargs) as bar:\r\n            yield UpdateWrapper(file, bar.update) \r\n```\r\n\r\nThe yielded `UpdateWrapper` goes out of scope quickly and `click.progressbar.__exit__()` is called. The cursor is made un-invisible. Hoewever `bar` is still live and so when the caller iterates on the yielded wrapper this invokes the bar's update method, calling `render_progress()`, each time printing the \"make cursor invisible\" escape code. The `progressbar.__exit__` function is not called again, so the cursor doesn't re-appear.\r\n\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239034903, "label": "CLI eats my cursor"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1793274350", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433", "id": 1793274350, "node_id": "IC_kwDOCGYnMM5q4zHu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:46:30Z", "updated_at": "2023-11-04T00:46:30Z", "author_association": "OWNER", "body": "And a GIF of the fix after applying:\r\n- #598\r\n\r\n![cursor-fix](https://github.com/simonw/sqlite-utils/assets/9599/35829aec-c9ac-4925-a8e6-ffe7c2ab0d96)\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239034903, "label": "CLI eats my cursor"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098548931", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/421", "id": 1098548931, "node_id": "IC_kwDOCGYnMM5BeobD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-13T22:41:59Z", "updated_at": "2022-04-13T22:41:59Z", "author_association": "OWNER", "body": "I'm going to close this ticket since it looks like this is a bug in the way the Dockerfile builds Python, but I'm going to ship a fix for that issue I found so the `LD_PRELOAD` workaround above should work OK with the next release of `sqlite-utils`. Thanks for the detailed bug report!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1180427792, "label": "\"Error: near \"(\": syntax error\" when using sqlite-utils indexes CLI"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030740653", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/399", "id": 1030740653, "node_id": "IC_kwDOCGYnMM49b9qt", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2022-02-06T02:57:17Z", "updated_at": "2022-02-06T02:57:17Z", "author_association": "CONTRIBUTOR", "body": "I like the idea of having stock conversions you could import. I'd actually move them to a dedicated module (call it `sqlite_utils.conversions` or something), because it's different from other utilities. Maybe they even take configuration, or they're composable.\r\n\r\n```python\r\nfrom sqlite_utils.conversions import LongitudeLatitude\r\n\r\ndb[\"places\"].insert(\r\n    {\r\n        \"name\": \"London\",\r\n        \"lng\": -0.118092,\r\n        \"lat\": 51.509865,\r\n    },\r\n    conversions={\"point\": LongitudeLatitude(\"lng\", \"lat\")},\r\n)\r\n```\r\n\r\nI would definitely use that for every CSV I get with lat/lng columns where I actually need GeoJSON.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1124731464, "label": "Make it easier to insert geometries, with documentation and maybe code"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1548913065", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/399", "id": 1548913065, "node_id": "IC_kwDOCGYnMM5cUomp", "user": {"value": 433780, "label": "chrislkeller"}, "created_at": "2023-05-16T03:11:03Z", "updated_at": "2023-05-16T03:11:52Z", "author_association": "NONE", "body": "Using this thread and some [other resources](https://sqlite-utils.datasette.io/en/stable/cli.html#spatialite-helpers) I managed to cobble together a couple of sqlite-utils lines to add a geometry column for a table that already has a lat/lng column.\r\n\r\n```\r\n# add a geometry column\r\nsqlite-utils add-geometry-column [db name] [table name] geometry --type POINT --srid 4326\r\n\r\n# add a point for each row to geometry column\r\nsqlite-utils --load-extension=spatialite [db name] 'update [table name] SET Geometry=MakePoint(longitude, latitude, 4326);'\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1124731464, "label": "Make it easier to insert geometries, with documentation and maybe code"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/385#issuecomment-1029285985", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/385", "id": 1029285985, "node_id": "IC_kwDOCGYnMM49Wahh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-02-03T18:37:48Z", "updated_at": "2022-02-03T18:37:48Z", "author_association": "OWNER", "body": "`from sqlite_utils.utils import find_spatialite` is part of the documented API already:\r\n\r\nhttps://sqlite-utils.datasette.io/en/3.22.1/python-api.html#finding-spatialite\r\n\r\nTo avoid needing to bump the major version number to 4 to indicate a backwards incompatible change, we should keep a `from .gis import find_spatialite` line at the top of `utils.py` such that any existing code with that documented import continues to work.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1102899312, "label": "Add new spatialite helper methods"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991378346", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991378346, "node_id": "IC_kwDOCGYnMM47Fzuq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T23:48:28Z", "updated_at": "2021-12-10T23:48:28Z", "author_association": "OWNER", "body": "One option: allow `CODE` to be a special value of `-` which means \"read from standard input\". It's a tiny bit of a hack but I think it would work here.\r\n\r\nIf you wanted to replace a column entirely with hyphens you would still be able to do this:\r\n\r\n    sqlite-utils convert my.db mytable col1 '\"-\"'", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/336#issuecomment-962411119", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/336", "id": 962411119, "node_id": "IC_kwDOCGYnMM45XTpv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-06T07:21:04Z", "updated_at": "2021-11-06T07:21:04Z", "author_association": "OWNER", "body": "I've never used `DEFAULT 'CURRENT_TIMESTAMP'` myself so this one should be an interesting bug to explore.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1044267332, "label": "sqlite-util tranform --column-order mangles columns of type \"timestamp\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/328#issuecomment-925296085", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/328", "id": 925296085, "node_id": "IC_kwDOCGYnMM43JuXV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-09-22T20:14:53Z", "updated_at": "2021-09-22T20:14:53Z", "author_association": "OWNER", "body": "The bug is in this code: https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/sqlite_utils/cli.py#L2205-L2227", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1004613267, "label": "Invalid JSON output when no rows"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/325#issuecomment-925321439", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/325", "id": 925321439, "node_id": "IC_kwDOCGYnMM43J0jf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-09-22T20:52:56Z", "updated_at": "2021-09-22T20:52:56Z", "author_association": "OWNER", "body": "Updated documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#running-queries-directly-against-csv-or-json\r\n\r\n> If two files have the same name they will be assigned a numeric suffix:\r\n> \r\n>     $ sqlite-utils memory foo/data.csv bar/data.csv \"select * from data_2\"", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 990844088, "label": "sqlite-utils memory can't deal with multiple files with the same name"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/298#issuecomment-891359751", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/298", "id": 891359751, "node_id": "IC_kwDOCGYnMM41IRIH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-02T21:55:16Z", "updated_at": "2021-08-02T21:55:16Z", "author_association": "OWNER", "body": "This is a feature already! You can do this:\r\n\r\n    sqlite-utils insert nl-demo.db mytable data.ndjson --nl\r\n\r\nSee https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-newline-delimited-json\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 951581763, "label": "Read lines with JSON object"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1160991031", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/297", "id": 1160991031, "node_id": "IC_kwDOCGYnMM5FM1E3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-06-21T00:35:20Z", "updated_at": "2022-06-21T00:35:20Z", "author_association": "OWNER", "body": "Relevant TIL: https://til.simonwillison.net/sqlite/one-line-csv-operations", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944846776, "label": "Option for importing CSV data using the SQLite .import mechanism"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1246977989", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/297", "id": 1246977989, "node_id": "IC_kwDOCGYnMM5KU1_F", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-14T15:57:09Z", "updated_at": "2022-09-14T15:57:09Z", "author_association": "OWNER", "body": "Should consider how this could best handle creating columns that are integer and float as opposed to just text.\r\n\r\nhttps://discord.com/channels/823971286308356157/823971286941302908/1019630014544748584 is a relevant discussion on Discord. Even if you create the schema in advance with the correct column types, this import mechanism can put empty strings in blank float/integer columns when ideally you would want to have nulls.\r\n\r\nRelated feature idea for `sqlite-utils transform`:\r\n- #488\r\n\r\nNot sure how best to handle this for `sqlite3 .import` imports.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944846776, "label": "Option for importing CSV data using the SQLite .import mechanism"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/278#issuecomment-864128489", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/278", "id": 864128489, "node_id": "MDEyOklzc3VlQ29tbWVudDg2NDEyODQ4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-18T15:46:24Z", "updated_at": "2021-06-18T15:46:24Z", "author_association": "OWNER", "body": "A workaround could be to define a bash or zsh alias of some sort.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 923697888, "label": "Support db as first parameter before subcommand, or as environment variable"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861987651", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861987651, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTk4NzY1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T02:27:20Z", "updated_at": "2021-06-16T02:27:20Z", "author_association": "OWNER", "body": "Solution: `sqlite-utils memory -` attempts to detect the input based on if it starts with a `{` or `[` (likely JSON) or if it doesn't use the `csv.Sniffer()` mechanism. Or you can use `sqlite-utils memory -:csv` to specifically indicate the type of input.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853567861", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/264", "id": 853567861, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MzU2Nzg2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-03T05:12:21Z", "updated_at": "2021-06-03T05:12:21Z", "author_association": "OWNER", "body": "I think this is more likely to happen in Datasette than in sqlite-utils - see https://github.com/simonw/datasette/issues/1356 for thoughts on this.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 907642546, "label": "Supporting additional output formats, like GeoJSON"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/258#issuecomment-843702392", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/258", "id": 843702392, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcwMjM5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T02:47:37Z", "updated_at": "2021-05-19T02:47:37Z", "author_association": "OWNER", "body": "I'm going to merge this and add a test - thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 868191959, "label": "Fixing insert from JSON containing strings with non-ascii characters \u2026"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/253#issuecomment-843718859", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/253", "id": 843718859, "node_id": "MDEyOklzc3VlQ29tbWVudDg0MzcxODg1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-05-19T03:31:47Z", "updated_at": "2021-05-19T03:31:47Z", "author_association": "OWNER", "body": "Fixed: https://simonwillison.net/2020/Sep/23/sqlite-advanced-alter-table/", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 847423559, "label": "fixtures.db example error in sql-utils blog post"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/249#issuecomment-803501756", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/249", "id": 803501756, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzUwMTc1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-21T02:33:45Z", "updated_at": "2021-03-21T02:33:45Z", "author_association": "OWNER", "body": "Did you run `enable-fts` before you inserted the data?\r\n\r\nIf so you'll need to run `populate-fts` after the insert to populate the FTS index.\r\n\r\nA better solution may be to add `--create-triggers` to the `enable-fts` command to add triggers that will automatically keep the index updated as you insert new records.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 836963850, "label": "Full text search possibly broken?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/220#issuecomment-761015218", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220", "id": 761015218, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTAxNTIxOA==", "user": {"value": 649467, "label": "mhalle"}, "created_at": "2021-01-15T15:40:08Z", "updated_at": "2021-01-15T15:40:08Z", "author_association": "NONE", "body": "Make sense. If you're coming from the sqlite3 side of things, rather than the datasette side, wanting the fts methods to work for views makes more sense. sqlite3 allows fts5 tables on views, so I was looking for CLI functionality to build the fts virtual tables. Ultimately, though, sharing fts virtual tables across tables and derivative views is likely more efficient. \r\n\r\nMaybe an explicit error message like, \"fts is not supported for views\" rather than just throwing an exception that the method doesn't exist\" might be helpful.  Not critical though.\r\n\r\nThanks.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 783778672, "label": "Better error message for *_fts methods against views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/21#issuecomment-496786354", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/21", "id": 496786354, "node_id": "MDEyOklzc3VlQ29tbWVudDQ5Njc4NjM1NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-05-29T05:09:01Z", "updated_at": "2019-05-29T05:09:01Z", "author_association": "OWNER", "body": "Shipped this feature in sqlite-utils 1.1: https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v1-1", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 448391492, "label": "Option to ignore inserts if primary key exists already"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/203#issuecomment-1404070841", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/203", "id": 1404070841, "node_id": "IC_kwDOCGYnMM5TsGu5", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-01-25T18:47:18Z", "updated_at": "2023-01-25T18:47:18Z", "author_association": "CONTRIBUTOR", "body": "i'll adopt this PR to make the changes @simonw suggested https://github.com/simonw/sqlite-utils/pull/203#issuecomment-753567932", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743384829, "label": "changes to allow for compound foreign keys"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/189#issuecomment-717359145", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/189", "id": 717359145, "node_id": "MDEyOklzc3VlQ29tbWVudDcxNzM1OTE0NQ==", "user": {"value": 35681, "label": "adamwolf"}, "created_at": "2020-10-27T16:20:32Z", "updated_at": "2020-10-27T16:20:32Z", "author_association": "CONTRIBUTOR", "body": "No problem. I added a test. Let me know if it looks sufficient or if you want me to to tweak something!\r\n\r\nIf you don't mind, would you tag this PR as \"hacktoberfest-accepted\"? If you do mind, no problem and I'm sorry for asking :) My kiddos like the shirts.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 729818242, "label": "Allow iterables other than Lists in m2m records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/178#issuecomment-701627158", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/178", "id": 701627158, "node_id": "MDEyOklzc3VlQ29tbWVudDcwMTYyNzE1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-30T20:29:11Z", "updated_at": "2020-09-30T20:29:11Z", "author_association": "OWNER", "body": "Thanks for the fix!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 709043182, "label": "Update README.md"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693199049", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/159", "id": 693199049, "node_id": "MDEyOklzc3VlQ29tbWVudDY5MzE5OTA0OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-16T06:20:26Z", "updated_at": "2020-09-16T06:20:26Z", "author_association": "OWNER", "body": "See #121 - I need to think harder about how this all interacts with transactions.\r\n\r\nYou can do this:\r\n\r\n```python\r\nwith db.conn:\r\n    db[\"mytable\"].delete_where()\r\n```\r\nBut that should be documented and maybe rethought.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 702386948, "label": ".delete_where() does not auto-commit (unlike .insert() or .upsert())"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/159#issuecomment-802032152", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/159", "id": 802032152, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMjAzMjE1Mg==", "user": {"value": 1025224, "label": "limar"}, "created_at": "2021-03-18T15:42:52Z", "updated_at": "2021-03-18T15:42:52Z", "author_association": "NONE", "body": "I confirm the bug. Happens for me in version 3.6. I use the call to delete all the records:\r\n`table.delete_where()`\r\nThis does not delete anything. \r\n\r\nI see that `delete()` method DOES use context manager `with self.db.conn:` which should help. You may want to align the code of both methods.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 702386948, "label": ".delete_where() does not auto-commit (unlike .insert() or .upsert())"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/142#issuecomment-683173375", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/142", "id": 683173375, "node_id": "MDEyOklzc3VlQ29tbWVudDY4MzE3MzM3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-28T22:29:02Z", "updated_at": "2020-08-28T22:29:02Z", "author_association": "OWNER", "body": "Yeah I think that failure is actually because there's a brand new release of Black out and it subtly changes some of the formatting rules. I'll merge this and then run Black against the entire codebase.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 688386219, "label": "insert_all(..., alter=True) should work for new columns introduced after the first 100 records"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/131#issuecomment-778510528", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/131", "id": 778510528, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODUxMDUyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-12T23:25:06Z", "updated_at": "2021-02-12T23:25:06Z", "author_association": "OWNER", "body": "If `-c` isn't available, maybe `-t` or `--type` would work for specifying column types:\r\n```\r\nsqlite-utils insert db.db images images.tsv \\\r\n  --tsv \\\r\n  --type id int \\\r\n  --type score float\r\n```\r\nor\r\n```\r\nsqlite-utils insert db.db images images.tsv \\\r\n  --tsv \\\r\n  -t id int \\\r\n  -t score float\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 675753042, "label": "sqlite-utils insert: options for column types"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/121#issuecomment-655652679", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/121", "id": 655652679, "node_id": "MDEyOklzc3VlQ29tbWVudDY1NTY1MjY3OQ==", "user": {"value": 79913, "label": "tsibley"}, "created_at": "2020-07-08T17:24:46Z", "updated_at": "2020-07-08T17:24:46Z", "author_association": "CONTRIBUTOR", "body": "Better transaction handling would be really great. Some of my thoughts on implementing better transaction discipline are in https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655239728.\r\n\r\nMy preferences:\r\n\r\n- Each CLI command should operate in a single transaction so that either the whole thing succeeds or the whole thing is rolled back. This avoids partially completed operations when an error occurs part way through processing. Partially completed operations are typically much harder to recovery from gracefully and may cause inconsistent data states.\r\n\r\n- The Python API should be transaction-agnostic and rely on the caller to coordinate transactions. Only the caller knows how individual insert, create, update, etc operations/methods should be bundled conceptually into transactions. When the caller is the CLI, for example, that bundling would be at the CLI command-level. Other callers might want to break up operations into multiple transactions. Transactions are usually most useful when controlled at the application-level (like logging configuration) instead of the library level. The library needs to provide an API that's conducive to transaction use, though.\r\n\r\n- The Python API should provide a context manager to provide consistent transactions handling with more useful defaults than Python's `sqlite3` module. The latter issues implicit `BEGIN` statements by default for most DML (`INSERT`, `UPDATE`, `DELETE`, \u2026 but not `SELECT`, I believe), but **not** DDL (`CREATE TABLE`, `DROP TABLE`, `CREATE VIEW`, \u2026). Notably, the `sqlite3` module doesn't issue the implicit `BEGIN` until the first DML statement. It _does not_ issue it when entering the `with conn` block, like other DBAPI2-compatible modules do. The `with conn` block for `sqlite3` only arranges to commit or rollback an existing transaction when exiting. Including DDL and `SELECT`s in transactions is important for operation consistency, though. There are several existing bugs.python.org tickets about this and future changes are in the works, but sqlite-utils can provide its own API sooner. sqlite-utils's `Database` class could itself be a context manager (built on the `sqlite3` connection context manager) which additionally issues an explicit `BEGIN` when entering. This would then let Python API callers do something like:\r\n\r\n```python\r\ndb = sqlite_utils.Database(path)\r\n\r\nwith db: # \u2190 BEGIN issued here by Database.__enter__\r\n    db.insert(\u2026)\r\n    db.create_view(\u2026)\r\n# \u2190 COMMIT/ROLLBACK issue here by sqlite3.connection.__exit__\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 652961907, "label": "Improved (and better documented) support for transactions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/121#issuecomment-655673896", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/121", "id": 655673896, "node_id": "MDEyOklzc3VlQ29tbWVudDY1NTY3Mzg5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-08T18:08:11Z", "updated_at": "2020-07-08T18:08:11Z", "author_association": "OWNER", "body": "I'm with you on most of this. Completely agreed that the CLI should do everything in a transaction.\r\n\r\nThe one thing I'm not keen on is forcing calling code to explicitly start a transaction, for a couple of reasons:\r\n\r\n1. It will break all of the existing code out there\r\n2. It doesn't match to how I most commonly use this library - as an interactive tool in a Jupyter notebook, where I'm generally working against a brand new scratch database and any errors don't actually matter\r\n\r\nSo... how about this: IF you wrap your code in a `with db:` block then the `.insert()` and suchlike methods expect you to manage transactions yourself. But if you don't use the context manager they behave like they do at the moment (or maybe a bit more sensibly).\r\n\r\nThat way existing code works as it does today, lazy people like me can call `.insert()` without thinking about transactions, but people writing actual production code (as opposed to Jupyter hacks) have a sensible way to take control of the transactions themselves.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 652961907, "label": "Improved (and better documented) support for transactions"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/991#issuecomment-712317638", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/991", "id": 712317638, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjMxNzYzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-19T17:30:56Z", "updated_at": "2020-10-19T17:30:56Z", "author_association": "OWNER", "body": "https://biglocal.datasettes.com/ is one of my larger Datasettes in terms of number of databases.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 714377268, "label": "Redesign application homepage"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/987#issuecomment-752714747", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/987", "id": 752714747, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MjcxNDc0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-12-30T18:23:08Z", "updated_at": "2020-12-30T18:23:20Z", "author_association": "OWNER", "body": "In terms of \"places to put your plugin content\", the simplest solution I can think of is something like this:\r\n```html\r\n<div id=\"plugin-content-pre-table\"></div>\r\n```\r\nAlternative designs:\r\n\r\n- A documented JavaScript function that returns the CSS selector where plugins should put their content\r\n- A documented JavaScript function that returns a DOM node where plugins should put their content. This would allow the JavaScript to create the element if it does not already exist (though it wouldn't be obvious WHERE that element should be created)\r\n- Documented JavaScript functions for things like \"append this node/HTML to the place-where-plugins-go\"\r\n\r\nI think the original option - an empty `<div>` with a known `id` attribute - is the right one to go with here. It's the simplest, it's very easy for custom template authors to understand and it acknowledges that plugins may have all kinds of extra crazy stuff they want to do - like checking in that div to see if another plugin has written to it already, for example.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 712984738, "label": "Documented HTML hooks for JavaScript plugin authors"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/986#issuecomment-702265255", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/986", "id": 702265255, "node_id": "MDEyOklzc3VlQ29tbWVudDcwMjI2NTI1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-01T16:51:45Z", "updated_at": "2020-10-01T16:51:45Z", "author_association": "OWNER", "body": "Thanks for taking a look! The fix ended up being a little different from this because I still want to disable faceting on regular single primary keys (since faceting by those won't ever produce interesting results) - here's what I used: https://github.com/simonw/datasette/commit/5d6bc4c268f9f155e59561671f8617addd3e91bc", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 712889459, "label": "Allow facet by primary keys, fixes #985"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/970#issuecomment-695896557", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/970", "id": 695896557, "node_id": "MDEyOklzc3VlQ29tbWVudDY5NTg5NjU1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-21T04:40:12Z", "updated_at": "2020-09-21T04:40:12Z", "author_association": "OWNER", "body": "The Python standard library has a module for this: https://docs.python.org/3/library/webbrowser.html", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 705108492, "label": "request an \"-o\" option on \"datasette server\" to open the default browser at the running url"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/97#issuecomment-392602334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/97", "id": 392602334, "node_id": "MDEyOklzc3VlQ29tbWVudDM5MjYwMjMzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-05-28T20:57:21Z", "updated_at": "2018-05-28T20:57:21Z", "author_association": "OWNER", "body": "The `/.json` endpoint is more of an implementation detail of the homepage at this point. A better, documented ( http://datasette.readthedocs.io/en/stable/introspection.html#inspect ) endpoint for finding all of the databases and tables is https://parlgov.datasettes.com/-/inspect.json", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 274022950, "label": "Link to JSON for the list of tables "}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/949#issuecomment-1791911093", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/949", "id": 1791911093, "node_id": "IC_kwDOBm6k_c5qzmS1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-03T05:28:09Z", "updated_at": "2023-11-03T05:28:58Z", "author_association": "OWNER", "body": "Datasette is using that now, see:\r\n- #1893", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684961449, "label": "Try out CodeMirror SQL hints"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-675718593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 675718593, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTcxODU5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T21:02:11Z", "updated_at": "2020-08-18T21:02:24Z", "author_association": "OWNER", "body": "Easiest solution: if you provide column metadata it gets displayed above the table, something like on https://fivethirtyeight.datasettes.com/fivethirtyeight/antiquities-act%2Factions_under_antiquities_act\r\n\r\n<img width=\"500\" alt=\"fivethirtyeight__antiquities-act_actions_under_antiquities_act__344_rows\" src=\"https://user-images.githubusercontent.com/9599/90565187-57d3e700-e15b-11ea-89c8-0270e3040a50.png\">\r\n\r\nHTML `title=` tooltips are also added to the table headers, which won't be visible on touch devices but that's OK because the information is visible on the page already.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-737463116", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 737463116, "node_id": "MDEyOklzc3VlQ29tbWVudDczNzQ2MzExNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-12-02T20:02:10Z", "updated_at": "2020-12-02T20:03:01Z", "author_association": "OWNER", "body": "My idea is that if you installed my proposed plugin you wouldn't need `metadata.json` at all - your metadata would instead live in a table in the connected SQLite database files - either one table per database (so the metadata can live in the same place as the data) or maybe also in a dedicated separate database file, for if you want to add metadata to an otherwise read-only database.\r\n\r\nThe plugin would then provide a UI for editing that metadata - maybe by configuring some writable canned queries or maybe something more custom than that. Or you could edit the metadata by manually editing the SQLite database file (or loading data into it using a tool like [yaml-to-sqlite](https://github.com/simonw/yaml-to-sqlite)).", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/942#issuecomment-897996296", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/942", "id": 897996296, "node_id": "IC_kwDOBm6k_c41hlYI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-08-12T22:01:36Z", "updated_at": "2021-08-12T22:01:36Z", "author_association": "OWNER", "body": "I'm going with `\"columns\": {\"name-of-column\": \"description-of-column\"}`.\r\n\r\nIf I decide to make `\"col\"` and `\"nocol\"` available in metadata I'll use those as the keys in the metadata, for consistency with the existing query string parameters.\r\n\r\nI'm OK with having both `\"columns\": ...` and `\"col\": ...` keys in the metadata, even though they could be a tiny bit confusing without the documentation.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681334912, "label": "Support column descriptions in metadata.json"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754215392", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754215392, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDIxNTM5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T20:59:20Z", "updated_at": "2021-01-04T21:03:14Z", "author_association": "OWNER", "body": "Updated `pyinstaller` recipe - lots of hidden imports needed now:\r\n```\r\npip install wheel\r\npip install datasette pyinstaller\r\n\r\nBASE=$(python -c 'import os; print(os.path.dirname(__import__(\"datasette\").__file__))') \\\r\n    pyinstaller -F \\\r\n        --add-data \"$BASE/templates:datasette/templates\" \\\r\n        --add-data \"$BASE/static:datasette/static\" \\\r\n        --hidden-import datasette.publish \\\r\n        --hidden-import datasette.publish.heroku \\\r\n        --hidden-import datasette.publish.cloudrun \\\r\n        --hidden-import datasette.facets \\\r\n        --hidden-import datasette.sql_functions \\\r\n        --hidden-import datasette.actor_auth_cookie \\\r\n        --hidden-import datasette.default_permissions \\\r\n        --hidden-import datasette.default_magic_parameters \\\r\n        --hidden-import datasette.blob_renderer \\\r\n        --hidden-import datasette.default_menu_links \\\r\n        --hidden-import uvicorn \\\r\n        --hidden-import uvicorn.logging \\\r\n        --hidden-import uvicorn.loops \\\r\n        --hidden-import uvicorn.loops.auto \\\r\n        --hidden-import uvicorn.protocols \\\r\n        --hidden-import uvicorn.protocols.http \\\r\n        --hidden-import uvicorn.protocols.http.auto \\\r\n        --hidden-import uvicorn.protocols.websockets \\\r\n        --hidden-import uvicorn.protocols.websockets.auto \\\r\n        --hidden-import uvicorn.lifespan \\\r\n        --hidden-import uvicorn.lifespan.on \\\r\n        $(which datasette)\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/913#issuecomment-754187326", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/913", "id": 754187326, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDE4NzMyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T20:03:50Z", "updated_at": "2021-01-04T20:03:50Z", "author_association": "OWNER", "body": "I renamed `--config` to `--setting` and changed it to work like this:\r\n\r\n    datasette --setting sql_time_limit_ms 1000\r\n\r\nNote the lack of colons.\r\n\r\nThis actually makes colons cleaner to use for plugins - I could support this:\r\n\r\n    datasette --setting datasette-insert:unsafe 1", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 670209331, "label": "Mechanism for passing additional options to `datasette my.db` that affect plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/877#issuecomment-652520496", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/877", "id": 652520496, "node_id": "MDEyOklzc3VlQ29tbWVudDY1MjUyMDQ5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-01T16:26:52Z", "updated_at": "2020-07-01T16:26:52Z", "author_association": "OWNER", "body": "Tokens get verified by plugins. So far there's only one: https://github.com/simonw/datasette-auth-tokens - which has you hard-coding plugins in a configuration file. I have a issue there to add support for database-backed tokens too: https://github.com/simonw/datasette-auth-tokens/issues/1", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648421105, "label": "Consider dropping explicit CSRF protection entirely?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/868#issuecomment-650600606", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/868", "id": 650600606, "node_id": "MDEyOklzc3VlQ29tbWVudDY1MDYwMDYwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-27T18:44:28Z", "updated_at": "2020-06-27T18:44:28Z", "author_association": "OWNER", "body": "This is really exciting! Thanks so much for looking into this.\r\n\r\nI'm interested in moving CI for this repo over to GitHub Actions, so I'd be fine with you getting this to work as an Action rather than through Travis. If you can get it working in Travis though I'll happily land that and figure out how to convert that to GitHub Actions later on.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 646448486, "label": "initial windows ci setup"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/865#issuecomment-726412057", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/865", "id": 726412057, "node_id": "MDEyOklzc3VlQ29tbWVudDcyNjQxMjA1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-12T23:49:23Z", "updated_at": "2020-11-12T23:49:23Z", "author_association": "OWNER", "body": "@tballison thanks, I've split that out into a new issue #1091", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 644582921, "label": "base_url doesn't seem to work when adding criteria and clicking \"apply\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/838#issuecomment-795895436", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/838", "id": 795895436, "node_id": "MDEyOklzc3VlQ29tbWVudDc5NTg5NTQzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-10T18:44:46Z", "updated_at": "2021-03-10T18:44:57Z", "author_association": "OWNER", "body": "Let's reopen this.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637395097, "label": "Incorrect URLs when served behind a proxy with base_url set"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-712569695", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 712569695, "node_id": "MDEyOklzc3VlQ29tbWVudDcxMjU2OTY5NQ==", "user": {"value": 222245, "label": "carlmjohnson"}, "created_at": "2020-10-20T03:45:48Z", "updated_at": "2020-10-20T03:46:14Z", "author_association": "NONE", "body": "I vote against headers. It has a lot of strikes against it: poor discoverability, new developers often don\u2019t know how to use them, makes CORS harder, makes it hard to use eg with JQ, needs ad hoc specification for each bit of metadata, etc. \r\n\r\nThe only advantage of headers is that you don\u2019t need to do .rows, but that\u2019s actually good as a data validation step anyway\u2014if .rows is missing assume there\u2019s an error and do your error handling path instead of parsing the rest.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-782789598", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 782789598, "node_id": "MDEyOklzc3VlQ29tbWVudDc4Mjc4OTU5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-21T03:30:02Z", "updated_at": "2021-02-21T03:30:02Z", "author_association": "OWNER", "body": "Another benefit to default:object - I could include a key that shows a list of available extras. I could then use that to power an interactive API explorer.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/778#issuecomment-702493047", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/778", "id": 702493047, "node_id": "MDEyOklzc3VlQ29tbWVudDcwMjQ5MzA0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-02T02:26:25Z", "updated_at": "2020-10-02T02:26:25Z", "author_association": "OWNER", "body": "I think this could work for arbitrary SQL queries too. Those would need querystring configuration that specifies which sorted column(s) should be used for the \"next\" cursor.\r\n\r\nOne example: I'd like to be able to offer a paginated list of counts of values in a table - e.g. this query:\r\n\r\nhttps://fivethirtyeight.datasettes.com/fivethirtyeight?sql=select+replies%2C+count%28*%29+from+%5Btwitter-ratio%2Fsenators%5D+group+by+replies+order+by+count%28*%29+desc%3B\r\n\r\nThat could even become a query that gets linked to from the column actions menu.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 626211658, "label": "Ability to configure keyset pagination for views and queries"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/766#issuecomment-741665253", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/766", "id": 741665253, "node_id": "MDEyOklzc3VlQ29tbWVudDc0MTY2NTI1Mw==", "user": {"value": 2181410, "label": "clausjuhl"}, "created_at": "2020-12-09T09:59:05Z", "updated_at": "2020-12-09T09:59:05Z", "author_association": "NONE", "body": "Hi Simon. Any news on using wildcard-searches with datasette? Thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 617323873, "label": "Enable wildcard-searches by default"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/758#issuecomment-624797119", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/758", "id": 624797119, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDc5NzExOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-06T17:53:46Z", "updated_at": "2020-05-06T17:53:46Z", "author_association": "OWNER", "body": "It's interesting to hear from someone who's using this feature - I'm considering moving it out into a plugin #647.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612382643, "label": "Question: Access to immutable database-path"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/757#issuecomment-624821090", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/757", "id": 624821090, "node_id": "MDEyOklzc3VlQ29tbWVudDYyNDgyMTA5MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-06T18:41:29Z", "updated_at": "2020-05-06T18:41:29Z", "author_association": "OWNER", "body": "OK, I just released 0.41 with that and a bunch of other stuff: https://datasette.readthedocs.io/en/latest/changelog.html#v0-41", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612378203, "label": "Question: Any fixed date for the release with the uft8-encoding fix?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/731#issuecomment-618155472", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/731", "id": 618155472, "node_id": "MDEyOklzc3VlQ29tbWVudDYxODE1NTQ3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-23T03:28:42Z", "updated_at": "2020-04-23T03:28:56Z", "author_association": "OWNER", "body": "As an alternative to `--static` this could work by letting you create the following:\r\n\r\n- `static/css/`\r\n- `static/js/`\r\n\r\nWhich would be automatically mounted at `/js/...` and `/css/...`\r\n\r\nOr maybe just mount `static/` at `/static/` instead? ", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 605110015, "label": "Option to automatically configure based on directory layout"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/717#issuecomment-610076073", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/717", "id": 610076073, "node_id": "MDEyOklzc3VlQ29tbWVudDYxMDA3NjA3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-04-06T22:47:21Z", "updated_at": "2020-04-06T22:47:21Z", "author_association": "OWNER", "body": "I'm confident it's possible to create a plugin that deploys to Now v2 now. I'll do the rest of the work in a separate repo: https://github.com/simonw/datasette-publish-now", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 594189527, "label": "See if I can get Datasette working on Zeit Now v2"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/696#issuecomment-809548363", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/696", "id": 809548363, "node_id": "MDEyOklzc3VlQ29tbWVudDgwOTU0ODM2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-29T17:04:19Z", "updated_at": "2021-03-29T17:04:19Z", "author_association": "OWNER", "body": "I tried this just now against Datasette 0.56 with the new Dockerfile from #1249 (that uses SQLite and SpatiaLite installed with `apt-get install`) and the tests all passed.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 576722115, "label": "Single failing unit test when run inside the Docker image"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/683#issuecomment-590679273", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/683", "id": 590679273, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MDY3OTI3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-25T04:37:21Z", "updated_at": "2020-02-25T04:37:21Z", "author_association": "OWNER", "body": "I'm happy with this now. I'm going to merge to master.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 570101428, "label": ".execute_write() and .execute_write_fn() methods on Database"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/682#issuecomment-590517338", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/682", "id": 590517338, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MDUxNzMzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-24T19:51:21Z", "updated_at": "2020-02-24T19:51:21Z", "author_association": "OWNER", "body": "I filed a question / feature request with Janus about supporting timeouts for `.get()` against async queues here: https://github.com/aio-libs/janus/issues/240\r\n\r\nI'm going to move ahead without needing that ability though. I figure SQLite writes are _fast_, and plugins can be trusted to implement just fast writes. So I'm going to support either fire-and-forget writes (they get added to the queue and a task ID is returned) or have the option to block awaiting the completion of the write (using Janus) but let callers decide which version they want. I may add optional timeouts some time in the future.\r\n\r\nI am going to make both `execute_write()` and `execute_write_fn()` awaitable functions though, for consistency with `.execute()` and to give me flexibility to change how they work in the future.\r\n\r\nI'll also add a `block=True` option to both of them which causes the function to wait for the write to be successfully executed - defaults to `False` (fire-and-forget mode).\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 569613563, "label": "Mechanism for writing to database via a queue"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/675#issuecomment-589908912", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/675", "id": 589908912, "node_id": "MDEyOklzc3VlQ29tbWVudDU4OTkwODkxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-22T02:38:21Z", "updated_at": "2020-02-22T02:38:21Z", "author_association": "OWNER", "body": "Interesting feature suggestion.\r\n\r\nMy initial instinct was that this would be better handled using the layered nature of Docker - so build a Docker image with `datasette package` and then have a separate custom script which takes that image, copies in the extra data and outputs a new image.\r\n\r\nBut... `datasette package` is already meant to be more convenient than messing around with Docker by hand like this - so actually having a `--copy` option like you describe here feels like it's within scope of what `datasette package` is meant to do.\r\n\r\nSo yeah - if you're happy to design this I think it would be worth us adding.\r\n\r\nSmall design suggestion: allow `--copy` to be applied multiple times, so you can do something like this:\r\n\r\n    datasette package \\\r\n        --copy ~/project/templates /templates \\\r\n        --copy ~/project/README.md /README.md \\\r\n        data.db\r\n\r\nAlso since Click arguments can take multiple options I don't think you need to have the `:` in there - although if it better matches Docker's own UI it might be more consistent to have it.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 567902704, "label": "--cp option for datasette publish and datasette package for shipping additional files and directories"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/675#issuecomment-592399256", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/675", "id": 592399256, "node_id": "MDEyOklzc3VlQ29tbWVudDU5MjM5OTI1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-28T08:09:12Z", "updated_at": "2020-02-28T08:09:12Z", "author_association": "OWNER", "body": "Sure, `--cp` looks good to me.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 567902704, "label": "--cp option for datasette publish and datasette package for shipping additional files and directories"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/670#issuecomment-1816642044", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/670", "id": 1816642044, "node_id": "IC_kwDOBm6k_c5sR8H8", "user": {"value": 16142258, "label": "tf13"}, "created_at": "2023-11-17T15:32:20Z", "updated_at": "2023-11-17T15:32:20Z", "author_association": "NONE", "body": "Any progress on this? It would be very helpful on my end as well. Thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 564833696, "label": "Prototoype for Datasette on PostgreSQL"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/662#issuecomment-579787057", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/662", "id": 579787057, "node_id": "MDEyOklzc3VlQ29tbWVudDU3OTc4NzA1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-29T14:43:46Z", "updated_at": "2020-01-29T14:43:46Z", "author_association": "OWNER", "body": "Can you share the exact queries you're having trouble with? The SQL itself or even just the full URL to the page (it doesn't matter if it's to a Datasette instance that isn't available online - I just need to see the URL parameters).", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 556814876, "label": "Escape_fts5_query-hookimplementation does not work with queries to standard tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/662#issuecomment-579798917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/662", "id": 579798917, "node_id": "MDEyOklzc3VlQ29tbWVudDU3OTc5ODkxNw==", "user": {"value": 2181410, "label": "clausjuhl"}, "created_at": "2020-01-29T15:08:57Z", "updated_at": "2020-01-29T15:08:57Z", "author_association": "NONE", "body": "Hi Simon\r\n\r\nThankt you for a quick reply. Here are a few examples of urls, where I search the 'cases_fts'-virtual table for tokens in the title-column. It returns the same results, wether the other query-params are present or not.\r\n\r\nSearching for sky\r\nhttp://localhost:8001/db-7596a4e/cases?_search_title=sky&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date\r\nReturns searchresults\r\n\r\nSearching for sky*\r\nhttp://localhost:8001/db-7596a4e/cases?_search_title=sky*&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date\r\nReturns searchresults\r\n\r\nSearching for sky-tog\r\nhttp://localhost:8001/db-7596a4e/cases?_search_title=sky-tog&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date\r\nThrows: No such column: tog\r\n\r\nsearching for sky+\r\nhttp://localhost:8001/db-7596a4e/cases?_search_title=sky%2B&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date\r\nThrows: Invalid SQL: fts5: syntax error near \"\"\r\n\r\nSearching for \"madpakke\" (including double quotes)\r\nhttp://localhost:8001/db-7596a4e/cases?_search_title=%22madpakke%22&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date\r\nReturns searchresults even though 'madpakke' only appears in the fulltextindex without quotes\r\n\r\nAs I said, my other plugins work just fine, and I just copied your sql_functions.py from the datasette-repo.\r\n\r\nThanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 556814876, "label": "Escape_fts5_query-hookimplementation does not work with queries to standard tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/662#issuecomment-579832857", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/662", "id": 579832857, "node_id": "MDEyOklzc3VlQ29tbWVudDU3OTgzMjg1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-29T16:12:08Z", "updated_at": "2020-01-29T16:12:08Z", "author_association": "OWNER", "body": "I think I see what's happening here.\r\n\r\nAdding the new plugin isn't quite enough: the change I made to master also alters the table view code to call the new function:\r\n\r\nhttps://github.com/simonw/datasette/commit/3c861f363df02a59a67c59036278338e4760d2ed#diff-5e0ffd62fced7d46339b9b2cd167c2f9\r\n\r\nIf you add the escape function as a plugin in Datasette 0.33 you will have to use a custom SQL query to run it, like this:\r\n\r\nhttps://latest.datasette.io/fixtures?sql=select+pk%2C+text1%2C+text2%2C+%5Bname+with+.+and+spaces%5D+from+searchable+where+rowid+in+%28select+rowid+from+searchable_fts+where+searchable_fts+match+escape_fts%28%3Asearch%29%29+order+by+pk+limit+101&search=Dog\r\n\r\nOr you can hold out for Datasette 0.34 which will have this fix and will hopefully ship within the next 24 hours.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 556814876, "label": "Escape_fts5_query-hookimplementation does not work with queries to standard tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/662#issuecomment-579864036", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/662", "id": 579864036, "node_id": "MDEyOklzc3VlQ29tbWVudDU3OTg2NDAzNg==", "user": {"value": 2181410, "label": "clausjuhl"}, "created_at": "2020-01-29T17:17:01Z", "updated_at": "2020-01-29T17:17:01Z", "author_association": "NONE", "body": "This is excellent news. I'll wait until version 0.34. It would be tiresome to rewrite all standard-queries into custom queries. Thank you!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 556814876, "label": "Escape_fts5_query-hookimplementation does not work with queries to standard tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/661#issuecomment-580028593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/661", "id": 580028593, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDAyODU5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-30T00:30:04Z", "updated_at": "2020-01-30T00:30:04Z", "author_association": "OWNER", "body": "This has now shipped as part of Datasette 0.34: https://datasette.readthedocs.io/en/stable/changelog.html#v0-34", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 555832585, "label": "--port option to expose a port other than 8001 in \"datasette package\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/658#issuecomment-580029288", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/658", "id": 580029288, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MDAyOTI4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-01-30T00:32:43Z", "updated_at": "2020-01-30T00:32:43Z", "author_association": "OWNER", "body": "Can you share how your file layout is working?\r\n\r\nYou should have something like this:\r\n\r\n`static/app.css` - a CSS file\r\n\r\nThen run Datasette like this:\r\n\r\n`datasette my.db --static-dir=static:static/`\r\n\r\nThen `http://127.0.0.1:8001/static/app.css` should serve your CSS.\r\n\r\nCould you share the command you're using to deploy to Heroku?", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 550293770, "label": "How do I use the app.css as style sheet?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/657#issuecomment-761179229", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/657", "id": 761179229, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTE3OTIyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-15T20:24:35Z", "updated_at": "2021-01-15T20:24:35Z", "author_association": "OWNER", "body": "I'm not sure how I missed this issue but it's almost a year later and I'm finally taking a look at your Parquet work.\r\n\r\nThis is yet more evidence that allowing plugins to provide their own custom `Database` objects would be a good idea.\r\n\r\nI started exploring what Datasette would like on PostgreSQL in #670 - my concern was that I would need to add a large amount of database abstraction code which would dramatically increase the complexity of the core project, but my thinking now is that it might be tractable - Datasette doesn't actually construct SQL in complex ways anywhere outside of the `TableView` class so abstracting away just that bit should be feasible.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 548591089, "label": "Allow creation of virtual tables at startup"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/646#issuecomment-561022224", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/646", "id": 561022224, "node_id": "MDEyOklzc3VlQ29tbWVudDU2MTAyMjIyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-12-03T06:30:42Z", "updated_at": "2019-12-03T06:30:42Z", "author_association": "OWNER", "body": "I don't think this is possible at the moment but you're right, it totally should be.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 531502365, "label": "Make database level information from metadata.json available in the index.html template"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/619#issuecomment-697973420", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/619", "id": 697973420, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Nzk3MzQyMA==", "user": {"value": 45416, "label": "obra"}, "created_at": "2020-09-23T21:07:58Z", "updated_at": "2020-09-23T21:07:58Z", "author_association": "NONE", "body": "I've just run into this after crafting a complex query and discovered that hitting back loses my query.\r\n\r\nEven showing me the whole bad query would be a huge improvement over the current status quo.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 520655983, "label": "\"Invalid SQL\" page should let you edit the SQL"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/595#issuecomment-541931047", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/595", "id": 541931047, "node_id": "MDEyOklzc3VlQ29tbWVudDU0MTkzMTA0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-10-14T21:25:38Z", "updated_at": "2019-10-14T21:25:38Z", "author_association": "OWNER", "body": "I like the conditional dependency for the moment - maybe until 3.5 becomes officially unsupported.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 506300941, "label": "bump uvicorn to 0.9.0 to be Python-3.8 friendly"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/595#issuecomment-552275668", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/595", "id": 552275668, "node_id": "MDEyOklzc3VlQ29tbWVudDU1MjI3NTY2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-11T03:09:43Z", "updated_at": "2019-11-11T03:09:43Z", "author_association": "OWNER", "body": "Glitch has been upgraded to Python 3.7. I think I'm happy to drop 3.5 support now - users who want Python 3.5 can get it by installing `datasette==0.30.2`", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 506300941, "label": "bump uvicorn to 0.9.0 to be Python-3.8 friendly"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/577#issuecomment-581758728", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/577", "id": 581758728, "node_id": "MDEyOklzc3VlQ29tbWVudDU4MTc1ODcyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-02-04T06:11:53Z", "updated_at": "2020-02-04T06:11:53Z", "author_association": "OWNER", "body": "For the moment I'm going to move it to `async def render_template()` on `datasette` but otherwise keep the implementation the same.\r\n\r\nThe new signature will be:\r\n\r\n    async def render_template(self, template, context=None, request=None, view_name=None):\r\n\r\n`template` can be a list of strings or a single string. If a list of strings a template will be selected from them.\r\n\r\nI'll reconsider the large list of default context variables later on in a separate ticket.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 497171390, "label": "Utility mechanism for plugins to render templates"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/567#issuecomment-549665423", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/567", "id": 549665423, "node_id": "MDEyOklzc3VlQ29tbWVudDU0OTY2NTQyMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-11-05T05:11:14Z", "updated_at": "2019-11-05T05:11:14Z", "author_association": "OWNER", "body": "@clausjuhl I wrote a bit about that here: https://simonwillison.net/2019/May/19/datasette-0-28/\r\n\r\nShort version: just point Datasette at a SQLite file and update it from another process - it should work fine! I do it all the time now - I'll have a script running that writes to a database and I'll use Datasette to monitor progress. ", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 476573875, "label": "Datasette Edit"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/557#issuecomment-511625212", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/557", "id": 511625212, "node_id": "MDEyOklzc3VlQ29tbWVudDUxMTYyNTIxMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-07-16T01:12:14Z", "updated_at": "2019-07-16T01:12:14Z", "author_association": "OWNER", "body": "This looks useful for dealing with the `The process cannot access the file because it is being used by another process` error: https://stackoverflow.com/a/28032829", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 466996584, "label": "Get tests running on Windows using Travis CI"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/556#issuecomment-510550279", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/556", "id": 510550279, "node_id": "MDEyOklzc3VlQ29tbWVudDUxMDU1MDI3OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-07-11T16:07:27Z", "updated_at": "2019-07-11T16:07:27Z", "author_association": "OWNER", "body": "This is a really neat trick, thanks!", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 465773546, "label": "Add support for running datasette as a module"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/527#issuecomment-505057520", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/527", "id": 505057520, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNTA1NzUyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-24T15:21:18Z", "updated_at": "2019-06-24T15:21:18Z", "author_association": "OWNER", "body": "I just released csvs-to-sqlite 0.9.1 with this bug fix: https://github.com/simonw/csvs-to-sqlite/releases/tag/0.9.1", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459936585, "label": "Unable to use rank when fts-table generated with csvs-to-sqlite"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/526#issuecomment-853567413", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/526", "id": 853567413, "node_id": "MDEyOklzc3VlQ29tbWVudDg1MzU2NzQxMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-03T05:11:27Z", "updated_at": "2021-06-03T05:11:27Z", "author_association": "OWNER", "body": "Another potential way to implement this would be to hold the SQLite connection open and execute the full query there.\r\n\r\nI've avoided this in the past due to concerns of resource exhaustion - if multiple requests attempt this at the same time all of the connections in the pool will become tied up and the site will be unable to respond to further requests.\r\n\r\nBut... now that Datasette has authentication there's the possibility of making this feature only available to specific authenticated users - the `--root` user for example. Which avoids the danger while unlocking a super-useful feature.\r\n\r\nNot to mention people who are running Datasette privately on their own laptop, or the proposed `--query` CLI feature in #1356.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459882902, "label": "Stream all results for arbitrary SQL and canned queries"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/526#issuecomment-1074019047", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/526", "id": 1074019047, "node_id": "IC_kwDOBm6k_c5ABDrn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-21T15:09:56Z", "updated_at": "2022-03-21T15:09:56Z", "author_association": "OWNER", "body": "I should research how much overhead creating a new connection costs - it may be that an easy way to solve this is to create A dedicated connection for the query and then close that connection at the end.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459882902, "label": "Stream all results for arbitrary SQL and canned queries"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/514#issuecomment-509154312", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/514", "id": 509154312, "node_id": "MDEyOklzc3VlQ29tbWVudDUwOTE1NDMxMg==", "user": {"value": 4363711, "label": "JesperTreetop"}, "created_at": "2019-07-08T09:36:25Z", "updated_at": "2019-07-08T09:40:33Z", "author_association": "NONE", "body": "@chrismp: Ports 1024 and under are privileged and can usually only be bound by a root or supervisor user, so it makes sense if you're running as the user `chris` that port 8000 works but 80 doesn't.\r\n\r\nSee [this generic question-and-answer](https://superuser.com/questions/710253/allow-non-root-process-to-bind-to-port-80-and-443) and [this systemd question-and-answer](https://stackoverflow.com/questions/40865775/linux-systemd-service-on-port-80) for more information about ways to skin this cat. Without knowing your specific circumstances, either extending those privileges to that service/executable/user, proxying them through something like nginx or indeed looking at what the nginx systemd job has to do to listen at port 80 all sound like good ways to start.\r\n\r\nAt this point, this is more generic systemd/Linux support than a Datasette issue, which is why a complete rando like me is able to contribute anything. ", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459397625, "label": "Documentation with recommendations on running Datasette in production without using Docker"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/499#issuecomment-498840129", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/499", "id": 498840129, "node_id": "MDEyOklzc3VlQ29tbWVudDQ5ODg0MDEyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-04T20:55:30Z", "updated_at": "2019-06-04T21:01:22Z", "author_association": "OWNER", "body": "I really want this too!\r\n\r\nIt's one of the goals of the Datasette Library #417 concept, which I'm hoping to turn into an actual feature in the coming months.\r\n\r\nIt's also going to be a major focus of my ten month JSK fellowship at Stanford, which starts in September. https://twitter.com/simonw/status/1123624552867565569", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 451585764, "label": "Accessibility for non-techie newsies? "}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/498#issuecomment-498839428", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/498", "id": 498839428, "node_id": "MDEyOklzc3VlQ29tbWVudDQ5ODgzOTQyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-04T20:53:21Z", "updated_at": "2019-06-04T20:53:21Z", "author_association": "OWNER", "body": "It does not, but that's a really great idea for a feature.\r\n\r\nOne challenge here is that FTS ranking calculations take overall table statistics into account, which means it's usually not possible to combine rankings from different tables in a sensible way. But that doesn't mean it's not possible to return grouped results.\r\n\r\nI think this makes a lot of sense as a plugin.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 451513541, "label": "Full text search of all tables at once?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/493#issuecomment-1689128911", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/493", "id": 1689128911, "node_id": "IC_kwDOBm6k_c5krg_P", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-23T01:29:20Z", "updated_at": "2023-08-23T01:29:20Z", "author_association": "OWNER", "body": "It's going to be called `datasette.json` and the concept of metadata will be split out separately. See:\r\n\r\n- #2149 ", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 449886319, "label": "Rename metadata.json to config.json"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/486#issuecomment-495659567", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/486", "id": 495659567, "node_id": "MDEyOklzc3VlQ29tbWVudDQ5NTY1OTU2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-05-24T14:41:45Z", "updated_at": "2019-05-24T14:41:45Z", "author_association": "OWNER", "body": "I'm really keen to offer this as a plugin hook once I have Datasette working on ASGI - #272 \r\n\r\nI'll hopefully have that working in the next few weeks, but in the meantime there are a couple of tricks you can use:\r\n\r\n- you can add static HTML files (no templates though) using the static route configuration options\r\n- you can link to external hosted pages using the `about_url` metadata option\r\n- you can add information to an existing page with a custom template. I do that here for example: https://russian-ira-facebook-ads.datasettes.com/", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 448189298, "label": "Ability to add extra routes and related templates"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/46#issuecomment-344161226", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/46", "id": 344161226, "node_id": "MDEyOklzc3VlQ29tbWVudDM0NDE2MTIyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2017-11-14T06:41:21Z", "updated_at": "2017-11-14T06:41:21Z", "author_association": "OWNER", "body": "Spatial extensions would be really useful too. https://www.gaia-gis.it/spatialite-2.1/SpatiaLite-manual.html", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 271301468, "label": "Dockerfile should build more recent SQLite with FTS5 and spatialite support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/437#issuecomment-505087020", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/437", "id": 505087020, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNTA4NzAyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-24T16:38:56Z", "updated_at": "2019-06-24T16:38:56Z", "author_association": "OWNER", "body": "Closing this because it doesn't really fit the new model of inspect (though we should discuss in #465 how to further evolve this feature) and because as-of #272 we no longer use Sanic - though #520 will implement the equivalent of `prepare_sanic` against ASGI.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 438048318, "label": "Add inspect and prepare_sanic hooks"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/419#issuecomment-473708941", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/419", "id": 473708941, "node_id": "MDEyOklzc3VlQ29tbWVudDQ3MzcwODk0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-03-17T19:58:11Z", "updated_at": "2019-03-17T19:58:11Z", "author_association": "OWNER", "body": "Some problems to solve:\r\n\r\n* Right now Datasette assumes it can always show the count of rows in a table, because this has been pre-calculated. If a database is mutable the pre-calculation trick no longer works, and for giant tables a `select count(*) from X` query can be expensive to run. Maybe we set a time limit on these? If time limit expires show \"many rows\"?\r\n* Maintaining a content hash of the table no longer makes sense if it is changing (though interestingly there's a `.sha3sum` built-in SQLite CLI command which takes a hash of the content and stays the same even through vacuum runs). Without that we need a different mechanism for calculating table colours. It also means that we can't do the special dbname-hash URL trick (see #418) at all if the database is opened as mutable.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 421551434, "label": "Default to opening files in mutable mode, special option for immutable files"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/397#issuecomment-453330680", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/397", "id": 453330680, "node_id": "MDEyOklzc3VlQ29tbWVudDQ1MzMzMDY4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-01-11T01:17:11Z", "updated_at": "2019-01-11T01:25:33Z", "author_association": "OWNER", "body": "If you pull [the latest image](https://hub.docker.com/r/datasetteproject/datasette) you should get the right SQLite version now:\r\n\r\n    docker pull datasetteproject/datasette\r\n    docker run -p 8001:8001 \\\r\n        datasetteproject/datasette \\\r\n        datasette -p 8001 -h 0.0.0.0\r\n\r\nhttp://0.0.0.0:8001/-/versions now gives me:\r\n\r\n```\r\n        \"version\": \"3.26.0\"\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 397129564, "label": "Update official datasetteproject/datasette Docker container to SQLite 3.26.0"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/394#issuecomment-602907207", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/394", "id": 602907207, "node_id": "MDEyOklzc3VlQ29tbWVudDYwMjkwNzIwNw==", "user": {"value": 127565, "label": "wragge"}, "created_at": "2020-03-23T23:12:18Z", "updated_at": "2020-03-23T23:12:18Z", "author_association": "CONTRIBUTOR", "body": "This would also be useful for running Datasette in Jupyter notebooks on [Binder](https://mybinder.org/). While you can use [Jupyter-server-proxy](https://github.com/jupyterhub/jupyter-server-proxy) to access Datasette on Binder, the links are broken.\r\n\r\nWhy run Datasette on Binder? I'm developing a [range of Jupyter notebooks](https://glam-workbench.github.io/) that are aimed at getting humanities researchers to explore data from libraries, archives, and museums. Many of them are aimed at researchers with limited digital skills, so being able to run examples in Binder without them installing anything is fantastic.\r\n\r\nFor example, there are a [series of notebooks](https://glam-workbench.github.io/trove-harvester/) that help researchers harvest digitised historical newspaper articles from Trove. The metadata from this harvest is saved as a CSV file that users can download. I've also provided some extra notebooks that use Pandas etc to demonstrate ways of analysing and visualising the harvested data.\r\n\r\nBut it would be really nice if, after completing a harvest, the user could spin up Datasette for some initial exploration of their harvested data without ever leaving their browser.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 396212021, "label": "base_url configuration setting"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/394#issuecomment-603631640", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/394", "id": 603631640, "node_id": "MDEyOklzc3VlQ29tbWVudDYwMzYzMTY0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-03-25T04:19:08Z", "updated_at": "2020-03-25T04:19:08Z", "author_association": "OWNER", "body": "Shipped in 0.39: https://datasette.readthedocs.io/en/latest/changelog.html#v0-39", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 396212021, "label": "base_url configuration setting"}, "performed_via_github_app": null}