{"id": 1907655261, "node_id": "I_kwDOBm6k_c5xtIJd", "number": 2193, "title": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-09-21T19:49:34Z", "updated_at": "2023-09-21T21:56:43Z", "closed_at": "2023-09-21T21:56:43Z", "author_association": "OWNER", "pull_request": null, "body": "> That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the \"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/2057#issuecomment-1730201226_\r\n ", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2193/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1876353656, "node_id": "I_kwDOBm6k_c5v1uJ4", "number": 2168, "title": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-08-31T21:42:04Z", "updated_at": "2023-09-10T17:54:08Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "There's a long justification for why this might be needed here:\r\n- https://github.com/simonw/datasette-auth-tokens/issues/10#issuecomment-1701820001\r\n\r\nShort version: it would be neat if it was possible to stash some data on the `request` object such that a later plugin/middleware-type-thing could use that to influence the final returned response - similar to the kinds of things you can do with Django middleware.\r\n\r\nThe `asgi_wrapper()` mechanism doesn't have access to the request or response objects - it gets `scope` and can mess around with `receive` and `send`, but those are pretty low-level primitives.\r\n\r\nSince Datasette has well-defined `request` and `response` objects now it might be nice to have a middleware layer that can manipulate those directly.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2168/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1886771493, "node_id": "I_kwDOCGYnMM5wddkl", "number": 592, "title": "`table.transform()` should preserve `rowid` values", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-09-08T00:42:38Z", "updated_at": "2023-09-10T17:46:41Z", "closed_at": "2023-09-09T00:45:32Z", "author_association": "OWNER", "pull_request": null, "body": "I just spotted a bug when using https://datasette.io/plugins/datasette-configure-fts and https://datasette.io/plugins/datasette-edit-schema at the same time.\r\n\r\nSteps to reproduce:\r\n\r\n- Configure FTS for a table, then run a test search\r\n- Edit the schema for that table and change the order of columns\r\n- Run the test search again\r\n\r\nI got the wrong search results, which I think is because the `_fts` table pointed to the first table by `rowid` but those `rowid` values were entirely rewritten as a consequence of running `table.transform()` on the table.\r\n\r\nReconfiguring FTS on the table fixed the problem.\r\n\r\nI think `table.transform()` should be able to preserve `rowid` values.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/592/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1886791100, "node_id": "I_kwDOBm6k_c5wdiW8", "number": 2180, "title": "Plugin hook: `actors_from_ids()`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-09-08T01:16:41Z", "updated_at": "2023-09-10T17:44:14Z", "closed_at": "2023-09-08T04:28:03Z", "author_association": "OWNER", "pull_request": null, "body": "In building Datasette Cloud we realized that a bunch of the features we are building need a way of resolving an actor ID to the actual actor, in order to display something more interesting than just an integer ID.\r\n\r\nSocial plugins in particular need this - comments by X, CSV uploaded by X, that kind of thing.\r\n\r\nI think the solution is a new plugin hook: `actors_from_ids(datasette, ids)` which can return a list of actor dictionaries.\r\n\r\nThe default implementation can return `[{\"id\": \"...\"}]` for the IDs passed to it.\r\n\r\nPluggy has a `firstresult=True` option which is relevant here, since this is the first plugin hook we will have implemented where only one plugin should provide an answer.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2180/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1874327336, "node_id": "PR_kwDOBm6k_c5ZLMSe", "number": 2165, "title": "DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-08-30T20:33:30Z", "updated_at": "2023-08-30T22:12:25Z", "closed_at": "2023-08-30T22:12:25Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/2165", "body": "- #2164\r\n\r\nTODO:\r\n\r\n- [x] Automated tests\r\n- [ ] Documentation\r\n- [x] Make sure `DATASETTE_LOAD_PLUGINS=''` works for loading zero plugins", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2165/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 685806511, "node_id": "MDU6SXNzdWU2ODU4MDY1MTE=", "number": 950, "title": "Private/secret databases: database files that are only visible to plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-08-25T20:46:17Z", "updated_at": "2023-08-24T22:26:09Z", "closed_at": "2023-08-24T22:26:08Z", "author_association": "OWNER", "pull_request": null, "body": "In thinking about the best way to implement https://github.com/simonw/datasette-auth-passwords/issues/6 (SQL-backed user accounts for `datasette-auth-passwords`) I realized that there are a few different use-cases where a plugin might want to store data that isn't visible to regular Datasette users:\r\n\r\n- Storing password hashes\r\n- Storing API tokens\r\n- Storing secrets that are used for data import integrations (secrets for talking to the Twitter API for example)\r\n\r\nIdea: allow one or more private database files to be attached to Datasette, something like this:\r\n\r\n datasette github.db linkedin.db -s secrets.db -m metadata.yml\r\n\r\nThe `secrets.db` file would not be visible using any of the Datasette's usual interface or API routes - but plugins would be able to run queries against it.\r\n\r\nSo `datasette-auth-passwords` might then be configured like this:\r\n\r\n```yaml\r\nplugins:\r\n datasette-auth-passwords:\r\n database: secrets\r\n sql: \"select password_hash from passwords where username = :username\"\r\n```\r\nThe plugin could even refuse to operate against a database that hadn't been loaded as a secret database.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/950/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 814595021, "node_id": "MDU6SXNzdWU4MTQ1OTUwMjE=", "number": 1241, "title": "Share button for copying current URL", "user": {"value": 7107523, "label": "Kabouik"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-02-23T15:55:40Z", "updated_at": "2023-08-24T20:09:52Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I use datasette in an `iframe` inside another HTML file that contains other ways to represent my data (mostly leaflets maps built with R on summarized data), and the datasette `iframe` is a tab in that page. \r\n\r\nThis particular use prevents users to access the full URLs of their datasette views and queries, which is a shame because the way datasette handles URLs to make every view or query easy to share is awesome. I know how to get the URL from the context menu of my browser, but I don't think many visitors would do it or even notice that datasette uses permalinks for pretty much every action they do. Would it be possible to add a \"Share link\" button to the interface, either in datasette itself or in a plugin?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1241/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1858228057, "node_id": "I_kwDOBm6k_c5uwk9Z", "number": 2147, "title": "Plugin hook for database queries that are run", "user": {"value": 18899, "label": "jackowayed"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-08-20T18:43:50Z", "updated_at": "2023-08-24T03:54:35Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I'm interested in making a plugin that saves every query that gets run to a table in the database. (I know about datasette-query-history but thought it would be good to have a server-side option.)\r\n\r\nAs far as I can tell reading the docs, there isn't really a hook setup to allow this.\r\n\r\nMaybe I could hack it with some of the hooks that are passed requests, but that doesn't seem good.\r\n\r\nI'm a little surprised this isn't possible, so I thought I would open an issue and see if that's a deeply considered decision or just \"haven't needed it yet.\" I'm potentially interested in implementing the hook if the latter.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2147/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1786258502, "node_id": "I_kwDOCGYnMM5qeCRG", "number": 565, "title": "Table renaming: db.rename_table() and sqlite-utils rename-table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-07-03T14:07:42Z", "updated_at": "2023-07-22T22:12:40Z", "closed_at": "2023-07-22T22:12:40Z", "author_association": "OWNER", "pull_request": null, "body": "> I find myself wanting two new features in `sqlite-utils`:\r\n> - The ability to have the new transformed table set to a specific name, while keeping the old table around\r\n> - The ability to rename a table (`sqlite-utils` doesn't have a table rename function at all right now)\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618375042_\r\n ", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/565/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1718607907, "node_id": "I_kwDOCGYnMM5mb-Aj", "number": 551, "title": "Make as many examples in the CLI docs as possible copy-and-pastable", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-05-21T19:04:10Z", "updated_at": "2023-05-21T21:04:04Z", "closed_at": "2023-05-21T20:57:24Z", "author_association": "OWNER", "pull_request": null, "body": "e.g. in this section:\r\n\r\nhttps://sqlite-utils.datasette.io/en/stable/cli.html#running-queries-directly-against-csv-or-json\r\n\r\n\"image\"\r\n\r\nThe little copy button will also copy the `$ ` which breaks the examples when copied.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/551/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1718517882, "node_id": "I_kwDOCGYnMM5mboB6", "number": 545, "title": "Try out Trogon for a tui interface", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-05-21T14:08:25Z", "updated_at": "2023-05-21T19:33:13Z", "closed_at": "2023-05-21T18:41:58Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/Textualize/trogon", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/545/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1661860507, "node_id": "PR_kwDOBm6k_c5N_bMw", "number": 2056, "title": "GitHub Action to lint Python code with ruff", "user": {"value": 3709715, "label": "cclauss"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-04-11T06:41:27Z", "updated_at": "2023-04-15T14:24:46Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/2056", "body": "[Ruff](https://beta.ruff.rs/) supports [over 500 lint rules](https://beta.ruff.rs/docs/rules) and can be used to replace [Flake8](https://pypi.org/project/flake8/) (plus dozens of plugins), [isort](https://pypi.org/project/isort/), [pydocstyle](https://pypi.org/project/pydocstyle/), [yesqa](https://github.com/asottile/yesqa), [eradicate](https://pypi.org/project/eradicate/), [pyupgrade](https://pypi.org/project/pyupgrade/), and [autoflake](https://pypi.org/project/autoflake/), all while executing (in Rust) tens or hundreds of times faster than any individual tool.\r\n\r\nThe ruff Action uses minimal steps to run in ~5 seconds, rapidly providing intuitive GitHub Annotations to contributors.\r\n\r\n![image](https://user-images.githubusercontent.com/3709715/223758136-afc386d2-70aa-4eff-953a-2c2d82ceea23.png)\r\n\r\n\r\n----\n:books: Documentation preview :books:: https://datasette--2056.org.readthedocs.build/en/2056/\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2056/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1617769847, "node_id": "I_kwDOJHON9s5gbTV3", "number": 7, "title": "Folder support", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-03-09T18:21:33Z", "updated_at": "2023-03-09T20:48:18Z", "closed_at": "2023-03-09T20:48:18Z", "author_association": "MEMBER", "pull_request": null, "body": "Notes can live in folders. These relationships should be exported too.", "repo": {"value": 611552758, "label": "apple-notes-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1615862295, "node_id": "I_kwDOBm6k_c5gUBoX", "number": 2036, "title": "`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2023-03-08T20:11:44Z", "updated_at": "2023-03-08T20:57:34Z", "closed_at": "2023-03-08T20:57:34Z", "author_association": "OWNER", "pull_request": null, "body": "See this issue:\r\n- https://github.com/simonw/datasette.io/issues/141", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2036/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 743371103, "node_id": "MDU6SXNzdWU3NDMzNzExMDM=", "number": 1099, "title": "Support linking to compound foreign keys", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-11-15T23:23:17Z", "updated_at": "2023-01-25T00:58:26Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Reported as a bug in #1098 because they caused 500 errors - but it would be even better if Datasette could hyperlink to related rows via compound foreign keys.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1099/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 664485022, "node_id": "MDU6SXNzdWU2NjQ0ODUwMjI=", "number": 46, "title": "Feature: pull request reviews and comments", "user": {"value": 1326704, "label": "bhrutledge"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-07-23T13:43:45Z", "updated_at": "2022-12-20T14:40:15Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi there! I saw your [presentation at Boston Python](https://www.meetup.com/bostonpython/events/271887195). I'm already a light user of Datasette (thank you!), but wasn't aware of this project.\r\n\r\nI've been working on a \"pull request dashboard\" to get a comprehensive view of the state of open PR's, esp. related to reviews (i.e., pending, approved, changes requested). Currently it's a CLI command, but I thought a Datasette UI might be fun.\r\n\r\nI see that PR's are available from the `issues` command, but I don't see reviews anywhere. From the [API docs](https://docs.github.com/en/rest/reference/pulls#reviews), it looks like there are separate endpoints for those (as well as pull requests in general). What do you think about adding that? Would you accept a PR? Any sense of the level of effort?", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1495431932, "node_id": "I_kwDOBm6k_c5ZInr8", "number": 1951, "title": "`datasette.create_token(...)` method for creating signed API tokens", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 6, "created_at": "2022-12-14T01:25:34Z", "updated_at": "2022-12-14T02:43:45Z", "closed_at": "2022-12-14T02:42:05Z", "author_association": "OWNER", "pull_request": null, "body": "I need this for:\r\n- #1947\r\n\r\nAnd I can refactor this to use it too:\r\n- #1855\r\n\r\nBy making this a documented internal API it can be used by other plugins too. It's also going to be really useful for writing tests.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1951/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1486011362, "node_id": "PR_kwDOBm6k_c5E3XqB", "number": 1940, "title": "register_permissions() plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8711695, "label": " Datasette 1.0a2"}, "comments": 6, "created_at": "2022-12-09T05:09:28Z", "updated_at": "2022-12-13T02:05:55Z", "closed_at": "2022-12-13T02:05:54Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1940", "body": "Refs #1939\r\n\r\nFrom this comment: https://github.com/simonw/datasette/issues/1939#issuecomment-1343872168\r\n\r\n- [x] Unit test for the registration plugin hook itself\r\n- [x] Use them in `check_permission_actions_are_documented` test in `conftest.py`\r\n- [x] Add description field to `Permissions` (and update tests and docs)\r\n- [x] Documentation for `datasette.permissions` dictionary\r\n- [x] If no `default=` provided in call to `permission_allowed()` then use default from `datasette.permissions` list\r\n- [x] Remove `default=` from a bunch of places\r\n- [x] Throw an error if two permissions are registered with the same name or abbreviation (but other attributes differ)\r\n- [x] Update authentication and permissions documentation to explain that permissions are now registered and have a registered default\r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://datasette--1940.org.readthedocs.build/en/1940/\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1940/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1468709531, "node_id": "I_kwDOBm6k_c5Xirqb", "number": 1915, "title": "Interactive demo of Datasette 1.0 write APIs", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-11-29T21:16:03Z", "updated_at": "2022-11-30T04:05:46Z", "closed_at": "2022-11-30T04:05:46Z", "author_association": "OWNER", "pull_request": null, "body": "I'm going to try to get this working on https://latest.datasette.io/ - it already has a way for people to sign in as root, but none of the databases there are writable.\r\n\r\nSo I'm going to build a plugin which adds a writable named in-memory database.\r\n\r\nAnd some kind of mechanism for clearing out that database on a regular basis - maybe tables in that database get deleted automatically an hour after they are created?\r\n\r\n(Would be neat to display their time-left-until-deleted too)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1915/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1439009231, "node_id": "I_kwDOBm6k_c5VxYnP", "number": 1884, "title": "Exclude virtual tables from datasette inspect", "user": {"value": 25778, "label": "eyeseast"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-11-07T21:26:01Z", "updated_at": "2022-11-21T04:40:56Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Ran `inspect` on a spatialite database and got these warnings:\r\n\r\n```\r\nERROR: conn=, sql = 'select count(*) from [SpatialIndex]', params = None: no such module: VirtualSpatialIndex\r\nERROR: conn=, sql = 'select count(*) from [ElementaryGeometries]', params = None: no such module: VirtualElementary\r\nERROR: conn=, sql = 'select count(*) from [KNN]', params = None: no such module: VirtualKNN\r\n```\r\n\r\nIt still worked, but probably want to catch this.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1884/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1363440999, "node_id": "I_kwDOBm6k_c5RRHVn", "number": 1804, "title": "Ability to set a custom facet_size per table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-09-06T15:11:40Z", "updated_at": "2022-09-07T00:21:56Z", "closed_at": "2022-09-06T18:06:53Z", "author_association": "OWNER", "pull_request": null, "body": "Suggestion from Discord: https://discord.com/channels/823971286308356157/823971286941302908/1016725586351247430\r\n\r\n> Is it possible to limit the facet size per database or even per table?\r\n\r\nThis is a really good idea, it could be done in `metadata.yml`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1804/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1348294436, "node_id": "PR_kwDOCGYnMM49qP2V", "number": 468, "title": "db[table].create(..., transform=True) and create-table --transform", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8355157, "label": "3.29"}, "comments": 6, "created_at": "2022-08-23T17:27:58Z", "updated_at": "2022-08-27T23:17:55Z", "closed_at": "2022-08-27T23:17:55Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/468", "body": "Work in progress. Still needs documentation and tests (and to cover more cases of things that might have changed).\r\n\r\nRefs:\r\n- #467\r\n\r\n\r\n----\r\n:books: Documentation preview :books:: https://sqlite-utils--468.org.readthedocs.build/en/468/\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/468/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1352932038, "node_id": "I_kwDOCGYnMM5QpBrG", "number": 470, "title": "Upgrade `--load-extension` to accept entrypoints like Datasette", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 8355157, "label": "3.29"}, "comments": 6, "created_at": "2022-08-27T03:53:20Z", "updated_at": "2022-08-27T05:55:49Z", "closed_at": "2022-08-27T05:55:48Z", "author_association": "OWNER", "pull_request": null, "body": "Imitate:\r\n- https://github.com/simonw/datasette/pull/1789\r\n```\r\n# would load default entrypoint like before\r\ndatasette data.db --load-extension ext\r\n\r\n# loads the extensions with the \"sqlite3_foo_init\" entrpoint\r\ndatasette data.db --load-extension ext:sqlite3_foo_init\r\n\r\n# loads the extensions with the \"sqlite3_bar_init\" entrpoint\r\ndatasette data.db --load-extension ext:sqlite3_bar_init\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/470/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1084193403, "node_id": "PR_kwDOBm6k_c4wDKmb", "number": 1574, "title": "introduce new option for datasette package to use a slim base image", "user": {"value": 33631, "label": "fs111"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-12-19T21:18:19Z", "updated_at": "2022-08-15T08:49:31Z", "closed_at": "2022-08-15T08:49:31Z", "author_association": "NONE", "pull_request": "simonw/datasette/pulls/1574", "body": "The official python images on docker hub come with a slim variant that is significantly smaller than the default. The diff does not change the default, but allows to switch to the `slim` variant with commandline switch (`--slim-base-image`)\r\n\r\nSize comparison:\r\n\r\n```\r\n$ datasette package some.db -t fat --install \"datasette-basemap datasette-cluster-map\"\r\n\r\n$ datasette package some.db -t slim --slim-base-image --install \"datasette-basemap datasette-cluster-map\"\r\n\r\n$ docker images\r\nREPOSITORY TAG IMAGE ID CREATED SIZE\r\nfat latest 807b393ace0d 9 seconds ago 978MB\r\nslim latest 31bc5e63505c 8 minutes ago 191MB\r\n\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1574/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1223699280, "node_id": "I_kwDOBm6k_c5I8CtQ", "number": 1739, "title": ".db downloads should be served with an ETag", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-05-03T05:11:21Z", "updated_at": "2022-05-04T18:21:18Z", "closed_at": "2022-05-03T14:59:51Z", "author_association": "OWNER", "pull_request": null, "body": "I noticed that my Pyodide Datasette prototype is downloading the same database file every single time rather than browser caching it:\r\n\r\n![image](https://user-images.githubusercontent.com/9599/166407074-dee19587-0667-4424-9e88-d3b5b90fd819.png)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1739/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1193090967, "node_id": "I_kwDOBm6k_c5HHR-X", "number": 1699, "title": "Proposal: datasette query", "user": {"value": 25778, "label": "eyeseast"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-04-05T12:36:43Z", "updated_at": "2022-04-11T01:32:12Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I started sketching out a plugin to add a `datasette query` subcommand to export data from the command line. This is based on discussions in #1356 and #1605. Before I get too far down this rabbit hole, I figure it's worth getting some feedback here (unless this should happen in `Discussions`). Here's what I'm thinking:\r\n\r\nAt its most basic, it will write the results of a query to STDOUT.\r\n\r\n```sh\r\ndatasette query -d data.db 'select * from data' > results.json\r\n```\r\n\r\nThis isn't much improvement over using [sqlite-utils](https://github.com/simonw/sqlite-utils). To make better use of datasette and its ecosystem, run `datasette query` using a canned query defined in a `metadata.yml` file.\r\n\r\nFor example, using the metadata file from [alltheplaces-datasette](https://github.com/eyeseast/alltheplaces-datasette/blob/main/metadata.yml):\r\n\r\n```sh\r\ncd alltheplaces-datasette\r\ndatasette query -d alltheplaces.db -m metadata.yml count_by_spider\r\n```\r\n\r\nThat query would be good to get as CSV, and we can auto-discover metadata and databases in the current directory:\r\n\r\n```sh\r\ncd alltheplaces-datasette\r\ndatasette query count_by_spider -f csv\r\n```\r\n\r\nIn this case, `count_by_spider` is a canned query defined on the `alltheplaces` database. If the same query is defined on multiple databases or its otherwise unclear which database `query` should use, pass the `-d` or `--database` option.\r\n\r\nIf a query takes parameters, I can pass them in at runtime, using the `--param` or `-p` option:\r\n\r\n```sh\r\ndatasette query -d data.db -p value something 'select * from neighborhoods where some_column = :value'\r\n```\r\n\r\nI'm very interested in feedback on this, including whether it should be a plugin or in Datasette core. (I don't have a strong opinion about this, but I'm prototyping it as a plugin to start.)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1699/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1174423568, "node_id": "I_kwDOBm6k_c5GAEgQ", "number": 1670, "title": "Ship Datasette 0.61", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-03-20T02:47:54Z", "updated_at": "2022-03-23T18:32:32Z", "closed_at": "2022-03-23T18:32:03Z", "author_association": "OWNER", "pull_request": null, "body": "Let the alpha bake for a while, since #1668 is a big last-minute change.\r\n\r\nAfter shipping, release a new `datasette-hashed-urls` that depends on it, also this:\r\n\r\n- https://github.com/simonw/datasette-hashed-urls/issues/11", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1670/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1126604194, "node_id": "I_kwDOBm6k_c5DJp2i", "number": 1632, "title": "datasette one.db one.db opens database twice, as one and one_2", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2022-02-07T23:14:47Z", "updated_at": "2022-03-19T04:04:49Z", "closed_at": "2022-02-07T23:50:01Z", "author_association": "OWNER", "pull_request": null, "body": "> ```\r\n> % mkdir /tmp/data\r\n> % cp ~/Dropbox/Development/datasette/fixtures.db /tmp/data \r\n> % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852\r\n> ...\r\n> INFO: Uvicorn running on http://127.0.0.1:8852 (Press CTRL+C to quit)\r\n> ^CINFO: Shutting down\r\n> % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852\r\n> ...\r\n> INFO: 127.0.0.1:49533 - \"GET / HTTP/1.1\" 200 OK\r\n> ```\r\n> The first time I ran Datasette I got two databases - `fixtures` and `created`\r\n> \r\n> BUT... when I ran Datasette the second time it looked like this:\r\n> \r\n> \"image\"\r\n> \r\n> This is the same result you get if you run:\r\n> \r\n> datasette /tmp/data/fixtures.db /tmp/data/created.db /tmp/data/created.db\r\n> \r\n> This is caused by this Datasette issue:\r\n> - https://github.com/simonw/datasette/issues/509\r\n> \r\n> So... either I teach Datasette to de-duplicate multiple identical file paths passed to the command, or I can't use `/data/*.db` in the `Dockerfile` here and I need to go back to other solutions for the challenge described in this comment: https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1031971831\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1032029874_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1632/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1145882578, "node_id": "I_kwDOCGYnMM5ETMfS", "number": 408, "title": "`deterministic=True` fails on versions of SQLite prior to 3.8.3", "user": {"value": 24938923, "label": "learning4life"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-02-21T14:36:43Z", "updated_at": "2022-03-13T16:54:09Z", "closed_at": "2022-03-02T00:38:11Z", "author_association": "NONE", "pull_request": null, "body": "Hi, love your work.\r\n\r\nI am unable to lookup indexes in a database using sqlite-utils:\r\n\r\n`\r\nsqlite-utils indexes city_spec.db --table`\r\n\r\nor\r\n\r\n`sqlite-utils indexes city_spec.db MyTable\r\n`\r\n\r\n**Software**\r\nsqlite-utils, version 3.24\r\nsqlite3 --version: 3.36.0 \r\n\r\n**Output:**\r\n\r\nTraceback (most recent call last):\r\n File \"/opt/app-root/bin/sqlite-utils\", line 8, in \r\n sys.exit(cli())\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/core.py\", line 1128, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/core.py\", line 1053, in main\r\n rv = self.invoke(ctx)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/core.py\", line 1659, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/core.py\", line 1395, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/core.py\", line 754, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/decorators.py\", line 26, in new_func\r\n return f(get_current_context(), *args, **kwargs)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/cli.py\", line 2123, in indexes\r\n ctx.invoke(\r\n File \"/opt/app-root/lib64/python3.8/site-packages/click/core.py\", line 754, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/cli.py\", line 1624, in query\r\n db.register_fts4_bm25()\r\n File \"/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py\", line 403, in register_fts4_bm25\r\n self.register_function(rank_bm25, deterministic=True)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py\", line 399, in register_function\r\n register(fn)\r\n File \"/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py\", line 392, in register\r\n self.conn.create_function(name, arity, fn, **kwargs)\r\nsqlite3.NotSupportedError: deterministic=True requires SQLite 3.8.3 or higher\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/408/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 678760988, "node_id": "MDU6SXNzdWU2Nzg3NjA5ODg=", "number": 932, "title": "End-user documentation", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2020-08-13T22:04:39Z", "updated_at": "2022-03-08T15:20:48Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Datasette's documentation is aimed at people who install and configure it.\r\n\r\nWhat about end users of preconfigured and deployed Datasette instances?\r\n\r\nSomething that can be linked to from the Datasette UI would be really useful.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/932/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 764059235, "node_id": "MDU6SXNzdWU3NjQwNTkyMzU=", "number": 1143, "title": "More flexible CORS support in core, to encourage good security practices", "user": {"value": 114388, "label": "yurivish"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2020-12-12T17:06:35Z", "updated_at": "2022-02-13T17:41:17Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "It would be nice if the `--cors` option accepted an origin regex to more securely allow secure local development.\r\n\r\nAs an example, Observable notebooks namespace every user's notebooks by their username and user content is served from username.observableusercontent.com, so you would set `--cors-origin username.observableusercontent.com` to restrict access to a local development Datasette instance to only your own notebooks, rather than exposing the data to any website that makes a request.\r\n\r\nThank you for all of your work on Datasette!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1143/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1121121305, "node_id": "I_kwDOBm6k_c5C0vQZ", "number": 1618, "title": "Reconsider policy on blocking queries containing the string \"pragma\"", "user": {"value": 770231, "label": "strada"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2022-02-01T19:39:46Z", "updated_at": "2022-02-02T19:42:03Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "First of all, thanks for creating this cool project, and also supporting publishing to various hosting services out of the box.\r\n\r\nWhile testing out, I noticed legitimate queries such as \r\n```\r\nselect * from books where title like 'Pragmatic%'\r\n```\r\nor\r\n```\r\nselect * from books where title = 'The Pragmatic Programmer'\r\n```\r\nare blocked, due to the regular expression check here:\r\nhttps://github.com/simonw/datasette/blob/main/datasette/utils/__init__.py#L185\r\n\r\nExample as seen from a Datasette instance:\r\nhttps://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+like+%27Pragmatic%25%27%0D%0A\r\n\r\nI'd propose a regular expression like \r\n```\r\nre.compile(f\"pragma_(?!({'|'.join(allowed_pragmas)}))\"),\r\n```\r\ninstead of\r\n```\r\nre.compile(f\"pragma(?!_({'|'.join(allowed_pragmas)}))\"),\r\n```\r\n\r\nI can create a pull request with this change, unless the maintainers think it would allow unwanted queries to be executed.\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1618/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1087913724, "node_id": "I_kwDOBm6k_c5A2D78", "number": 1577, "title": "Drop support for Python 3.6", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2021-12-23T18:17:03Z", "updated_at": "2022-01-25T23:30:03Z", "closed_at": "2022-01-20T04:31:41Z", "author_association": "OWNER", "pull_request": null, "body": "*Original title: Decide when to drop support for Python 3.6*\r\n\r\n> `context_vars` can solve this but they were introduced in Python 3.7: https://www.python.org/dev/peps/pep-0567/\r\n>\r\n> Python 3.6 support ends in a few days time, and it looks like Glitch has updated to 3.7 now - so maybe I can get away with Datasette needing 3.7 these days?\r\n>\r\n> Tweeted about that here: https://twitter.com/simonw/status/1473761478155010048\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1576#issuecomment-999878907_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1577/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1083669410, "node_id": "I_kwDOBm6k_c5Al3ui", "number": 1566, "title": "Release Datasette 0.60", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7571612, "label": "Datasette 0.60"}, "comments": 6, "created_at": "2021-12-17T22:58:12Z", "updated_at": "2022-01-14T01:59:55Z", "closed_at": "2022-01-14T01:59:55Z", "author_association": "OWNER", "pull_request": null, "body": "Using this as a tracking issue. I'm hoping to get the bulk of the JSON redesign work from the refactor in #1554 in for this release.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1566/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1076388044, "node_id": "I_kwDOBm6k_c5AKGDM", "number": 1547, "title": "Writable canned queries fail to load custom templates", "user": {"value": 127565, "label": "wragge"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7571612, "label": "Datasette 0.60"}, "comments": 6, "created_at": "2021-12-10T03:31:48Z", "updated_at": "2022-01-13T22:27:59Z", "closed_at": "2021-12-19T21:12:00Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I've created a canned query with `\"write\": true` set. I've also created a custom template for it, but the template doesn't seem to be found. If I look in the HTML I see (`stock_exchange` is the db name):\r\n\r\n``\r\n\r\nMy non-writeable canned queries pick up custom templates as expected, and if I look at their HTML I see the canned query name added to the templates considered (the canned query here is `date_search`):\r\n\r\n``\r\n\r\nSo it seems like the writeable canned query is behaving differently for some reason. Is it an authentication thing? I'm using the built in `--root` authentication.\r\n\r\nThanks!\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1547/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1067771698, "node_id": "I_kwDOCGYnMM4_pOcy", "number": 348, "title": "Command for creating an empty database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7558727, "label": "3.21"}, "comments": 6, "created_at": "2021-11-30T23:24:27Z", "updated_at": "2022-01-13T07:06:59Z", "closed_at": "2022-01-09T20:33:20Z", "author_association": "OWNER", "pull_request": null, "body": "I sometimes find the need to create an empty SQLite database file - for example if I want to enable WAL on it before using it with another script. I currently do that like this:\r\n\r\n sqlite3 my.db vacuum\r\n sqlite-utils enable-wal my.db\r\n\r\nIt would be nice if `sqlite-utils` had a convenience command for doing this.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/348/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1097128334, "node_id": "I_kwDOCGYnMM5BZNmO", "number": 371, "title": "Support mutating row in `--convert` without returning it", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 7558727, "label": "3.21"}, "comments": 6, "created_at": "2022-01-09T07:38:44Z", "updated_at": "2022-01-10T19:27:30Z", "closed_at": "2022-01-09T20:06:15Z", "author_association": "OWNER", "pull_request": null, "body": "Currently you have to do this:\r\n```\r\n$ sqlite-utils insert dogs.db dogs dogs.json --convert '\r\nrow[\"is_good\"] = 1\r\nreturn row'\r\n```\r\nWould be neat if this worked too:\r\n```\r\n$ sqlite-utils insert dogs.db dogs dogs.json \\\r\n --convert 'row[\"is_good\"] = 1'\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/371/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 1082743068, "node_id": "PR_kwDOBm6k_c4v-izc", "number": 1559, "title": "filters_from_request plugin hook, now used in TableView", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-12-16T23:59:33Z", "updated_at": "2021-12-17T23:09:41Z", "closed_at": "2021-12-17T19:02:15Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1559", "body": "New plugin hook, refs #473\r\n\r\nUsed it to extract the logic from TableView that handles _search and\r\n_through and _where - refs #1518", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1559/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1068791148, "node_id": "I_kwDOBm6k_c4_tHVs", "number": 1540, "title": "Idea: hover to reveal details of linked row", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-12-01T19:28:07Z", "updated_at": "2021-12-09T23:38:39Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "\"fara__item_version__7_rows_where_where__item___5236\"\r\n\r\nHovering over that could work a little bit like GitHub issue links:\r\n\r\n![hover](https://user-images.githubusercontent.com/9599/144300537-9cd9e9af-ac16-42db-842f-37661bc94063.gif)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1540/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1052851176, "node_id": "I_kwDOBm6k_c4-wTvo", "number": 1507, "title": "ReadTheDocs build failed for 0.59.2 release", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-11-14T05:24:34Z", "updated_at": "2021-11-14T05:41:55Z", "closed_at": "2021-11-14T05:41:55Z", "author_association": "OWNER", "pull_request": null, "body": "I had to cancel the 0.59.2 release because ReadTheDocs was failing to build the documentation.\r\n\r\nhttps://readthedocs.org/projects/datasette/builds/15268454/\r\n\r\n```\r\n /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html\r\nRunning Sphinx v1.8.5\r\nloading translations [en]... done\r\nmaking output directory...\r\nbuilding [mo]: targets for 0 po files that are out of date\r\nbuilding [html]: targets for 27 source files that are out of date\r\nupdating environment: 27 added, 0 changed, 0 removed\r\nreading sources... [ 3%] authentication\r\n\r\nTraceback (most recent call last):\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/cmd/build.py\", line 304, in build_main\r\n app.build(args.force_all, filenames)\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/application.py\", line 341, in build\r\n self.builder.build_update()\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py\", line 347, in build_update\r\n len(to_build))\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py\", line 360, in build\r\n updated_docnames = set(self.read())\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py\", line 468, in read\r\n self._read_serial(docnames)\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py\", line 490, in _read_serial\r\n self.read_doc(docname)\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py\", line 534, in read_doc\r\n doctree = read_doc(self.app, self.env, self.env.doc2path(docname))\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/io.py\", line 318, in read_doc\r\n pub.publish()\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py\", line 219, in publish\r\n self.apply_transforms()\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py\", line 200, in apply_transforms\r\n self.document.transformer.apply_transforms()\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py\", line 90, in apply_transforms\r\n Transformer.apply_transforms(self)\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/transforms/__init__.py\", line 171, in apply_transforms\r\n transform.apply(**kwargs)\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py\", line 245, in apply\r\n apply_source_workaround(n)\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py\", line 94, in apply_source_workaround\r\n for classifier in reversed(node.parent.traverse(nodes.classifier)):\r\nTypeError: argument to reversed() must be a sequence\r\n\r\nException occurred:\r\n File \"/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py\", line 94, in apply_source_workaround\r\n for classifier in reversed(node.parent.traverse(nodes.classifier)):\r\nTypeError: argument to reversed() must be a sequence\r\nThe full traceback has been saved in /tmp/sphinx-err-vkl0oE.log, if you want to report the issue to the developers.\r\nPlease also report this if it was a user error, so that a better error message can be provided next time.\r\nA bug report can be filed in the tracker at . Thanks! \r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1507/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 707478649, "node_id": "MDU6SXNzdWU3MDc0Nzg2NDk=", "number": 173, "title": "Progress bar for sqlite-utils insert", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-09-23T15:43:56Z", "updated_at": "2021-11-01T08:42:24Z", "closed_at": "2020-10-27T18:16:04Z", "author_association": "OWNER", "pull_request": null, "body": "It would be nice if `sqlite-utils insert` had a progress bar, for when it's churning through huge CSV files.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/173/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 991191951, "node_id": "MDU6SXNzdWU5OTExOTE5NTE=", "number": 1464, "title": "clean checkout & clean environment has test failures", "user": {"value": 51016, "label": "ctb"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-09-08T14:16:23Z", "updated_at": "2021-09-13T22:17:17Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I followed the instructions [here](https://docs.datasette.io/en/stable/contributing.html#setting-up-a-development-environment), and even after running `python update-docs-help.py` I get the following failed tests -- any thoughts?\r\n\r\n```\r\nFAILED tests/test_api.py::test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]\r\nFAILED tests/test_api.py::test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1]\r\nFAILED tests/test_api.py::test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2]\r\n```\r\n\r\nThis is with python 3.9.7 and lots of other packages, as in attached environment listing from `conda list`.\r\n[conda-installed.txt](https://github.com/simonw/datasette/files/7129487/conda-installed.txt)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1464/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 907645813, "node_id": "MDU6SXNzdWU5MDc2NDU4MTM=", "number": 57, "title": "Error: Use either --since or --since_id, not both", "user": {"value": 42904, "label": "rubenv"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-05-31T18:11:04Z", "updated_at": "2021-08-20T00:01:31Z", "closed_at": "2021-08-20T00:01:31Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I'm using the following command:\r\n\r\n```\r\ntwitter-to-sqlite user-timeline -a twitter-auth.json twitter/tweets.db --since\r\n```\r\n\r\nWhich gives the following error:\r\n```\r\nError: Use either --since or --since_id, not both\r\n```\r\n\r\nRunning without `--since`.\r\n\r\n```\r\nTraceback (most recent call last):\r\n File \"/usr/local/bin/twitter-to-sqlite\", line 8, in \r\n sys.exit(cli())\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1137, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1062, in main\r\n rv = self.invoke(ctx)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1668, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1404, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 763, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/twitter_to_sqlite/cli.py\", line 317, in user_timeline\r\n for tweet in bar:\r\n File \"/usr/local/lib/python3.9/site-packages/click/_termui_impl.py\", line 328, in generator\r\n for rv in self.iter:\r\n File \"/usr/local/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 234, in fetch_user_timeline\r\n yield from fetch_timeline(\r\n File \"/usr/local/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 202, in fetch_timeline\r\n raise Exception(str(tweets[\"errors\"]))\r\nException: [{'code': 44, 'message': 'since_id parameter is invalid.'}]\r\n```\r\n\r\n```\r\nPython 3.9.5\r\ntwitter-to-sqlite, version 0.21.3\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57/reactions\", \"total_count\": 4, \"+1\": 4, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 974987856, "node_id": "MDU6SXNzdWU5NzQ5ODc4NTY=", "number": 1442, "title": "Mechanism to cause specific branches to deploy their own demos", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-08-19T19:41:39Z", "updated_at": "2021-08-19T21:11:45Z", "closed_at": "2021-08-19T21:09:40Z", "author_association": "OWNER", "pull_request": null, "body": "A useful capability would be if it was super-easy to say \"any pushes to branch X should be deployed to `latest-X.datasette.io`\".\r\n\r\nI'd like to use this for the column query information work in #1434", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1442/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 465815372, "node_id": "MDU6SXNzdWU0NjU4MTUzNzI=", "number": 37, "title": "Experiment with type hints", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-07-09T14:30:34Z", "updated_at": "2021-08-18T21:48:57Z", "closed_at": "2021-08-18T21:48:57Z", "author_association": "OWNER", "pull_request": null, "body": "Since it's designed to be used in Jupyter or for rapid prototyping in an IDE (and it's still pretty small) `sqlite-utils` feels like a great candidate for me to finally try out Python type hints.\r\n\r\nhttps://veekaybee.github.io/2019/07/08/python-type-hints/ is good.\r\n\r\nIt suggests the mypy docs for getting started: https://mypy.readthedocs.io/en/latest/existing_code.html plus this tutorial: https://pymbook.readthedocs.io/en/latest/typehinting.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/37/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 944326512, "node_id": "MDU6SXNzdWU5NDQzMjY1MTI=", "number": 296, "title": "`table.search(..., quote=True)` parameter and `sqlite-utils search --quote` option", "user": {"value": 32427188, "label": "deafmute1"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-07-14T11:26:47Z", "updated_at": "2021-08-18T20:13:12Z", "closed_at": "2021-08-18T20:10:48Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nRecently got this error:\r\n```\r\nTraceback (most recent call last):\r\n File \"\", line 1, in \r\n File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/__init__.py\", line 38, in \r\n start(\"/home/ethan/git/music-metadata-indexer/sample\", \"/home/ethan/git/music-metadata-indexer/test.db\")\r\n File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/__init__.py\", line 23, in start\r\n scanner.build_database()\r\n File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/scan.py\", line 79, in build_database\r\n _import_song(self.db, Path(dirpath).joinpath(f), self.logger) \r\n File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/scan.py\", line 23, in _import_song\r\n db.add_song(filepath)\r\n File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/index.py\", line 166, in add_song\r\n for match in self.search(\"albums\", album): \r\n File \"/home/ethan/git/music-metadata-indexer/env/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1625, in search\r\n cursor = self.db.execute(\r\n File \"/home/ethan/git/music-metadata-indexer/env/lib/python3.9/site-packages/sqlite_utils/db.py\", line 243, in execute\r\n return self.conn.execute(sql, parameters)\r\nsqlite3.OperationalError: fts5: syntax error near \".\" \r\n```\r\nSo, the error seems to suggest there was a \".\" character somewhere in the SQL command that was causing the error. I did a little digging and found this in the docs: https://www.sqlite.org/fts5.html#fts5_strings. \".\" is one of the many prohibited characters.\r\n\r\nMy solution was to just strip these out of the query using this line\r\n`query = query.translate({e: None for e in itertools.chain(range(0,26), range(27, 48), range(58,65), range(91,95), [96], range(123,128))})`\r\n\r\nPerhaps this could be included into the `table.search()` function?\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/296/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 963897111, "node_id": "MDU6SXNzdWU5NjM4OTcxMTE=", "number": 309, "title": "sqlite-utils insert errors should show SQL and parameters, if possible", "user": {"value": 16622642, "label": "scaleoutsean"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-08-09T11:24:14Z", "updated_at": "2021-08-09T23:40:29Z", "closed_at": "2021-08-09T22:25:58Z", "author_association": "NONE", "pull_request": null, "body": "I've tried several approaches, but this is the current one:\r\n\r\n```sh\r\necho $json-line | sqlite-utils insert json.db jsontable --truncate --alter --detect-types -\r\n```\r\nIn all cases, I get this error:\r\n\r\n```sh\r\nOverflowError: Python int too large to convert to SQLite INTEGER\r\nTraceback (most recent call last):\r\n File \"/home/sean/.local/bin/sqlite-utils\", line 8, in \r\n sys.exit(cli())\r\n File \"/usr/lib/python3/dist-packages/click/core.py\", line 764, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/usr/lib/python3/dist-packages/click/core.py\", line 717, in main\r\n rv = self.invoke(ctx)\r\n File \"/usr/lib/python3/dist-packages/click/core.py\", line 1137, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/usr/lib/python3/dist-packages/click/core.py\", line 956, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/usr/lib/python3/dist-packages/click/core.py\", line 555, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/home/sean/.local/lib/python3.8/site-packages/sqlite_utils/cli.py\", line 841, in insert\r\n insert_upsert_implementation(\r\n File \"/home/sean/.local/lib/python3.8/site-packages/sqlite_utils/cli.py\", line 780, in insert_upsert_implementation\r\n db[table].insert_all(\r\n File \"/home/sean/.local/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2145, in insert_all\r\n self.insert_chunk(\r\n File \"/home/sean/.local/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1957, in insert_chunk\r\n result = self.db.execute(query, params)\r\n File \"/home/sean/.local/lib/python3.8/site-packages/sqlite_utils/db.py\", line 257, in execute\r\n return self.conn.execute(sql, parameters)\r\n```\r\n\r\nI googled the error and checked SO answers and advice, all good. I changed my JSON file to not use integers so I no longer get this error. Of course, that makes using the database a bit harder, so I also tried to solve the problem by modifying DB structure (while using integers in JSON).\r\n\r\nIf change all `INTEGER` Data Types to something else (`STRING`, `TEXT`) and try to import again using `--truncate`, I still get this error. I suppose I should tell sqlite-utils which columns should use non-INTEGER Data Type rather than rely on it to check my SQL table configuration.\r\n\r\nIf that is the case, can this error be a bit more specific for easier troubleshooting - maybe tell us which which record caused the problem when that error is thrown? \r\n\r\nMy table has 60+ columns, many of which use 64-bit integers (not all records are large or known in advance), so while I can modify JSON to use strings instead of integers, it decreases usability and finding out which records have values for which SQLite integers aren't sufficient requires some work (I'm thinking about parsing all integers with `jq` and sorting output by length to identify those columns, but I'd prefer if sqlite-utils could tell me that). \r\n\r\nMy environment:\r\n\r\n- Python 3.8.10\r\n - sqlite-utils 3.14 \r\n - pandas 1.3.1 \r\n - numpy 1.21.1 \r\n - sqlite-fts4 1.0.1\r\n- sqlite 3.31.1-4ubuntu0.2\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/309/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 940077168, "node_id": "MDU6SXNzdWU5NDAwNzcxNjg=", "number": 1389, "title": "\"searchmode\": \"raw\" in table metadata", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-07-08T17:32:10Z", "updated_at": "2021-07-10T18:33:13Z", "closed_at": "2021-07-10T18:33:13Z", "author_association": "OWNER", "pull_request": null, "body": "> http://localhost:8001/index/summary?_search=language%3Aeng&_sort=title&_searchmode=raw\r\n>\r\n> But I'm not able to manage it in the metadata file. Here is mine (note that the sort column is taken into account)\r\n> Here it is:\r\n>\r\n> ```\r\n> {\r\n> \"databases\": {\r\n> \"index\": {\r\n> \"tables\": {\r\n> \"summary\": {\r\n> \"sort\": \"title\",\r\n> \"searchmode\": \"raw\"\r\n> }\r\n> }\r\n> }\r\n> }\r\n> }\r\n\r\n_Originally posted by @Krazybug in https://github.com/simonw/datasette/issues/759#issuecomment-624860451_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1389/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 926777310, "node_id": "MDU6SXNzdWU5MjY3NzczMTA=", "number": 290, "title": "`db.query()` method (renamed `db.execute_returning_dicts()`)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-06-22T03:03:54Z", "updated_at": "2021-06-24T23:17:38Z", "closed_at": "2021-06-24T22:54:43Z", "author_association": "OWNER", "pull_request": null, "body": "Most of this library deals with lists of Python dictionaries - `.insert_all()`, `.rows`, `.rows_where()`, `.search()`.\r\n\r\nThe `db.execute()` method is the only thing that returns a `sqlite3` cursor.\r\n\r\nThere is a clumsily named `db.execute_returning_dicts(sql)` method but it's not currently mentioned in the documentation.\r\n\r\nIt needs a better name, and needs to be properly documented.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/290/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 395236066, "node_id": "MDU6SXNzdWUzOTUyMzYwNjY=", "number": 393, "title": "CSV export in \"Advanced export\" pane doesn't respect query", "user": {"value": 1727065, "label": "ltrgoddard"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-01-02T12:39:41Z", "updated_at": "2021-06-17T18:14:24Z", "closed_at": "2019-01-03T02:44:10Z", "author_association": "NONE", "pull_request": null, "body": "It looks like there's an inconsistency when exporting to CSV via the the web interface. Say I'm looking at [songs released in 1989](https://fivethirtyeight.datasettes.com/fivethirtyeight-c300360/classic-rock%2Fclassic-rock-song-list?Release+Year__exact=1989) in the `classic-rock/classic-rock-song-list` table from the Five Thirty Eight data. The JSON and CSV export links at the top of the page both give me filtered data using `Release+Year__exact=1989` in the URL. In the `Advanced export` tab, though, the CSV option gives me the whole data set, while the JSON options preserve the query.\r\n\r\nIt may be that this is intended behaviour related to the streaming CSV stuff [discussed here](https://github.com/simonw/datasette/issues/266), but if that's the case then I think it should be a little clearer.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/393/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 906356331, "node_id": "MDU6SXNzdWU5MDYzNTYzMzE=", "number": 263, "title": "`sqlite-utils indexes` command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-05-29T04:52:34Z", "updated_at": "2021-06-03T04:34:38Z", "closed_at": "2021-06-03T04:34:38Z", "author_association": "OWNER", "pull_request": null, "body": "While working on #260 I realized there's no command to show indexes in a database, even though there is one for showing tables and one for triggers.\r\n\r\nI should implement #261 first.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/263/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 520667773, "node_id": "MDU6SXNzdWU1MjA2Njc3NzM=", "number": 620, "title": "Mechanism for indicating foreign key relationships in the table and query page URLs", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-11-10T22:26:27Z", "updated_at": "2021-04-05T03:57:22Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Datasette currently only inflates foreign keys (into names hyperlinks) if it detects them as foreign key constraints in the underlying database.\r\n\r\nIt would be useful if you could specify additional \"foreign keys\" using both `metadata.json` and the querystring - similar time how you can pass `?_fts_table=x` https://datasette.readthedocs.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/620/reactions\", \"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "draft": null, "state_reason": null} {"id": 724369025, "node_id": "MDExOlB1bGxSZXF1ZXN0NTA1NzY5NDYy", "number": 1031, "title": "Fallback to databases in inspect-data.json when no -i options are passed", "user": {"value": 299380, "label": "frankier"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-10-19T07:51:06Z", "updated_at": "2021-03-29T01:46:45Z", "closed_at": "2021-03-29T00:23:41Z", "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/1031", "body": "Currenlty `Datasette.__init__` checks immutables against None to decide whether to fallback to inspect-data.json. This patch modifies the serve command to pass None when no -i options are passed so this fallback works correctly.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1031/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 826613352, "node_id": "MDExOlB1bGxSZXF1ZXN0NTg4NjAxNjI3", "number": 1254, "title": "Update Docker Spatialite version to 5.0.1 + add support for Spatialite topology functions", "user": {"value": 3200608, "label": "durkie"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-03-09T20:49:08Z", "updated_at": "2021-03-10T18:27:45Z", "closed_at": "2021-03-09T22:04:23Z", "author_association": "NONE", "pull_request": "simonw/datasette/pulls/1254", "body": "This requires adding the RT Topology library (Spatialite changed to RT Topology from LWGEOM between 4.4 and 5.0), as well as upgrading the GEOS version (which is the reason for switching to `python:3.7.10-slim-buster` as the base image.)\r\n\r\n`autoconf` and `libtool` are added to build RT Topology, and Spatialite is now built with `--disable-minizip` (minizip wasn't an option in 4.4 and I didn't want to add another dependency) and `--disable-dependency-tracking` which, according to Spatialite, \"speeds up one-time builds\"", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1254/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 763320133, "node_id": "MDExOlB1bGxSZXF1ZXN0NTM3NzkxNjc1", "number": 208, "title": "sqlite-utils analyze-tables command and table.analyze_column() method", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-12-12T05:27:49Z", "updated_at": "2020-12-13T07:20:16Z", "closed_at": "2020-12-13T07:20:12Z", "author_association": "OWNER", "pull_request": "simonw/sqlite-utils/pulls/208", "body": "Refs #207\r\n\r\n- [x] Improve design of CLI output\r\n- [x] Truncate long values in least/most common\r\n- [x] Add a `-c` column selection option\r\n- [x] Tests\r\n- [x] Documentation", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/208/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 753767911, "node_id": "MDExOlB1bGxSZXF1ZXN0NTI5NzgzMjc1", "number": 1117, "title": "Support for generated columns", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-11-30T20:10:46Z", "updated_at": "2020-11-30T22:23:19Z", "closed_at": "2020-11-30T21:29:58Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1117", "body": "Refs #1116. My first attempt at this worked on my laptop but broke in CI, so I'm going to iterate on it in a pull request instead.", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1117/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 714449879, "node_id": "MDU6SXNzdWU3MTQ0NDk4Nzk=", "number": 992, "title": "Change \"--config foo:bar\" to \"--setting foo bar\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6055094, "label": "Datasette 0.52"}, "comments": 6, "created_at": "2020-10-05T01:27:45Z", "updated_at": "2020-11-24T20:01:54Z", "closed_at": "2020-11-24T20:01:54Z", "author_association": "OWNER", "pull_request": null, "body": "I designed the config format before I had a good feel for CLI design using Click. `--config max_page_size 2000` is better than `--config max_page_size:2000`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/992/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 737476423, "node_id": "MDU6SXNzdWU3Mzc0NzY0MjM=", "number": 198, "title": "Support order by relevance against FTS4", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-11-06T05:36:31Z", "updated_at": "2020-11-06T18:30:44Z", "closed_at": "2020-11-06T18:30:44Z", "author_association": "OWNER", "pull_request": null, "body": "For #192 and #197 I've decided I want to be able to order by relevance in FTS4 as well as FTS5.\r\n\r\nThis means I need to port over my work on bm25() from https://github.com/simonw/sqlite-fts4 (since I don't want to add a full dependency).", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/198/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 733796942, "node_id": "MDU6SXNzdWU3MzM3OTY5NDI=", "number": 1075, "title": "PrefixedUrlString mechanism broke everything", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 6, "created_at": "2020-10-31T19:58:05Z", "updated_at": "2020-10-31T20:48:51Z", "closed_at": "2020-10-31T20:48:51Z", "author_association": "OWNER", "pull_request": null, "body": "Added in 7a67bc7a569509d65b3a8661e0ad2c65f0b09166 refs #1026. Lots of tests are failing now.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1075/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 732905360, "node_id": "MDU6SXNzdWU3MzI5MDUzNjA=", "number": 1067, "title": "Table actions menu on view pages, not on query pages", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 6, "created_at": "2020-10-30T05:56:39Z", "updated_at": "2020-10-31T17:51:31Z", "closed_at": "2020-10-31T17:40:14Z", "author_association": "OWNER", "pull_request": null, "body": "Follow-on from #1066.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1067/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 733303548, "node_id": "MDExOlB1bGxSZXF1ZXN0NTEzMTA2MDI2", "number": 1069, "title": "load_template() plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 6, "created_at": "2020-10-30T15:59:45Z", "updated_at": "2020-10-30T17:47:20Z", "closed_at": "2020-10-30T17:47:19Z", "author_association": "OWNER", "pull_request": "simonw/datasette/pulls/1069", "body": "Refs #1042", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1069/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 729096595, "node_id": "MDU6SXNzdWU3MjkwOTY1OTU=", "number": 1051, "title": "Better display of binary data on arbitrary query results page", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-10-25T19:38:06Z", "updated_at": "2020-10-29T22:12:16Z", "closed_at": "2020-10-29T22:01:39Z", "author_association": "OWNER", "pull_request": null, "body": "https://latest.datasette.io/fixtures?sql=select+rowid%2C+data+from+binary_data+order+by+rowid+limit+101\r\n\r\n\"fixtures__select_rowid__data_from_binary_data_order_by_rowid_limit_101_and_Switch_to__blob_render_extension_for_BLOB_downloads_\u00b7_Issue__1050_\u00b7_simonw_datasette\"\r\n\r\nProblem: if these were larger fields that HTML page could have multiple megabytes of Python binary string representations on it.\r\n\r\nIt should behave more like the regular table view does:\r\n\r\n\"fixtures__binary_data__2_rows\"\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1051/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 718723543, "node_id": "MDU6SXNzdWU3MTg3MjM1NDM=", "number": 1014, "title": "Add Link: pagination HTTP headers", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 6, "created_at": "2020-10-10T23:42:40Z", "updated_at": "2020-10-23T19:44:05Z", "closed_at": "2020-10-11T00:18:51Z", "author_association": "OWNER", "pull_request": null, "body": "Spun off from #782. These can go on all of the JSON endpoints that support pagination.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1014/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 721068929, "node_id": "MDU6SXNzdWU3MjEwNjg5Mjk=", "number": 1020, "title": "Method for datasette.client() to forward on authentication", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-10-14T01:47:49Z", "updated_at": "2020-10-19T22:45:01Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I stumbled into this while working on Dogsheep Beta: the requests it re-dispatched through `TableView` did not carry authentication cookies, and since this was against a private instance they were thus denied.\r\n\r\nhttps://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L223-L231\r\n\r\nThis made me think that `datasette.client.get()` (which Dogsheep Beta will start using shortly) could benefit from some kind of utility mechanism for passing through the cookies and general authenticated state from the current request.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1020/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 718938889, "node_id": "MDU6SXNzdWU3MTg5Mzg4ODk=", "number": 5, "title": "Figure out how to display images from tags inline in Datasette", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-10-11T22:17:03Z", "updated_at": "2020-10-16T20:16:28Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Relates to #1. Evernote XML looks like this:\r\n\r\n```xml\r\n\r\n\r\n
This note includes two images.
\r\n
\r\n The Python logo\r\n
\r\n
\r\n \r\n
\r\n
\r\n The Evernote logo\r\n
\r\n
\r\n \r\n
\r\n\r\n```\r\nThat hash is the md5 we use to store resources. It should be possible to turn these into embedded image tags, especially if done in conjunction with the https://github.com/simonw/datasette-media plugin.", "repo": {"value": 303218369, "label": "evernote-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 710506708, "node_id": "MDU6SXNzdWU3MTA1MDY3MDg=", "number": 978, "title": "Rendering glitch with column headings on mobile", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5971510, "label": "Datasette 0.50"}, "comments": 6, "created_at": "2020-09-28T19:04:45Z", "updated_at": "2020-10-08T23:54:40Z", "closed_at": "2020-09-28T22:43:01Z", "author_association": "OWNER", "pull_request": null, "body": "![6348FED5-1310-47DA-BF54-810E63C96C6F](https://user-images.githubusercontent.com/9599/94474607-bc368d00-0182-11eb-9bb1-9a0e455353f1.jpeg)\r\n\r\nhttps://latest-with-plugins.datasette.io/fixtures?sql=select%0D%0A++dateutil_parse%28%2210+october+2020+3pm%22%29%2C%0D%0A++dateutil_easter%28%222020%22%29%2C%0D%0A++dateutil_parse_fuzzy%28%22This+is+due+10+september%22%29%2C%0D%0A++dateutil_parse%28%221%2F2%2F2020%22%29%2C%0D%0A++dateutil_parse%28%222020-03-04%22%29%2C%0D%0A++dateutil_parse_dayfirst%28%222020-03-04%22%29%2C%0D%0A++dateutil_easter%282020%29", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/978/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 314506446, "node_id": "MDU6SXNzdWUzMTQ1MDY0NDY=", "number": 214, "title": "Ability for plugins to define extra JavaScript and CSS", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-04-16T05:29:34Z", "updated_at": "2020-09-30T20:36:11Z", "closed_at": "2018-04-18T03:13:03Z", "author_association": "OWNER", "pull_request": null, "body": "This can hook in to the existing `extra_css_urls` and `extra_js_urls` mechanism:\r\n\r\nhttps://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L304-L305\r\n\r\nThe plugins should be able to bundle their own assets though, so it will also have to integrate with the `/static/` static mounts mechanism somehow:\r\n\r\nhttps://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L1255-L1257\r\n\r\nRefs #14", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/214/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 691265198, "node_id": "MDU6SXNzdWU2OTEyNjUxOTg=", "number": 7, "title": "Mechanism for differentiating between \"by me\" and \"liked by me\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-09-02T17:44:37Z", "updated_at": "2020-09-02T21:06:28Z", "closed_at": "2020-09-02T21:06:28Z", "author_association": "MEMBER", "pull_request": null, "body": "Some of the content I'm indexing is by me - photos I've taken, tweets I wrote, commits, comments I posted.\r\n\r\nSome of it is stuff that I've \"liked\" or \"bookmarked\" in some way - favourited tweets, Pocket articles, starred GitHub repos.\r\n\r\nIt woud be useful to be able to differentiate between the two.", "repo": {"value": 197431109, "label": "dogsheep-beta"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 679779797, "node_id": "MDU6SXNzdWU2Nzk3Nzk3OTc=", "number": 939, "title": "extra_ plugin hooks should take the same arguments", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-08-16T16:04:54Z", "updated_at": "2020-08-16T18:25:05Z", "closed_at": "2020-08-16T16:50:29Z", "author_association": "OWNER", "pull_request": null, "body": "- [x] `extra_css_urls(template, database, table, datasette)`\r\n- [x] `extra_js_urls(template, database, table, datasette)`\r\n- [x] `extra_body_script(template, database, table, view_name, datasette)`\r\n- [x] `extra_template_vars(template, database, table, view_name, request, datasette)`\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/938#issuecomment-674544691_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/939/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 677326155, "node_id": "MDU6SXNzdWU2NzczMjYxNTU=", "number": 930, "title": "Datasette sdist is missing templates (hence broken when installing from Homebrew)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-08-12T02:20:16Z", "updated_at": "2020-08-12T03:30:59Z", "closed_at": "2020-08-12T03:30:59Z", "author_association": "OWNER", "pull_request": null, "body": "Pretty nasty bug this: I'm getting 500 errors for all pages that try to render a template after installing the newly released Datasette 0.47 - both from `pip install` and via Homebrew.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/930/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 644309017, "node_id": "MDU6SXNzdWU2NDQzMDkwMTc=", "number": 864, "title": "datasette.add_message() doesn't work inside plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5533512, "label": "Datasette 0.45"}, "comments": 6, "created_at": "2020-06-24T04:30:06Z", "updated_at": "2020-06-29T00:51:01Z", "closed_at": "2020-06-29T00:51:01Z", "author_association": "OWNER", "pull_request": null, "body": "Similar problem to #863 - calling `datasette.add_message()` in a view registered using the `register_routes()` plugin hook doesn't work, because the code that writes accumulated messages to the `ds_messages` signed cookie lives in the `BaseView` class here:\r\n\r\nhttps://github.com/simonw/datasette/blob/28bb1c51897f3956861755e345e18b8e0b1423ac/datasette/views/base.py#L94-L97", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/864/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 637342551, "node_id": "MDU6SXNzdWU2MzczNDI1NTE=", "number": 834, "title": "startup() plugin hook", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5533512, "label": "Datasette 0.45"}, "comments": 6, "created_at": "2020-06-11T21:48:14Z", "updated_at": "2020-06-28T19:38:50Z", "closed_at": "2020-06-13T17:56:12Z", "author_association": "OWNER", "pull_request": null, "body": "It might be useful to have an `startup` hook which gets passed the `datasette` object as soon as Datasette has finished initializing.\r\n\r\nMy initial use-case for this is configuration verification - checking that the `\"plugins\"` configuration block for this plugin contains valid details.\r\n\r\nI imagine there are plenty of other potential uses for this as well.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/834/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 348043884, "node_id": "MDU6SXNzdWUzNDgwNDM4ODQ=", "number": 357, "title": "Plugin hook for loading metadata.json", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-08-06T19:00:01Z", "updated_at": "2020-06-21T22:19:58Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "For https://github.com/simonw/russian-ira-facebook-ads-datasette/tree/af6d956995e14afd585c35a6a06bb01da32043ba I wrote a script to convert YAML to JSON because YAML is a better format for embedding multi-line HTML descriptions and canned SQL statements.\r\n\r\nExample yaml metadata file: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/af6d956995e14afd585c35a6a06bb01da32043ba/russian-ads-metadata.yaml\r\n\r\nIt would be useful if Datasette could be fed a YAML file directly:\r\n\r\n datasette -m metadata.yaml\r\n\r\nQuestion is... should this be a native feature (hence adding a YAML dependency) or should it be handled by a `datasette-metadata-yaml` plugin, using a new plugin hook for loading metadata? If so, what would other use-cases for that plugin hook be?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/357/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 529429214, "node_id": "MDU6SXNzdWU1Mjk0MjkyMTQ=", "number": 642, "title": "Provide a cookiecutter template for creating new plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2019-11-27T15:46:36Z", "updated_at": "2020-06-20T03:20:33Z", "closed_at": "2020-06-20T03:20:25Z", "author_association": "OWNER", "pull_request": null, "body": "See this conversation: https://twitter.com/psychemedia/status/1199707352540368896", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/642/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 638241779, "node_id": "MDU6SXNzdWU2MzgyNDE3Nzk=", "number": 846, "title": "\"Too many open files\" error running tests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-06-13T22:11:40Z", "updated_at": "2020-06-14T00:26:31Z", "closed_at": "2020-06-14T00:26:31Z", "author_association": "OWNER", "pull_request": null, "body": "I got this on my laptop:\r\n```pytest\r\n...\r\n/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/loaders.py:171: in get_source\r\n f = open_if_exists(filename)\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nfilename = '/Users/simon/Dropbox/Development/datasette/datasette/templates/400.html', mode = 'rb'\r\n\r\n def open_if_exists(filename, mode='rb'):\r\n \"\"\"Returns a file descriptor for the filename if that file exists,\r\n otherwise `None`.\r\n \"\"\"\r\n try:\r\n> return open(filename, mode)\r\nE OSError: [Errno 24] Too many open files: '/Users/simon/Dropbox/Development/datasette/datasette/templates/400.html'\r\n\r\n/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/utils.py:154: OSError\r\n```\r\nBased on the conversation in https://github.com/pytest-dev/pytest/issues/2970 I'm worried that my tests are opening too many files without closing them.\r\n\r\nIn particular... I call `sqlite3.connect(filepath)` a LOT - and I don't ever call `conn.close()` on those opened connections:\r\n\r\nhttps://github.com/simonw/datasette/blob/cf7a2bdb404734910ec07abc7571351a2d934828/datasette/database.py#L58-L60\r\n\r\nCould this be resulting in my tests eventually opening too many unclosed file handles? How could I confirm this?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/846/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 631932926, "node_id": "MDU6SXNzdWU2MzE5MzI5MjY=", "number": 801, "title": "allow_by_query setting for configuring permissions with a SQL statement", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2020-06-05T20:30:19Z", "updated_at": "2020-06-11T18:58:56Z", "closed_at": "2020-06-11T18:58:49Z", "author_association": "OWNER", "pull_request": null, "body": "> Idea: an `\"allow_sql\"` key with a SQL query that gets passed the actor JSON as `:actor` and can extract the relevant keys from it and return 1 or 0.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-639787304_\r\n\r\nSee also #800", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/801/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 636426530, "node_id": "MDU6SXNzdWU2MzY0MjY1MzA=", "number": 829, "title": "Ability to set ds_actor cookie such that it expires", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 6, "created_at": "2020-06-10T17:31:40Z", "updated_at": "2020-06-10T19:41:35Z", "closed_at": "2020-06-10T19:40:05Z", "author_association": "OWNER", "pull_request": null, "body": "I need this for `datasette-auth-github`: https://github.com/simonw/datasette-auth-github/issues/62#issuecomment-642152076", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/829/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 632673972, "node_id": "MDU6SXNzdWU2MzI2NzM5NzI=", "number": 804, "title": "python tests/fixtures.py command has a bug", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 6, "created_at": "2020-06-06T19:17:36Z", "updated_at": "2020-06-09T20:01:30Z", "closed_at": "2020-06-09T19:58:34Z", "author_association": "OWNER", "pull_request": null, "body": "This command is meant to write out `fixtures.db`, `metadata.json` and a plugins directory:\r\n```\r\n$ python tests/fixtures.py /tmp/fixtures.db /tmp/metadata.json /tmp/plugins/\r\nTest tables written to /tmp/fixtures.db\r\n- metadata written to /tmp/metadata.json\r\nTraceback (most recent call last):\r\n File \"tests/fixtures.py\", line 833, in \r\n (\"my_plugin.py\", PLUGIN1),\r\nNameError: name 'PLUGIN1' is not defined\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/804/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 635147716, "node_id": "MDU6SXNzdWU2MzUxNDc3MTY=", "number": 825, "title": "Way to enable a default=False permission for anonymous users", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 6, "created_at": "2020-06-09T06:26:27Z", "updated_at": "2020-06-09T17:19:19Z", "closed_at": "2020-06-09T17:01:10Z", "author_association": "OWNER", "pull_request": null, "body": "I'd like plugins to be able to ship with a default that says \"anonymous users cannot do this\", but allow site administrators to over-ride that such that anonymous users can use the feature after all.\r\n\r\nThis is tricky because right now the anonymous user doesn't have an actor dictionary at all, so there's no key to match to an allow block.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/825/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 634139848, "node_id": "MDU6SXNzdWU2MzQxMzk4NDg=", "number": 813, "title": "Mechanism for specifying allow_sql permission in metadata.json", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 6, "created_at": "2020-06-08T04:57:19Z", "updated_at": "2020-06-09T00:09:57Z", "closed_at": "2020-06-09T00:07:19Z", "author_association": "OWNER", "pull_request": null, "body": "Split from #811. It would be useful if finely-grained permissions configured in `metadata.json` could be used to specify if a user is allowed to execute arbitrary SQL queries.\r\n\r\nWe have a permission check call for this already: https://github.com/simonw/datasette/blob/9397d718345c4b35d2a5c55bfcbd1468876b5ab9/datasette/views/database.py#L159\r\n\r\nBut there's currently no way to implement this check without writing a plugin.\r\n\r\nI think a `\"allow_sql\": {...}` block at the database level in `metadata.json` (sibling to the current `\"allow\"` block for that database implemented in #811) would be a good option for this.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/813/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 585633142, "node_id": "MDU6SXNzdWU1ODU2MzMxNDI=", "number": 706, "title": "Documentation for the \"request\" object", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2020-03-22T02:55:50Z", "updated_at": "2020-05-30T13:20:00Z", "closed_at": "2020-05-27T22:31:22Z", "author_association": "OWNER", "pull_request": null, "body": "Since that object is passed to the `extra_template_vars` hooks AND the classes registered by `register_facet_classes` it should be part of the documented interface on https://datasette.readthedocs.io/en/stable/internals.html\r\n\r\nI could also start passing it to the `register_output_renderer` callback.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/706/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 612382643, "node_id": "MDU6SXNzdWU2MTIzODI2NDM=", "number": 758, "title": "Question: Access to immutable database-path", "user": {"value": 2181410, "label": "clausjuhl"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-05-05T07:01:18Z", "updated_at": "2020-05-28T08:23:27Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi Simon\r\n\r\nIs there anywhere in the app-context where one can access the hashed urlpath of the database? Currently it's included in the template-context (`databases[0][\"path\")` when rendering urls of the database (eg. `/db-44b06v9/cases`...), but where can I find the hashed url when rendering the index-page? I'm trying to avoid redirects. Thanks!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/758/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 613755043, "node_id": "MDU6SXNzdWU2MTM3NTUwNDM=", "number": 110, "title": "Support decimal.Decimal type", "user": {"value": 134771, "label": "dvhthomas"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-05-07T03:57:19Z", "updated_at": "2020-05-11T01:58:20Z", "closed_at": "2020-05-11T01:50:11Z", "author_association": "NONE", "pull_request": null, "body": "Decimal types in Postgres cause a failure in db.py data type selection\r\n---\r\nI have a Django app using a MoneyField, which uses a `numeric(14,0)` data type in Postgres (https://www.postgresql.org/docs/9.3/datatype-numeric.html). When attempting to export that table I get the following error:\r\n\r\n```bash\r\n$ db-to-sqlite --table isaweb_proposal \"postgres://connection\" test.db\r\n....\r\n column_type=COLUMN_TYPE_MAPPING[column_type],\r\nKeyError: \r\n```\r\n\r\nLooking at `sql_utils.db.py` at 292-ish it's clear that there is no matching type for what I assume SQLAlchemy interprets as Python decimal.Decimal.\r\n\r\nFrom the [SQLite docs](https://www.sqlite.org/datatype3.html#affinity_name_examples) it looks like DECIMAL in other DBs are considered numeric.\r\n\r\nI'm not quite sure if it's as simple as adding a data type to that list or if there are repercussions beyond it.\r\n\r\nThanks for a great tool!", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/110/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610408908, "node_id": "MDU6SXNzdWU2MTA0MDg5MDg=", "number": 34, "title": "Command for retrieving dependents for a repo", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-04-30T21:47:51Z", "updated_at": "2020-05-03T15:53:01Z", "closed_at": "2020-05-03T15:53:01Z", "author_association": "MEMBER", "pull_request": null, "body": "I really, really want to start grabbing this data: https://github.com/simonw/datasette/network/dependents", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 573583971, "node_id": "MDU6SXNzdWU1NzM1ODM5NzE=", "number": 689, "title": "\"Templates considered\" comment broken in >=0.35", "user": {"value": 35075, "label": "chrishas35"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-03-01T17:31:21Z", "updated_at": "2020-04-05T19:39:44Z", "closed_at": "2020-04-05T19:39:44Z", "author_association": "NONE", "pull_request": null, "body": "Noticed that the \"Templates Considered\" comment is missing in 0.37. Believe I traced it back to #664 as you can see it in https://v0-34.datasette.io/ but not https://v0-35.datasette.io/. Looking at the template context debug between the two you can see what is missing from 0.35 vs. 0.34:\r\n\r\n```diff\r\n< \"datasette_version\": \"0.34\",\r\n< \"app_css_hash\": \"ffa51a\",\r\n< \"select_templates\": [\r\n< \"*index.html\"\r\n< ],\r\n< \"zip\": \"\",\r\n< \"body_scripts\": [],\r\n< \"extra_css_urls\": \"\",\r\n< \"extra_js_urls\": \"\",\r\n< \"format_bytes\": \"\",\r\n< \"database_url\": \">\",\r\n< \"database_color\": \">\"\r\n---\r\n> \"datasette_version\": \"0.35\",\r\n> \"database_url\": \">\",\r\n> \"database_color\": \">\"\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/689/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 592829135, "node_id": "MDU6SXNzdWU1OTI4MjkxMzU=", "number": 713, "title": "Support YAML in metadata - metadata.yaml", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-04-02T18:10:05Z", "updated_at": "2020-04-02T19:36:17Z", "closed_at": "2020-04-02T19:30:55Z", "author_association": "OWNER", "pull_request": null, "body": "I was originally going to do this with a plugin - see #357 - but the more I work with `metadata.json` the more I want it to just accept YAML as an optional alternative to JSON.\r\n\r\nThe best example why is still this one: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/master/russian-ads-metadata.yaml\r\n\r\nYAML is just SO much better than JSON for multi-line strings - in particular HTML and SQL, both of which are common in `metadata.json` files.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/713/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 545407916, "node_id": "MDU6SXNzdWU1NDU0MDc5MTY=", "number": 73, "title": "upsert_all() throws issue when upserting to empty table", "user": {"value": 82988, "label": "psychemedia"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-01-05T11:58:57Z", "updated_at": "2020-01-31T14:21:09Z", "closed_at": "2020-01-05T17:20:18Z", "author_association": "NONE", "pull_request": null, "body": "If I try to add a list of `dict`s to an empty table using `upsert_all`, I get an error:\r\n\r\n```python\r\nimport sqlite3\r\nfrom sqlite_utils import Database\r\nimport pandas as pd\r\n\r\nconx = sqlite3.connect(':memory')\r\ncx = conx.cursor()\r\ncx.executescript('CREATE TABLE \"test\" (\"Col1\" TEXT);')\r\n\r\nq=\"SELECT * FROM test;\"\r\npd.read_sql(q, conx) #shows empty table\r\n\r\ndb = Database(conx)\r\ndb['test'].upsert_all([{'Col1':'a'},{'Col1':'b'}])\r\n\r\n---------------------------------------------------------------------------\r\nTypeError Traceback (most recent call last)\r\n in \r\n 1 db = Database(conx)\r\n----> 2 db['test'].upsert_all([{'Col1':'a'},{'Col1':'b'}])\r\n\r\n/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in upsert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, extracts)\r\n 1157 alter=alter,\r\n 1158 extracts=extracts,\r\n-> 1159 upsert=True,\r\n 1160 )\r\n 1161 \r\n\r\n/usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, upsert)\r\n 1040 sql = \"INSERT OR IGNORE INTO [{table}]({pks}) VALUES({pk_placeholders});\".format(\r\n 1041 table=self.name,\r\n-> 1042 pks=\", \".join([\"[{}]\".format(p) for p in pks]),\r\n 1043 pk_placeholders=\", \".join([\"?\" for p in pks]),\r\n 1044 )\r\n\r\nTypeError: 'NoneType' object is not iterable\r\n\r\n```\r\n\r\nA hacky workaround in use is:\r\n\r\n```python\r\ntry:\r\n db['test'].upsert_all([{'Col1':'a'},{'Col1':'b'}])\r\nexcept:\r\n db['test'].insert_all([{'Col1':'a'},{'Col1':'b'}])\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/73/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 527670799, "node_id": "MDU6SXNzdWU1Mjc2NzA3OTk=", "number": 639, "title": "updating metadata.json without recreating the app", "user": {"value": 172847, "label": "pkoppstein"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-11-24T09:19:53Z", "updated_at": "2019-11-30T06:08:50Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I've sucessfully \"uploaded\" an SQLite database (with a metadata.json file) to heroku using:\r\n\r\n $ datasette publish heroku so-sales.db -m metadata.json -n so-sales\r\n\r\nThe question is: how can I modify the (small) metadata.json file without having to upload the (large) SQLite database.\r\n\r\nThe directions on heroku indicate I should run:\r\n\r\n heroku git:clone -a so-sales\r\n\r\nBut this just results in an empty directory with a warning:\r\nwarning: You appear to have cloned an empty repository.\r\n\r\nI've been able to \"clone\" the heroku \"app\" using the command:\r\n\r\n $ heroku slugs:download -a so-sales\r\n\r\nbut this is not a git repository....\r\n\r\nIdeally, it seems to me, there'd be an option of the `datasette` CLI to allow a file\r\nto be updated, or there'd be some way to create a local git \"clone\" of the app\r\nso that the heroku instructions for \"Deploying with git\" would apply.\r\n\r\n(p.s. I ran `datasette publish heroku -m metadata.json -n so-sales`\r\nin the hope that that would not cause the .db file to be wiped, but of course\r\nit was.)\r\n\r\n(p.p.s. Thanks for Datasette!)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/639/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 513008936, "node_id": "MDU6SXNzdWU1MTMwMDg5MzY=", "number": 608, "title": "Improve UI of \"datasette publish cloudrun\" to reduce chances of accidentally over-writing a service", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-10-27T19:21:28Z", "updated_at": "2019-11-08T02:51:36Z", "closed_at": "2019-11-08T02:48:46Z", "author_association": "OWNER", "pull_request": null, "body": "The concept of a \"service\" in Cloud Run is crucial: if you deploy to the same service, you will over-write what you deployed there last!\r\n\r\nAs such, I'd like to make service a required positional argument for `publish cloudrun`:\r\n\r\n datasette publish cloudrun my-service one.db two.db three.db\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/608/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 512996469, "node_id": "MDU6SXNzdWU1MTI5OTY0Njk=", "number": 607, "title": "Ways to improve fuzzy search speed on larger data sets?", "user": {"value": 8431341, "label": "zeluspudding"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-10-27T17:31:37Z", "updated_at": "2019-11-07T03:38:10Z", "closed_at": "2019-11-07T03:38:10Z", "author_association": "NONE", "pull_request": null, "body": "I have an sqlite table with 16 million rows in it. Having read @simonw article \"[Fast Autocomplete Search for Your Website](https://24ways.org/2018/fast-autocomplete-search-for-your-website/)\" I was curious to try datasette to see what kind of query performance I could get out of it. In truth I don't need to do full text search since all I would like to do is give my users a way to search for the names of investors such as \"Warren Buffet\", or \"Tim Cook\" (who's names are in a single column).\r\n\r\nOn the first search, Datasette takes over 20 seconds to return all records associated with `elon musk`:\r\n\r\n> ![image](https://user-images.githubusercontent.com/8431341/67638889-a86e1100-f8b7-11e9-9f7e-a9d13a42e988.png)\r\n\r\n> ![image](https://user-images.githubusercontent.com/8431341/67638825-ed457800-f8b6-11e9-94d1-b44f1a40ee8c.png)\r\n\r\nIf I rerun the same search, it then takes almost 9 seconds:\r\n> ![image](https://user-images.githubusercontent.com/8431341/67638908-e4a17180-f8b7-11e9-9d00-748c80ef1f21.png)\r\n\r\nThat's far to slow to implement an autocomplete feature. I could reduce the latency by making a special table of only unique investor names, thereby reducing the search space to less than a million rows (then I'd need to implement a way to add only new investor names to the table as I received new data.. about 4,000 rows a day). If I did that, I'm still concerned the new table wouldn't be lean enough to lookup investor names quickly. Plus, even if I can implement the autocomplete feature, I would still finally have to lookup records for that investors which would take between 8 - 20 seconds. \r\n\r\nAre there any tricks for speeding this up?\r\n\r\nHere's my hardware:\r\n> ![image](https://user-images.githubusercontent.com/8431341/67638861-55945980-f8b7-11e9-96a8-ca76c7c68c5d.png)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/607/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 488833975, "node_id": "MDU6SXNzdWU0ODg4MzM5NzU=", "number": 3, "title": "Command for running a search and saving tweets for that search", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-09-03T21:29:56Z", "updated_at": "2019-11-04T05:31:56Z", "closed_at": "2019-11-04T05:31:16Z", "author_association": "MEMBER", "pull_request": null, "body": " $ twitter-to-sqlite search dogsheep", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 459621683, "node_id": "MDU6SXNzdWU0NTk2MjE2ODM=", "number": 521, "title": "Easier way of creating custom row templates", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-06-23T21:49:27Z", "updated_at": "2019-07-03T03:23:56Z", "closed_at": "2019-07-03T03:23:56Z", "author_association": "OWNER", "pull_request": null, "body": "I was messing around with a custom `_rows_and_columns.html` template and ended up with this:\r\n```html\r\n{% for row in display_rows %}\r\n
\r\n
\r\n {% for cell in row %}\r\n {% if cell.column == \"First_Name\" %}\r\n

{{ cell.value }}\r\n {% elif cell.column == \"Last_Name\" %}\r\n {{ cell.value }}

\r\n {% elif cell.column == \"Short_Description\" %}\r\n

{{ cell.column }}: {{ cell.value }}

\r\n

\r\n {% else %}\r\n {{ cell.column }}: {{ cell.value }}   \r\n {% endif %}\r\n {% endfor %}\r\n

\r\n{% endfor %}\r\n```\r\nThis is nasty. I'd like to be able to do something like this instead:\r\n```\r\n{% for row in display_rows %}\r\n

{{ row[\"First_Name\"] }} {{ row[\"Last_Name\"] }}

\r\n ...\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/521/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 449818897, "node_id": "MDU6SXNzdWU0NDk4MTg4OTc=", "number": 24, "title": "Additional Column Constraints?", "user": {"value": 98555, "label": "IgnoredAmbience"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-05-29T13:47:03Z", "updated_at": "2019-06-13T06:47:17Z", "closed_at": "2019-06-13T06:30:26Z", "author_association": "NONE", "pull_request": null, "body": "I'm looking to import data from XML with a pre-defined schema that maps fairly closely to a relational database.\r\nIn particular, it has explicit annotations for when fields are required, optional, or when a default value should be inferred.\r\n\r\nWould there be value in adding the ability to define `NOT NULL` and `DEFAULT` column constraints to sqlite-utils?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/24/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 349827640, "node_id": "MDU6SXNzdWUzNDk4Mjc2NDA=", "number": 359, "title": "Faceted browse against a JSON list of tags", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-08-12T17:01:14Z", "updated_at": "2019-05-29T21:39:12Z", "closed_at": "2019-05-03T00:21:44Z", "author_association": "OWNER", "pull_request": null, "body": "If a table has a `[\"foo\", \"bar\", \"baz\"]` JSON column allow that to be faceted against.\r\n\r\n- [x] Support `?column__arraycontains=x` filter queries\r\n- [x] Support `?_facet_array=column` faceting", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/359/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 322787470, "node_id": "MDU6SXNzdWUzMjI3ODc0NzA=", "number": 259, "title": "inspect() should detect many-to-many relationships", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-05-14T12:03:58Z", "updated_at": "2019-05-23T03:55:37Z", "closed_at": "2019-05-23T03:55:37Z", "author_association": "OWNER", "pull_request": null, "body": "Relates to #255 - in particular supporting facets across M2M relationships.\r\n\r\nIt should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables.\r\n\r\nWhen rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side.\r\n\r\nSince SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/259/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 310533258, "node_id": "MDU6SXNzdWUzMTA1MzMyNTg=", "number": 191, "title": "Figure out how to bundle a more up-to-date SQLite", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-04-02T16:33:25Z", "updated_at": "2018-07-10T17:46:13Z", "closed_at": "2018-07-10T17:46:13Z", "author_association": "OWNER", "pull_request": null, "body": "The version of SQLite that ships with Python 3 is a bit limited - it doesn't support row values for example https://www.sqlite.org/rowvalue.html\r\n\r\nFigure out how to bundle a more recent SQLite engine with datasette. We need to figure out two cases:\r\n\r\n* Bundling a recent version in a Dockerfile build. I expect this to be quite easy.\r\n* Making a more recent version available to people hacking around in Mac OS X. I have no idea how to start on this.\r\n\r\nI want it working on Mac OS X too because I don't want to force Docker as a dependency for anyone who just want to hack around with Datasette a little and run the test suite.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/191/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 333086005, "node_id": "MDU6SXNzdWUzMzMwODYwMDU=", "number": 313, "title": "Deploy demo of Datasette on every commit that passes tests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-06-17T19:19:12Z", "updated_at": "2018-06-17T21:52:58Z", "closed_at": "2018-06-17T21:52:58Z", "author_association": "OWNER", "pull_request": null, "body": "We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/313/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 314455877, "node_id": "MDExOlB1bGxSZXF1ZXN0MTgxNzIzMzAz", "number": 209, "title": " Don't duplicate simple primary keys in the link column", "user": {"value": 45057, "label": "russss"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-04-15T21:56:15Z", "updated_at": "2018-04-18T08:40:37Z", "closed_at": "2018-04-18T01:13:04Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/datasette/pulls/209", "body": "When there's a simple (single-column) primary key, it looks weird to duplicate it in the link column.\r\n\r\nThis change removes the second PK column and treats the link column as if it were the PK column from a header/sorting perspective. \r\n\r\nThis might make it a bit more difficult to tell what the link for the row is, I'm not sure yet. I feel like the alternative is to change the link column to just have the text \"view\" or something, instead of repeating the PK. (I doubt it makes much more sense with compound PKs.)\r\n\r\nBonus change in this PR: fix urlencoding of links in the displayed HTML.\r\n\r\nBefore:\r\n![image](https://user-images.githubusercontent.com/45057/38783830-e2ababb4-40ff-11e8-97fb-25e286a8c920.png)\r\n\r\nAfter:\r\n![image](https://user-images.githubusercontent.com/45057/38783835-ebf6b48e-40ff-11e8-8c47-6a864cf21ccc.png)", "repo": {"value": 107914493, "label": "datasette"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/209/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 314471743, "node_id": "MDU6SXNzdWUzMTQ0NzE3NDM=", "number": 211, "title": "Load plugins from a `--plugins-dir=plugins/` directory", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2018-04-16T01:17:43Z", "updated_at": "2018-04-16T05:22:02Z", "closed_at": "2018-04-16T05:22:02Z", "author_association": "OWNER", "pull_request": null, "body": "In #14 and 33c7c53ff87c2 I've added working support for setuptools entry_points plugins. These can be installed from PyPI using `pip install ...`.\r\n\r\nI imagine some projects will benefit from being able to add plugins without first publishing them to PyPI. Datasette already supports [loading custom templates](http://datasette.readthedocs.io/en/latest/custom_templates.html#custom-templates) like so:\r\n\r\n datasette serve --template-dir=mytemplates/ mydb.db\r\n\r\nI propose an additional option, `--plugins-dir=` which specifies a directory full of `blah.py` files which will be loaded into Datasette when the application server starts.\r\n\r\n datasette serve --plugins-dir=myplugins/ mydb.db\r\n\r\nThis will also need to be supported by `datasette publish` as those Python files should be copied up as part of the deployment.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/211/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 280315352, "node_id": "MDU6SXNzdWUyODAzMTUzNTI=", "number": 167, "title": "Nasty bug: last column not being correctly displayed", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2949431, "label": "Custom templates edition"}, "comments": 6, "created_at": "2017-12-07T23:23:46Z", "updated_at": "2017-12-10T01:00:21Z", "closed_at": "2017-12-10T01:00:20Z", "author_association": "OWNER", "pull_request": null, "body": "e.g. https://datasette-bwnojrhmmg.now.sh/dk3-bde9a9a/dk?source__contains=http\r\n\r\n![2017-12-07 at 3 22 pm](https://user-images.githubusercontent.com/9599/33743613-7ee97be0-db62-11e7-8e81-9b9ec69d93f0.png)\r\n\r\nThe JSON output shows that the column is there, but is being displayed incorrectly:\r\n\r\nhttps://datasette-bwnojrhmmg.now.sh/dk3-bde9a9a/dk.jsono?source__contains=http\r\n\r\n![2017-12-07 at 3 23 pm](https://user-images.githubusercontent.com/9599/33743645-9489b302-db62-11e7-898b-72e812e8855d.png)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/167/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}