github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
633578769 | MDU6SXNzdWU2MzM1Nzg3Njk= | 811 | Support "allow" block on root, databases and tables, not just queries | 9599 | closed | 0 | 5512395 | 16 | 2020-06-07T17:01:09Z | 2020-06-08T19:34:00Z | 2020-06-08T19:32:36Z | OWNER | No reason not to expand the "allow" mechanism [described here](https://github.com/simonw/datasette/blob/86dec9e8fffd6c4efec928ae9b5713748dec7e74/docs/authentication.rst#permissions-for-canned-queries) to the root of `metadata.json` plus to databases and tables. Refs #810 and #800. ```json { "databases": { "mydatabase": { "allow": { "id": ["root"] } } } } ``` TODO: - [x] Instance level - [x] Database level - [x] Table level - [x] Query level - [x] Affects list of queries - [x] Affects list of tables on database page - [x] Affects truncated list of tables on index page - [x] Affects list of SQL views on database page - [x] Affects list of databases on index page - [x] Show π in header on index page for private instances - [x] Show π in header on private database page - [x] Show π in header on private table page - [x] Show π in header on private query page - [x] Move `assert_permissions_checked()` calls from `test_html.py` to `test_permissions.py` - [x] Update documentation | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/811/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
628499086 | MDU6SXNzdWU2Mjg0OTkwODY= | 790 | "flash messages" mechanism | 9599 | closed | 0 | 5512395 | 20 | 2020-06-01T14:55:44Z | 2020-06-08T19:33:59Z | 2020-06-02T21:14:03Z | OWNER | > Passing `?_success` like this isn't necessarily the best approach. Potential improvements include: > > - Signing this message so it can't be tampered with (I could generate a signing secret on startup) > - Using a cookie with a temporary flash message in it instead > - Using HTML5 history API to remove the `?_success=` from the URL bar when the user lands on the page > > If I add an option to redirect the user to another page after success I may need a mechanism to show a flash message on that page as well, in which case I'll need a general flash message solution that works for any page. _Originally posted by @simonw in https://github.com/simonw/datasette/pull/703_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/790/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
634783573 | MDU6SXNzdWU2MzQ3ODM1NzM= | 816 | Come up with a new example for extra_template_vars plugin | 9599 | closed | 0 | 5512395 | 2 | 2020-06-08T16:57:59Z | 2020-06-08T19:06:44Z | 2020-06-08T19:06:11Z | OWNER | This example is obsolete, it's from a time before `request.actor` and authentication as a built-in concept (#699): https://github.com/simonw/datasette/blob/0c064c5fe220b7b3d8dcf85b02b4e60452c47232/docs/plugins.rst#L696-L700 https://github.com/simonw/datasette/blob/0c064c5fe220b7b3d8dcf85b02b4e60452c47232/docs/plugins.rst#extra_template_varstemplate-database-table-view_name-request-datasette | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/816/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
634844634 | MDU6SXNzdWU2MzQ4NDQ2MzQ= | 817 | Drop resource_type from permission_allowed system | 9599 | closed | 0 | 1 | 2020-06-08T18:41:37Z | 2020-06-08T19:00:12Z | 2020-06-08T19:00:12Z | OWNER | Current signature: permission_allowed(datasette, actor, action, resource_type, resource_identifier) It turns out the `resource_type` is always the same thing for any given action, so it's not actually useful. I'm going to drop it. New signature will be: permission_allowed(datasette, actor, action, resource) Refs #811. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/817/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
396215043 | MDU6SXNzdWUzOTYyMTUwNDM= | 395 | Find a cleaner pattern for fixtures with arguments | 9599 | closed | 0 | 1 | 2019-01-06T00:31:22Z | 2020-06-07T21:23:22Z | 2020-06-07T21:23:22Z | OWNER | A lot of Datasette tests look like this: https://github.com/simonw/datasette/blob/b65d97792a53f78cb14b226231063209d22c4602/tests/test_api.py#L438-L444 The loop here isn't actually expected to loop - it's there because the `make_app_client` function yields a value and then cleans it up afterwards. This pattern works, but it is a little confusing. It would be nice to replace it with something less strange looking. The answer may be to switch to the "factories as fixtures" pattern described here: https://docs.pytest.org/en/latest/fixture.html#factories-as-fixtures In particular some variant of this example: ``` @pytest.fixture def make_customer_record(): created_records = [] def _make_customer_record(name): record = models.Customer(name=name, orders=[]) created_records.append(record) return record yield _make_customer_record for record in created_records: record.destroy() def test_customer_records(make_customer_record): customer_1 = make_customer_record("Lisa") customer_2 = make_customer_record("Mike") customer_3 = make_customer_record("Meredith") ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/395/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
633066114 | MDU6SXNzdWU2MzMwNjYxMTQ= | 810 | Refactor permission check for canned query | 9599 | closed | 0 | 5512395 | 1 | 2020-06-07T05:33:05Z | 2020-06-07T17:03:15Z | 2020-06-07T17:03:15Z | OWNER | This code here (TODO is follow-on from #808). https://github.com/simonw/datasette/blob/86dec9e8fffd6c4efec928ae9b5713748dec7e74/datasette/views/database.py#L133-L142 I can improve this with extra code in https://github.com/simonw/datasette/blob/86dec9e8fffd6c4efec928ae9b5713748dec7e74/datasette/default_permissions.py | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/810/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
631931408 | MDU6SXNzdWU2MzE5MzE0MDg= | 800 | Canned query permissions mechanism | 9599 | closed | 0 | 5512395 | 14 | 2020-06-05T20:28:21Z | 2020-06-07T16:22:53Z | 2020-06-07T16:22:53Z | OWNER | > Idea: default is anyone can execute a query. > > Or you can specify the following: > > ```json > > { > "databases": { > "my-database": { > "queries": { > "add_twitter_handle": { > "sql": "insert into twitter_handles (username) values (:username)", > "write": true, > "allow": { > "id": ["simon"], > "role": ["staff"] > } > } > } > } > } > } > ``` > These get matched against the actor JSON. If any of the fields in any of the keys of `"allow"` match a key on the actor, the query is allowed. > > `"id": "*"` matches any actor with an `id` key. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-639784651_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/800/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
632918799 | MDU6SXNzdWU2MzI5MTg3OTk= | 808 | Permission check for every view in Datasette (plus docs) | 9599 | closed | 0 | 5512395 | 2 | 2020-06-07T01:59:23Z | 2020-06-07T05:30:49Z | 2020-06-07T05:30:49Z | OWNER | Every view in Datasette should perform a permission check to see if the current user/actor is allowed to view that page. This permission check will default to allowed, but having this check will allow plugins to lock down access selectively or even to everything in a Datasette instance. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/808/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
582517965 | MDU6SXNzdWU1ODI1MTc5NjU= | 698 | Ability for a canned query to write to the database | 9599 | closed | 0 | 5512395 | 26 | 2020-03-16T18:31:59Z | 2020-06-06T19:43:49Z | 2020-06-06T19:43:48Z | OWNER | Canned queries are currently read-only: https://datasette.readthedocs.io/en/0.38/sql_queries.html#canned-queries Add a `"write": true` option to their definition in `metadata.json` which turns them into queries that are submitted via POST and send their queries to the write queue. Then they can be used as a really quick way to define a writable interface and JSON API! | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/698/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
582526961 | MDU6SXNzdWU1ODI1MjY5NjE= | 699 | Authentication (and permissions) as a core concept | 9599 | closed | 0 | 5512395 | 40 | 2020-03-16T18:48:00Z | 2020-06-06T19:42:11Z | 2020-06-06T19:42:11Z | OWNER | Right now Datasette authentication is provided exclusively by plugins: * https://github.com/simonw/datasette-auth-github * https://github.com/simonw/datasette-auth-existing-cookies This is an all-or-nothing approach: either your Datasette instance requires authentication at the top level or it does not. But... as I build new plugins like https://github.com/simonw/datasette-configure-fts and https://github.com/simonw/datasette-edit-tables I increasingly have individual features which should be reserved for logged-in users while still wanting other parts of Datasette to be open to all. This is too much for plugins to own independently of Datasette core. Datasette needs to ship a single "user is authenticated" concept (independent of how users actually sign in) so that different plugins can integrate with it. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/699/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
632645865 | MDExOlB1bGxSZXF1ZXN0NDI5MzY2NjQx | 803 | Canned query permissions | 9599 | closed | 0 | 0 | 2020-06-06T18:20:00Z | 2020-06-06T19:40:21Z | 2020-06-06T19:40:20Z | OWNER | simonw/datasette/pulls/803 | Refs #800. Closes #786 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/803/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
628087971 | MDU6SXNzdWU2MjgwODc5NzE= | 786 | Documentation page describing Datasette's authentication system | 9599 | closed | 0 | 5512395 | 2 | 2020-06-01T01:10:06Z | 2020-06-06T19:40:20Z | 2020-06-06T19:40:20Z | OWNER | _Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636562999_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/786/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
629524205 | MDU6SXNzdWU2Mjk1MjQyMDU= | 793 | CSRF protection for /-/messages tool and writable canned queries | 9599 | closed | 0 | 5512395 | 3 | 2020-06-02T21:22:21Z | 2020-06-06T00:43:41Z | 2020-06-05T19:05:59Z | OWNER | > The `/-/messages` debug tool will need CSRF protection or people will be able to add messages using a hidden form on another website. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/790#issuecomment-637790860_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/793/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
631300342 | MDExOlB1bGxSZXF1ZXN0NDI4MjEyNDIx | 798 | CSRF protection | 9599 | closed | 0 | 5512395 | 5 | 2020-06-05T04:22:35Z | 2020-06-06T00:43:41Z | 2020-06-05T19:05:58Z | OWNER | simonw/datasette/pulls/798 | Refs #793 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/798/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
628025100 | MDU6SXNzdWU2MjgwMjUxMDA= | 785 | Datasette secret mechanism - initially for signed cookies | 9599 | closed | 0 | 5512395 | 11 | 2020-05-31T19:14:52Z | 2020-06-06T00:43:40Z | 2020-06-01T00:18:40Z | OWNER | See comment in https://github.com/simonw/datasette/issues/784#issuecomment-636514974 Datasette needs to be able to set signed cookies - which means it needs a mechanism for safely handling a signing secret. Since Datasette is a long-running process the default behaviour here can be to create a random secret on startup. This means that if the server restarts any signed cookies will be invalidated. If the user wants a persistent secret they'll have to generate it themselves - maybe by setting an environment variable? | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/785/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
628121234 | MDU6SXNzdWU2MjgxMjEyMzQ= | 788 | /-/permissions debugging tool | 9599 | closed | 0 | 5512395 | 2 | 2020-06-01T03:13:47Z | 2020-06-06T00:43:40Z | 2020-06-01T05:01:01Z | OWNER | > Debugging tool idea: `/-/permissions` page which shows you the actor and lets you type in the strings for `action`, `resource_type` and `resource_identifier` - then shows you EVERY plugin hook that would have executed and what it would have said, plus when the chain would have terminated. > > Bonus: if you're logged in as the `root` user (or a user that matches some kind of permission check, maybe a check for `permissions_debug`) you get to see a rolling log of the last 30 permission checks and what the results were across the whole of Datasette. This should make figuring out permissions policies a whole lot easier. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636576603_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/788/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
632056825 | MDU6SXNzdWU2MzIwNTY4MjU= | 802 | "datasette plugins" command is broken | 9599 | closed | 0 | 1 | 2020-06-05T23:33:01Z | 2020-06-05T23:46:43Z | 2020-06-05T23:46:43Z | OWNER | I broke it in https://github.com/simonw/datasette/commit/a7137dfe069e5fceca56f78631baebd4a6a19967 - and it turns out there was no test coverage so I didn't realize it was broken. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/802/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
631789422 | MDU6SXNzdWU2MzE3ODk0MjI= | 799 | TestResponse needs to handle multiple set-cookie headers | 9599 | closed | 0 | 2 | 2020-06-05T17:39:52Z | 2020-06-05T18:34:10Z | 2020-06-05T18:34:10Z | OWNER | Seeing this test failure on #798: ``` _______________________ test_auth_token _______________________ app_client = <tests.fixtures.TestClient object at 0x11285c910> def test_auth_token(app_client): "The /-/auth-token endpoint sets the correct cookie" assert app_client.ds._root_token is not None path = "/-/auth-token?token={}".format(app_client.ds._root_token) response = app_client.get(path, allow_redirects=False,) assert 302 == response.status assert "/" == response.headers["Location"] > assert {"id": "root"} == app_client.ds.unsign(response.cookies["ds_actor"], "actor") E KeyError: 'ds_actor' datasette/tests/test_auth.py:12: KeyError ``` It looks like that's happening because the ASGI middleware is adding another set-cookie header - but those two set-cookie headers are combined into one when the TestResponse is constructed: https://github.com/simonw/datasette/blob/0c064c5fe220b7b3d8dcf85b02b4e60452c47232/tests/fixtures.py#L113-L127 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/799/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
570301333 | MDU6SXNzdWU1NzAzMDEzMzM= | 684 | Add documentation on Database introspection methods to internals.rst | 9599 | closed | 0 | 3268330 | 4 | 2020-02-25T04:20:24Z | 2020-06-04T18:56:15Z | 2020-05-30T18:40:39Z | OWNER | `internals.rst` will be landing as part of #683 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/684/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
275082158 | MDU6SXNzdWUyNzUwODIxNTg= | 119 | Build an "export this data to google sheets" plugin | 9599 | closed | 0 | 1 | 2017-11-18T14:14:51Z | 2020-06-04T18:46:40Z | 2020-06-04T18:46:39Z | OWNER | Inspired by https://github.com/kren1/tosheets It should be a plug-in because I'd like to keep all interactions with proprietary / non-open-source software encapsulated in plugins rather than shipped as part of core. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/119/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
629595228 | MDExOlB1bGxSZXF1ZXN0NDI2ODkxNDcx | 796 | New WIP writable canned queries | 9599 | closed | 0 | 3268330 | 9 | 2020-06-03T00:08:00Z | 2020-06-03T15:16:52Z | 2020-06-03T15:16:50Z | OWNER | simonw/datasette/pulls/796 | Refs #698. Replaces #703 Still todo: - [x] Unit tests - ~~Figure out `.json` mode~~ - [x] Flash message solution - ~~CSRF protection~~ - [x] Better error message display on errors - [x] Documentation - ~~Maybe widgets?~~ I'll do these later | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/796/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
585597133 | MDExOlB1bGxSZXF1ZXN0MzkxOTI0NTA5 | 703 | WIP implementation of writable canned queries | 9599 | closed | 0 | 3 | 2020-03-21T22:23:51Z | 2020-06-03T00:08:14Z | 2020-06-02T23:57:35Z | OWNER | simonw/datasette/pulls/703 | Refs #698. | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/703/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1 | |||||
629535669 | MDU6SXNzdWU2Mjk1MzU2Njk= | 794 | Show hooks implemented by each plugin on /-/plugins | 9599 | closed | 0 | 3268330 | 2 | 2020-06-02T21:44:38Z | 2020-06-02T22:30:17Z | 2020-06-02T21:50:10Z | OWNER | e.g. ```json { "name": "qs_actor.py", "static": false, "templates": false, "version": null, "hooks": [ "actor_from_request" ] } ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/794/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
629459637 | MDU6SXNzdWU2Mjk0NTk2Mzc= | 792 | Replace response.body.decode("utf8") with response.text in tests | 9599 | closed | 0 | 0 | 2020-06-02T19:32:24Z | 2020-06-02T21:29:58Z | 2020-06-02T21:29:58Z | OWNER | Make use of the `response.text` property to clean up the tests a tiny bit: https://github.com/simonw/datasette/blob/57cf5139c552cb7feab9947daa949ca434cc0a66/tests/fixtures.py#L26-L38 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/792/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
628156527 | MDU6SXNzdWU2MjgxNTY1Mjc= | 789 | Mechanism for enabling pluggy tracing | 9599 | open | 0 | 2 | 2020-06-01T05:10:14Z | 2020-06-01T05:11:03Z | OWNER | Could be useful for debugging plugins: https://pluggy.readthedocs.io/en/latest/#call-tracing I tried this out by adding these two lines in `plugins.py`: ```python pm = pluggy.PluginManager("datasette") pm.add_hookspecs(hookspecs) # Added these: pm.trace.root.setwriter(print) pm.enable_tracing() ``` Output looked something like this: ``` INFO: 127.0.0.1:52724 - "GET /-/-/static/app.css HTTP/1.1" 404 Not Found actor_from_request [hook] datasette: <datasette.app.Datasette object at 0x106277ad0> request: <datasette.utils.asgi.Request object at 0x106550a50> finish actor_from_request --> [] [hook] extra_body_script [hook] template: show_json.html database: None table: None view_name: json_data datasette: <datasette.app.Datasette object at 0x106277ad0> finish extra_body_script --> [] [hook] extra_template_vars [hook] template: show_json.html database: None table: None view_name: json_data request: <datasette.utils.asgi.Request object at 0x1065504d0> datasette: <datasette.app.Datasette object at 0x106277ad0> finish extra_template_vars --> [] [hook] extra_css_urls [hook] template: show_json.html database: None table: None datasette: <datasette.app.Datasette object at 0x106277ad0> finish extra_css_urls --> [] [hook] extra_js_urls [hook] template: show_json.html database: None table: None datasette: <datasette.app.Datasette object at 0x106277ad0> finish extra_js_urls --> [] [hook] INFO: 127.0.0.1:52724 - "GET /-/actor HTTP/1.1" 200 OK actor_from_request [hook] datasette: <datasette.app.Datasette object at 0x106277ad0> request: <datasette.utils.asgi.Request object at 0x1065500d0> finish actor_from_request --> [] [hook] ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/789/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
627836898 | MDExOlB1bGxSZXF1ZXN0NDI1NTMxMjA1 | 783 | Authentication: plugin hooks plus default --root auth mechanism | 9599 | closed | 0 | 0 | 2020-05-30T22:25:47Z | 2020-06-01T01:16:44Z | 2020-06-01T01:16:43Z | OWNER | simonw/datasette/pulls/783 | See #699 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/783/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
570101428 | MDExOlB1bGxSZXF1ZXN0Mzc5MTkyMjU4 | 683 | .execute_write() and .execute_write_fn() methods on Database | 9599 | closed | 0 | 3268330 | 14 | 2020-02-24T19:51:58Z | 2020-05-30T18:40:20Z | 2020-02-25T04:45:08Z | OWNER | simonw/datasette/pulls/683 | See #682 - [x] Come up with design for `.execute_write()` and `.execute_write_fn()` - [x] Build some quick demo plugins to exercise the design - [x] Write some unit tests - [x] Write the documentation | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/683/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
268453968 | MDU6SXNzdWUyNjg0NTM5Njg= | 37 | Ability to serialize massive JSON without blocking event loop | 9599 | closed | 0 | 2 | 2017-10-25T15:58:03Z | 2020-05-30T17:29:20Z | 2020-05-30T17:29:20Z | OWNER | We run the risk of someone attempting a select statement that returns thousands of rows and hence takes several seconds just to JSON encode the response, effectively blocking the event loop and pausing all other traffic. The Twisted community have a solution for this, can we adapt that in some way? http://as.ynchrono.us/2010/06/asynchronous-json_18.html?m=1 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
570309546 | MDU6SXNzdWU1NzAzMDk1NDY= | 685 | Document (and reconsider design of) Database.execute() and Database.execute_against_connection_in_thread() | 9599 | closed | 0 | 3268330 | 15 | 2020-02-25T04:49:44Z | 2020-05-30T13:20:50Z | 2020-05-08T17:42:18Z | OWNER | In #683 I started a new section of internals documentation covering the `Database` class: https://datasette.readthedocs.io/en/latest/internals.html#database-class I decided not to document `.execute()` and `.execute_against_connection_in_thread()` yet because I'm not 100% happy with their API design yet. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/685/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
585633142 | MDU6SXNzdWU1ODU2MzMxNDI= | 706 | Documentation for the "request" object | 9599 | closed | 0 | 3268330 | 6 | 2020-03-22T02:55:50Z | 2020-05-30T13:20:00Z | 2020-05-27T22:31:22Z | OWNER | Since that object is passed to the `extra_template_vars` hooks AND the classes registered by `register_facet_classes` it should be part of the documented interface on https://datasette.readthedocs.io/en/stable/internals.html I could also start passing it to the `register_output_renderer` callback. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/706/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
626078521 | MDU6SXNzdWU2MjYwNzg1MjE= | 774 | Consolidate request.raw_args and request.args | 9599 | closed | 0 | 3268330 | 8 | 2020-05-27T22:30:59Z | 2020-05-29T23:27:35Z | 2020-05-29T23:22:38Z | OWNER | `request.raw_args` is not documented, and I'd like to remove it entirely. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/706#issuecomment-634975252_ I use it in a few places in other projects though, so I'll have to fix those first: https://github.com/search?q=user%3Asimonw+raw_args&type=Code | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/774/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
345469355 | MDU6SXNzdWUzNDU0NjkzNTU= | 351 | Automatically create a GitHub release linking to release notes for every tagged release | 9599 | closed | 0 | 1 | 2018-07-28T18:31:12Z | 2020-05-28T18:56:16Z | 2020-05-28T18:56:15Z | OWNER | Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
455965174 | MDU6SXNzdWU0NTU5NjUxNzQ= | 508 | Ability to set default sort order for a table or view in metadata.json | 9599 | closed | 0 | 9599 | 1 | 2019-06-13T21:40:51Z | 2020-05-28T18:53:03Z | 2020-05-28T18:53:02Z | OWNER | It can go here in the documentation: https://datasette.readthedocs.io/en/stable/metadata.html#setting-which-columns-can-be-used-for-sorting Also need to fix this sentence which is no longer true: > By default, database views in Datasette do not support sorting | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/508/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
626663119 | MDU6SXNzdWU2MjY2NjMxMTk= | 781 | request.url and request.scheme should obey force_https_urls config setting | 9599 | closed | 0 | 3 | 2020-05-28T16:54:47Z | 2020-05-28T17:39:54Z | 2020-05-28T17:10:13Z | OWNER | I'm trying to get the https://www.niche-museums.com/browse/feed.atom feed to validate and I git this from https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom > This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations. > > [line 6](https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom#l6), column 73: Self reference doesn't match document location [[help](https://validator.w3.org/feed/docs/warning/SelfDoesntMatchLocation.html "more information about this error")] > > <link href="http://www.niche-museums.com/browse/feed.atom" rel="self"/> I tried to fix this using `force_https_urls` ([commit](https://github.com/simonw/museums/commit/5dc8e2c717c59f9e949b65e47a59878e01f929e4)) but it didn't work - because that setting isn't respected by the Request class: https://github.com/simonw/datasette/blob/40885ef24e32d91502b6b8bbad1c7376f50f2830/datasette/utils/asgi.py#L15-L32 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/781/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
626582657 | MDU6SXNzdWU2MjY1ODI2NTc= | 779 | Make human_description_en explicitly available to output renderers | 9599 | open | 0 | 0 | 2020-05-28T14:59:54Z | 2020-05-28T14:59:54Z | OWNER | `datasette-atom` uses this: https://github.com/simonw/datasette-atom/blob/df98a6c43a443224b6cd232f84703ec297ef046b/datasette_atom/__init__.py#L36-L37 ```python if data.get("human_description_en"): title += ": " + data["human_description_en"] ``` It's a nice way to generate a useful title for a filtered table. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/779/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
625930207 | MDU6SXNzdWU2MjU5MzAyMDc= | 770 | register_output_renderer can_render mechanism | 9599 | closed | 0 | 5471110 | 4 | 2020-05-27T18:29:14Z | 2020-05-28T05:57:16Z | 2020-05-28T05:57:16Z | OWNER | I would like is the ability for renderers to opt-in / opt-out of being displayed as options on the page. https://www.niche-museums.com/browse/museums for example shows a atom link because the datasette-atom plugin is installed... but clicking it will give you a 400 error because the correct columns are not present. <img width="1113" alt="browse__museums__102_rows" src="https://user-images.githubusercontent.com/9599/83058543-62278e80-a00d-11ea-9e04-298e02745118.png"> Here's the code that passes a list of renderers to the template: https://github.com/simonw/datasette/blob/2d099ad9c657d2cab59de91cdb8bfed2da236ef6/datasette/views/base.py#L411-L423 A renderer is currently defined as a two-key dictionary: ```python @hookimpl def register_output_renderer(datasette): return { 'extension': 'test', 'callback': render_test } ``` I can add a third key, `"should_suggest"` which is a function that returns `True` or `False` for a given query. If that key is missing it is assumed to return `True`. One catch: what arguments should be passed to the `should_suggest(...)` function? UPDATE: now calling it `can_render` instead. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/581#issuecomment-634856748_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/770/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
611540797 | MDU6SXNzdWU2MTE1NDA3OTc= | 751 | Ability to set custom default _size on a per-table basis | 9599 | closed | 0 | 5471110 | 4 | 2020-05-04T00:13:03Z | 2020-05-28T05:00:22Z | 2020-05-28T05:00:20Z | OWNER | I have some tables where I'd like the default page size to be 10, without affecting the rest of my Datasette instance. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/751/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
626001501 | MDU6SXNzdWU2MjYwMDE1MDE= | 773 | All plugin hooks should have unit tests | 9599 | closed | 0 | 5471110 | 2 | 2020-05-27T20:17:41Z | 2020-05-28T04:12:11Z | 2020-05-28T04:09:25Z | OWNER | Four hooks currently missing tests: - [x] prepare_jinja2_environment - [x] publish_subcommand - [x] register_facet_classes - [x] register_output_renderer ``` $ pytest -k test_plugin_hooks_have_tests -vv ====================================== test session starts ====================================== platform darwin -- Python 3.7.7, pytest-5.2.4, py-1.8.1, pluggy-0.13.1 -- /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/python cachedir: .pytest_cache rootdir: /Users/simon/Dropbox/Development/datasette, inifile: pytest.ini plugins: asyncio-0.10.0 collected 486 items / 475 deselected / 11 selected tests/test_plugins.py::test_plugin_hooks_have_tests[asgi_wrapper] XPASS [ 9%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_body_script] XPASS [ 18%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_css_urls] XPASS [ 27%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_js_urls] XPASS [ 36%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_template_vars] XPASS [ 45%] tests/test_plugins.py::test_plugin_hooks_have_tests[prepare_connection] XPASS [ 54%] tests/test_plugins.py::test_plugin_hooks_have_tests[prepare_jinja2_environment] XFAIL [ 63%] tests/test_plugins.py::test_plugin_hooks_have_tests[publish_subcommand] XFAIL [ 72%] tests/test_plugins.py::test_plugin_hooks_have_tests[register_facet_classes] XFAIL [ 81%] tests/test_plugins.py::test_plugin_hooks_have_tests[register_output_renderer] XFAIL [ 90%] tests/test_plugins.py::test_plugin_hooks_have_tests[render_cell] XPASS [100%] ========================= 475 deselected, 4 xfailed, 7 xpassed in 1.70s ========================= _Originally posted by @simonw in https://github.com/simonw/datasette/issues/771#issuecomment-634915104_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/773/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
626163974 | MDU6SXNzdWU2MjYxNjM5NzQ= | 776 | register_output_renderer render callback should be optionally awaitable | 9599 | closed | 0 | 5471110 | 1 | 2020-05-28T02:26:29Z | 2020-05-28T02:43:36Z | 2020-05-28T02:43:36Z | OWNER | In #581 I made a bunch of improvements to this, including making `datasette` available to it so it could execute queries. But... it needs to be able to `await` in order to do that. Which means it should be optionally-awaitable. Original idea here: https://github.com/simonw/datasette/issues/645#issuecomment-560036740 | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/776/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
502993509 | MDU6SXNzdWU1MDI5OTM1MDk= | 581 | Redesign register_output_renderer callback | 9599 | closed | 0 | 5471110 | 24 | 2019-10-05T17:43:23Z | 2020-05-28T02:24:14Z | 2020-05-28T02:21:50Z | OWNER | In building https://github.com/simonw/datasette-atom it became clear that the callback function (which currently accepts just args, data and view_name) would also benefit from access to a mechanism to render templates and a `datasette` instance so it can execute SQL. To maintain backwards compatibility with existing plugins, we can introspect the callback function to see if it wants those new arguments or not. At a minimum I want to make `datasette` and ASGI `scope` available. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/581/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
530653633 | MDU6SXNzdWU1MzA2NTM2MzM= | 645 | Mechanism for register_output_renderer to suggest extension or not | 9599 | closed | 0 | 4 | 2019-12-01T01:26:27Z | 2020-05-28T02:22:18Z | 2020-05-28T02:22:12Z | OWNER | [datasette-atom](https://github.com/simonw/datasette-atom) only works if the user constructs a SQL query with specific output columns (`atom_id` ,`atom_updated` etc). It would be good if the `.atom` link wasn't shown on the query/table page unless those columns were present. Right now you get a link which results in a 400 error: <img width="622" alt="browse__museums__51_rows" src="https://user-images.githubusercontent.com/9599/69908058-81af6880-1396-11ea-99d7-19297fa1bb9b.png"> See also #581. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/645/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
626131309 | MDU6SXNzdWU2MjYxMzEzMDk= | 775 | Move test plugins into datasette/tests/plugins/ directory | 9599 | closed | 0 | 1 | 2020-05-28T00:46:58Z | 2020-05-28T00:57:31Z | 2020-05-28T00:57:31Z | OWNER | Right now the plugins used during test runs are baked into strings. It would be nicer if they were actual files on disk. Will make #581 easier to write tests for. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/775/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
622672640 | MDExOlB1bGxSZXF1ZXN0NDIxNDkxODEw | 768 | Use dirs_exist_ok=True | 9599 | closed | 0 | 5471110 | 0 | 2020-05-21T17:53:44Z | 2020-05-27T20:21:56Z | 2020-05-21T17:53:51Z | OWNER | simonw/datasette/pulls/768 | Refs #744 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/768/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
625922239 | MDExOlB1bGxSZXF1ZXN0NDI0MDMyNDQ1 | 769 | Backport of Python 3.8 shutil.copytree | 9599 | closed | 0 | 5471110 | 0 | 2020-05-27T18:17:15Z | 2020-05-27T20:21:56Z | 2020-05-27T18:17:44Z | OWNER | simonw/datasette/pulls/769 | Closes #744 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/769/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
625991831 | MDExOlB1bGxSZXF1ZXN0NDI0MDg1MjY0 | 772 | Test that plugin hooks are unit tested | 9599 | closed | 0 | 5471110 | 0 | 2020-05-27T20:01:32Z | 2020-05-27T20:21:56Z | 2020-05-27T20:16:03Z | OWNER | simonw/datasette/pulls/772 | Refs #771 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/772/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||
616012427 | MDU6SXNzdWU2MTYwMTI0Mjc= | 764 | Add PyPI project urls to setup.py | 9599 | closed | 0 | 5471110 | 3 | 2020-05-11T16:23:08Z | 2020-05-27T20:21:36Z | 2020-05-11T18:28:55Z | OWNER | Spotted this example here: ```python project_urls={ "Issues": "https://gitlab.com/Cyb3r-Jak3/ExifReader/issues", "Source Code": "https://gitlab.com/Cyb3r-Jak3/ExifReader/-/tree/publish", "CI": "https://gitlab.com/Cyb3r-Jak3/ExifReader/pipelines", "Releases": "https://github.com/Cyb3r-Jak3/ExifReader" }, ``` Results in this on https://pypi.org/project/ExifReader/ <img width="333" alt="ExifReader_Β·_PyPI" src="https://user-images.githubusercontent.com/9599/81585430-03a1b580-9369-11ea-9b58-e2f732cc13f8.png"> | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/764/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
625980317 | MDU6SXNzdWU2MjU5ODAzMTc= | 771 | Unit test that checks that all plugin hooks have corresponding unit tests | 9599 | closed | 0 | 5471110 | 5 | 2020-05-27T19:42:35Z | 2020-05-27T20:21:36Z | 2020-05-27T20:17:13Z | OWNER | Turns out some hooks are missing unit test coverage: https://github.com/simonw/datasette/issues/581#issuecomment-634893744_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/771/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
613006393 | MDU6SXNzdWU2MTMwMDYzOTM= | 20 | Ability to serve thumbnailed Apple Photo from its place on disk | 9599 | closed | 0 | 10 | 2020-05-06T02:17:50Z | 2020-05-25T20:14:22Z | 2020-05-25T20:09:41Z | MEMBER | A custom Datasette plugin that can be run locally on a Mac laptop which knows how to serve photos such that they can be seen in the browser. _Originally posted by @simonw in https://github.com/dogsheep/photos-to-sqlite/issues/19#issuecomment-624406285_ | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
621332242 | MDU6SXNzdWU2MjEzMzIyNDI= | 25 | Create a public demo | 9599 | closed | 0 | 5 | 2020-05-19T22:47:20Z | 2020-05-21T22:26:16Z | 2020-05-20T05:54:18Z | MEMBER | So I can show people what this does, using some of my photos. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
621486115 | MDU6SXNzdWU2MjE0ODYxMTU= | 27 | photos_with_apple_metadata view should include labels | 9599 | open | 0 | 0 | 2020-05-20T06:06:17Z | 2020-05-20T06:06:17Z | MEMBER | https://dogsheep-photos.dogsheep.net/public/photos_with_apple_metadata?place_city=New+Orleans&_facet=place_city&_facet_array=albums&_facet_array=persons Here's one way to add that: ```sql select rowid, photo, ( select json_group_array( json_object( 'label', normalized_string, 'href', '/photos/labelled?_hide_sql=1&label=' || normalized_string ) ) from labels where labels.uuid = photos_with_apple_metadata.uuid ) as labels, date, ``` | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
621323348 | MDU6SXNzdWU2MjEzMjMzNDg= | 24 | Configurable URL for images | 9599 | open | 0 | 1 | 2020-05-19T22:25:56Z | 2020-05-20T06:00:29Z | MEMBER | This is hard-coded at the moment, which is bad: https://github.com/dogsheep/photos-to-sqlite/blob/d5d69b9019703c47bc251444838578dd752801e2/photos_to_sqlite/cli.py#L269-L272 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/24/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
621444763 | MDU6SXNzdWU2MjE0NDQ3NjM= | 26 | Rename project to dogsheep-photos | 9599 | closed | 0 | 8 | 2020-05-20T04:12:34Z | 2020-05-20T04:31:02Z | 2020-05-20T04:30:40Z | MEMBER | `photos-to-sqlite` doesn't really capture the full scope of this project anymore. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
621280529 | MDU6SXNzdWU2MjEyODA1Mjk= | 23 | create-subset command for creating a publishable subset of a photos database | 9599 | closed | 0 | 1 | 2020-05-19T20:58:20Z | 2020-05-19T22:32:48Z | 2020-05-19T22:32:37Z | MEMBER | I want to share a subset of my photos, without sharing everything. Idea: $ photos-to-sqlite create-subset photos.db public.db "select sha256 from ... where ..." So the command takes a SQL query that returns sha256 hashes, then creates a new file called `public.db` containing just the data corresponding to those photos. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613002220 | MDU6SXNzdWU2MTMwMDIyMjA= | 19 | apple-photos command should work even if upload has not run | 9599 | closed | 0 | 1 | 2020-05-06T02:02:25Z | 2020-05-19T20:59:59Z | 2020-05-19T20:59:59Z | MEMBER | I want people to be able to query their Apple Photos metadata without having to first run `upload` to upload all of their files to their own S3 bucket. To do this I can have `apple-photos` calculate SHA256 hashes of each photo if the `uploads` table does not yet exist (or does not contain that photo). | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
603295970 | MDU6SXNzdWU2MDMyOTU5NzA= | 729 | Visually distinguish integer and text columns | 9599 | closed | 0 | 8 | 2020-04-20T14:47:26Z | 2020-05-18T17:20:02Z | 2020-05-15T18:16:56Z | OWNER | It would be useful if I could tell from looking at the table page if a column was a integer or a text (or a float I guess?). This is particularly important for knowing if it safe to sort by that column. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/729/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
615626118 | MDU6SXNzdWU2MTU2MjYxMTg= | 22 | Try out ExifReader | 9599 | open | 0 | 4 | 2020-05-11T06:32:13Z | 2020-05-14T05:59:53Z | MEMBER | https://pypi.org/project/ExifReader/ New fork that should be able to handle EXIF in HEIC files. Forked here: https://github.com/ianare/exif-py/issues/102#issuecomment-626376522 Refs #3 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
461215118 | MDU6SXNzdWU0NjEyMTUxMTg= | 30 | Option to open database in read-only mode | 9599 | closed | 0 | 1 | 2019-06-26T22:50:38Z | 2020-05-11T19:17:17Z | 2020-05-11T19:17:17Z | OWNER | Would this make it 100% safe to run reads against a database file that is being written to by another process? | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/30/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
616087149 | MDU6SXNzdWU2MTYwODcxNDk= | 765 | publish heroku should default to currently tagged version | 9599 | open | 0 | 1 | 2020-05-11T18:24:06Z | 2020-05-11T18:25:43Z | OWNER | Had a report that deploying to Heroku was using the previously installed version of Datasette, not the latest. Could be because of this: https://github.com/simonw/datasette/blob/af6c6c5d6f929f951c0e63bfd1c82e37a071b50f/datasette/publish/heroku.py#L172-L179 Heroku documentation recommends pinning to specific versions https://devcenter.heroku.com/articles/python-pip So... we could ensure we default to an install value of `["datasette>=current_tag"]`. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/765/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
615477131 | MDU6SXNzdWU2MTU0NzcxMzE= | 111 | sqlite-utils drop-table and drop-view commands | 9599 | closed | 0 | 2 | 2020-05-10T21:10:42Z | 2020-05-11T01:58:36Z | 2020-05-11T00:44:26Z | OWNER | Would be useful to be able to drop views and tables from the CLI. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/111/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
534492501 | MDU6SXNzdWU1MzQ0OTI1MDE= | 648 | Mechanism for adding arbitrary pages like /about | 9599 | closed | 0 | 13 | 2019-12-08T04:55:19Z | 2020-05-07T15:21:19Z | 2020-04-26T18:46:45Z | OWNER | For www.niche-museums.com I solved this by creating an empty `about.db` database file - see https://simonwillison.net/2019/Nov/25/niche-museums/ I want a neater mechanism for this. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/648/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613467382 | MDU6SXNzdWU2MTM0NjczODI= | 761 | Allow-list pragma_table_info(tablename) and similar | 9599 | closed | 0 | 8 | 2020-05-06T16:54:14Z | 2020-05-07T03:09:05Z | 2020-05-06T17:18:38Z | OWNER | It would be great if `pragma_table_info(tablename)` was allowed to be used in queries. See also https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md > `select * from pragma_table_info(tablename);` is currently disallowed for user-provided queries via a regex restriction - but could help here too. > > https://github.com/simonw/datasette/blob/d349d57cdf3d577afb62bdf784af342a4d5be660/datasette/utils/__init__.py#L174 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/760#issuecomment-624729459_ | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/761/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613491342 | MDU6SXNzdWU2MTM0OTEzNDI= | 762 | Experiment with PRAGMA hard_heap_limit | 9599 | open | 0 | 0 | 2020-05-06T17:33:23Z | 2020-05-07T03:08:44Z | OWNER | This was added in SQLite 2020-01-22 (3.31.0): https://www.sqlite.org/changes.html#version_3_31_0 > Add the [sqlite3_hard_heap_limit64()](https://www.sqlite.org/c3ref/hard_heap_limit64.html) interface and the corresponding [PRAGMA hard_heap_limit](https://www.sqlite.org/pragma.html#pragma_hard_heap_limit) command. This sounds like it could be a nice extra safety measure. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/762/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
613422636 | MDU6SXNzdWU2MTM0MjI2MzY= | 760 | Way of seeing full schema for a database | 9599 | open | 0 | 3 | 2020-05-06T15:46:08Z | 2020-05-06T23:49:06Z | OWNER | I find myself wanting to quickly figure out all of the BLOB columns in a database. A `/-/schema` page showing the full schema (actually since it's per-database probably `/dbname/-/schema` or `/-/schema/dbname`) would be really handy. It would need to be carefully constructed from various queries against `sqlite_master` - just doing `select * from sqlite_master where type='table'` isn't quite enough because I also want to show indexes, triggers etc. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/760/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
612860758 | MDU6SXNzdWU2MTI4NjA3NTg= | 18 | Switch CI solution to GitHub Actions with a macOS runner | 9599 | open | 0 | 1 | 2020-05-05T20:03:50Z | 2020-05-05T23:49:18Z | MEMBER | Refs #17. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
612860531 | MDU6SXNzdWU2MTI4NjA1MzE= | 17 | Only install osxphotos if running on macOS | 9599 | closed | 0 | 3 | 2020-05-05T20:03:26Z | 2020-05-05T20:20:05Z | 2020-05-05T20:11:23Z | MEMBER | The build is broken right now because you can't `pip install osxphotos` on Ubuntu. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612658444 | MDU6SXNzdWU2MTI2NTg0NDQ= | 109 | table.create_index(..., ignore=True) | 9599 | closed | 0 | 1 | 2020-05-05T14:44:21Z | 2020-05-05T14:46:53Z | 2020-05-05T14:46:53Z | OWNER | Option to silently do nothing if the index already exists. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/109/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612287234 | MDU6SXNzdWU2MTIyODcyMzQ= | 16 | Import machine-learning detected labels (dog, llama etc) from Apple Photos | 9599 | open | 0 | 13 | 2020-05-05T02:45:43Z | 2020-05-05T05:38:16Z | MEMBER | Follow-on from #1. Apple Photos runs some very sophisticated machine learning on-device to figure out if photos are of dogs, llamas and so on. I really want to extract those labels out into my own database. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 1, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
612089949 | MDU6SXNzdWU2MTIwODk5NDk= | 756 | Add pipx to installation documentation | 9599 | closed | 0 | 2 | 2020-05-04T18:49:01Z | 2020-05-04T19:19:06Z | 2020-05-04T19:10:33Z | OWNER | Add to this page: https://datasette.readthedocs.io/en/stable/installation.html Here's how to install plugins: https://twitter.com/simonw/status/1257348687979778050 ``` $ datasette plugins [] $ pipx inject datasette datasette-json-html injected package datasette-json-html into venv datasette done! β¨ π β¨ $ datasette plugins [ { "name": "datasette-json-html", "static": false, "templates": false, "version": "0.6" } ] ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/756/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612082842 | MDU6SXNzdWU2MTIwODI4NDI= | 755 | Fix "no such column: id" output in tests | 9599 | closed | 0 | 1 | 2020-05-04T18:37:49Z | 2020-05-04T18:42:14Z | 2020-05-04T18:42:14Z | OWNER | ``` pytest ... tests/test_custom_pages.py ........ [ 33%] tests/test_database.py ......no such column: id ... [ 35%] ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/755/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611997130 | MDU6SXNzdWU2MTE5OTcxMzA= | 754 | Clean up aiofiles warnings on 3.8 | 9599 | closed | 0 | 2 | 2020-05-04T16:14:59Z | 2020-05-04T16:22:30Z | 2020-05-04T16:22:30Z | OWNER | https://travis-ci.org/github/simonw/datasette/jobs/682624476 Lots of warnings like this: ``` /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33 /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33 /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def method(self, *args, **kwargs): /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/__init__.py:27 /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/__init__.py:27: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def _open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/754/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
602533300 | MDU6SXNzdWU2MDI1MzMzMDA= | 1 | Import photo metadata from Apple Photos into SQLite | 9599 | open | 0 | 5324096 | 8 | 2020-04-18T19:23:26Z | 2020-05-04T02:41:40Z | MEMBER | Faces, albums, locations, that kind of thing. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
|||||||
610408908 | MDU6SXNzdWU2MTA0MDg5MDg= | 34 | Command for retrieving dependents for a repo | 9599 | closed | 0 | 6 | 2020-04-30T21:47:51Z | 2020-05-03T15:53:01Z | 2020-05-03T15:53:01Z | MEMBER | I really, really want to start grabbing this data: https://github.com/simonw/datasette/network/dependents | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611222968 | MDU6SXNzdWU2MTEyMjI5Njg= | 107 | sqlite-utils create-view CLI command | 9599 | closed | 0 | 2 | 2020-05-02T16:15:13Z | 2020-05-03T15:36:58Z | 2020-05-03T15:36:37Z | OWNER | Can go with #27 - `sqlite-utils create-table`. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/107/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
455496504 | MDU6SXNzdWU0NTU0OTY1MDQ= | 27 | sqlite-utils create-table command | 9599 | closed | 0 | 8 | 2019-06-13T01:43:30Z | 2020-05-03T15:26:15Z | 2020-05-03T15:26:15Z | OWNER | Spun off from #24 - it would be useful if CLI users could create new tables (with explicit column types, not null rules and defaults) without having to insert an example record. - [x] Get it working - [x] Support `--pk` - [x] Support `--not-null` - [x] Support `--default` - [x] Support `--fk colname othertable othercol` - [x] Support `--replace` and `--ignore` - [x] Documentation | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611326701 | MDU6SXNzdWU2MTEzMjY3MDE= | 108 | Documentation unit tests for CLI commands | 9599 | closed | 0 | 2 | 2020-05-03T03:58:42Z | 2020-05-03T04:13:57Z | 2020-05-03T04:13:57Z | OWNER | Have a test that ensures all CLI commands are documented. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/108/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
493670730 | MDU6SXNzdWU0OTM2NzA3MzA= | 4 | Command to fetch stargazers for one or more repos | 9599 | closed | 0 | 8 | 2019-09-14T21:58:22Z | 2020-05-02T21:30:27Z | 2020-05-02T21:30:27Z | MEMBER | Maybe this: $ github-to-sqlite stargazers github.db simonw/datasette It could accept more than one repos. Maybe have options similar to `--sql` in [twitter-to-sqlite](https://github.com/dogsheep/twitter-to-sqlite) so you can e.g. fetch all stargazers for all of the repos you have fetched into the database already (or all of the repos belonging to owner X) | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516763727 | MDExOlB1bGxSZXF1ZXN0MzM1OTgwMjQ2 | 8 | stargazers command, refs #4 | 9599 | closed | 0 | 5 | 2019-11-03T00:37:36Z | 2020-05-02T20:00:27Z | 2020-05-02T20:00:26Z | MEMBER | dogsheep/github-to-sqlite/pulls/8 | Needs tests. Refs #4. | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
611252244 | MDU6SXNzdWU2MTEyNTIyNDQ= | 750 | Add notlike table filter | 9599 | closed | 0 | 3 | 2020-05-02T18:54:36Z | 2020-05-02T19:10:44Z | 2020-05-02T19:10:44Z | OWNER | I found myself wanting that for applying the opposite of this: https://github-to-sqlite.dogsheep.net/github/dependent_repos?dependent__like=%25simonw%2F%25&_sort_desc=dependent_stars | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/750/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
520756546 | MDU6SXNzdWU1MjA3NTY1NDY= | 12 | Add this view for seeing new releases | 9599 | closed | 0 | 5 | 2019-11-11T06:00:12Z | 2020-05-02T18:58:18Z | 2020-05-02T18:58:17Z | MEMBER | ```sql CREATE VIEW recent_releases AS select json_object("label", repos.full_name, "href", repos.html_url) as repo, json_object( "href", releases.html_url, "label", releases.name ) as release, substr(releases.published_at, 0, 11) as date, releases.body as body_markdown, releases.published_at from releases join repos on repos.id = releases.repo order by releases.published_at desc ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610511450 | MDU6SXNzdWU2MTA1MTE0NTA= | 35 | Create index on issue_comments(user) and other foreign keys | 9599 | closed | 0 | 3 | 2020-05-01T02:06:56Z | 2020-05-02T18:26:24Z | 2020-05-02T18:26:24Z | MEMBER | ``` create index issue_comments_user on issue_comments(user) ``` I'm sure there are other user columns that could benefit from an index. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610842926 | MDU6SXNzdWU2MTA4NDI5MjY= | 36 | Add view for better display of dependent repos | 9599 | closed | 0 | 2 | 2020-05-01T16:33:44Z | 2020-05-02T16:50:31Z | 2020-05-02T16:30:11Z | MEMBER | ```sql select repos.full_name as repo, 'https://github.com/' || repos2.full_name as dependent, repos2.created_at as dependent_repo_created, repos2.updated_at as dependent_repo_updated, repos2.stargazers_count as dependent_repo_stars, repos2.watchers_count as dependent_repo_watchers from dependents join repos as repos2 on dependents.dependent = repos2.id join repos on dependents.repo = repos.id order by repos2.created_at desc ``` https://dogsheep.simonwillison.net/github?sql=select%0D%0A++repos.full_name+as+repo%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos2.full_name+as+dependent%2C%0D%0A++repos2.created_at+as+dependent_repo_created%2C%0D%0A++repos2.updated_at+as+dependent_repo_updated%2C%0D%0A++repos2.stargazers_count+as+dependent_repo_stars%2C%0D%0A++repos2.watchers_count+as+dependent_repo_watchers%0D%0Afrom%0D%0A++dependents%0D%0A++join+repos+as+repos2+on+dependents.dependent+%3D+repos2.id%0D%0A++join+repos+on+dependents.repo+%3D+repos.id%0D%0Aorder+by%0D%0A++repos2.created_at+desc | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/36/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516967682 | MDU6SXNzdWU1MTY5Njc2ODI= | 10 | Add this repos_starred view | 9599 | closed | 0 | 3 | 2019-11-04T05:44:38Z | 2020-05-02T16:37:36Z | 2020-05-02T16:37:36Z | MEMBER | ```sql create view repos_starred as select stars.starred_at, users.login, repos.* from repos join stars on repos.id = stars.repo join users on repos.owner = users.id order by starred_at desc; ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610843136 | MDU6SXNzdWU2MTA4NDMxMzY= | 37 | Mechanism for creating views if they don't yet exist | 9599 | closed | 0 | 3 | 2020-05-01T16:34:10Z | 2020-05-02T16:19:47Z | 2020-05-02T16:19:31Z | MEMBER | Needed for #36 #10 #12 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611216862 | MDU6SXNzdWU2MTEyMTY4NjI= | 106 | create_view(..., ignore=True, replace=True) parameters | 9599 | closed | 0 | 1 | 2020-05-02T15:45:21Z | 2020-05-02T16:04:51Z | 2020-05-02T16:02:10Z | OWNER | Two new parameters which specify what should happen if the view already exists. I want this for https://github.com/dogsheep/github-to-sqlite/issues/37 Here's the current `create_view()` implementation: https://github.com/simonw/sqlite-utils/blob/b4d953d3ccef28bb81cea40ca165a647b59971fa/sqlite_utils/db.py#L325-L332 `ignore=True` will not do anything if the view exists already. `replace=True` will drop and redefine the view - but only if its SQL definition differs, otherwise it will be left alone. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/106/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
602569315 | MDU6SXNzdWU2MDI1NjkzMTU= | 102 | Can't store an array or dictionary containing a bytes value | 9599 | closed | 0 | 0 | 2020-04-18T22:49:21Z | 2020-05-01T20:45:45Z | 2020-05-01T20:45:45Z | OWNER | ``` In [1]: import sqlite_utils In [2]: db = sqlite_utils.Database(memory=True) In [3]: db["t"].insert({"id": 1, "data": {"foo": b"bytes"}}) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-3-a8ab1f72c72c> in <module> ----> 1 db["t"].insert({"id": 1, "data": {"foo": b"bytes"}}) ~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert(self, record, pk, foreign_keys, column_order, not_null, defaults, hash_id, alter, ignore, replace, extracts, conversions, columns) 950 extracts=extracts, 951 conversions=conversions, --> 952 columns=columns, 953 ) 954 ~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, conversions, columns, upsert) 1052 for key in all_columns: 1053 value = jsonify_if_needed( -> 1054 record.get(key, None if key != hash_id else _hash(record)) 1055 ) 1056 if key in extracts: ~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in jsonify_if_needed(value) 1318 def jsonify_if_needed(value): 1319 if isinstance(value, (dict, list, tuple)): -> 1320 return json.dumps(value) 1321 elif isinstance(value, (datetime.time, datetime.date, datetime.datetime)): 1322 return value.isoformat() /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw) 229 cls is None and indent is None and separators is None and 230 default is None⦠| 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/102/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610853576 | MDU6SXNzdWU2MTA4NTM1NzY= | 105 | "sqlite-utils views" command | 9599 | closed | 0 | 1 | 2020-05-01T16:56:11Z | 2020-05-01T20:40:07Z | 2020-05-01T20:38:36Z | OWNER | Similar to `sqlite-utils tables`. See also #104. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/105/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610853393 | MDU6SXNzdWU2MTA4NTMzOTM= | 104 | --schema option to "sqlite-utils tables" | 9599 | closed | 0 | 0 | 2020-05-01T16:55:49Z | 2020-05-01T17:12:37Z | 2020-05-01T17:12:37Z | OWNER | Adds output showing the table schema. | 140912432 | issue | { "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/104/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610342575 | MDU6SXNzdWU2MTAzNDI1NzU= | 748 | ?_searchmode=raw should be documented on full-text search page | 9599 | closed | 0 | 0 | 2020-04-30T19:50:06Z | 2020-04-30T21:06:12Z | 2020-04-30T21:06:12Z | OWNER | It's currently documented here: https://datasette.readthedocs.io/en/stable/json_api.html#special-table-arguments But it should also be described here: https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-view-api | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/748/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610192152 | MDU6SXNzdWU2MTAxOTIxNTI= | 747 | Directory configuration mode should support metadata.yaml | 9599 | closed | 0 | 4 | 2020-04-30T16:05:30Z | 2020-04-30T19:04:19Z | 2020-04-30T19:04:19Z | OWNER | Refs #739 - `metadata.yml` or `metadata.yaml` should be detected in the same way as `metadata.json` is. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/747/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610284471 | MDU6SXNzdWU2MTAyODQ0NzE= | 46 | Error running 'search' for the first time | 9599 | closed | 0 | 0 | 2020-04-30T18:11:20Z | 2020-04-30T18:11:58Z | 2020-04-30T18:11:58Z | MEMBER | ``` % twitter-to-sqlite search infodemic.db '#infodemic' Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/bin/twitter-to-sqlite", line 11, in <module> load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')() File "/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py", line 867, in search for tweet in tweets: File "/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py", line 165, in fetch_timeline [since_type_id, since_key], sqlite3.OperationalError: no such table: since_ids ``` | 206156866 | issue | { "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/46/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
608752766 | MDExOlB1bGxSZXF1ZXN0NDEwNDY5Mjcy | 746 | shutil.Error, not OSError | 9599 | closed | 0 | 1 | 2020-04-29T03:30:51Z | 2020-04-29T07:07:24Z | 2020-04-29T07:07:23Z | OWNER | simonw/datasette/pulls/746 | Refs #744 | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/746/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
522334771 | MDU6SXNzdWU1MjIzMzQ3NzE= | 633 | Publish to Heroku is broken: "WARNING: You must pass the application as an import string to enable 'reload' or 'workers" | 9599 | closed | 0 | 3 | 2019-11-13T16:32:11Z | 2020-04-28T20:37:50Z | 2019-11-13T16:43:23Z | OWNER | ``` 2019-11-13T16:27:59.821483+00:00 heroku[web.1]: Starting process with command `datasette serve --host 0.0.0.0 -i fixtures.db --cors --port 36817 --inspect-file inspect-data.json` 2019-11-13T16:28:01.856471+00:00 heroku[web.1]: State changed from starting to crashed 2019-11-13T16:28:01.750253+00:00 app[web.1]: Serve! files=() (immutables=('fixtures.db',)) on port 36817 2019-11-13T16:28:01.771524+00:00 app[web.1]: WARNING: You must pass the application as an import string to enable 'reload' or 'workers'. 2019-11-13T16:28:01.837839+00:00 heroku[web.1]: Process exited with status 1 ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/633/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
608512747 | MDU6SXNzdWU2MDg1MTI3NDc= | 14 | Annotate photos using the Google Cloud Vision API | 9599 | open | 0 | 5 | 2020-04-28T18:09:03Z | 2020-04-28T18:19:06Z | MEMBER | It can detect faces, run OCR, do image labeling (it knows what a lemur is!) and do object localization where it identifies objects and returns bounding polygons for them. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14/reactions", "total_count": 3, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
||||||||
602533352 | MDU6SXNzdWU2MDI1MzMzNTI= | 2 | Ability to convert HEIC images to JPEG | 9599 | closed | 0 | 5324096 | 1 | 2020-04-18T19:23:43Z | 2020-04-28T16:47:21Z | 2020-04-28T16:47:21Z | MEMBER | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
607888367 | MDU6SXNzdWU2MDc4ODgzNjc= | 13 | Also upload movie files | 9599 | open | 0 | 2 | 2020-04-27T22:11:25Z | 2020-04-28T00:39:45Z | MEMBER | The `upload` command currently only handles static images: https://github.com/dogsheep/photos-to-sqlite/blob/d939455af00e07866686457ee2fcb9b2d1b7194e/photos_to_sqlite/utils.py#L26-L33 Need to cover movies taken by my phone and DSLR too. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
||||||||
607770595 | MDU6SXNzdWU2MDc3NzA1OTU= | 743 | escape_fts() does not correctly escape * wildcards | 9599 | closed | 0 | 4 | 2020-04-27T18:48:53Z | 2020-04-27T19:11:30Z | 2020-04-27T19:11:01Z | OWNER | Spotted in #732. This should not return any results... but it does: https://latest.datasette.io/fixtures/searchable?_search=bar%2A&_trace=1 <img width="694" alt="fixtures__searchable__1_row_where_where_search_matches__bar__" src="https://user-images.githubusercontent.com/9599/80408992-f66ecc00-887c-11ea-852f-08dd9d115206.png"> The query from trace is: ``` "sql": "select count(*) from searchable where rowid in (select rowid from searchable_fts where searchable_fts match escape_fts(:search))", "params": { "search": "bar*" } ``` | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/743/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
607211058 | MDU6SXNzdWU2MDcyMTEwNTg= | 740 | Don't throw 500 error on attempted directory browse | 9599 | closed | 0 | 1 | 2020-04-27T03:50:11Z | 2020-04-27T18:29:15Z | 2020-04-27T18:29:15Z | OWNER | <img width="725" alt="Error_500" src="https://user-images.githubusercontent.com/9599/80332338-76eee780-87ff-11ea-85b3-32704d715cd9.png"> This should be a 403 error instead, because the `--static` mechanism doesn't allow directory browsing. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/740/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
607243940 | MDU6SXNzdWU2MDcyNDM5NDA= | 742 | Speed up tests with scope="session"? | 9599 | closed | 0 | 1 | 2020-04-27T05:23:54Z | 2020-04-27T18:24:53Z | 2020-04-27T18:24:53Z | OWNER | Tests are pretty slow - could I speed them up with pytest `scope="session"` on some of the fixtures? Eg https://travis-ci.org/github/simonw/datasette/jobs/679940036 ran 452 tests in 3m53s - the `test_html` ones seem particularly slow. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/742/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
605110015 | MDU6SXNzdWU2MDUxMTAwMTU= | 731 | Option to automatically configure based on directory layout | 9599 | closed | 0 | 9 | 2020-04-22T22:17:47Z | 2020-04-27T16:32:44Z | 2020-04-27T16:30:26Z | OWNER | My Datasette projects increasingly take on the following structure: - `metadata.json` with the metadata - One or more `something.db` database files - A `templates/` folder with some custom templates - A `plugins/` folder with some custom plugins Then I have to run Datasette like this: datasette *.db -m metadata.json --template-dir=templates --plugins-dir=plugins It would be really interesting if Datasette had a special mode where you could point it at a directory with the above layout and it would automatically configure itself based on the contents. Maybe even allow `datasette serve` to detect if it was passed a single argument that's a directory, not a file, and kick in to "directory layout configuration mode" in that case: datasette . | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/731/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
607107849 | MDExOlB1bGxSZXF1ZXN0NDA5MTUzODcw | 739 | Configuration directory mode | 9599 | closed | 0 | 3 | 2020-04-26T20:37:46Z | 2020-04-27T16:30:25Z | 2020-04-27T16:30:25Z | OWNER | simonw/datasette/pulls/739 | Refs #731 TODO: - [x] Decide how to combine explicit command-line options with items detected from the directory structure - [x] Add unit tests - [x] Implement `inspect-data.json` mechanism for populating `immutables` - [x] Add documentation | 107914493 | pull | { "url": "https://api.github.com/repos/simonw/datasette/issues/739/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
607086780 | MDU6SXNzdWU2MDcwODY3ODA= | 738 | Pass a request object to custom page templates | 9599 | closed | 0 | 1 | 2020-04-26T18:57:48Z | 2020-04-26T19:01:54Z | 2020-04-26T19:01:54Z | OWNER | Follow-up to #648. I'm not passing a request object to `.render_template()` at the moment, which breaks any other custom plugins using e.g. `extra_template_vars()` that were expecting to be able to access the request. | 107914493 | issue | { "url": "https://api.github.com/repos/simonw/datasette/issues/738/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |