github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1016#issuecomment-706788010 | https://api.github.com/repos/simonw/datasette/issues/1016 | 706788010 | MDEyOklzc3VlQ29tbWVudDcwNjc4ODAxMA== | 9599 | 2020-10-11T23:50:39Z | 2020-10-11T23:50:39Z | OWNER | For consistency can reuse the icon used on selected facets: <img width="721" alt="evernote__notes__8_rows_where_where_application-data____ACD4B543-4663-40B2-BAF5-666FA54D4204__and_source-url_is_not_blank_sorted_by_id" src="https://user-images.githubusercontent.com/9599/95693564-e0ae5280-0be1-11eb-881b-fa0cec8397f5.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718953669 | |
https://github.com/simonw/datasette/issues/1015#issuecomment-706756879 | https://api.github.com/repos/simonw/datasette/issues/1015 | 706756879 | MDEyOklzc3VlQ29tbWVudDcwNjc1Njg3OQ== | 9599 | 2020-10-11T19:35:03Z | 2020-10-11T19:35:03Z | OWNER | Since plugins are installed via pip this would require Datasette to be restarted. This StackOverflow thread looks relevant to that: https://stackoverflow.com/questions/11329917/restart-python-script-from-within-itself This recipe looks promising: ```python import os import sys import psutil import logging def restart_program(): """Restarts the current program, with file objects and descriptors cleanup """ try: p = psutil.Process(os.getpid()) for handler in p.get_open_files() + p.connections(): os.close(handler.fd) except Exception, e: logging.error(e) python = sys.executable os.execl(python, python, *sys.argv) ``` https://docs.python.org/3/library/os.html#os.execl says about `os.execl`: > These functions all execute a new program, replacing the current process; they do not return. On Unix, the new executable is loaded into the current process, and will have the same process id as the caller | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718910318 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706745236 | https://api.github.com/repos/simonw/datasette/issues/782 | 706745236 | MDEyOklzc3VlQ29tbWVudDcwNjc0NTIzNg== | 9599 | 2020-10-11T18:16:05Z | 2020-10-11T18:16:05Z | OWNER | Here's the `datasette-json-preview` plugin I'll be using to experiment with different formats: https://github.com/simonw/datasette-json-preview | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706740250 | https://api.github.com/repos/simonw/datasette/issues/782 | 706740250 | MDEyOklzc3VlQ29tbWVudDcwNjc0MDI1MA== | 9599 | 2020-10-11T17:40:48Z | 2020-10-11T17:43:07Z | OWNER | Building this plugin reminded me of an oddity of the `register_output_renderer()` plugin hook: one of the arguments that can be passed to it is `data`, which is the default internal data structure created by Datasette - but I deliberately avoided documenting that on https://docs.datasette.io/en/stable/plugin_hooks.html#register-output-renderer-datasette because it's not a stable interface. That's not ideal. I'd like custom renderers to be able to access this data to get at things like suggested facets, on an opt-in basis. So maybe that kind of stuff is re-implemented as "extras" which are awaitable callables - then renderer plugins can call the extras that they need to as part of their execution. To illustrate the problem (in this case the need to access `next_url`) here's my first prototype of the plugin: ```python from datasette import hookimpl from datasette.utils.asgi import Response @hookimpl def register_output_renderer(datasette): return { "extension": "json-preview", "render": json_preview, } def json_preview(data, columns, rows): next_url = data.get("next_url") headers = {} if next_url: headers["link"] = '<{}>; rel="next"'.format(next_url) return Response.json([dict(zip(columns, row)) for row in rows], headers=headers) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706738020 | https://api.github.com/repos/simonw/datasette/issues/782 | 706738020 | MDEyOklzc3VlQ29tbWVudDcwNjczODAyMA== | 9599 | 2020-10-11T17:23:18Z | 2020-10-11T17:23:48Z | OWNER | I'm going to prototype what it would look like if the default shape was a list of objects and `?_extra=` turns that into an object with a `rows` key, in a plugin. As a separate extension (maybe `.json-preview`). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706735341 | https://api.github.com/repos/simonw/datasette/issues/782 | 706735341 | MDEyOklzc3VlQ29tbWVudDcwNjczNTM0MQ== | 9599 | 2020-10-11T17:03:29Z | 2020-10-11T17:15:34Z | OWNER | Maybe `.jsonfull` becomes a new renderer that returns ALL of the defined `?_extra=` blocks. Or... `?_extra=all` turns on ALL of the available information blocks (some of which can come from plugins). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706735200 | https://api.github.com/repos/simonw/datasette/issues/782 | 706735200 | MDEyOklzc3VlQ29tbWVudDcwNjczNTIwMA== | 9599 | 2020-10-11T17:02:11Z | 2020-10-11T17:14:51Z | OWNER | Since the total count can be expensive to calculate, I'm inclined to make that an opt-in extra - maybe `?_extra=count`. Based on that, the default JSON shape could look something like this: ```json { "rows": [{"id": 1}, {"id": 2}], "next": "2", "next_url": "/db/table?_next=2" } ``` And with `?_extra=count`: ```json { "rows": [{"id": 1}, {"id": 2}], "next": "2", "next_url": "/db/table?_next=2", "count": 31 } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706736541 | https://api.github.com/repos/simonw/datasette/issues/782 | 706736541 | MDEyOklzc3VlQ29tbWVudDcwNjczNjU0MQ== | 9599 | 2020-10-11T17:12:27Z | 2020-10-11T17:12:27Z | OWNER | The core issue that I keep reconsidering is whether the default `.json` representation should be an object or a list. Arguments in favour of a list: - It's what I always want. Almost all of the code that I've written against the API myself uses `?_shape=array`. - It's really easy to use. You can pipe it to e.g. `sqlite-utils insert`, you can load it into JavaScript without thinking about it. Arguments against: - Nowhere to put pagination or total counts. I added pagination to the `link:` HTTP header in #1014 (inspired by the WordPress and GitHub APIs) but I haven't solved for total count, and there's other stuff that's useful like `"truncated": true` to indicate that more than 1000 results were returned and they were truncated. - An array is inherently non-extensible: if the root item is an object it's easy to add new features to it in a backwards-compatible way in the future. An array is a fixed format. But maybe that last point is a positive? It ensures the default `.json` format remains completely predictable forever. If `.json` DID default to an array of objects, the `?_shape=` argument could still be used to get back alternative formats. Maybe `.json?_extra=total` changes the shape of that default to be this instead: ```json { "rows": [{"id": 1}, {"id": 2}], "total": 104 } ``` The thing I care about most though is `next_url`. That could be provided like so: `.json?_extra=total&_extra=next` - alternative syntax `.json?_extra=total,next`: ```json { "rows": [{"id": 1}, {"id": 2}], "total": 104, "next": "2", "next_url": "/db/table.json?_extra=total&_extra=next&_next=2" } ``` This is feeling a bit verbose for a common combination though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-706735280 | https://api.github.com/repos/simonw/datasette/issues/782 | 706735280 | MDEyOklzc3VlQ29tbWVudDcwNjczNTI4MA== | 9599 | 2020-10-11T17:03:01Z | 2020-10-11T17:03:01Z | OWNER | Should that default also include `"columns"` as a list of strings? That would be duplicate data of the keys in the `"rows"` list of objects, and I've never found myself wanting it in my own code - so I'm going to say no. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/1014#issuecomment-706631006 | https://api.github.com/repos/simonw/datasette/issues/1014 | 706631006 | MDEyOklzc3VlQ29tbWVudDcwNjYzMTAwNg== | 9599 | 2020-10-11T00:36:43Z | 2020-10-11T00:36:43Z | OWNER | Demo using [paginate-json](https://github.com/simonw/paginate-json): ``` % paginate-json 'https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array' | jq '. | length' https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cd%2Cv http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Ch%2Cr http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cl%2Cn http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cp%2Cj http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Ct%2Cf http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cx%2Cb http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ca%2Cx http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ce%2Ct http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ci%2Cp http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Cm%2Cl 1001 ``` New documentation: https://docs.datasette.io/en/latest/json_api.html#pagination | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718723543 |