github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/514#issuecomment-504684709 | https://api.github.com/repos/simonw/datasette/issues/514 | 504684709 | MDEyOklzc3VlQ29tbWVudDUwNDY4NDcwOQ== | 7936571 | 2019-06-22T17:36:25Z | 2019-06-22T17:36:25Z | NONE | > WorkingDirectory=/path/to/data @russss, Which directory does this represent? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504663766 | https://api.github.com/repos/simonw/datasette/issues/514 | 504663766 | MDEyOklzc3VlQ29tbWVudDUwNDY2Mzc2Ng== | 45057 | 2019-06-22T12:57:59Z | 2019-06-22T12:57:59Z | CONTRIBUTOR | > This example is useful to - I like how it has a Makefile that knows how to set up systemd: https://github.com/pikesley/Queube I wasn't even aware it was possible to add a systemd service at an arbitrary path, but it seems a little messy to me. Maybe worth noting that systemd does support [per-user services](https://wiki.archlinux.org/index.php/Systemd/User) which don't require root access. Cool but probably overkill for most people (especially when you're going to need root to listen on port 80 anyway, directly or via a reverse proxy). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504663390 | https://api.github.com/repos/simonw/datasette/issues/514 | 504663390 | MDEyOklzc3VlQ29tbWVudDUwNDY2MzM5MA== | 9599 | 2019-06-22T12:52:27Z | 2019-06-22T12:52:27Z | OWNER | This example is useful to - I like how it has a Makefile that knows how to set up systemd: https://github.com/pikesley/Queube | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504663201 | https://api.github.com/repos/simonw/datasette/issues/514 | 504663201 | MDEyOklzc3VlQ29tbWVudDUwNDY2MzIwMQ== | 9599 | 2019-06-22T12:49:56Z | 2019-06-22T12:49:56Z | OWNER | Here are some partial notes I have saved from an nginx configuration I've used in the past: ``` cat /etc/nginx/sites-available/default server { listen 80 default_server; listen [::]:80 default_server; location / { proxy_pass http://127.0.0.1:8001/; proxy_set_header Host $host; } ... ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504662987 | https://api.github.com/repos/simonw/datasette/issues/514 | 504662987 | MDEyOklzc3VlQ29tbWVudDUwNDY2Mjk4Nw== | 9599 | 2019-06-22T12:46:39Z | 2019-06-22T12:46:39Z | OWNER | ... and @russss also suggested systemd 21 seconds before I posted that! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504662931 | https://api.github.com/repos/simonw/datasette/issues/514 | 504662931 | MDEyOklzc3VlQ29tbWVudDUwNDY2MjkzMQ== | 9599 | 2019-06-22T12:45:47Z | 2019-06-22T12:45:47Z | OWNER | I asked about this on Twitter and got a solid recommendation for systemd and this tutorial: https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units - via https://twitter.com/sil/status/1142412145990221825 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504662904 | https://api.github.com/repos/simonw/datasette/issues/514 | 504662904 | MDEyOklzc3VlQ29tbWVudDUwNDY2MjkwNA== | 45057 | 2019-06-22T12:45:21Z | 2019-06-22T12:45:39Z | CONTRIBUTOR | On most modern Linux distros, systemd is the easiest answer. Example systemd unit file (save to `/etc/systemd/system/datasette.service`): ``` [Unit] Description=Datasette After=network.target [Service] Type=simple User=<username> WorkingDirectory=/path/to/data ExecStart=/path/to/datasette serve -h 0.0.0.0 ./my.db Restart=on-failure [Install] WantedBy=multi-user.target ``` Activate it with: ```bash $ sudo systemctl daemon-reload $ sudo systemctl enable datasette $ sudo systemctl start datasette ``` Logs are best viewed using `journalctl -u datasette -f`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504662063 | https://api.github.com/repos/simonw/datasette/issues/514 | 504662063 | MDEyOklzc3VlQ29tbWVudDUwNDY2MjA2Mw== | 9599 | 2019-06-22T12:31:58Z | 2019-06-22T12:31:58Z | OWNER | This is also relevant to Datasette Library #417 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504661990 | https://api.github.com/repos/simonw/datasette/issues/514 | 504661990 | MDEyOklzc3VlQ29tbWVudDUwNDY2MTk5MA== | 9599 | 2019-06-22T12:30:47Z | 2019-06-22T12:30:47Z | OWNER | A section in the Datasette docs that acts as recommendations plus a tutorial for running Datasette on a VPS without using a Docker would be excellent. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/514#issuecomment-504661909 | https://api.github.com/repos/simonw/datasette/issues/514 | 504661909 | MDEyOklzc3VlQ29tbWVudDUwNDY2MTkwOQ== | 9599 | 2019-06-22T12:29:46Z | 2019-06-22T12:29:46Z | OWNER | I'm still trying to figure this out myself. I'm confident that running nginx on port 80 and using it to proxy traffic to Datasette is a sensible way to solve the port problem. As for running Datasette itself: the two options that seem best to me are some kind of Init.d service or running it under supervisord. I have to admit I haven't worked out the necessary incantation for either of those yet: the solitary instance I have that's not running as a Docker container is sitting in a "screen" instance for the moment! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459397625 | |
https://github.com/simonw/datasette/issues/272#issuecomment-503369834 | https://api.github.com/repos/simonw/datasette/issues/272 | 503369834 | MDEyOklzc3VlQ29tbWVudDUwMzM2OTgzNA== | 9599 | 2019-06-19T01:26:24Z | 2019-06-19T01:26:24Z | OWNER | I need to be able to define the URL routes once and have them work for both Sanic and ASGI. I'm going to extract the web application bits out of the `Datasette` class into a `DatasetteServer` class. Then I can have a `add_route()` method on that class, then have `DatasetteSanic` and `DatasetteAsgi` subclasses which redefine that method. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/272#issuecomment-502393107 | https://api.github.com/repos/simonw/datasette/issues/272 | 502393107 | MDEyOklzc3VlQ29tbWVudDUwMjM5MzEwNw== | 9599 | 2019-06-15T19:25:54Z | 2019-06-19T01:20:14Z | OWNER | OK, time for a solid implementation plan. As soon as https://github.com/django/asgiref/pull/92 is merged (hopefully very soon) the ASGI spec will have support for an optional `raw_path` - which means we can continue to use `table%2Fnames` with embedded `/` without being unable to tell if a path has been decoded or not. Steps to implement: ## Refactor classes, then add .asgi() method to BaseView Add a `.asgi(self, scope, receive, send)` method to my base view class. This will expose an ASGI interface to the outside world: the method itself will construct a request object and call the existing `.get()` method. My only true shared base class is actually `RenderMixin` because the `IndexView` doesn't extend `BaseView`. I'm going to refactor the class hierarchy a bit here - `AsgiView` will be my top level class with the `.asgi()` method on it. `RenderMixin` will be renamed `BaseView(AsgiView)`, while existing `BaseView` will be renamed `DataView(BaseView)` since it mainly exists to introduce the handy `.data()` abstraction. So... * `AsgiView` - has `.asgi()` method, extends Sanic `HTTPMethodView` (for the moment) * `BaseView(AsgiView)` - defines utility methods currently on `RenderMixin` * `IndexView(BaseView)` - the current `IndexView` * `DataView(BaseView)` - defines the utilities currently on `BaseView`, including `data()` * Everything else subclasses `DataView` ## Extract routing logic out into a new `DatasetteView` I considered calling this `RouteView`, but one of the goals of this project is to allow other ASGI apps to import Datasette itself and reuse it as its own ASGI function. So `DatasetteView` will subclass `BaseView` and will do all of the routing logic. That logic currently lives here: https://github.com/simonw/datasette/blob/aa911122feab13f8e65875c98edb00fd3832b7b8/datasette/app.py#L594-L640 ## For tests: Implement a version of app_client.get() that calls ASGI instead Almost all of the unit tests currently use `app_client.get("/path...")`. I want to be able to r… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/272#issuecomment-503351966 | https://api.github.com/repos/simonw/datasette/issues/272 | 503351966 | MDEyOklzc3VlQ29tbWVudDUwMzM1MTk2Ng== | 9599 | 2019-06-18T23:45:17Z | 2019-06-18T23:45:17Z | OWNER | Uvicorn 0.8.1 is our and supports `raw_path`! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/513#issuecomment-503249999 | https://api.github.com/repos/simonw/datasette/issues/513 | 503249999 | MDEyOklzc3VlQ29tbWVudDUwMzI0OTk5OQ== | 7936571 | 2019-06-18T18:11:36Z | 2019-06-18T18:11:36Z | NONE | Ah, so basically put the SQLite databases on Linode, for example, and run `datasette serve` on there? I'm comfortable with that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
457201907 | |
https://github.com/simonw/datasette/issues/502#issuecomment-503237884 | https://api.github.com/repos/simonw/datasette/issues/502 | 503237884 | MDEyOklzc3VlQ29tbWVudDUwMzIzNzg4NA== | 7936571 | 2019-06-18T17:39:18Z | 2019-06-18T17:46:08Z | NONE | It appears that I cannot reopen this issue but the proposed solution did not solve it. The link is not there. I have full text search enabled for a bunch of tables in my database and even clicking the link to reveal hidden tables did not show the download DB link. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453131917 | |
https://github.com/simonw/datasette/issues/512#issuecomment-503236800 | https://api.github.com/repos/simonw/datasette/issues/512 | 503236800 | MDEyOklzc3VlQ29tbWVudDUwMzIzNjgwMA== | 7936571 | 2019-06-18T17:36:37Z | 2019-06-18T17:36:37Z | NONE | Oh I didn't know the `description` field could be used for a database's metadata. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
457147936 | |
https://github.com/simonw/datasette/issues/512#issuecomment-503200024 | https://api.github.com/repos/simonw/datasette/issues/512 | 503200024 | MDEyOklzc3VlQ29tbWVudDUwMzIwMDAyNA== | 9599 | 2019-06-18T15:58:18Z | 2019-06-18T15:58:18Z | OWNER | The `about` and `license` and `source` keys are all intended to be used for links - so if you provide a `about_url` it will be displayed as a URL, and you can then use the `about` key to customize the link label. There are `description` and `title` fields which can be used to display text without linking to anything. I'm definitely open to reconsidering how these work - I don't think they quite serve people's needs as they are right now, so suggestions for improving them would be very welcome. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
457147936 | |
https://github.com/simonw/datasette/issues/513#issuecomment-503199253 | https://api.github.com/repos/simonw/datasette/issues/513 | 503199253 | MDEyOklzc3VlQ29tbWVudDUwMzE5OTI1Mw== | 9599 | 2019-06-18T15:56:29Z | 2019-06-18T15:56:29Z | OWNER | Unfortunately not - I really wish this was possible. I have not yet found a great serverless solution for publishing 1GB+ databases - they're too big for Heroku, Cloud Run OR Zeit Now. Once databases get that big the only option I've found is to run a VPS (or an EC2 instance) with a mounted hard drive volume and execute `datasette serve` on that instance, with an nginx running on port 80 that proxies traffic back to Datasette. I'd love to figure out a way to make hosting larger databases as easy as it currently is to host small ones. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
457201907 | |
https://github.com/simonw/datasette/issues/272#issuecomment-503195217 | https://api.github.com/repos/simonw/datasette/issues/272 | 503195217 | MDEyOklzc3VlQ29tbWVudDUwMzE5NTIxNw== | 9599 | 2019-06-18T15:46:31Z | 2019-06-18T15:54:18Z | OWNER | How should file serving work? Starlette and Sanic both use `aiofiles` - https://github.com/Tinche/aiofiles - which is a small wrapper around file operations which runs them all in an executor thread. It doesn't have any C dependencies so it looks like a good option. [Quart uses it too](https://gitlab.com/pgjones/quart/blob/317562ea660edb7159efc20fa57b95223d408ea0/quart/wrappers/response.py#L122-169). `aiohttp` does things differently: it has [an implementation based on sendfile](https://github.com/aio-libs/aiohttp/blob/7a324fd46ff7dc9bb0bb1bc5afb326e04cf7cef0/aiohttp/web_fileresponse.py#L46-L122) with [an alternative fallback](https://github.com/aio-libs/aiohttp/blob/7a324fd46ff7dc9bb0bb1bc5afb326e04cf7cef0/aiohttp/web_fileresponse.py#L175-L200) which reads chunks from a file object and yields them one chunk at a time, | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/272#issuecomment-502466466 | https://api.github.com/repos/simonw/datasette/issues/272 | 502466466 | MDEyOklzc3VlQ29tbWVudDUwMjQ2NjQ2Ng== | 9599 | 2019-06-16T16:28:10Z | 2019-06-16T16:28:10Z | OWNER | I have an open pull request to Uvicorn with an implementation of `raw_path`: https://github.com/encode/uvicorn/pull/372 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/511#issuecomment-502420036 | https://api.github.com/repos/simonw/datasette/issues/511 | 502420036 | MDEyOklzc3VlQ29tbWVudDUwMjQyMDAzNg== | 9599 | 2019-06-16T04:51:39Z | 2019-06-16T04:52:14Z | OWNER | It looks like AppVeyor is the right solution for Windows CI. I've set up a Datasette project there but I do t yet have the configuration figured out (plus Datasette won't work on windows yet anyway): https://ci.appveyor.com/project/simonw/datasette | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
456578474 | |
https://github.com/simonw/datasette/issues/507#issuecomment-502402586 | https://api.github.com/repos/simonw/datasette/issues/507 | 502402586 | MDEyOklzc3VlQ29tbWVudDUwMjQwMjU4Ng== | 9599 | 2019-06-15T22:00:14Z | 2019-06-15T22:00:14Z | OWNER | I need to find a good tool for taking screenshots on OSX that are reliably the same dimension across multiple sessions. Maybe in Firefox? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455852801 | |
https://github.com/simonw/datasette/issues/187#issuecomment-502401636 | https://api.github.com/repos/simonw/datasette/issues/187 | 502401636 | MDEyOklzc3VlQ29tbWVudDUwMjQwMTYzNg== | 9599 | 2019-06-15T21:44:23Z | 2019-06-15T21:44:23Z | OWNER | I'm closing his as a duplicate of the new #511 - I hope to have his working very shortly as a result of #272 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
309033998 | |
https://github.com/simonw/datasette/issues/272#issuecomment-502401078 | https://api.github.com/repos/simonw/datasette/issues/272 | 502401078 | MDEyOklzc3VlQ29tbWVudDUwMjQwMTA3OA== | 9599 | 2019-06-15T21:35:26Z | 2019-06-15T21:35:26Z | OWNER | Started sketching out the router in the `asgi` branch: https://github.com/simonw/datasette/commit/7cdc55c6836fe246b1ca8a13a965a39991c9ffec | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/272#issuecomment-502395689 | https://api.github.com/repos/simonw/datasette/issues/272 | 502395689 | MDEyOklzc3VlQ29tbWVudDUwMjM5NTY4OQ== | 9599 | 2019-06-15T20:05:26Z | 2019-06-15T20:05:26Z | OWNER | For the routing component: I'm going to base my implementation on the one from Django Channels. https://github.com/django/channels/blob/507cb54fcb36df63282dd19653ea743036e7d63c/channels/routing.py#L123-L149 Documented here: https://channels.readthedocs.io/en/latest/topics/routing.html#urlrouter Particularly relevant: my view classes need access to the components that were already parsed out of the URL by the router. I'm going to copy the Django Channels mechanism of stashing those in `scope["url_route"]["kwargs"]`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/272#issuecomment-502394420 | https://api.github.com/repos/simonw/datasette/issues/272 | 502394420 | MDEyOklzc3VlQ29tbWVudDUwMjM5NDQyMA== | 9599 | 2019-06-15T19:45:46Z | 2019-06-15T19:45:46Z | OWNER | For reference, here's some WIP code I wrote last year against the old ASGI 2 spec: https://github.com/simonw/datasette/commit/4fd36ba2f3f91da7258859808616078e3464fb97 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/509#issuecomment-502393573 | https://api.github.com/repos/simonw/datasette/issues/509 | 502393573 | MDEyOklzc3VlQ29tbWVudDUwMjM5MzU3Mw== | 9599 | 2019-06-15T19:32:56Z | 2019-06-15T19:32:56Z | OWNER | Experimental exploratory patch: ```diff diff --git a/datasette/app.py b/datasette/app.py index 2ef7da4..ca51866 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -164,8 +164,10 @@ class Datasette: is_memory = True is_mutable = path not in self.immutables db = Database(self, path, is_mutable=is_mutable, is_memory=is_memory) + i = 1 if db.name in self.databases: - raise Exception("Multiple files with same stem: {}".format(db.name)) + db.stem = db.name + "-" + str(i) + i += 1 self.databases[db.name] = db self.cache_headers = cache_headers self.cors = cors diff --git a/datasette/database.py b/datasette/database.py index e491577..75c8681 100644 --- a/datasette/database.py +++ b/datasette/database.py @@ -14,6 +14,8 @@ from .inspect import inspect_hash class Database: + stem = None + def __init__(self, ds, path=None, is_mutable=False, is_memory=False): self.ds = ds self.path = path @@ -73,6 +75,8 @@ class Database: def name(self): if self.is_memory: return ":memory:" + elif self.stem: + return self.stem else: return Path(self.path).stem ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
456568880 | |
https://github.com/simonw/datasette/issues/272#issuecomment-502393267 | https://api.github.com/repos/simonw/datasette/issues/272 | 502393267 | MDEyOklzc3VlQ29tbWVudDUwMjM5MzI2Nw== | 9599 | 2019-06-15T19:28:27Z | 2019-06-15T19:28:27Z | OWNER | I'll probably revert 9fdb47ca952b93b7b60adddb965ea6642b1ff523 from https://github.com/simonw/datasette/issues/272#issuecomment-494192779 since I won't need it now that ASGI is getting `raw_path` support. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
324188953 | |
https://github.com/simonw/datasette/issues/111#issuecomment-502135123 | https://api.github.com/repos/simonw/datasette/issues/111 | 502135123 | MDEyOklzc3VlQ29tbWVudDUwMjEzNTEyMw== | 9599 | 2019-06-14T14:39:59Z | 2019-06-14T14:39:59Z | OWNER | This may be the feature that causes me to add dateutilas a dependency (so I can use dateutil.parser.parse) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274615452 | |
https://github.com/simonw/datasette/issues/111#issuecomment-502134699 | https://api.github.com/repos/simonw/datasette/issues/111 | 502134699 | MDEyOklzc3VlQ29tbWVudDUwMjEzNDY5OQ== | 9599 | 2019-06-14T14:38:58Z | 2019-06-14T14:38:58Z | OWNER | I think I'll just call it "updated" to avoid the ugly underscore. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274615452 | |
https://github.com/simonw/datasette/issues/111#issuecomment-502134167 | https://api.github.com/repos/simonw/datasette/issues/111 | 502134167 | MDEyOklzc3VlQ29tbWVudDUwMjEzNDE2Nw== | 9599 | 2019-06-14T14:37:35Z | 2019-06-14T14:37:35Z | OWNER | We have per-database and per-table metadata now. I think it's time to make this actually happen. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274615452 | |
https://github.com/simonw/datasette/issues/498#issuecomment-501903071 | https://api.github.com/repos/simonw/datasette/issues/498 | 501903071 | MDEyOklzc3VlQ29tbWVudDUwMTkwMzA3MQ== | 7936571 | 2019-06-13T22:35:06Z | 2019-06-13T22:35:06Z | NONE | I'd like to start working on this. I've made a custom template for `index.html` that contains a `form` that contains a search `input`. But I'm not sure where to go from here. When user enters a search term, I'd like for that term to go into a function I'll make that will search all tables with full text search enabled. Can I make additional custom Python scripts for this or must I edit datasette's files directly? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451513541 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501541902 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501541902 | MDEyOklzc3VlQ29tbWVudDUwMTU0MTkwMg== | 9599 | 2019-06-13T04:15:22Z | 2019-06-13T16:55:42Z | OWNER | So maybe something like this: ``` curl https://api.github.com/repos/simonw/datasette/pulls?state=all | \ sqlite-utils insert git.db pulls - \ --flatten=base \ --flatten=head \ --extract=user:users:id \ --extract=head_repo.license:licenses:key \ --extract=head_repo.owner:users \ --extract=head_repo --extract=base_repo.license:licenses:key \ --extract=base_repo.owner:users \ --extract=base_repo ``` Is the order of those nested `--extract` lines significant I wonder? It would be nice if the order didn't matter and the code figured out the right execution plan on its own. | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/24#issuecomment-501572149 | https://api.github.com/repos/simonw/sqlite-utils/issues/24 | 501572149 | MDEyOklzc3VlQ29tbWVudDUwMTU3MjE0OQ== | 9599 | 2019-06-13T06:47:17Z | 2019-06-13T06:47:17Z | OWNER | @IgnoredAmbience this is now shipped in sqlite-utils 1.2 - documentation here: * https://sqlite-utils.readthedocs.io/en/latest/python-api.html#python-api-defaults-not-null * https://sqlite-utils.readthedocs.io/en/latest/cli.html#cli-defaults-not-null | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449818897 | |
https://github.com/simonw/sqlite-utils/issues/25#issuecomment-501548676 | https://api.github.com/repos/simonw/sqlite-utils/issues/25 | 501548676 | MDEyOklzc3VlQ29tbWVudDUwMTU0ODY3Ng== | 9599 | 2019-06-13T04:58:12Z | 2019-06-13T04:58:12Z | OWNER | I'm going to reuse the `ForeignKey` named tuple here: https://github.com/simonw/sqlite-utils/blob/d645032cfa4edbccd0542eecdddca29edf9f7b07/sqlite_utils/db.py#L17-L19 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449848803 | |
https://github.com/simonw/sqlite-utils/issues/25#issuecomment-501548097 | https://api.github.com/repos/simonw/sqlite-utils/issues/25 | 501548097 | MDEyOklzc3VlQ29tbWVudDUwMTU0ODA5Nw== | 9599 | 2019-06-13T04:54:33Z | 2019-06-13T04:54:33Z | OWNER | Still need to add this mechanism to `.create_table()` - this code here is all that needs to be modified - it needs to learn to deal with the alternative syntax for foreign keys and guess the missing data if necessary: https://github.com/simonw/sqlite-utils/blob/d645032cfa4edbccd0542eecdddca29edf9f7b07/sqlite_utils/db.py#L115-L119 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449848803 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501543688 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501543688 | MDEyOklzc3VlQ29tbWVudDUwMTU0MzY4OA== | 9599 | 2019-06-13T04:26:15Z | 2019-06-13T04:26:15Z | OWNER | I may ignore `--flatten` for the moment - users can do their own flattening using `jq` if they need that. ``` curl https://api.github.com/repos/simonw/datasette/pulls?state=all | jq " [.[] | . + { base_label: .base.label, base_ref: .base.ref, base_sha: .base.sha, base_user: .base.user, base_repo: .base.repo, head_label: .head.label, head_ref: .head.ref, head_sha: .head.sha, head_user: .head.user, head_repo: .head.repo } | del(.base, .head, ._links)] " ``` Output: https://gist.github.com/simonw/2703ed43fcfe96eb8cfeee7b558b61e1 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501542025 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501542025 | MDEyOklzc3VlQ29tbWVudDUwMTU0MjAyNQ== | 9599 | 2019-06-13T04:16:10Z | 2019-06-13T04:16:42Z | OWNER | So for `--extract` the format is `path-to-property:table-to-extract-to:primary-key` If we find an array (as opposed to a direct nested object) at the end of the dotted path we do a m2m table. And if `primary-key` is omitted maybe we do the rowid thing with a foreign key back to ourselves. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501539452 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501539452 | MDEyOklzc3VlQ29tbWVudDUwMTUzOTQ1Mg== | 9599 | 2019-06-13T03:59:32Z | 2019-06-13T03:59:32Z | OWNER | Another complexity from the https://api.github.com/repos/simonw/datasette/pulls example: <img width="552" alt="Mozilla_Firefox" src="https://user-images.githubusercontent.com/9599/59402680-55eee680-8d54-11e9-9694-9443c8bbbf8a.png"> We don't actually want `head` and `base` to be pulled out into a separate table. Our ideal table design would probably look something like this: - `url`: ... - `id`: `285698310` - ... - `user_id`: `9599` -> refs `users` - `head_label`: `simonw:travis-38dev` - `head_ref`: `travis-38dev` - `head_sha`: `f274f9004302c5ca75ce89d0abfd648457957e31` - `head_user_id`: `9599` -> refs `users` - `head_repo_id`: `107914493` -> refs `repos` - `base_label`: `simonw:master` - `base_ref`: `master` - `base_sha`: `5e8fbf7f6fbc0b63d0479da3806dd9ccd6aaa945` - `base_user_id`: `9599` -> refs `users` - `base_repo_id`: `107914493` -> refs `repos` So the nested `head` and `base` sections here, instead of being extracted into another table, were flattened into their own columns. So perhaps we need a flatten-nested-into-columns mechanism which can be used in conjunction with a extract-to-tables mechanism. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501538100 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501538100 | MDEyOklzc3VlQ29tbWVudDUwMTUzODEwMA== | 9599 | 2019-06-13T03:51:27Z | 2019-06-13T03:51:27Z | OWNER | I like the term "extract" for what we are doing here, partly because that's the terminology I used in `csvs-to-sqlite`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501537812 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501537812 | MDEyOklzc3VlQ29tbWVudDUwMTUzNzgxMg== | 9599 | 2019-06-13T03:49:37Z | 2019-06-13T03:50:39Z | OWNER | There's an interesting difference here between nested objects with a primary-key style ID and nested objects without. If a nested object does not have a primary key, we could still shift it out to another table but it would need to be in a context where it has an automatic foreign key back to our current record. A good example of something where that would be useful is the `outageDevices` key in https://github.com/simonw/pge-outages/blob/d890d09ff6e2997948028528e06c82e1efe30365/pge-outages.json#L13-L25 ```json { "outageNumber": "407367", "outageStartTime": "1560355216", "crewCurrentStatus": "PG&E repair crew is on-site working to restore power.", "currentEtor": "1560376800", "cause": "Our preliminary determination is that your outage was caused by scheduled maintenance work.", "estCustAffected": "3", "lastUpdateTime": "1560355709", "hazardFlag": "0", "latitude": "37.35629", "longitude": "-119.70469", "outageDevices": [ { "latitude": "37.35409", "longitude": "-119.70575" }, { "latitude": "37.35463", "longitude": "-119.70525" }, { "latitude": "37.35562", "longitude": "-119.70467" } ], "regionName": "Ahwahnee" } ``` These could either be inserted into an `outageDevices` table that uses `rowid`... or we could have a mechanism where we automatically derive a primary key for them based on a hash of their data, hence avoiding creating duplicates even though we don't have a provided primary key. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501536495 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501536495 | MDEyOklzc3VlQ29tbWVudDUwMTUzNjQ5NQ== | 9599 | 2019-06-13T03:40:21Z | 2019-06-13T03:40:21Z | OWNER | I think I can do something here with a very simple `head.repo.owner` path syntax. Normally this kind of syntax would have to take the difference between dictionaries and lists into account but I don't think that matters here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/sqlite-utils/issues/25#issuecomment-501517535 | https://api.github.com/repos/simonw/sqlite-utils/issues/25 | 501517535 | MDEyOklzc3VlQ29tbWVudDUwMTUxNzUzNQ== | 9599 | 2019-06-13T01:50:34Z | 2019-06-13T01:50:34Z | OWNER | If I'm going to do this then I should make the `other_table` and `other_column` arguments optional here too: https://github.com/simonw/sqlite-utils/blob/2fed87da6ea990d295672e4db2c8ae97b787913e/sqlite_utils/cli.py#L201-L215 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449848803 | |
https://github.com/simonw/sqlite-utils/issues/25#issuecomment-501516797 | https://api.github.com/repos/simonw/sqlite-utils/issues/25 | 501516797 | MDEyOklzc3VlQ29tbWVudDUwMTUxNjc5Nw== | 9599 | 2019-06-13T01:46:36Z | 2019-06-13T01:47:35Z | OWNER | Maybe `foreign_keys` could even optionally just be a list of columns - it could then attempt to detect the related tables based on some rules-of-thumb and raise an error if there's no obvious candidate. Rules: * If the column name ends in `_id`, remove that suffix and look for a matching table. * Try for a table which is the column name without the `_id` suffix with an `s` appended to it * Try for a table that's the exact match for the column name If none of these rules match, raise an error. So the above example could be further simplified to: ```python db["usages"].insert_all( usages_to_insert, foreign_keys=["line_id", "definition_id"] ) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449848803 | |
https://github.com/simonw/sqlite-utils/issues/24#issuecomment-501516028 | https://api.github.com/repos/simonw/sqlite-utils/issues/24 | 501516028 | MDEyOklzc3VlQ29tbWVudDUwMTUxNjAyOA== | 9599 | 2019-06-13T01:42:36Z | 2019-06-13T01:42:36Z | OWNER | Maybe it's time to create a `sqlite-utils create-table` command here too, rather than forcing people to create tables only by inserting example data. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449818897 | |
https://github.com/simonw/sqlite-utils/issues/24#issuecomment-501515609 | https://api.github.com/repos/simonw/sqlite-utils/issues/24 | 501515609 | MDEyOklzc3VlQ29tbWVudDUwMTUxNTYwOQ== | 9599 | 2019-06-13T01:40:12Z | 2019-06-13T01:40:47Z | OWNER | But what to do for creating a table? For the Python function I could do this: ```python db["cats"].create({ "id": int, "name": str, "score": int, "weight": float, }, pk="id", not_null={"weight"}, defaults={"score": 1}) ``` The CLI tool only every creates tables as a side-effect of a `sqlite-utils insert` or `sqlite-utils upsert`. I can have them accept optional arguments, `--not-null colname` and `--default colname value`: ``` echo '{"name": "Cleo", "age": 4, "score": 2}' | \ sqlite-utils insert dogs.db dogs - \ --not-null age \ --not-null name \ --default score 1 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449818897 | |
https://github.com/simonw/sqlite-utils/issues/24#issuecomment-501514575 | https://api.github.com/repos/simonw/sqlite-utils/issues/24 | 501514575 | MDEyOklzc3VlQ29tbWVudDUwMTUxNDU3NQ== | 9599 | 2019-06-13T01:34:55Z | 2019-06-13T01:34:55Z | OWNER | Since you can't have one without the other, I'm going with `--not-null-default=` and `not_null_default=` for the add column versions of this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449818897 | |
https://github.com/simonw/sqlite-utils/issues/24#issuecomment-501509642 | https://api.github.com/repos/simonw/sqlite-utils/issues/24 | 501509642 | MDEyOklzc3VlQ29tbWVudDUwMTUwOTY0Mg== | 9599 | 2019-06-13T01:06:09Z | 2019-06-13T01:06:09Z | OWNER | Hmm... we need the ability to pass `--not-null` when we are creating a table as well. If you attempt to add `NOT NULL` to a column after a table has first been created you get this error: ```sqlite3.OperationalError: Cannot add a NOT NULL column with default value NULL``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449818897 | |
https://github.com/simonw/sqlite-utils/issues/26#issuecomment-501508302 | https://api.github.com/repos/simonw/sqlite-utils/issues/26 | 501508302 | MDEyOklzc3VlQ29tbWVudDUwMTUwODMwMg== | 9599 | 2019-06-13T00:57:52Z | 2019-06-13T00:57:52Z | OWNER | Two challenges here: 1. We need a way to specify which tables should be used - e.g. "put records from the `"user"` key in a `users` table, put multiple records from the `"labels"` key in a table called `labels`" (we can pick an automatic name for the m2m table, though it might be nice to have an option to customize it) 2. Should we deal with nested objects? Consider https://api.github.com/repos/simonw/datasette/pulls for example: <img width="620" alt="Mozilla_Firefox" src="https://user-images.githubusercontent.com/9599/59395956-73fc1d00-8d3b-11e9-8fbe-41a04ac8ff41.png"> Here we have `head.user` as a user, `head.repo` as a repo, and `head.repo.owner` as another user. Ideally our mechanism for specifying which table things should be pulled out into would handle this, but it's getting a bit complicated. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455486286 | |
https://github.com/simonw/datasette/issues/502#issuecomment-500975113 | https://api.github.com/repos/simonw/datasette/issues/502 | 500975113 | MDEyOklzc3VlQ29tbWVudDUwMDk3NTExMw== | 9599 | 2019-06-11T18:50:42Z | 2019-06-11T18:50:42Z | OWNER | There should be a "Download SQLite DB" link at the bottom of each database page - e.g. on https://latest.datasette.io/fixtures <img width="815" alt="fixtures" src="https://user-images.githubusercontent.com/9599/59298300-1bdff080-8c3f-11e9-8d75-b3eb6e1ae385.png"> Does that solve this? Re-open the issue if it doesn't! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453131917 | |
https://github.com/simonw/datasette/pull/500#issuecomment-500974354 | https://api.github.com/repos/simonw/datasette/issues/500 | 500974354 | MDEyOklzc3VlQ29tbWVudDUwMDk3NDM1NA== | 9599 | 2019-06-11T18:48:43Z | 2019-06-11T18:48:43Z | OWNER | Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451705509 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500903287 | https://api.github.com/repos/simonw/datasette/issues/506 | 500903287 | MDEyOklzc3VlQ29tbWVudDUwMDkwMzI4Nw== | 9599 | 2019-06-11T15:48:27Z | 2019-06-11T15:48:27Z | OWNER | Calling out to Tika does make me a little nervous, but that's why Datasette has plugins! A plugin that calls Tika (and caches the results) could be really interesting. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500238035 | https://api.github.com/repos/simonw/datasette/issues/506 | 500238035 | MDEyOklzc3VlQ29tbWVudDUwMDIzODAzNQ== | 1059677 | 2019-06-09T19:21:18Z | 2019-06-09T19:21:18Z | NONE | If you don't mind calling out to Java, then Apache Tika is able to tell you what a load of "binary stuff" is, plus render it to XHTML where possible. There's a python wrapper around the Apache Tika server, but for a more typical datasette usecase you'd probably just want to grab the Tika CLI jar, and call it with `--detect` and/or `--xhtml` to process the unknown binary blob | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500224864 | https://api.github.com/repos/simonw/datasette/issues/506 | 500224864 | MDEyOklzc3VlQ29tbWVudDUwMDIyNDg2NA== | 9599 | 2019-06-09T16:07:39Z | 2019-06-09T16:07:39Z | OWNER | Shipped 0.1 of the plugin! I'm pretty happy with this display format: <img width="543" alt="many-photos-tables__RKFaceCrop__58_rows" src="https://user-images.githubusercontent.com/9599/59161307-011e4800-8a96-11e9-9768-a168320d8979.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500220862 | https://api.github.com/repos/simonw/datasette/issues/506 | 500220862 | MDEyOklzc3VlQ29tbWVudDUwMDIyMDg2Mg== | 9599 | 2019-06-09T15:25:18Z | 2019-06-09T15:26:23Z | OWNER | I'm going to call this `datasette-render-binary`: https://github.com/simonw/datasette-render-binary | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500220646 | https://api.github.com/repos/simonw/datasette/issues/506 | 500220646 | MDEyOklzc3VlQ29tbWVudDUwMDIyMDY0Ng== | 9599 | 2019-06-09T15:22:12Z | 2019-06-09T15:22:12Z | OWNER | ![3C9CCDBA-F346-47CB-BFEC-964B0426E728](https://user-images.githubusercontent.com/9599/59160835-789ca900-8a8f-11e9-9767-0f50890d17fe.jpeg) New idea: show essentially this but differentiate the escape sequences in some way. Maybe wrap them in `<code>` or put the non-escape sequences in bold? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500183948 | https://api.github.com/repos/simonw/datasette/issues/506 | 500183948 | MDEyOklzc3VlQ29tbWVudDUwMDE4Mzk0OA== | 9599 | 2019-06-09T04:22:58Z | 2019-06-09T04:26:53Z | OWNER | This is quite nice: ``` $ od -c /tmp/Thumb64Segment_11.data | head -n 10 0000000 \0 \0 @ \0 \0 \0 005 5 X T S F \0 \0 \0 001 0000020 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 0010000 025 030 . 377 026 032 . 377 027 033 - 377 031 035 0 377 0010020 032 036 1 377 036 5 377 037 ! 8 377 036 " : 377 0010040 036 8 377 $ : 377 ! & ; 377 $ * ? 377 0010060 ' - ? 377 % * < 377 % , > 377 - 3 E 377 0010100 6 ; M 377 : @ O 377 = C R 377 @ G V 377 0010120 @ I X 377 < B Q 377 8 @ N 377 8 @ P 377 0010140 : C T 377 ; C U 377 : C V 377 9 C W 377 ``` Here's a rough Python equivalent http://code.activestate.com/recipes/579120-data_dumppy-like-the-unix-od-octal-dump-command/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500183106 | https://api.github.com/repos/simonw/datasette/issues/506 | 500183106 | MDEyOklzc3VlQ29tbWVudDUwMDE4MzEwNg== | 9599 | 2019-06-09T03:59:12Z | 2019-06-09T03:59:12Z | OWNER | Another cheap trick is the equivalent of the Unix `strings` command - https://stackoverflow.com/questions/6804582/extract-strings-from-a-binary-file-in-python | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500183002 | https://api.github.com/repos/simonw/datasette/issues/506 | 500183002 | MDEyOklzc3VlQ29tbWVudDUwMDE4MzAwMg== | 9599 | 2019-06-09T03:56:06Z | 2019-06-09T03:56:06Z | OWNER | What are some other interesting tricks we can use to make binary data a bit more interesting to look at? https://martin.varela.fi/2017/09/09/simple-binary-data-visualization/ has some really clever visualization tricks - probably a bit much for this plugin though. See also https://codisec.com/binary-visualization-explained/ https://github.com/tryexceptpass/perceptio is some much simpler code for rendering an image for a binary. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500173881 | https://api.github.com/repos/simonw/datasette/issues/506 | 500173881 | MDEyOklzc3VlQ29tbWVudDUwMDE3Mzg4MQ== | 9599 | 2019-06-08T23:55:52Z | 2019-06-08T23:55:52Z | OWNER | Possible plugin direction: ```python import filetype # https://pypi.org/project/filetype/ @hookimpl(trylast=True) def render_cell(value): if isinstance(value, bytes): info = repr(value) # May still want to truncate this on table view (but not on row page) guess = filetype.guess(value) if guess is not None: # Need jinja2 markup here for \n to display info = "Guess: mime={}, extension={}\n\n{}".format( guess.mime, guess.extension, info ) return info return None ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/506#issuecomment-500173464 | https://api.github.com/repos/simonw/datasette/issues/506 | 500173464 | MDEyOklzc3VlQ29tbWVudDUwMDE3MzQ2NA== | 9599 | 2019-06-08T23:44:59Z | 2019-06-08T23:44:59Z | OWNER | This could also be handled by a plugin. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453846217 | |
https://github.com/simonw/datasette/issues/503#issuecomment-500145486 | https://api.github.com/repos/simonw/datasette/issues/503 | 500145486 | MDEyOklzc3VlQ29tbWVudDUwMDE0NTQ4Ng== | 9599 | 2019-06-08T17:52:38Z | 2019-06-08T17:52:38Z | OWNER | This should definitely work - hopefully not a big fix. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
453243459 | |
https://github.com/simonw/datasette/issues/394#issuecomment-499923145 | https://api.github.com/repos/simonw/datasette/issues/394 | 499923145 | MDEyOklzc3VlQ29tbWVudDQ5OTkyMzE0NQ== | 13896256 | 2019-06-07T15:10:57Z | 2019-06-07T15:11:07Z | CONTRIBUTOR | Putting this here in case anyone else encounters the same issue with nginx, I was able to resolve it by passing the header in the nginx proxy config (i.e., `proxy_set_header Host $host`). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
396212021 | |
https://github.com/simonw/sqlite-utils/issues/24#issuecomment-499603495 | https://api.github.com/repos/simonw/sqlite-utils/issues/24 | 499603495 | MDEyOklzc3VlQ29tbWVudDQ5OTYwMzQ5NQ== | 9599 | 2019-06-06T18:02:20Z | 2019-06-06T18:02:20Z | OWNER | This seems reasonable. It could look like this: ```python db["dogs"].add_column("instagram", str, not_null=True) db["dogs"].add_column("is_good_dog", bool, default=True) ``` And this: ```bash $ sqlite-utils add-column mydb.db dogs instagram --not-null $ sqlite-utils add-column mydb.db dogs is_good_dog integer --default=1 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449818897 | |
https://github.com/simonw/datasette/issues/394#issuecomment-499320973 | https://api.github.com/repos/simonw/datasette/issues/394 | 499320973 | MDEyOklzc3VlQ29tbWVudDQ5OTMyMDk3Mw== | 13896256 | 2019-06-06T02:07:59Z | 2019-06-06T02:07:59Z | CONTRIBUTOR | Hey was this ever merged? Trying to run this behind nginx, and encountering this issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
396212021 | |
https://github.com/simonw/datasette/issues/498#issuecomment-499262397 | https://api.github.com/repos/simonw/datasette/issues/498 | 499262397 | MDEyOklzc3VlQ29tbWVudDQ5OTI2MjM5Nw== | 7936571 | 2019-06-05T21:28:32Z | 2019-06-05T21:28:32Z | NONE | Thinking about this more, I'd probably have to make a template page to go along with this, right? I'm guessing there's no way to add an all-databases-all-tables search to datasette's "home page" except by copying the "home page" template and editing it? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451513541 | |
https://github.com/simonw/datasette/issues/499#issuecomment-499260727 | https://api.github.com/repos/simonw/datasette/issues/499 | 499260727 | MDEyOklzc3VlQ29tbWVudDQ5OTI2MDcyNw== | 7936571 | 2019-06-05T21:22:55Z | 2019-06-05T21:22:55Z | NONE | I was thinking of having some kind of GUI in which regular reporters can upload a CSV and choose how to name the tables, columns and whatnot. Maybe it's possible to make such a GUI using Jinja template language? I ask because I'm unsure how to pursue this but I'd like to try. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451585764 | |
https://github.com/simonw/datasette/issues/499#issuecomment-498840129 | https://api.github.com/repos/simonw/datasette/issues/499 | 498840129 | MDEyOklzc3VlQ29tbWVudDQ5ODg0MDEyOQ== | 9599 | 2019-06-04T20:55:30Z | 2019-06-04T21:01:22Z | OWNER | I really want this too! It's one of the goals of the Datasette Library #417 concept, which I'm hoping to turn into an actual feature in the coming months. It's also going to be a major focus of my ten month JSK fellowship at Stanford, which starts in September. https://twitter.com/simonw/status/1123624552867565569 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451585764 | |
https://github.com/simonw/datasette/issues/499#issuecomment-498840597 | https://api.github.com/repos/simonw/datasette/issues/499 | 498840597 | MDEyOklzc3VlQ29tbWVudDQ5ODg0MDU5Nw== | 9599 | 2019-06-04T20:56:53Z | 2019-06-04T20:56:53Z | OWNER | Have you played with Datasette on Glitch yet? https://simonwillison.net/2019/Apr/23/datasette-glitch/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451585764 | |
https://github.com/simonw/datasette/issues/498#issuecomment-498839428 | https://api.github.com/repos/simonw/datasette/issues/498 | 498839428 | MDEyOklzc3VlQ29tbWVudDQ5ODgzOTQyOA== | 9599 | 2019-06-04T20:53:21Z | 2019-06-04T20:53:21Z | OWNER | It does not, but that's a really great idea for a feature. One challenge here is that FTS ranking calculations take overall table statistics into account, which means it's usually not possible to combine rankings from different tables in a sensible way. But that doesn't mean it's not possible to return grouped results. I think this makes a lot of sense as a plugin. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
451513541 | |
https://github.com/simonw/datasette/issues/446#issuecomment-498838841 | https://api.github.com/repos/simonw/datasette/issues/446 | 498838841 | MDEyOklzc3VlQ29tbWVudDQ5ODgzODg0MQ== | 9599 | 2019-06-04T20:51:37Z | 2019-06-04T20:51:37Z | OWNER | So the Python 3.7 way to do this would be dataclasses, but I still want to support 3.5. This excellent essay has made me think attrs might be the way to go: https://www.revsys.com/tidbits/dataclasses-and-attrs-when-and-why/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
440134714 | |
https://github.com/simonw/datasette/issues/496#issuecomment-497885590 | https://api.github.com/repos/simonw/datasette/issues/496 | 497885590 | MDEyOklzc3VlQ29tbWVudDQ5Nzg4NTU5MA== | 1740337 | 2019-05-31T23:05:05Z | 2019-05-31T23:05:05Z | NONE | Upon doing a "fix" which allowed a longer build timeout the cloudrun container was too slow when it actually ran. So I would say if your sqlite database is over 1 GB heroku and cloudrun are not good options. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
450862577 | |
https://github.com/simonw/datasette/issues/495#issuecomment-497194179 | https://api.github.com/repos/simonw/datasette/issues/495 | 497194179 | MDEyOklzc3VlQ29tbWVudDQ5NzE5NDE3OQ== | 9599 | 2019-05-30T04:04:01Z | 2019-05-30T04:04:01Z | OWNER | I think there are a few steps to solving this: * Define a variant of `?_facet_m2m={..}` that takes a JSON object which specifies which middle and destination table should be used, and via which columns * Teach the Facet `ManyToManyFacet.suggest()` method to check for multiple conflicting m2m relationships and use the new JSON URL for them. Also pick non-conflicting display names for those facets. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
450032134 | |
https://github.com/simonw/datasette/issues/485#issuecomment-497116074 | https://api.github.com/repos/simonw/datasette/issues/485 | 497116074 | MDEyOklzc3VlQ29tbWVudDQ5NzExNjA3NA== | 9599 | 2019-05-29T21:29:16Z | 2019-05-29T21:29:16Z | OWNER | Another good rule of thumb: look for text fields with a unique constraint? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/492#issuecomment-496970219 | https://api.github.com/repos/simonw/datasette/issues/492 | 496970219 | MDEyOklzc3VlQ29tbWVudDQ5Njk3MDIxOQ== | 9599 | 2019-05-29T14:50:10Z | 2019-05-29T14:50:10Z | OWNER | Code in question: https://github.com/simonw/datasette/blob/2a4b892d6c0f6609ea48df0dc393397af2b7b1c1/datasette/templates/table.html#L84-L86 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449854604 | |
https://github.com/simonw/datasette/issues/120#issuecomment-496966227 | https://api.github.com/repos/simonw/datasette/issues/120 | 496966227 | MDEyOklzc3VlQ29tbWVudDQ5Njk2NjIyNw== | 26342344 | 2019-05-29T14:40:52Z | 2019-05-29T14:40:52Z | NONE | I would really like this. If you give me some pointers @simonw I'm willing to PR! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
275087397 | |
https://github.com/simonw/sqlite-utils/issues/16#issuecomment-496787958 | https://api.github.com/repos/simonw/sqlite-utils/issues/16 | 496787958 | MDEyOklzc3VlQ29tbWVudDQ5Njc4Nzk1OA== | 9599 | 2019-05-29T05:17:58Z | 2019-05-29T05:17:58Z | OWNER | Shipped this feature in sqlite-utils 1.1: https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v1-1 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
413867537 | |
https://github.com/simonw/sqlite-utils/issues/21#issuecomment-496786354 | https://api.github.com/repos/simonw/sqlite-utils/issues/21 | 496786354 | MDEyOklzc3VlQ29tbWVudDQ5Njc4NjM1NA== | 9599 | 2019-05-29T05:09:01Z | 2019-05-29T05:09:01Z | OWNER | Shipped this feature in sqlite-utils 1.1: https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v1-1 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
448391492 | |
https://github.com/simonw/sqlite-utils/issues/16#issuecomment-496757010 | https://api.github.com/repos/simonw/sqlite-utils/issues/16 | 496757010 | MDEyOklzc3VlQ29tbWVudDQ5Njc1NzAxMA== | 9599 | 2019-05-29T02:16:17Z | 2019-05-29T02:16:17Z | OWNER | In that case `--fk othertable` detects the primary key. If you want to point at a different column, use `--fk-col X` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
413867537 | |
https://github.com/simonw/sqlite-utils/issues/16#issuecomment-496756758 | https://api.github.com/repos/simonw/sqlite-utils/issues/16 | 496756758 | MDEyOklzc3VlQ29tbWVudDQ5Njc1Njc1OA== | 9599 | 2019-05-29T02:14:50Z | 2019-05-29T02:14:50Z | OWNER | Hmm... probably not: > For options, only a fixed number of arguments is supported. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
413867537 | |
https://github.com/simonw/sqlite-utils/issues/16#issuecomment-496756348 | https://api.github.com/repos/simonw/sqlite-utils/issues/16 | 496756348 | MDEyOklzc3VlQ29tbWVudDQ5Njc1NjM0OA== | 9599 | 2019-05-29T02:12:43Z | 2019-05-29T02:12:43Z | OWNER | For the CLI I'm thinking: sqlite-utils add-column data.db mytable col text --fk othertable id So the foreign key is added using `--fk othertable othercolumn` If the user doesn't provide the column type we can derive it from the foreign key. Can we make the `othercolumn` optional? That would be neat - then we could automatically detect it using the sole primary key of the referenced table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
413867537 | |
https://github.com/simonw/sqlite-utils/issues/21#issuecomment-496665669 | https://api.github.com/repos/simonw/sqlite-utils/issues/21 | 496665669 | MDEyOklzc3VlQ29tbWVudDQ5NjY2NTY2OQ== | 9599 | 2019-05-28T19:57:47Z | 2019-05-28T19:57:47Z | OWNER | https://www.sqlite.org/lang_insert.html Looks like it's as simple as `INSERT OR IGNORE` - and we already have code related to that here: https://github.com/simonw/sqlite-utils/blob/1e28eeee8ce55ea68eddb228294a1eff6785b497/sqlite_utils/db.py#L498-L501 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
448391492 | |
https://github.com/simonw/datasette/issues/485#issuecomment-496367866 | https://api.github.com/repos/simonw/datasette/issues/485 | 496367866 | MDEyOklzc3VlQ29tbWVudDQ5NjM2Nzg2Ng== | 9599 | 2019-05-28T05:14:06Z | 2019-05-28T05:14:06Z | OWNER | I'm going to generate statistics for every TEXT column. Any column with more than 90% distinct rows (compared to the total count of rows) will be a candidate for the label. I will then pick the candidate column with the shortest average length. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/489#issuecomment-496345660 | https://api.github.com/repos/simonw/datasette/issues/489 | 496345660 | MDEyOklzc3VlQ29tbWVudDQ5NjM0NTY2MA== | 9599 | 2019-05-28T02:48:57Z | 2019-05-28T02:48:57Z | OWNER | Fixed: https://latest.datasette.io/fixtures/roadside_attraction_characteristics?_sort=attraction_id&_size=2 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
448977444 | |
https://github.com/simonw/datasette/issues/473#issuecomment-496339819 | https://api.github.com/repos/simonw/datasette/issues/473 | 496339819 | MDEyOklzc3VlQ29tbWVudDQ5NjMzOTgxOQ== | 9599 | 2019-05-28T02:13:44Z | 2019-05-28T02:13:44Z | OWNER | I'm leaning towards supporting both hooks. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
445850934 | |
https://github.com/simonw/datasette/issues/473#issuecomment-496339574 | https://api.github.com/repos/simonw/datasette/issues/473 | 496339574 | MDEyOklzc3VlQ29tbWVudDQ5NjMzOTU3NA== | 9599 | 2019-05-28T02:12:13Z | 2019-05-28T02:13:07Z | OWNER | Here's an older version of what that custom table filtering plugin might look like: https://github.com/simonw/datasette/commit/5116c4ec8aed5091e1f75415424b80f613518dc6 ```python from datasette.utils import TableFilter @hookimpl def table_filter(): async def inner(view, name, table, request): extra_human_descriptions = [] where_clauses = [] params = {} # ... build those things here return TableFilter( human_description_extras=extra_human_descriptions, where_clauses=where_clauses, params=params, ) return inner ``` I built this for the https://github.com/simonw/russian-ira-facebook-ads-datasette project. It's pretty neat. Maybe I should go with that? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
445850934 | |
https://github.com/simonw/datasette/issues/473#issuecomment-496338808 | https://api.github.com/repos/simonw/datasette/issues/473 | 496338808 | MDEyOklzc3VlQ29tbWVudDQ5NjMzODgwOA== | 9599 | 2019-05-28T02:07:23Z | 2019-05-28T02:07:23Z | OWNER | Assuming I do go ahead with this plugin hook, the existing `InFilter` makes for a nice simple example that illustrates the two key methods: `.where_clause()` and `.human_clause()`: ```python class InFilter(Filter): key = "in" display = "in" def split_value(self, value): if value.startswith("["): return json.loads(value) else: return [v.strip() for v in value.split(",")] def where_clause(self, table, column, value, param_counter): values = self.split_value(value) params = [":p{}".format(param_counter + i) for i in range(len(values))] sql = "{} in ({})".format(escape_sqlite(column), ", ".join(params)) return sql, values def human_clause(self, column, value): return "{} in {}".format(column, json.dumps(self.split_value(value))) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
445850934 | |
https://github.com/simonw/datasette/issues/473#issuecomment-496338666 | https://api.github.com/repos/simonw/datasette/issues/473 | 496338666 | MDEyOklzc3VlQ29tbWVudDQ5NjMzODY2Ng== | 9599 | 2019-05-28T02:06:23Z | 2019-05-28T02:06:23Z | OWNER | I'm having trouble coming up with interesting column-based filters which don't make sense to ship as default behaviour. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
445850934 | |
https://github.com/simonw/datasette/issues/473#issuecomment-496338533 | https://api.github.com/repos/simonw/datasette/issues/473 | 496338533 | MDEyOklzc3VlQ29tbWVudDQ5NjMzODUzMw== | 9599 | 2019-05-28T02:05:39Z | 2019-05-28T02:05:39Z | OWNER | I wonder if this is the right hook? The more likely case is that we need a hook that registers a new type of lookup entirely - `?_spatial_within={geojson}` for example. These aren't necessarily attached to a specific column and may have entirely custom syntax - but they still need to be able to influence the query (like the `_where=` clause does) and the human-readable description of the page. Is there a strong case for supporting both custom filter plugins AND custom table where plugins, or could those where plugins cover both bases? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
445850934 | |
https://github.com/simonw/datasette/issues/473#issuecomment-496337533 | https://api.github.com/repos/simonw/datasette/issues/473 | 496337533 | MDEyOklzc3VlQ29tbWVudDQ5NjMzNzUzMw== | 9599 | 2019-05-28T01:59:19Z | 2019-05-28T01:59:19Z | OWNER | It would be nice if this plugin was passed the current database/table so it can decide to enable new filters only for specific tables. This will require a bit of refactoring because the filters list is static at the moment - it would instead have to be returned by a function that runs when the table view is rendered. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
445850934 | |
https://github.com/simonw/datasette/issues/490#issuecomment-496322262 | https://api.github.com/repos/simonw/datasette/issues/490 | 496322262 | MDEyOklzc3VlQ29tbWVudDQ5NjMyMjI2Mg== | 9599 | 2019-05-27T23:51:12Z | 2019-05-27T23:51:12Z | OWNER | I'll call it `QueryInterrupted` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
448978907 | |
https://github.com/simonw/datasette/issues/485#issuecomment-496283728 | https://api.github.com/repos/simonw/datasette/issues/485 | 496283728 | MDEyOklzc3VlQ29tbWVudDQ5NjI4MzcyOA== | 9599 | 2019-05-27T18:44:07Z | 2019-05-27T18:44:07Z | OWNER | This code now lives in a method on the new `datasette.database.Database` class, which should make it easier to write unit tests for. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/487#issuecomment-496077220 | https://api.github.com/repos/simonw/datasette/issues/487 | 496077220 | MDEyOklzc3VlQ29tbWVudDQ5NjA3NzIyMA== | 9599 | 2019-05-27T04:56:14Z | 2019-05-27T04:56:14Z | OWNER | I'm going to rename `ConnectedDatabase` to just `Database` and move it out of the `app.py` module too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
448664792 | |
https://github.com/simonw/datasette/issues/485#issuecomment-496039483 | https://api.github.com/repos/simonw/datasette/issues/485 | 496039483 | MDEyOklzc3VlQ29tbWVudDQ5NjAzOTQ4Mw== | 9599 | 2019-05-26T23:22:53Z | 2019-05-26T23:22:53Z | OWNER | Comparing these two SQL queries (the one with union and the one without) using explain: With union: https://latest.datasette.io/fixtures?sql=explain+select+%27name%27+as+column%2C+count+%28distinct+name%29+as+count_distinct%2C+avg%28length%28name%29%29+as+avg_length+from+roadside_attractions%0D%0A++union%0D%0Aselect+%27address%27+as+column%2C+count%28distinct+address%29+as+count_distinct%2C+avg%28length%28address%29%29+as+avg_length+from+roadside_attractions produces 52 rows Without union: https://latest.datasette.io/fixtures?sql=explain+select%0D%0A++count+(distinct+name)+as+count_distinct_column_1%2C%0D%0A++avg(length(name))+as+avg_length_column_1%2C%0D%0A++count(distinct+address)+as+count_distinct_column_2%2C%0D%0A++avg(length(address))+as+avg_length_column_2%0D%0Afrom+roadside_attractions produces 32 rows So I'm going to use the one without the union. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/485#issuecomment-496039267 | https://api.github.com/repos/simonw/datasette/issues/485 | 496039267 | MDEyOklzc3VlQ29tbWVudDQ5NjAzOTI2Nw== | 9599 | 2019-05-26T23:19:38Z | 2019-05-26T23:20:10Z | OWNER | Thinking about that union query: I imagine doing this with union could encourage multiple full table scans. Maybe this query would only do one? https://latest.datasette.io/fixtures?sql=select%0D%0A++count+%28distinct+name%29+as+count_distinct_column_1%2C%0D%0A++avg%28length%28name%29%29+as+avg_length_column_1%2C%0D%0A++count%28distinct+address%29+as+count_distinct_column_2%2C%0D%0A++avg%28length%28address%29%29+as+avg_length_column_2%0D%0Afrom+roadside_attractions ``` select count (distinct name) as count_distinct_column_1, avg(length(name)) as avg_length_column_1, count(distinct address) as count_distinct_column_2, avg(length(address)) as avg_length_column_2 from roadside_attractions ``` <img width="800" alt="fixtures__select_count__distinct_name__as_count_distinct_column_1__avg_length_name___as_avg_length_column_1__count_distinct_address__as_count_distinct_column_2__avg_length_address___as_avg_length_column_2_from_roadside_attractions" src="https://user-images.githubusercontent.com/9599/58388316-201ad580-7fd2-11e9-95c3-c98e2758fc1e.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/485#issuecomment-495085021 | https://api.github.com/repos/simonw/datasette/issues/485 | 495085021 | MDEyOklzc3VlQ29tbWVudDQ5NTA4NTAyMQ== | 9599 | 2019-05-23T06:27:57Z | 2019-05-26T23:15:51Z | OWNER | I could attempt to calculate the statistics needed for this in a time limited SQL query something like this one: https://latest.datasette.io/fixtures?sql=select+%27name%27+as+column%2C+count+%28distinct+name%29+as+count_distinct%2C+avg%28length%28name%29%29+as+avg_length+from+roadside_attractions%0D%0A++union%0D%0Aselect+%27address%27+as+column%2C+count%28distinct+address%29+as+count_distinct%2C+avg%28length%28address%29%29+as+avg_length+from+roadside_attractions ``` select 'name' as column, count (distinct name) as count_distinct, avg(length(name)) as avg_length from roadside_attractions union select 'address' as column, count(distinct address) as count_distinct, avg(length(address)) as avg_length from roadside_attractions ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/485#issuecomment-496038601 | https://api.github.com/repos/simonw/datasette/issues/485 | 496038601 | MDEyOklzc3VlQ29tbWVudDQ5NjAzODYwMQ== | 9599 | 2019-05-26T23:08:41Z | 2019-05-26T23:08:41Z | OWNER | The code currently assumes the primary key is called "id" or "pk" - improving it to detect the primary key using database introspection should work much better. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
447469253 | |
https://github.com/simonw/datasette/issues/356#issuecomment-495931140 | https://api.github.com/repos/simonw/datasette/issues/356 | 495931140 | MDEyOklzc3VlQ29tbWVudDQ5NTkzMTE0MA== | 9599 | 2019-05-25T16:30:59Z | 2019-05-25T16:30:59Z | OWNER | I went with a much more simple URL scheme: `?_facet_m2m=destination_table` (it then figures out which the middle table is by looking at the foreign keys). This can be extended to be more complicated in the future if needed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
346028655 | |
https://github.com/simonw/datasette/issues/356#issuecomment-495931078 | https://api.github.com/repos/simonw/datasette/issues/356 | 495931078 | MDEyOklzc3VlQ29tbWVudDQ5NTkzMTA3OA== | 9599 | 2019-05-25T16:30:09Z | 2019-05-25T16:30:09Z | OWNER | Implemented in https://github.com/simonw/datasette/commit/d923d847545e829bf946bb9170bebfc7c3f9d993 Documentation here: https://datasette.readthedocs.io/en/latest/facets.html#facet-by-many-to-many Demo here: https://latest.datasette.io/fixtures/roadside_attractions | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
346028655 | |
https://github.com/simonw/datasette/pull/365#issuecomment-495929743 | https://api.github.com/repos/simonw/datasette/issues/365 | 495929743 | MDEyOklzc3VlQ29tbWVudDQ5NTkyOTc0Mw== | 9599 | 2019-05-25T16:09:54Z | 2019-05-25T16:09:54Z | OWNER | That last commit referenced the wrong ticket - it was meant to reference #356 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
361764460 | |
https://github.com/simonw/sqlite-utils/issues/22#issuecomment-495823219 | https://api.github.com/repos/simonw/sqlite-utils/issues/22 | 495823219 | MDEyOklzc3VlQ29tbWVudDQ5NTgyMzIxOQ== | 9599 | 2019-05-25T01:18:27Z | 2019-05-25T01:18:27Z | OWNER | https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v1-0 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
448395665 |