github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1150#issuecomment-747754229 | https://api.github.com/repos/simonw/datasette/issues/1150 | 747754229 | MDEyOklzc3VlQ29tbWVudDc0Nzc1NDIyOQ== | 9599 | 2020-12-17T23:04:38Z | 2020-12-17T23:04:38Z | OWNER | Open question: will this work for hundreds of database files, or is the overhead of connecting to each of 100 databases in turn to run `PRAGMA schema_version` too high? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
770436876 | |
https://github.com/simonw/datasette/issues/1150#issuecomment-747754082 | https://api.github.com/repos/simonw/datasette/issues/1150 | 747754082 | MDEyOklzc3VlQ29tbWVudDc0Nzc1NDA4Mg== | 9599 | 2020-12-17T23:04:13Z | 2020-12-17T23:04:13Z | OWNER | Pages that need a list of all databases - the index page and /-/databases for example - could trigger a "check for new directories in the configured directories" scan. That scan would run at most once every 5 (n) seconds - the check is triggered if it’s run more recently than that it doesn’t run. Hopefully this means it could be done as a blocking operation, rather than trying to run it in a thread. When it runs it scans for *.db or *.sqlite files (maybe one or two other extensions) that it hasn’t seen before. It also checks that the existing list of known database files still exists. If it finds any new ones it connects to them once to run `.schema`. It also runs `PRAGMA schema_version` on each known database so that it can compare the schema version number to the last one it saw. That's how it detects if there are new tables or if the cached schema needs to be updated. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
770436876 | |
https://github.com/simonw/datasette/issues/461#issuecomment-747734273 | https://api.github.com/repos/simonw/datasette/issues/461 | 747734273 | MDEyOklzc3VlQ29tbWVudDc0NzczNDI3Mw== | 9599 | 2020-12-17T22:14:46Z | 2020-12-17T22:14:46Z | OWNER | I've been thinking about this a bunch. For Datasette to be useful as a private repository of data (Datasette Library, #417) it's crucial that it can handle a much, much larger number of databases. This makes me worry about how many connections (and open file handles) it makes sense to have open at one time. I realize now that this is much less of a problem for private instances. Public instances on the internet could get traffic to any database at any time, so connections could easily get out of control. A private instance with only a few users could instead get away with only opening connections to databases in "active use". This does however make it even more important for Datasette to maintain a cached set of metadata about the tables - which is also needed to power this feature. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
443021509 | |
https://github.com/simonw/datasette/issues/1005#issuecomment-747209115 | https://api.github.com/repos/simonw/datasette/issues/1005 | 747209115 | MDEyOklzc3VlQ29tbWVudDc0NzIwOTExNQ== | 9599 | 2020-12-17T05:11:04Z | 2020-12-17T05:11:04Z | OWNER | Tracking ticket for the next HTTPX release is https://github.com/encode/httpx/pull/1403 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718259202 | |
https://github.com/simonw/datasette/issues/741#issuecomment-747208543 | https://api.github.com/repos/simonw/datasette/issues/741 | 747208543 | MDEyOklzc3VlQ29tbWVudDc0NzIwODU0Mw== | 9599 | 2020-12-17T05:09:03Z | 2020-12-17T05:09:03Z | OWNER | I really like this in `datasette-publish-vercel` - I'm definitely going to bring this to the other publish implementations as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
607223136 | |
https://github.com/simonw/datasette/issues/1149#issuecomment-747207787 | https://api.github.com/repos/simonw/datasette/issues/1149 | 747207787 | MDEyOklzc3VlQ29tbWVudDc0NzIwNzc4Nw== | 9599 | 2020-12-17T05:06:16Z | 2020-12-17T05:06:16Z | OWNER | So, an idea: what if Datasette's default CSS applied only to elements with classes - or maybe to childen of a `body class="datasette"` element? In such a way that you could write your own custom HTML that reused elements of Datasette's CSS - the cog menu styling for example - but only on an opt-in basis? | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
769520939 | |
https://github.com/simonw/datasette/issues/1149#issuecomment-747207487 | https://api.github.com/repos/simonw/datasette/issues/1149 | 747207487 | MDEyOklzc3VlQ29tbWVudDc0NzIwNzQ4Nw== | 9599 | 2020-12-17T05:05:08Z | 2020-12-17T05:05:08Z | OWNER | I think what I want is for it to be easy to reuse portions of Datasette's CSS - the bit that styles the cog menu for example - without pulling in the whole thing. I tried linking in the `<link rel="stylesheet" href="/-/static/app.css">` stylesheet and the page broke, wildly: <img width="1030" alt="content__categories__3_rows_and_Datasette_search__pytest" src="https://user-images.githubusercontent.com/9599/102446222-2290b600-3fe2-11eb-9d3a-e32ca72d1dac.png"> That's because Datasette's [built-in CSS](https://github.com/simonw/datasette/blob/0.53/datasette/static/app.css) applies styles directly to a whole bunch of different tags - `body`, `header`, `footer` etc - which means that if you import that stylesheet it can play havoc with the site you have already built. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
769520939 | |
https://github.com/simonw/datasette/issues/675#issuecomment-747070709 | https://api.github.com/repos/simonw/datasette/issues/675 | 747070709 | MDEyOklzc3VlQ29tbWVudDc0NzA3MDcwOQ== | 9599 | 2020-12-16T22:09:15Z | 2020-12-16T22:09:15Z | OWNER | The other way this could work is passing a single argument - the file (or directory) to be copied in - and assuming it should always go in the `/app` root. Something like: datasette publish cloudrun my.db --include src/ --include dogsheep-beta.yml Which would add `/app/src/...` and `/app/dogsheep-beta.yml`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
567902704 | |
https://github.com/simonw/datasette/issues/675#issuecomment-747068624 | https://api.github.com/repos/simonw/datasette/issues/675 | 747068624 | MDEyOklzc3VlQ29tbWVudDc0NzA2ODYyNA== | 9599 | 2020-12-16T22:04:42Z | 2020-12-16T22:04:42Z | OWNER | I can't just use `COPY /path/to/blah.yml /app` in the `Dockerfile` because it runs on the Google Cloud Build servers, not on the user's laptop - so I need to first copy the files they specify to that temporary directory that gets uploaded to the cloud, then rewrite the `COPY` lines in the `Dockerfile` to copy from there. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
567902704 | |
https://github.com/simonw/datasette/issues/675#issuecomment-747067864 | https://api.github.com/repos/simonw/datasette/issues/675 | 747067864 | MDEyOklzc3VlQ29tbWVudDc0NzA2Nzg2NA== | 9599 | 2020-12-16T22:02:55Z | 2020-12-16T22:02:55Z | OWNER | But since we're already running `COPY . /app` anything that's made it into the temporary directory will get copied into `/app`. But... I feel the usability of the command will be better if users can use absolute paths on the `target` side: datasette publish cloudrun my.db --cp dogsheep-beta.yml /app | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
567902704 | |
https://github.com/simonw/datasette/issues/675#issuecomment-747066629 | https://api.github.com/repos/simonw/datasette/issues/675 | 747066629 | MDEyOklzc3VlQ29tbWVudDc0NzA2NjYyOQ== | 9599 | 2020-12-16T21:59:58Z | 2020-12-16T22:00:48Z | OWNER | Note that `datasette publish cloudrun` uses a working directory of `/app` - so users will need to copy their files into `/app` if that's where they need to live. https://github.com/simonw/datasette/blob/17cbbb1f7f230b39650afac62dd16476626001b5/datasette/utils/__init__.py#L348-L357 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
567902704 | |
https://github.com/simonw/datasette/issues/1148#issuecomment-747065487 | https://api.github.com/repos/simonw/datasette/issues/1148 | 747065487 | MDEyOklzc3VlQ29tbWVudDc0NzA2NTQ4Nw== | 9599 | 2020-12-16T21:57:29Z | 2020-12-16T21:57:29Z | OWNER | I filed a new public bug in their issue tracker here: https://github.com/vercel/vercel/issues/5575 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
767561886 | |
https://github.com/simonw/datasette/issues/1148#issuecomment-747062909 | https://api.github.com/repos/simonw/datasette/issues/1148 | 747062909 | MDEyOklzc3VlQ29tbWVudDc0NzA2MjkwOQ== | 9599 | 2020-12-16T21:51:54Z | 2020-12-16T21:51:54Z | OWNER | This is a really frustrating bug with Vercel: https://github.com/simonw/datasette-publish-vercel/issues/28 `+` characters in URLs get translated into spaces before they get to Datasette. They know about the bug and said they were working on a fix a few months ago, but looks like it's still a problem. A workaround is to avoid `+` and use `-` instead - I think this SQL query does the same thing as yours: https://aws-partners-singapore.vercel.app/partners?sql=select%0D%0A++A.launch_rank%2C%0D%0A++A.partner_info%0D%0Afrom%0D%0A++summary+A%0D%0A++INNER+JOIN+summary+B+ON+A.launch_rank+%3E%3D+B.launch_rank+-+3%0D%0A++AND+A.launch_rank+-4+%3C%3D+B.launch_rank%0D%0AWHERE%0D%0A++B.%22partner_info%22+LIKE+%27%25Palo+Alto%25%27 ```sql select A.launch_rank, A.partner_info from summary A INNER JOIN summary B ON A.launch_rank >= B.launch_rank - 3 AND A.launch_rank -4 <= B.launch_rank WHERE B."partner_info" LIKE '%Palo Alto%' ``` I've been moving projects from Vercel to Cloud Run when they run into this, but that's not a great situation to be in. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
767561886 | |
https://github.com/simonw/datasette/issues/675#issuecomment-747059277 | https://api.github.com/repos/simonw/datasette/issues/675 | 747059277 | MDEyOklzc3VlQ29tbWVudDc0NzA1OTI3Nw== | 9599 | 2020-12-16T21:43:52Z | 2020-12-16T21:43:52Z | OWNER | It turns out I need this for a couple of projects: - [datasette-ripgrep](https://github.com/simonw/datasette-ripgrep) needs to ship a whole bunch of source code files up in a known location. I worked around this with a nasty hack involving `--static` but it would be better if I wasn't doing that. - [dogsheep-beta](https://github.com/dogsheep/dogsheep-beta) uses an additional `dogsheep-beta.yml` configuration file in the project root (a sibling to `metadata.yml`) which needs to be included when publishing - see https://github.com/simonw/datasette.io/issues/21#issuecomment-747058067 I want this for `datasette publish cloudrun`, not just for `datasette package`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
567902704 | |
https://github.com/simonw/datasette/issues/1143#issuecomment-746827083 | https://api.github.com/repos/simonw/datasette/issues/1143 | 746827083 | MDEyOklzc3VlQ29tbWVudDc0NjgyNzA4Mw== | 9599 | 2020-12-16T18:56:07Z | 2020-12-16T18:56:07Z | OWNER | I think the right way to do this is to support multiple optional `--cors-origin=` pattern values, like you suggested. | { "total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
764059235 | |
https://github.com/simonw/datasette/issues/1143#issuecomment-744757558 | https://api.github.com/repos/simonw/datasette/issues/1143 | 744757558 | MDEyOklzc3VlQ29tbWVudDc0NDc1NzU1OA== | 9599 | 2020-12-14T22:42:10Z | 2020-12-14T22:42:10Z | OWNER | This may involve a breaking change to the CLI settings interface, so I'm adding this to the 1.0 milestone. | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
764059235 | |
https://github.com/simonw/datasette/issues/1143#issuecomment-744756861 | https://api.github.com/repos/simonw/datasette/issues/1143 | 744756861 | MDEyOklzc3VlQ29tbWVudDc0NDc1Njg2MQ== | 9599 | 2020-12-14T22:40:28Z | 2020-12-14T22:40:28Z | OWNER | That's a very convincing argument. I'm keen on making sure Datasette is "secure by default" so you're right, encouraging finely grains CORS rules in core rather than leaving that to a plugin sounds like the right call. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
764059235 | |
https://github.com/simonw/datasette/issues/1142#issuecomment-744576894 | https://api.github.com/repos/simonw/datasette/issues/1142 | 744576894 | MDEyOklzc3VlQ29tbWVudDc0NDU3Njg5NA== | 9599 | 2020-12-14T17:03:13Z | 2020-12-14T17:03:13Z | OWNER | I'm not sure about the radio boxes for JSON, just because you can't right-click on a radio box and copy it to your clipboard like you can with links. Worth trying it out though. The radio boxes for that CSV option are definitely the right way to go. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763361458 | |
https://github.com/simonw/datasette/issues/1142#issuecomment-744563209 | https://api.github.com/repos/simonw/datasette/issues/1142 | 744563209 | MDEyOklzc3VlQ29tbWVudDc0NDU2MzIwOQ== | 9599 | 2020-12-14T16:41:11Z | 2020-12-14T16:41:11Z | OWNER | To check out and start the server: /tmp % git clone git@github.com:nitinpaul/datasette Cloning into 'datasette'... remote: Enumerating objects: 124, done. # ... datasette % python3 -m venv venv datasette % source venv/bin/activate (venv) datasette % pip install -e '.[test]' Obtaining file:///private/tmp/datasette Collecting asgiref<3.4.0,>=3.2.10 Using cached asgiref-3.3.1-py3-none-any.whl (19 kB) # ... (venv) datasette % datasette INFO: Started server process [24002] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) And to run the tests: (venv) datasette % pytest ======================================================================== test session starts ======================================================================== platform darwin -- Python 3.9.1, pytest-6.1.2, py-1.10.0, pluggy-0.13.1 SQLite: 3.34.0 rootdir: /private/tmp/datasette, configfile: pytest.ini plugins: asyncio-0.14.0, timeout-1.4.2 collected 841 items tests/test_package.py .. [ 0%] | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763361458 | |
https://github.com/simonw/datasette/issues/1142#issuecomment-744251252 | https://api.github.com/repos/simonw/datasette/issues/1142 | 744251252 | MDEyOklzc3VlQ29tbWVudDc0NDI1MTI1Mg== | 9599 | 2020-12-14T07:56:38Z | 2020-12-14T07:56:38Z | OWNER | That's a really solid design for this! I'd be very happy to review a pull request - you should be able to implement this with just template edits and some CSS changes I think. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763361458 | |
https://github.com/simonw/datasette/issues/1143#issuecomment-744249157 | https://api.github.com/repos/simonw/datasette/issues/1143 | 744249157 | MDEyOklzc3VlQ29tbWVudDc0NDI0OTE1Nw== | 9599 | 2020-12-14T07:53:15Z | 2020-12-14T07:53:15Z | OWNER | Does this plugin do everything you need? https://github.com/simonw/datasette-cors I'm open to arguments as to why this should be in core rather than in a plugin - I'm on the fence about that at the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
764059235 | |
https://github.com/simonw/datasette/issues/741#issuecomment-744142692 | https://api.github.com/repos/simonw/datasette/issues/741 | 744142692 | MDEyOklzc3VlQ29tbWVudDc0NDE0MjY5Mg== | 9599 | 2020-12-14T03:28:56Z | 2020-12-14T03:28:56Z | OWNER | I'm going to try this out on `datasette-publish-vercel` first. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
607223136 | |
https://github.com/simonw/datasette/issues/983#issuecomment-744066249 | https://api.github.com/repos/simonw/datasette/issues/983 | 744066249 | MDEyOklzc3VlQ29tbWVudDc0NDA2NjI0OQ== | 9599 | 2020-12-13T20:47:52Z | 2020-12-13T20:47:52Z | OWNER | @yozlet just spotted this comment. Wow that is interesting! With the right plugin hooks on the page (see also #987) one relatively simple way to do that could be with bookmarklets - users could install bookmarklets which, when executed against a Datasette page in their browser, use the existing JavaScript plugin integration points to add all kinds of functionality. Doing full sandboxing is certainly daunting, but it looks like Figma figured it out so TIL it's technically feasible. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
712260429 | |
https://github.com/simonw/sqlite-utils/issues/207#issuecomment-743966801 | https://api.github.com/repos/simonw/sqlite-utils/issues/207 | 743966801 | MDEyOklzc3VlQ29tbWVudDc0Mzk2NjgwMQ== | 9599 | 2020-12-13T07:25:23Z | 2020-12-13T07:25:23Z | OWNER | CLI documentation: https://sqlite-utils.readthedocs.io/en/latest/cli.html#analyzing-tables Python library documentation: https://sqlite-utils.readthedocs.io/en/latest/python-api.html#analyzing-a-column | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763283616 | |
https://github.com/simonw/sqlite-utils/pull/203#issuecomment-743966289 | https://api.github.com/repos/simonw/sqlite-utils/issues/203 | 743966289 | MDEyOklzc3VlQ29tbWVudDc0Mzk2NjI4OQ== | 9599 | 2020-12-13T07:20:51Z | 2020-12-13T07:20:51Z | OWNER | Sorry for not reviewing this yet! I'll try to carve out time to look at it in the next few days. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
743384829 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743956666 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743956666 | MDEyOklzc3VlQ29tbWVudDc0Mzk1NjY2Ng== | 9599 | 2020-12-13T05:44:49Z | 2020-12-13T05:44:49Z | OWNER | Example output: ``` % sqlite-utils analyze-tables github.db tags tags.repo: (1/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 14 Most common: 88: 107914493 75: 140912432 27: 206156866 21: 207052882 17: 197431109 8: 197882382 5: 256834907 5: 205429375 4: 248903544 3: 206202864 Least common: 1: 209590345 2: 206649770 2: 303218369 3: 206202864 3: 213286752 4: 248903544 5: 205429375 5: 256834907 8: 197882382 17: 197431109 tags.name: (2/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 175 Most common: 10: 0.2 9: 0.1 7: 0.3 6: 0.4 5: 0.7 5: 0.5 5: 0.1a 4: 0.9 4: 0.8 4: 0.6 Least common: 1: 0.1.1 1: 0.11.1 1: 0.1a2 1: 0.20.1 1: 0.21.1 1: 0.21.2 1: 0.21.3 1: 0.22 1: 0.22.1 1: 0.23 tags.sha: (3/3) Total rows: 261 Null rows: 0 Blank rows: 0 Distinct values: 261 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/datasette/issues/1142#issuecomment-743913004 | https://api.github.com/repos/simonw/datasette/issues/1142 | 743913004 | MDEyOklzc3VlQ29tbWVudDc0MzkxMzAwNA== | 9599 | 2020-12-12T22:17:46Z | 2020-12-12T22:17:46Z | OWNER | You're actually choosing between two options here: the 100 rows you can see on the screen, or the x,000 rows that match the current query. Maybe a radio box would be more obvious? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763361458 | |
https://github.com/simonw/datasette/issues/1142#issuecomment-743912875 | https://api.github.com/repos/simonw/datasette/issues/1142 | 743912875 | MDEyOklzc3VlQ29tbWVudDc0MzkxMjg3NQ== | 9599 | 2020-12-12T22:16:38Z | 2020-12-12T22:16:38Z | OWNER | Yeah, maybe with the number of rows to make it completely clear. `Include all 2,455 rows` perhaps. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763361458 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708524 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708524 | MDEyOklzc3VlQ29tbWVudDc0MzcwODUyNA== | 9599 | 2020-12-12T05:48:20Z | 2020-12-12T05:48:32Z | OWNER | ``` % sqlite-utils analyze-tables ../datasette/fixtures.db facetable --column pk 1/1: ColumnDetails(table='facetable', column='pk', total_rows=15, num_null=0, num_blank=0, num_distinct=15, most_common=None, least_common=None) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708325 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708325 | MDEyOklzc3VlQ29tbWVudDc0MzcwODMyNQ== | 9599 | 2020-12-12T05:46:27Z | 2020-12-12T05:46:27Z | OWNER | It would be neat if you could optionally specify a subset of columns to analyze, using `-c` or `--column`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708169 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708169 | MDEyOklzc3VlQ29tbWVudDc0MzcwODE2OQ== | 9599 | 2020-12-12T05:44:46Z | 2020-12-12T05:44:46Z | OWNER | If there are less than ten values is it worth outputting them twice, once in `most_common` and then in reverse in `least_common`? Feels redundant - I think I should leave `least_common` empty in that case. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743708080 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743708080 | MDEyOklzc3VlQ29tbWVudDc0MzcwODA4MA== | 9599 | 2020-12-12T05:43:45Z | 2020-12-12T05:43:45Z | OWNER | CLI output looks like this at the moment, which is bad: ``` % sqlite-utils analyze-tables ../datasette/fixtures.db facetable 1/10: ColumnDetails(table='facetable', column='pk', total_rows=15, num_null=0, num_blank=0, num_distinct=15, most_common=None, least_common=None) 2/10: ColumnDetails(table='facetable', column='created', total_rows=15, num_null=0, num_blank=0, num_distinct=4, most_common=[('2019-01-17 08:00:00', 4), ('2019-01-15 08:00:00', 4), ('2019-01-14 08:00:00', 4), ('2019-01-16 08:00:00', 3)], least_common=[('2019-01-16 08:00:00', 3), ('2019-01-14 08:00:00', 4), ('2019-01-15 08:00:00', 4), ('2019-01-17 08:00:00', 4)]) 3/10: ColumnDetails(table='facetable', column='planet_int', total_rows=15, num_null=0, num_blank=0, num_distinct=2, most_common=[(1, 14), (2, 1)], least_common=[(2, 1), (1, 14)]) 4/10: ColumnDetails(table='facetable', column='on_earth', total_rows=15, num_null=0, num_blank=0, num_distinct=2, most_common=[(1, 14), (0, 1)], least_common=[(0, 1), (1, 14)]) 5/10: ColumnDetails(table='facetable', column='state', total_rows=15, num_null=0, num_blank=0, num_distinct=3, most_common=[('CA', 10), ('MI', 4), ('MC', 1)], least_common=[('MC', 1), ('MI', 4), ('CA', 10)]) 6/10: ColumnDetails(table='facetable', column='city_id', total_rows=15, num_null=0, num_blank=0, num_distinct=4, most_common=[(1, 6), (3, 4), (2, 4), (4, 1)], least_common=[(4, 1), (2, 4), (3, 4), (1, 6)]) 7/10: ColumnDetails(table='facetable', column='neighborhood', total_rows=15, num_null=0, num_blank=0, num_distinct=14, most_common=[('Downtown', 2), ('Tenderloin', 1), ('SOMA', 1), ('Mission', 1), ('Mexicantown', 1), ('Los Feliz', 1), ('Koreatown', 1), ('Hollywood', 1), ('Hayes Valley', 1), ('Greektown', 1)], least_common=[('Arcadia Planitia', 1), ('Bernal Heights', 1), ('Corktown', 1), ('Dogpatch', 1), ('Greektown', 1), ('Hayes Valley', 1), ('Hollywood', 1), ('Koreatown', 1), ('Los Feliz', 1), ('Mexicantown', 1)]) 8/10: ColumnDetails(table='facetable', column='tags', total_rows=15, num_null=0, num_blank=0, num_distinct=3,… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/pull/208#issuecomment-743707969 | https://api.github.com/repos/simonw/sqlite-utils/issues/208 | 743707969 | MDEyOklzc3VlQ29tbWVudDc0MzcwNzk2OQ== | 9599 | 2020-12-12T05:42:26Z | 2020-12-12T05:43:06Z | OWNER | Should truncate values in the least/most common JSON array to a sensible length, otherwise you end up with stuff like this: ```json [ [ "b'\\x00\\x05barry\\x03\\x01\\x02\\x00\\x00\\x03cat\\x03\\x01\\x03\\x00\\x00\\x03dog\\x08\\x01\\x01\\x01\\x03\\x00\\x01\\x03\\x00\\x00\\x07panther\\x05\\x01\\x01\\x02\\x02\\x00\\x01\\x03uma\\x05\\x02\\x01\\x02\\x02\\x00\\x00\\x04sara\\x05\\x02\\x01\\x01\\x02\\x00\\x00\\x05terry\\x08\\x01\\x01\\x01\\x02\\x00\\x01\\x02\\x00\\x00\\x06weasel\\x05\\x02\\x01\\x01\\x03\\x00'", 1 ] ] ``` This example also shows that binary values (like those in `_fts` tables) look a bit weird, but I think I'm OK with that since binary data can't be represented neatly in JSON anyway. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763320133 | |
https://github.com/simonw/sqlite-utils/issues/207#issuecomment-743701697 | https://api.github.com/repos/simonw/sqlite-utils/issues/207 | 743701697 | MDEyOklzc3VlQ29tbWVudDc0MzcwMTY5Nw== | 9599 | 2020-12-12T04:39:51Z | 2020-12-12T04:39:51Z | OWNER | CLI could be: sqlite-utils analyze-tables To analyze all tables or: sqlite-utils analyze-tables table1 table2 To analyze specific tables. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763283616 | |
https://github.com/simonw/sqlite-utils/issues/207#issuecomment-743701599 | https://api.github.com/repos/simonw/sqlite-utils/issues/207 | 743701599 | MDEyOklzc3VlQ29tbWVudDc0MzcwMTU5OQ== | 9599 | 2020-12-12T04:38:52Z | 2020-12-12T04:39:07Z | OWNER | I'll add a `table.analyze_column(column)` method which is used by the CLI tool - with a note that this is an unstable interface which may change in the future. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763283616 | |
https://github.com/simonw/sqlite-utils/issues/207#issuecomment-743701422 | https://api.github.com/repos/simonw/sqlite-utils/issues/207 | 743701422 | MDEyOklzc3VlQ29tbWVudDc0MzcwMTQyMg== | 9599 | 2020-12-12T04:37:14Z | 2020-12-12T04:38:25Z | OWNER | Prototype: ```python from collections import namedtuple ColumnDetails = namedtuple("ColumnDetails", ("column", "num_null", "num_blank", "num_distinct", "most_common", "least_common")) def analyze_column(db, table, column, values=10): num_null = db.execute("select count(*) from [{}] where [{}] is null".format(table, column)).fetchone()[0] num_blank = db.execute("select count(*) from [{}] where [{}] = ''".format(table, column)).fetchone()[0] num_distinct = db.execute("select count(distinct [{}]) from [{}]".format(column, table)).fetchone()[0] most_common = None least_common = None if num_distinct != 1: most_common = [(r[0], r[1]) for r in db.execute( "select [{}], count(*) from [{}] group by [{}] order by count(*) desc limit ".format(column, table, column, values) ).fetchall()] if num_distinct <= values: # No need to run the query if it will just return the results in revers order least_common = most_common[::-1] else: least_common = [(r[0], r[1]) for r in db.execute( "select [{}], count(*) from [{}] group by [{}] order by count(*) limit {}".format(column, table, column, values) ).fetchall()] return ColumnDetails(column, num_null, num_blank, num_distinct, most_common, least_common) def analyze_table(db, table): for column in db[table].columns: details = analyze_column(db, table, column.name) print(details) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
763283616 | |
https://github.com/simonw/sqlite-utils/issues/205#issuecomment-742737794 | https://api.github.com/repos/simonw/sqlite-utils/issues/205 | 742737794 | MDEyOklzc3VlQ29tbWVudDc0MjczNzc5NA== | 9599 | 2020-12-10T19:18:22Z | 2020-12-10T19:18:22Z | OWNER | Yup, it looks like you're using a window function that was added in SQLite 3.25.0: https://www.sqlite.org/changes.html#version_3_25_0 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760960559 | |
https://github.com/simonw/datasette/issues/1091#issuecomment-741992106 | https://api.github.com/repos/simonw/datasette/issues/1091 | 741992106 | MDEyOklzc3VlQ29tbWVudDc0MTk5MjEwNg== | 9599 | 2020-12-09T19:19:54Z | 2020-12-09T20:27:45Z | OWNER | Could you try removing the `ProxyPassReverse /datasette http://0.0.0.0:8001` line? My hunch is that `ProxyPassReverse` is rewriting some of the links in the HTML (or maybe in the HTTP headers) in a way that breaks things. Normally you would need `ProxyPassReverse` to compensate for the underlying application being unable to rewrite its links - but Datasette's `base_url` setting causes Datasette to rewrite all of the links for you, so `ProxyPassReverse` should be unneccessary. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
742011049 | |
https://github.com/simonw/datasette/issues/1134#issuecomment-742024588 | https://api.github.com/repos/simonw/datasette/issues/1134 | 742024588 | MDEyOklzc3VlQ29tbWVudDc0MjAyNDU4OA== | 9599 | 2020-12-09T20:19:59Z | 2020-12-09T20:20:33Z | OWNER | https://byraadsarkivet.aarhus.dk/db/cases?_searchmode=raw&_search=sundhedsfrem%2A is an absolutely beautiful example of a themed Datasette! Very excited to show this to people. | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760312579 | |
https://github.com/simonw/datasette/issues/1134#issuecomment-742023775 | https://api.github.com/repos/simonw/datasette/issues/1134 | 742023775 | MDEyOklzc3VlQ29tbWVudDc0MjAyMzc3NQ== | 9599 | 2020-12-09T20:18:23Z | 2020-12-09T20:18:23Z | OWNER | A fix for this should be available if you upgrade to 0.52.5 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760312579 | |
https://github.com/simonw/datasette/issues/1091#issuecomment-742023541 | https://api.github.com/repos/simonw/datasette/issues/1091 | 742023541 | MDEyOklzc3VlQ29tbWVudDc0MjAyMzU0MQ== | 9599 | 2020-12-09T20:17:54Z | 2020-12-09T20:17:54Z | OWNER | OK that is really weird. I'll have another go at replicating this locally. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
742011049 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742023111 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742023111 | MDEyOklzc3VlQ29tbWVudDc0MjAyMzExMQ== | 9599 | 2020-12-09T20:17:02Z | 2020-12-09T20:17:02Z | OWNER | Documentation for this procedure is now here: https://docs.datasette.io/en/latest/contributing.html#releasing-bug-fixes-from-a-branch | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742022222 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742022222 | MDEyOklzc3VlQ29tbWVudDc0MjAyMjIyMg== | 9599 | 2020-12-09T20:15:24Z | 2020-12-09T20:15:51Z | OWNER | Used this procedure for the first time for 0.52.5 - deploy run here: https://github.com/simonw/datasette/actions/runs/411465648 - PyPI release here: https://pypi.org/project/datasette/0.52.5/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742017622 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742017622 | MDEyOklzc3VlQ29tbWVudDc0MjAxNzYyMg== | 9599 | 2020-12-09T20:06:47Z | 2020-12-09T20:06:47Z | OWNER | Then I can ship the release directly from that branch, creating the tag as part of the release process: <img width="1057" alt="New_release_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/101681442-ffc93500-3a16-11eb-8801-3a0e357a2bc1.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742014881 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742014881 | MDEyOklzc3VlQ29tbWVudDc0MjAxNDg4MQ== | 9599 | 2020-12-09T20:01:27Z | 2020-12-09T20:01:27Z | OWNER | I'll write the release notes in the branch, then cherry-pick them over to `main`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742014366 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742014366 | MDEyOklzc3VlQ29tbWVudDc0MjAxNDM2Ng== | 9599 | 2020-12-09T20:00:35Z | 2020-12-09T20:00:35Z | OWNER | Actually I'll start from 0.52.4 and then cherry-pick the fixes. git branch 0.52.x 0.52.4 git checkout 0.52.x git cherry-pick COMMIT | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1134#issuecomment-742012324 | https://api.github.com/repos/simonw/datasette/issues/1134 | 742012324 | MDEyOklzc3VlQ29tbWVudDc0MjAxMjMyNA== | 9599 | 2020-12-09T19:57:05Z | 2020-12-09T19:57:05Z | OWNER | Thanks for the bug report! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760312579 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742009294 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742009294 | MDEyOklzc3VlQ29tbWVudDc0MjAwOTI5NA== | 9599 | 2020-12-09T19:51:18Z | 2020-12-09T19:51:18Z | OWNER | Likewise, Read The Docs publishes as stable the docs from the latest tagged release, so I would expect that to work fine as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742009101 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742009101 | MDEyOklzc3VlQ29tbWVudDc0MjAwOTEwMQ== | 9599 | 2020-12-09T19:50:53Z | 2020-12-09T19:50:53Z | OWNER | My concern is if this will break anything about CI. I don't think it will - the code that deploys the latest `main` to https://latest.datasette.io/ should be unaffected, and the checkout code in `publish.yml` should check out the correct code based on the tag used for that release. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1136#issuecomment-742008087 | https://api.github.com/repos/simonw/datasette/issues/1136 | 742008087 | MDEyOklzc3VlQ29tbWVudDc0MjAwODA4Nw== | 9599 | 2020-12-09T19:48:56Z | 2020-12-09T19:48:56Z | OWNER | I think I'm going to create a branch called `0.52.x` that starts with `8ae0f9f7f0d644b0161165a1084f53acd2786f7c` and then tag the release from there. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
760621356 | |
https://github.com/simonw/datasette/issues/1133#issuecomment-740850920 | https://api.github.com/repos/simonw/datasette/issues/1133 | 740850920 | MDEyOklzc3VlQ29tbWVudDc0MDg1MDkyMA== | 9599 | 2020-12-08T18:55:59Z | 2020-12-08T18:55:59Z | OWNER | Inspiration was this script: https://gist.github.com/simonw/f6e3cd29fde5d15ea9cd746c942046ba - which pipes output through `tail -n +2` to strip off the headers. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
759695780 | |
https://github.com/simonw/datasette/issues/1133#issuecomment-740850057 | https://api.github.com/repos/simonw/datasette/issues/1133 | 740850057 | MDEyOklzc3VlQ29tbWVudDc0MDg1MDA1Nw== | 9599 | 2020-12-08T18:55:29Z | 2020-12-08T18:55:29Z | OWNER | Can work on this as part of #1062. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
759695780 | |
https://github.com/simonw/sqlite-utils/pull/204#issuecomment-740796067 | https://api.github.com/repos/simonw/sqlite-utils/issues/204 | 740796067 | MDEyOklzc3VlQ29tbWVudDc0MDc5NjA2Nw== | 9599 | 2020-12-08T17:49:22Z | 2020-12-08T17:49:22Z | OWNER | Great catch, thank you. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
752888228 | |
https://github.com/simonw/datasette/issues/815#issuecomment-740385032 | https://api.github.com/repos/simonw/datasette/issues/815 | 740385032 | MDEyOklzc3VlQ29tbWVudDc0MDM4NTAzMg== | 9599 | 2020-12-08T05:26:09Z | 2020-12-08T05:26:16Z | OWNER | Sure! It's a bit of a fiddle one - I've not found an approach that I like, but I also haven't thought about it since June. I'd love to see what you come up with! | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 1, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
634663505 | |
https://github.com/simonw/datasette/issues/1132#issuecomment-740228858 | https://api.github.com/repos/simonw/datasette/issues/1132 | 740228858 | MDEyOklzc3VlQ29tbWVudDc0MDIyODg1OA== | 9599 | 2020-12-07T22:50:36Z | 2020-12-07T22:50:36Z | OWNER | Documented here: https://docs.datasette.io/en/latest/json_api.html#column-filter-arguments Demo: https://latest.datasette.io/fixtures/facetable?tags__arraynotcontains=tag2 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
758899581 | |
https://github.com/simonw/datasette/issues/1131#issuecomment-739414118 | https://api.github.com/repos/simonw/datasette/issues/1131 | 739414118 | MDEyOklzc3VlQ29tbWVudDczOTQxNDExOA== | 9599 | 2020-12-05T20:48:33Z | 2020-12-05T20:48:33Z | OWNER | Oddly enough, I tried fixing this with `sys.stderr.write("{}\n".format(e))` - but my Click `CLIRunner` tests failed because `result.stderr` was an empty string. Adding `sys.stderr.flush()` to the code that output errors fixed that issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
757481949 | |
https://github.com/simonw/datasette/issues/398#issuecomment-739357330 | https://api.github.com/repos/simonw/datasette/issues/398 | 739357330 | MDEyOklzc3VlQ29tbWVudDczOTM1NzMzMA== | 9599 | 2020-12-05T19:36:27Z | 2020-12-05T19:36:27Z | OWNER | This was fixed in #749 by 88ac538b41a4753c3de9b509c3a0e13077f66182 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
398011658 | |
https://github.com/simonw/datasette/pull/1128#issuecomment-739355855 | https://api.github.com/repos/simonw/datasette/issues/1128 | 739355855 | MDEyOklzc3VlQ29tbWVudDczOTM1NTg1NQ== | 9599 | 2020-12-05T19:34:57Z | 2020-12-05T19:34:57Z | OWNER | Thanks for this! | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756867924 | |
https://github.com/simonw/datasette/issues/1131#issuecomment-739083673 | https://api.github.com/repos/simonw/datasette/issues/1131 | 739083673 | MDEyOklzc3VlQ29tbWVudDczOTA4MzY3Mw== | 9599 | 2020-12-05T00:02:10Z | 2020-12-05T00:02:10Z | OWNER | https://clig.dev/#the-basics > **Send messaging to stderr**. Log messages, errors, and so on should all be sent to stderr. This means that when commands are piped together, these messages are displayed to the user and not fed into the next command. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
757481949 | |
https://github.com/simonw/datasette/issues/1131#issuecomment-739083472 | https://api.github.com/repos/simonw/datasette/issues/1131 | 739083472 | MDEyOklzc3VlQ29tbWVudDczOTA4MzQ3Mg== | 9599 | 2020-12-05T00:01:12Z | 2020-12-05T00:01:12Z | OWNER | Here's why: https://github.com/simonw/datasette/blob/37f87b5e52e7f8ddd1c4ffcf368bd7a62a406a6d/datasette/database.py#L158-L163 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
757481949 | |
https://github.com/simonw/datasette/issues/188#issuecomment-738905376 | https://api.github.com/repos/simonw/datasette/issues/188 | 738905376 | MDEyOklzc3VlQ29tbWVudDczODkwNTM3Ng== | 9599 | 2020-12-04T17:18:34Z | 2020-12-04T17:18:34Z | OWNER | This is likely to be covered by plugin hooks: #860 for the metadata and after investigating in #1042 it looks like the existing `prepare_jinja2_environment` hook may already be enough to load templates from the database. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
309047460 | |
https://github.com/simonw/datasette/issues/111#issuecomment-738904347 | https://api.github.com/repos/simonw/datasette/issues/111 | 738904347 | MDEyOklzc3VlQ29tbWVudDczODkwNDM0Nw== | 9599 | 2020-12-04T17:16:56Z | 2020-12-04T17:16:56Z | OWNER | This is STILL a good idea. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274615452 | |
https://github.com/simonw/datasette/pull/1130#issuecomment-738897582 | https://api.github.com/repos/simonw/datasette/issues/1130 | 738897582 | MDEyOklzc3VlQ29tbWVudDczODg5NzU4Mg== | 9599 | 2020-12-04T17:03:30Z | 2020-12-04T17:03:30Z | OWNER | I deployed this to https://datasette-issue-1129.vercel.app/ (using `datasette publish vercel fixtures.db --branch 8d4c69c6fb0ef741a19070f5172017ea3522e83c --about_url https://github.com/simonw/datasette/issues/1129 --about datasette/issues/1129 --project datasette-issue-1129`) - weirdly, on Mobile Safari the footer appears just below the visible window: ![RPReplay_Final1607100726](https://user-images.githubusercontent.com/9599/101191950-336b2000-360f-11eb-8f14-ed83bd86515c.gif) I've seen other problems with fixed footers on Mobile Safari too: at Eventbrite this was a really nasty problem for us to figure out: https://www.eventbrite.com/engineering/mobile-safari-why/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756876238 | |
https://github.com/simonw/datasette/issues/1125#issuecomment-738554392 | https://api.github.com/repos/simonw/datasette/issues/1125 | 738554392 | MDEyOklzc3VlQ29tbWVudDczODU1NDM5Mg== | 9599 | 2020-12-04T04:16:57Z | 2020-12-04T04:16:57Z | OWNER | https://latest.datasette.io/-/versions now shows this: ```json { "python": { "version": "3.8.6", "full": "3.8.6 (default, Nov 18 2020, 13:49:49) \n[GCC 8.3.0]" }, "datasette": { "version": "0.52.3", "note": "49d8fc056844d5a537d6cfd96dab0dd5686fe718" }, "asgi": "3.0", "uvicorn": "0.12.3", "sqlite": { "version": "3.33.0", "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "extensions": { "json1": null }, "compile_options": [] }, "pysqlite3": "0.4.4" } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756622648 | |
https://github.com/simonw/datasette/issues/1125#issuecomment-738551280 | https://api.github.com/repos/simonw/datasette/issues/1125 | 738551280 | MDEyOklzc3VlQ29tbWVudDczODU1MTI4MA== | 9599 | 2020-12-04T04:03:54Z | 2020-12-04T04:03:54Z | OWNER | I'm going to check `pkg_resources.get_distribution("pysqlite3-binary").version` too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756622648 | |
https://github.com/simonw/datasette/issues/1125#issuecomment-738550588 | https://api.github.com/repos/simonw/datasette/issues/1125 | 738550588 | MDEyOklzc3VlQ29tbWVudDczODU1MDU4OA== | 9599 | 2020-12-04T04:01:10Z | 2020-12-04T04:01:10Z | OWNER | Urgh, figuring out the version of `pysqlite3` is WAY harder than I expected. The `getversion` module looks like the smartest attempt at solving this problem generally, but I'd like to avoid adding another dependency just for this: https://github.com/smarie/python-getversion | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756622648 | |
https://github.com/simonw/datasette/issues/1125#issuecomment-738548693 | https://api.github.com/repos/simonw/datasette/issues/1125 | 738548693 | MDEyOklzc3VlQ29tbWVudDczODU0ODY5Mw== | 9599 | 2020-12-04T03:52:51Z | 2020-12-04T03:52:51Z | OWNER | That didn't work - https://latest.datasette.io/-/versions isn't showing the package. I bet that's because I'm actually installing `pysqlite3-binary` here: https://github.com/simonw/datasette/blob/e2fea36540e952d8d72c1bd0af7144b85b7a4671/.github/workflows/deploy-latest.yml#L57 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756622648 | |
https://github.com/simonw/datasette/issues/1126#issuecomment-738548393 | https://api.github.com/repos/simonw/datasette/issues/1126 | 738548393 | MDEyOklzc3VlQ29tbWVudDczODU0ODM5Mw== | 9599 | 2020-12-04T03:51:38Z | 2020-12-04T03:51:38Z | OWNER | That worked. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756761963 | |
https://github.com/simonw/datasette/issues/1125#issuecomment-738347171 | https://api.github.com/repos/simonw/datasette/issues/1125 | 738347171 | MDEyOklzc3VlQ29tbWVudDczODM0NzE3MQ== | 9599 | 2020-12-03T22:04:52Z | 2020-12-03T22:04:52Z | OWNER | ``` pkg_resources.get_distribution("pysqlite3").version Out[14]: '0.4.4' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756622648 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738215686 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738215686 | MDEyOklzc3VlQ29tbWVudDczODIxNTY4Ng== | 9599 | 2020-12-03T18:50:48Z | 2020-12-03T21:42:02Z | OWNER | I'm going to punt on writing a unit test for this (not sure how I'd simulate those symlinks) - I'll manually test it and push out a dot release instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738215487 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738215487 | MDEyOklzc3VlQ29tbWVudDczODIxNTQ4Nw== | 9599 | 2020-12-03T18:50:26Z | 2020-12-03T21:41:25Z | OWNER | This fix works - calling `.resolve()` on the `root_path` before the comparison to ensure symlinks are resolved: ```python # Ensure full_path is within root_path to avoid weird "../" tricks try: print("full_path={}, root_path={}".format(full_path, root_path)) full_path.relative_to(root_path.resolve()) except ValueError as e: print(" ValueError:", e) await asgi_send_html(send, "404", 404) return ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738213342 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738213342 | MDEyOklzc3VlQ29tbWVudDczODIxMzM0Mg== | 9599 | 2020-12-03T18:46:22Z | 2020-12-03T21:40:51Z | OWNER | I replaced that function with this code: ```python def asgi_static(root_path, chunk_size=4096, headers=None, content_type=None): async def inner_static(request, send): path = request.scope["url_route"]["kwargs"]["path"] print("path =", path) try: full_path = (Path(root_path) / path).resolve().absolute() except FileNotFoundError as e: print("FileNotFoundError:", e) await asgi_send_html(send, "404", 404) return if full_path.is_dir(): await asgi_send_html(send, "403: Directory listing is not allowed", 403) return # Ensure full_path is within root_path to avoid weird "../" tricks try: print("full_path={}, root_path={}".format(full_path, root_path)) full_path.relative_to(root_path) except ValueError as e: print(" ValueError:", e) await asgi_send_html(send, "404", 404) return try: await asgi_send_file(send, full_path, chunk_size=chunk_size) except FileNotFoundError: await asgi_send_html(send, "404", 404) return return inner_static ``` Edited using `vi /home/ec2-user/.local/pipx/venvs/datasette/lib/python3.7/site-packages/datasette/utils/asgi.py` The output shows me what the bug is: ``` $ datasette --get /-/static/app.css --pdb app_root = /home/ec2-user/.local/pipx/venvs/datasette/lib64/python3.7/site-packages path = app.css full_path=/home/ec2-user/.local/pipx/venvs/datasette/lib/python3.7/site-packages/datasette/static/app.css, root_path=/home/ec2-user/.local/pipx/venvs/datasette/lib64/python3.7/site-packages/datasette/static ValueError: '/home/ec2-user/.local/pipx/venvs/datasette/lib/python3.7/site-packages/datasette/static/app.css' does not start with '/home/ec2-user/.local/pipx/venvs/datasette/lib64/python3.7/site-packages/datasette/static' 404 ``` ` ValueError: '/home/ec2-user/.local/pipx/venvs/datasette/lib/python… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738313399 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738313399 | MDEyOklzc3VlQ29tbWVudDczODMxMzM5OQ== | 9599 | 2020-12-03T21:10:54Z | 2020-12-03T21:10:54Z | OWNER | Confirmed that installing a fresh copy of Datasette 0.52.3 on that server works correctly as expected. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738224865 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738224865 | MDEyOklzc3VlQ29tbWVudDczODIyNDg2NQ== | 9599 | 2020-12-03T19:01:52Z | 2020-12-03T19:01:52Z | OWNER | https://github.com/simonw/datasette/runs/1494631261 ``` /home/runner/work/datasette/datasette/tests/test_html.py:81: AssertionError ----------------------------- Captured stderr call ----------------------------- Traceback (most recent call last): File "/home/runner/work/datasette/datasette/datasette/app.py", line 1039, in route_path response = await view(request, send) File "/home/runner/work/datasette/datasette/datasette/utils/asgi.py", line 297, in inner_static full_path.relative_to(root_path.resolve()) AttributeError: 'str' object has no attribute 'resolve' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738220067 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738220067 | MDEyOklzc3VlQ29tbWVudDczODIyMDA2Nw== | 9599 | 2020-12-03T18:58:17Z | 2020-12-03T18:58:17Z | OWNER | I tested this by running: pipx uninstall datasette pipx install 'https://github.com/simonw/datasette/archive/6b4c55efea3e9d34d92cbe5f0066553ad9b14071.zip' To replace that version of Datasette (in the correct virtual environment) with this patch. It worked! ``` [ec2-user@ip-172-31-30-7 ~]$ datasette --get /-/static/app.css /* Reset and Page Setup ==================================================== */ ... ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738211776 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738211776 | MDEyOklzc3VlQ29tbWVudDczODIxMTc3Ng== | 9599 | 2020-12-03T18:43:21Z | 2020-12-03T18:43:21Z | OWNER | I'm suspicious of this code here:https://github.com/simonw/datasette/blob/e048791a9a2686f47d81a2c8aa88aa1966d82521/datasette/utils/asgi.py#L284-L307 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738211152 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738211152 | MDEyOklzc3VlQ29tbWVudDczODIxMTE1Mg== | 9599 | 2020-12-03T18:42:12Z | 2020-12-03T18:42:12Z | OWNER | Added a line to print out `app_root` from https://github.com/simonw/datasette/blob/e048791a9a2686f47d81a2c8aa88aa1966d82521/datasette/app.py#L848-L853 ``` app_root = /home/ec2-user/.local/pipx/venvs/datasette/lib64/python3.7/site-packages ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1124#issuecomment-738209642 | https://api.github.com/repos/simonw/datasette/issues/1124 | 738209642 | MDEyOklzc3VlQ29tbWVudDczODIwOTY0Mg== | 9599 | 2020-12-03T18:39:19Z | 2020-12-03T18:39:19Z | OWNER | The CSS files are in the expected location: ``` [ec2-user@ip-172-31-30-7 ~]$ find /home/ec2-user/.local/pipx/venvs/datasette | grep css /home/ec2-user/.local/pipx/venvs/datasette/lib/python3.7/site-packages/datasette/static/app.css /home/ec2-user/.local/pipx/venvs/datasette/lib/python3.7/site-packages/datasette/static/codemirror-5.57.0.min.css ``` Wow it's running an ANCIENT version of SQLite: ``` [ec2-user@ip-172-31-30-7 ~]$ datasette --get /-/versions.json {"python": {"version": "3.7.9", "full": "3.7.9 (default, Aug 27 2020, 21:58:41) \n[GCC 7.3.1 20180712 (Red Hat 7.3.1-9)]"}, "datasette": {"version": "0.52.2"}, "asgi": "3.0", "uvicorn": "0.12.3", "sqlite": {"version": "3.7.17", "fts_versions": ["FTS4", "FTS3"], "extensions": {}, "compile_options": ["DISABLE_DIRSYNC", "ENABLE_COLUMN_METADATA", "ENABLE_FTS3", "ENABLE_RTREE", "ENABLE_UNLOCK_NOTIFY", "SECURE_DELETE", "TEMP_STORE=1", "THREADSAFE=1"]}} ``` http://www.sqlite.org/releaselog/3_7_17.html - SQLite Release 3.7.17 On 2013-05-20 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
756439516 | |
https://github.com/simonw/datasette/issues/1121#issuecomment-737591281 | https://api.github.com/repos/simonw/datasette/issues/1121 | 737591281 | MDEyOklzc3VlQ29tbWVudDczNzU5MTI4MQ== | 9599 | 2020-12-03T01:03:18Z | 2020-12-03T01:03:18Z | OWNER | Demo: https://latest.datasette.io/fixtures?_bot=1 <img width="729" alt="fixtures" src="https://user-images.githubusercontent.com/9599/100950004-43c0b500-34c0-11eb-918c-aa959376461f.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
754178780 | |
https://github.com/simonw/datasette/issues/1100#issuecomment-737589314 | https://api.github.com/repos/simonw/datasette/issues/1100 | 737589314 | MDEyOklzc3VlQ29tbWVudDczNzU4OTMxNA== | 9599 | 2020-12-03T00:57:35Z | 2020-12-03T00:57:35Z | OWNER | Fixed in the demo: ``` % curl -XOPTIONS https://latest.datasette.io/fixtures.json ok% ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
747702144 | |
https://github.com/simonw/datasette/issues/1123#issuecomment-737586248 | https://api.github.com/repos/simonw/datasette/issues/1123 | 737586248 | MDEyOklzc3VlQ29tbWVudDczNzU4NjI0OA== | 9599 | 2020-12-03T00:47:37Z | 2020-12-03T00:47:37Z | OWNER | Affected tests: ``` FAILED tests/test_plugins.py::test_hook_table_actions[facetable] - AssertionE... FAILED tests/test_plugins.py::test_hook_table_actions[simple_view] - Assertio... ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
755721275 | |
https://github.com/simonw/datasette/issues/1100#issuecomment-737581719 | https://api.github.com/repos/simonw/datasette/issues/1100 | 737581719 | MDEyOklzc3VlQ29tbWVudDczNzU4MTcxOQ== | 9599 | 2020-12-03T00:35:23Z | 2020-12-03T00:35:23Z | OWNER | Replicated this against the live demo as well: ``` /tmp % curl -XOPTIONS https://latest.datasette.io/fixtures.json {"ok": false, "error": "object Response can't be used in 'await' expression", "status": 500, "title": null}% /tmp % ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
747702144 | |
https://github.com/simonw/datasette/pull/1122#issuecomment-737580813 | https://api.github.com/repos/simonw/datasette/issues/1122 | 737580813 | MDEyOklzc3VlQ29tbWVudDczNzU4MDgxMw== | 9599 | 2020-12-03T00:33:09Z | 2020-12-03T00:33:09Z | OWNER | This is a very neat fix, thank you. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
754179035 | |
https://github.com/simonw/datasette/issues/749#issuecomment-737580084 | https://api.github.com/repos/simonw/datasette/issues/749 | 737580084 | MDEyOklzc3VlQ29tbWVudDczNzU4MDA4NA== | 9599 | 2020-12-03T00:31:14Z | 2020-12-03T00:31:14Z | OWNER | This works! ``` /tmp % wget 'https://covid-19.datasettes.com/covid.db' --2020-12-02 16:28:02-- https://covid-19.datasettes.com/covid.db Resolving covid-19.datasettes.com (covid-19.datasettes.com)... 172.217.5.83 Connecting to covid-19.datasettes.com (covid-19.datasettes.com)|172.217.5.83|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/octet-stream] Saving to: ‘covid.db’ covid.db [ <=> ] 306.42M 3.27MB/s in 98s 2020-12-02 16:29:40 (3.13 MB/s) - ‘covid.db’ saved [321306624] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
610829227 | |
https://github.com/simonw/datasette/issues/749#issuecomment-737563699 | https://api.github.com/repos/simonw/datasette/issues/749 | 737563699 | MDEyOklzc3VlQ29tbWVudDczNzU2MzY5OQ== | 9599 | 2020-12-02T23:45:42Z | 2020-12-02T23:45:42Z | OWNER | I asked about this on Twitter - https://twitter.com/steren/status/1334281184965140483 > You simply need to send the `Transfer-Encoding: chunked` header. | { "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
610829227 | |
https://github.com/simonw/datasette/issues/942#issuecomment-737463116 | https://api.github.com/repos/simonw/datasette/issues/942 | 737463116 | MDEyOklzc3VlQ29tbWVudDczNzQ2MzExNg== | 9599 | 2020-12-02T20:02:10Z | 2020-12-02T20:03:01Z | OWNER | My idea is that if you installed my proposed plugin you wouldn't need `metadata.json` at all - your metadata would instead live in a table in the connected SQLite database files - either one table per database (so the metadata can live in the same place as the data) or maybe also in a dedicated separate database file, for if you want to add metadata to an otherwise read-only database. The plugin would then provide a UI for editing that metadata - maybe by configuring some writable canned queries or maybe something more custom than that. Or you could edit the metadata by manually editing the SQLite database file (or loading data into it using a tool like [yaml-to-sqlite](https://github.com/simonw/yaml-to-sqlite)). | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681334912 | |
https://github.com/simonw/datasette/issues/942#issuecomment-737402392 | https://api.github.com/repos/simonw/datasette/issues/942 | 737402392 | MDEyOklzc3VlQ29tbWVudDczNzQwMjM5Mg== | 9599 | 2020-12-02T18:08:55Z | 2020-12-02T18:08:55Z | OWNER | SQLite does let you add comments in your CREATE TABLE statements: ```sql CREATE TABLE something ( id integer primary key, -- integer primary key created text -- created date as ISO datetime ); ``` But the only mechanism for reading those back is to retrieve that `CREATE TABLE` block of SQL from the `sqlite_master` table and run a parser against it. I've so far resisted adding a SQL syntax parser to Datasette for complexity reasons - though I'm increasingly thinking I'll need to do it at some point. I think I'll leave this to plugins. I'm definitely going to build a plugin that lets you store metadata for tables and columns in a SQLite database table, which will then support interactively editing metadata through a UI. A plugin which extracts column comments from the SQLite CREATE TABLE comments would be feasible too, if I design the plugin hooks well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681334912 | |
https://github.com/simonw/datasette/issues/1119#issuecomment-736142201 | https://api.github.com/repos/simonw/datasette/issues/1119 | 736142201 | MDEyOklzc3VlQ29tbWVudDczNjE0MjIwMQ== | 9599 | 2020-12-01T00:41:14Z | 2020-12-01T00:41:14Z | OWNER | On my laptop: <img width="1123" alt="fixtures__generated_columns__1_row" src="https://user-images.githubusercontent.com/9599/100681839-8734eb00-3329-11eb-8aea-813bdc18efa1.png"> https://latest.datasette.io/-/versions is running SQLite 3.27.2 at the moment so it won't show that table until it gets to 3.31.0. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753876808 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-736030599 | https://api.github.com/repos/simonw/datasette/issues/1116 | 736030599 | MDEyOklzc3VlQ29tbWVudDczNjAzMDU5OQ== | 9599 | 2020-11-30T20:41:41Z | 2020-11-30T20:41:41Z | OWNER | Here's the problem: https://www.sqlite.org/changes.html#version_3_26_0 > ### 2018-12-01 (3.26.0) > > - Added [PRAGMA table_xinfo](https://www.sqlite.org/pragma.html#pragma_table_xinfo) that works just like [PRAGMA table_info](https://www.sqlite.org/pragma.html#pragma_table_info) except that it also shows [hidden columns](https://www.sqlite.org/vtab.html#hiddencol) in virtual tables. CI is running 3.22.0. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/pull/1117#issuecomment-736029337 | https://api.github.com/repos/simonw/datasette/issues/1117 | 736029337 | MDEyOklzc3VlQ29tbWVudDczNjAyOTMzNw== | 9599 | 2020-11-30T20:39:06Z | 2020-11-30T20:39:06Z | OWNER | Here's the problem: https://www.sqlite.org/changes.html#version_3_26_0 > ### 2018-12-01 (3.26.0) > > - Added [PRAGMA table_xinfo](https://www.sqlite.org/pragma.html#pragma_table_xinfo) that works just like [PRAGMA table_info](https://www.sqlite.org/pragma.html#pragma_table_info) except that it also shows [hidden columns](https://www.sqlite.org/vtab.html#hiddencol) in virtual tables. CI is running 3.22.0. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753767911 | |
https://github.com/simonw/datasette/pull/1117#issuecomment-736028726 | https://api.github.com/repos/simonw/datasette/issues/1117 | 736028726 | MDEyOklzc3VlQ29tbWVudDczNjAyODcyNg== | 9599 | 2020-11-30T20:37:50Z | 2020-11-30T20:37:50Z | OWNER | This kind of problem is why I have a `tmate` workflow: <img width="1086" alt="Actions_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/100660845-72dff680-3307-11eb-89dc-211d226f68dd.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753767911 | |
https://github.com/simonw/datasette/pull/1117#issuecomment-736023089 | https://api.github.com/repos/simonw/datasette/issues/1117 | 736023089 | MDEyOklzc3VlQ29tbWVudDczNjAyMzA4OQ== | 9599 | 2020-11-30T20:26:27Z | 2020-11-30T20:26:27Z | OWNER | On my laptop: ``` platform darwin -- Python 3.8.6, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 SQLite: 3.33.0 ``` In CI they are all SQLite: 3.22.0 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753767911 | |
https://github.com/simonw/datasette/pull/1117#issuecomment-736018609 | https://api.github.com/repos/simonw/datasette/issues/1117 | 736018609 | MDEyOklzc3VlQ29tbWVudDczNjAxODYwOQ== | 9599 | 2020-11-30T20:17:31Z | 2020-11-30T20:17:31Z | OWNER | I need to replicate these failures on my laptop. My hunch is that this is down to the version of SQLite available to Python. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753767911 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-736015487 | https://api.github.com/repos/simonw/datasette/issues/1116 | 736015487 | MDEyOklzc3VlQ29tbWVudDczNjAxNTQ4Nw== | 9599 | 2020-11-30T20:11:07Z | 2020-11-30T20:11:07Z | OWNER | Working on this in a pull request: https://github.com/simonw/datasette/pull/1117 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-736014372 | https://api.github.com/repos/simonw/datasette/issues/1116 | 736014372 | MDEyOklzc3VlQ29tbWVudDczNjAxNDM3Mg== | 9599 | 2020-11-30T20:08:48Z | 2020-11-30T20:08:48Z | OWNER | Ouch, the tests pass on my laptop but failed in CI: https://github.com/simonw/datasette/actions/runs/392367997 Lots of failures look like this: ``` ERROR: conn=<sqlite3.Connection object at 0x7f44f0494030>, sql = 'select rowid, from facetable order by rowid limit 51', params = {}: near "from": syntax error ``` Note the `select rowid, from...` - so it looks like invalid SQL queries are being constructed maybe due to mis-detecting columns somehow. I wonder why it didn't fail on my laptop? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-736010720 | https://api.github.com/repos/simonw/datasette/issues/1116 | 736010720 | MDEyOklzc3VlQ29tbWVudDczNjAxMDcyMA== | 9599 | 2020-11-30T20:01:53Z | 2020-11-30T20:01:53Z | OWNER | I'm OK exposing hidden columns, unless someone comes up with a pressing reason not to. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-736004383 | https://api.github.com/repos/simonw/datasette/issues/1116 | 736004383 | MDEyOklzc3VlQ29tbWVudDczNjAwNDM4Mw== | 9599 | 2020-11-30T19:51:51Z | 2020-11-30T19:51:51Z | OWNER | This change will also have an impact on how hidden virtual FTS tables are displayed, since apparently those have some hidden columns: https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_xinfo%28%27searchable_fts%27%29 | cid | name | type | notnull | dflt_value | pk | hidden | | --- | --- | --- | --- | --- | --- | --- | | 0 | text1 | | 0 | | 0 | 0 | | 1 | text2 | | 0 | | 0 | 0 | | 2 | name with . and spaces | | 0 | | 0 | 0 | | 3 | searchable_fts | | 0 | | 0 | 1 | | 4 | docid | | 0 | | 0 | 1 | | 5 | __langid | | 0 | | 0 | 1 | | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-735995695 | https://api.github.com/repos/simonw/datasette/issues/1116 | 735995695 | MDEyOklzc3VlQ29tbWVudDczNTk5NTY5NQ== | 9599 | 2020-11-30T19:34:15Z | 2020-11-30T19:34:15Z | OWNER | Generated column support was added in SQLite 3.31.0, so any unit tests I write for this should use skipIf to only run on that version or later. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-735993935 | https://api.github.com/repos/simonw/datasette/issues/1116 | 735993935 | MDEyOklzc3VlQ29tbWVudDczNTk5MzkzNQ== | 9599 | 2020-11-30T19:30:44Z | 2020-11-30T19:32:15Z | OWNER | It looks like `PRAGMA table_info` skips "hidden" columns: https://www.sqlite.org/pragma.html#pragma_table_info But `PRAGMA table_xinfo` does not: https://www.sqlite.org/pragma.html#pragma_table_xinfo Compare https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_info%28%27searchable%27%29 to https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_xinfo%28%27searchable%27%29 - the `xinfo` one has an additional `hidden` column. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/1116#issuecomment-735992106 | https://api.github.com/repos/simonw/datasette/issues/1116 | 735992106 | MDEyOklzc3VlQ29tbWVudDczNTk5MjEwNg== | 9599 | 2020-11-30T19:27:10Z | 2020-11-30T19:27:10Z | OWNER | I'm treating this as a bug - these columns should definitely be visible in Datasette. I created my own test database using SQLite from Homebrew like this: ``` /usr/local/Cellar/sqlite/3.33.0/bin/sqlite3 deeds.db << EOF CREATE TABLE deeds ( body TEXT, id INT GENERATED ALWAYS AS (json_extract(body, '$.id')) STORED, consideration INT GENERATED ALWAYS AS (json_extract(body, '$.consideration')) STORED ); INSERT INTO deeds (body) VALUES ('{ "id": 1, "consideration": "This is the consideration" }'); EOF ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753668177 | |
https://github.com/simonw/datasette/issues/263#issuecomment-735960132 | https://api.github.com/repos/simonw/datasette/issues/263 | 735960132 | MDEyOklzc3VlQ29tbWVudDczNTk2MDEzMg== | 9599 | 2020-11-30T18:25:17Z | 2020-11-30T18:25:17Z | OWNER | Fixing this would unblock this issue for switching `datasette-graphql` to using `datasette.client` internally: https://github.com/simonw/datasette-graphql/issues/61 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323671577 |