github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/916#issuecomment-812742462 | https://api.github.com/repos/simonw/datasette/issues/916 | 812742462 | MDEyOklzc3VlQ29tbWVudDgxMjc0MjQ2Mg== | 1111743 | 2021-04-02T22:37:27Z | 2021-04-02T22:37:27Z | NONE | Yes, this would be nice! I using Datasette v0.56 and don't see a previous page button. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
672421411 | |
https://github.com/simonw/datasette/issues/1245#issuecomment-812711365 | https://api.github.com/repos/simonw/datasette/issues/1245 | 812711365 | MDEyOklzc3VlQ29tbWVudDgxMjcxMTM2NQ== | 1111743 | 2021-04-02T20:53:35Z | 2021-04-02T20:53:35Z | NONE | Yes, I agree. Alternatively, maybe the header could be at the top and bottom, above the next page button. Maybe even have the header 50 records down? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
817544251 | |
https://github.com/simonw/datasette/issues/1255#issuecomment-812710120 | https://api.github.com/repos/simonw/datasette/issues/1255 | 812710120 | MDEyOklzc3VlQ29tbWVudDgxMjcxMDEyMA== | 1111743 | 2021-04-02T20:50:08Z | 2021-04-02T20:50:08Z | NONE | Hello again, I was able to get my facets running with this `settings.json`, which was lifted from one of Simon's datasette's and slightly modified. ``` { "default_page_size": 100, "max_returned_rows": 1000, "num_sql_threads": 3, "sql_time_limit_ms": 9000, "default_facet_size": 10, "facet_time_limit_ms": 9000, "facet_suggest_time_limit_ms": 500, "hash_urls": false, "allow_facet": true, "suggest_facets": false, "default_cache_ttl": 5, "default_cache_ttl_hashed": 31536000, "cache_size_kb": 0, "allow_csv_stream": true, "max_csv_mb": 100, "truncate_cells_html": 2048, "template_debug": false, "base_url": "/" } ``` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
826700095 | |
https://github.com/simonw/datasette/issues/1255#issuecomment-812680519 | https://api.github.com/repos/simonw/datasette/issues/1255 | 812680519 | MDEyOklzc3VlQ29tbWVudDgxMjY4MDUxOQ== | 1111743 | 2021-04-02T19:37:57Z | 2021-04-02T19:37:57Z | NONE | Hello, I'm also experiencing a timeout in my environment. I don't know if it's because I need more indexes or a more powerful system. My data has 1,271,111 and when I try to create a facet, there's a time out. I've tried this on two different rows that should significantly filter down data: `CITY` and `PARTY_REG`. Simon's johns_hopkins_csse_daily_reports has more rows and it setup with two facets on load. He does have four indexes created, though. Do I need more indexes? I have one simple one so far: ``` CREATE INDEX [idx_party_reg] ON [county_active] ([PARTY_REG]); ``` I'm running Datasette 0.56 installed via pip with Python 3.7.3. `4.19.0-10-amd64 #1 SMP Debian 4.19.132-1 (2020-07-24) x86_64 GNU/Linux` ``` $ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
826700095 | |
https://github.com/simonw/datasette/issues/1286#issuecomment-812679221 | https://api.github.com/repos/simonw/datasette/issues/1286 | 812679221 | MDEyOklzc3VlQ29tbWVudDgxMjY3OTIyMQ== | 192568 | 2021-04-02T19:34:01Z | 2021-04-02T19:34:01Z | CONTRIBUTOR | This shows the city in a different color (and not the comma), but I get the idea, and I like it. (Ooh, could be nice to have the gear have an option in array fields to show as bullets or commas or semicolons...) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849220154 | |
https://github.com/simonw/datasette/issues/1287#issuecomment-812665092 | https://api.github.com/repos/simonw/datasette/issues/1287 | 812665092 | MDEyOklzc3VlQ29tbWVudDgxMjY2NTA5Mg== | 9599 | 2021-04-02T18:54:29Z | 2021-04-02T18:54:29Z | OWNER | `python:3.9.3-slim-buster` isn't on Docker Hub yet either. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849396758 | |
https://github.com/simonw/datasette/issues/1286#issuecomment-812664443 | https://api.github.com/repos/simonw/datasette/issues/1286 | 812664443 | MDEyOklzc3VlQ29tbWVudDgxMjY2NDQ0Mw== | 9599 | 2021-04-02T18:52:45Z | 2021-04-02T18:52:51Z | OWNER | Idea: default to displaying single-dimension JSON arrays of strings as a comma-separated list but show the comma in a different colour - something like this: <img width="561" alt="fixtures__facetable__15_rows" src="https://user-images.githubusercontent.com/9599/113445005-dce9d480-93a9-11eb-901c-0c2fb079c150.png"> I used this HTML for the prototype (re-using `.type-int` just to get the colour): ```html <td class="col-tags type-str">tag1<span class="type-int">, </span>tag2</td> ``` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849220154 | |
https://github.com/simonw/datasette/issues/1286#issuecomment-812663107 | https://api.github.com/repos/simonw/datasette/issues/1286 | 812663107 | MDEyOklzc3VlQ29tbWVudDgxMjY2MzEwNw== | 9599 | 2021-04-02T18:49:22Z | 2021-04-02T18:49:22Z | OWNER | This makes senses - showing an array as `["blah", "blah2", "blah3"]` isn't particularly human-friendly! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849220154 | |
https://github.com/simonw/datasette/issues/1287#issuecomment-812662583 | https://api.github.com/repos/simonw/datasette/issues/1287 | 812662583 | MDEyOklzc3VlQ29tbWVudDgxMjY2MjU4Mw== | 9599 | 2021-04-02T18:47:59Z | 2021-04-02T18:47:59Z | OWNER | Once again having tests for the Dockerfile as seen in #1272 would be useful here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849396758 | |
https://github.com/simonw/datasette/issues/1287#issuecomment-812662026 | https://api.github.com/repos/simonw/datasette/issues/1287 | 812662026 | MDEyOklzc3VlQ29tbWVudDgxMjY2MjAyNg== | 9599 | 2021-04-02T18:46:20Z | 2021-04-02T18:46:20Z | OWNER | https://devcenter.heroku.com/articles/python-support#supported-runtimes looks like Heroku still only have 3.9.2 for the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849396758 | |
https://github.com/simonw/datasette/issues/1287#issuecomment-812661269 | https://api.github.com/repos/simonw/datasette/issues/1287 | 812661269 | MDEyOklzc3VlQ29tbWVudDgxMjY2MTI2OQ== | 9599 | 2021-04-02T18:45:08Z | 2021-04-02T18:45:19Z | OWNER | A few places: https://github.com/simonw/datasette/blob/7b1a9a1999eb9326ce8ec830d75ac200e5279c46/Dockerfile#L1 https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/__init__.py#L350 https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/tests/test_package.py#L14 https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/publish/heroku.py#L177-L178 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
849396758 | |
https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-811362316 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31 | 811362316 | MDEyOklzc3VlQ29tbWVudDgxMTM2MjMxNg== | 871250 | 2021-03-31T19:14:39Z | 2021-03-31T19:14:39Z | NONE | 👋 could I help somehow for this to be merged? As Big Sur is going to be more used as the time goes I think it would be nice to merge and publish a new version. Nice work! | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771511344 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-811209922 | https://api.github.com/repos/simonw/datasette/issues/1276 | 811209922 | MDEyOklzc3VlQ29tbWVudDgxMTIwOTkyMg== | 1314318 | 2021-03-31T16:27:26Z | 2021-03-31T16:27:26Z | NONE | Fantastic. Thank you! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/issues/526#issuecomment-810943882 | https://api.github.com/repos/simonw/datasette/issues/526 | 810943882 | MDEyOklzc3VlQ29tbWVudDgxMDk0Mzg4Mg== | 701 | 2021-03-31T10:03:55Z | 2021-03-31T10:03:55Z | NONE | +1 on using nested queries to achieve this! Would be great as streaming CSV is an amazing feature. Some UX/DX details: I was expecting it to work to simply add `&_stream=on` to custom SQL queries because the docs say > Any Datasette table, view or **custom SQL query** can be exported as CSV. After a bit of testing back and forth I realized streaming only works for full tables. Would love this feature because I'm using `pandas.read_csv` to paint graphs from custom queries and the graphs are cut off because of the 1000 row limit. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459882902 | |
https://github.com/simonw/datasette/issues/1284#issuecomment-810779928 | https://api.github.com/repos/simonw/datasette/issues/1284 | 810779928 | MDEyOklzc3VlQ29tbWVudDgxMDc3OTkyOA== | 192568 | 2021-03-31T05:40:12Z | 2021-03-31T05:40:12Z | CONTRIBUTOR | Maybe the addition of two template files: 'one_database_index.html' and 'one_table_index.html' would be a better idea than the documentation diff idea. (They could include commented instructions to rename the preferred template 'index.html', along with any other necessary guidance.) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
845794436 | |
https://github.com/simonw/datasette/issues/1284#issuecomment-810740486 | https://api.github.com/repos/simonw/datasette/issues/1284 | 810740486 | MDEyOklzc3VlQ29tbWVudDgxMDc0MDQ4Ng== | 9599 | 2021-03-31T03:57:55Z | 2021-03-31T03:57:55Z | OWNER | You're right, doing this is really hard at the moment - I'm not sure I know how I would tackle this either, and it's something I've wanted in the past! I'll have a think about this one. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
845794436 | |
https://github.com/simonw/datasette/pull/1282#issuecomment-809670294 | https://api.github.com/repos/simonw/datasette/issues/1282 | 809670294 | MDEyOklzc3VlQ29tbWVudDgwOTY3MDI5NA== | 9599 | 2021-03-29T19:57:29Z | 2021-03-29T19:57:29Z | OWNER | Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
843739658 | |
https://github.com/simonw/datasette/pull/1282#issuecomment-809667320 | https://api.github.com/repos/simonw/datasette/issues/1282 | 809667320 | MDEyOklzc3VlQ29tbWVudDgwOTY2NzMyMA== | 22429695 | 2021-03-29T19:52:35Z | 2021-03-29T19:52:35Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1282?src=pr&el=h1) Report > Merging [#1282](https://codecov.io/gh/simonw/datasette/pull/1282?src=pr&el=desc) (08f7427) into [main](https://codecov.io/gh/simonw/datasette/commit/0486303b60ce2784fd2e2ecdbecf304b7d6e6659?el=desc) (0486303) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1282/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1282?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1282 +/- ## ======================================= Coverage 91.51% 91.51% ======================================= Files 34 34 Lines 4255 4255 ======================================= Hits 3894 3894 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1282?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1282?src=pr&el=footer). Last update [0486303...08f7427](https://codecov.io/gh/simonw/datasette/pull/1282?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
843739658 | |
https://github.com/simonw/datasette/issues/696#issuecomment-809548363 | https://api.github.com/repos/simonw/datasette/issues/696 | 809548363 | MDEyOklzc3VlQ29tbWVudDgwOTU0ODM2Mw== | 9599 | 2021-03-29T17:04:19Z | 2021-03-29T17:04:19Z | OWNER | I tried this just now against Datasette 0.56 with the new Dockerfile from #1249 (that uses SQLite and SpatiaLite installed with `apt-get install`) and the tests all passed. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
576722115 | |
https://github.com/simonw/datasette/pull/1031#issuecomment-809010713 | https://api.github.com/repos/simonw/datasette/issues/1031 | 809010713 | MDEyOklzc3VlQ29tbWVudDgwOTAxMDcxMw== | 9599 | 2021-03-29T01:46:45Z | 2021-03-29T01:46:45Z | OWNER | Sorry I didn't get to this PR sooner. I've joint-credited you in the release notes for this fix: https://docs.datasette.io/en/stable/changelog.html#v0-56 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
724369025 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-809009580 | https://api.github.com/repos/simonw/datasette/issues/1281 | 809009580 | MDEyOklzc3VlQ29tbWVudDgwOTAwOTU4MA== | 9599 | 2021-03-29T01:41:48Z | 2021-03-29T01:41:48Z | OWNER | https://github.com/simonw/datasette/runs/2214871602?check_suite_focus=true worked: <img width="1001" alt="Banners_and_Alerts_and_Actions_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/112776676-270f3680-8ff5-11eb-90d1-533d4e82f677.png"> Here's the 0.56 image on Docker Hub: https://hub.docker.com/layers/datasetteproject/datasette/0.56/images/sha256-701fc0f299a0ea79434a4852c46dab351254b9ac25dbe3c5f36fd5360caf52f9?context=explore | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-809008760 | https://api.github.com/repos/simonw/datasette/issues/1281 | 809008760 | MDEyOklzc3VlQ29tbWVudDgwOTAwODc2MA== | 9599 | 2021-03-29T01:38:21Z | 2021-03-29T01:38:21Z | OWNER | Got this error: ``` "docker tag" requires exactly 2 arguments. See 'docker tag --help'. Usage: docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG] Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-809007255 | https://api.github.com/repos/simonw/datasette/issues/1281 | 809007255 | MDEyOklzc3VlQ29tbWVudDgwOTAwNzI1NQ== | 9599 | 2021-03-29T01:32:18Z | 2021-03-29T01:32:18Z | OWNER | I'm going to build a new GitHub Actions workflow for this that lets me manually specify a tag to build and push as a Docker image. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-809001653 | https://api.github.com/repos/simonw/datasette/issues/1281 | 809001653 | MDEyOklzc3VlQ29tbWVudDgwOTAwMTY1Mw== | 9599 | 2021-03-29T01:08:31Z | 2021-03-29T01:08:31Z | OWNER | I'm going to attempt to fix this manually for the 0.56 release, by building and tagging it by hand and then pushing the 0.56 tag to Docker Hub. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-809001273 | https://api.github.com/repos/simonw/datasette/issues/1281 | 809001273 | MDEyOklzc3VlQ29tbWVudDgwOTAwMTI3Mw== | 9599 | 2021-03-29T01:06:45Z | 2021-03-29T01:06:45Z | OWNER | https://docs.docker.com/engine/reference/commandline/push/#push-all-tags-of-an-image > Use the `-a` (or `--all-tags`) option to push all tags of a local image. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-809000903 | https://api.github.com/repos/simonw/datasette/issues/1281 | 809000903 | MDEyOklzc3VlQ29tbWVudDgwOTAwMDkwMw== | 9599 | 2021-03-29T01:05:10Z | 2021-03-29T01:05:10Z | OWNER | https://github.com/simonw/datasette/runs/1763835467?check_suite_focus=true for Datasette 0.54 worked, and the output included this: ``` Successfully tagged ***/datasette:0.54 The push refers to repository [docker.io/***/datasette] aedd33c6b161: Preparing ... aedd33c6b161: Pushed 0.54: digest: sha256:65c7e579d1c29755dac5c1ca86b1e97fa88c48bd3d724ac3e02988d0da296140 size: 2005 aedd33c6b161: Preparing ... 5dacd731af1b: Layer already exists latest: digest: sha256:65c7e579d1c29755dac5c1ca86b1e97fa88c48bd3d724ac3e02988d0da296140 size: 2005 ``` Here's that same section of output from the 0.56 release: ``` Successfully tagged ***/datasette:0.56 Using default tag: latest The push refers to repository [docker.io/***/datasette] 4d4a9976adcc: Preparing ... 9b2132a0d5cf: Pushed latest: digest: sha256:2250d0fbe57b1d615a8d6df0c9d43deb9533532e00bac68854773d8ff8dcf00a size: 1793 ``` The difference here is the "Using default tag: latest" bit. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1281#issuecomment-808999525 | https://api.github.com/repos/simonw/datasette/issues/1281 | 808999525 | MDEyOklzc3VlQ29tbWVudDgwODk5OTUyNQ== | 9599 | 2021-03-29T01:00:38Z | 2021-03-29T01:00:38Z | OWNER | Here's the diff between `Dockerfile` in 0.54.1 and 0.56: https://github.com/simonw/datasette/compare/0.54.1...0.56#diff-551d1fcf87f78cc3bc18a7b332a4dc5d8773a512062df881c5aba28a6f5c48d7 <img width="1357" alt="Banners_and_Alerts_and_Comparing_0_54_1___0_56_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/112774760-79e5ef80-8fef-11eb-886a-5d546d6d2f3d.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842881221 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-808998719 | https://api.github.com/repos/simonw/datasette/issues/1249 | 808998719 | MDEyOklzc3VlQ29tbWVudDgwODk5ODcxOQ== | 9599 | 2021-03-29T00:57:13Z | 2021-03-29T00:57:13Z | OWNER | I just shipped Datasette 0.56 - here's the CI run: https://github.com/simonw/datasette/runs/2214701802?check_suite_focus=true It pushed a new `latest` tag to https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated docker pull datasetteproject/datasette:latest And then: docker run datasetteproject/datasette:latest datasette \ --load-extension=spatialite \ --get /-/versions.json | jq .sqlite Outputs: ```json { "version": "3.27.2", "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "extensions": { "json1": null, "spatialite": "5.0.1" }, "compile_options": [ "COMPILER=gcc-8.3.0", "ENABLE_COLUMN_METADATA", "ENABLE_DBSTAT_VTAB", "ENABLE_FTS3", "ENABLE_FTS3_PARENTHESIS", "ENABLE_FTS3_TOKENIZER", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_JSON1", "ENABLE_LOAD_EXTENSION", "ENABLE_PREUPDATE_HOOK", "ENABLE_RTREE", "ENABLE_SESSION", "ENABLE_STMTVTAB", "ENABLE_UNLOCK_NOTIFY", "ENABLE_UPDATE_DELETE_LIMIT", "HAVE_ISNAN", "LIKE_DOESNT_MATCH_BLOBS", "MAX_SCHEMA_RETRY=25", "MAX_VARIABLE_NUMBER=250000", "OMIT_LOOKASIDE", "SECURE_DELETE", "SOUNDEX", "TEMP_STORE=1", "THREADSAFE=1", "USE_URI" ] } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/pull/1031#issuecomment-808989067 | https://api.github.com/repos/simonw/datasette/issues/1031 | 808989067 | MDEyOklzc3VlQ29tbWVudDgwODk4OTA2Nw== | 9599 | 2021-03-29T00:23:41Z | 2021-03-29T00:23:41Z | OWNER | This bug should have been fixed in #1229 - let me know if that's not the case! Thanks | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
724369025 | |
https://github.com/simonw/datasette/pull/1260#issuecomment-808988697 | https://api.github.com/repos/simonw/datasette/issues/1260 | 808988697 | MDEyOklzc3VlQ29tbWVudDgwODk4ODY5Nw== | 9599 | 2021-03-29T00:22:21Z | 2021-03-29T00:22:21Z | OWNER | This is interesting! I've decided to apply a subset of these - the `if` and `elif` blocks are a deliberate style choice from me, because I find code clearer when it has if/else as opposed to relying on early termination. Likewise the iteration against `.keys()` on dictionaries. I like the other fixes though, I'm about to land them in a separate commit that credits you. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
831163537 | |
https://github.com/simonw/datasette/pull/1229#issuecomment-780830464 | https://api.github.com/repos/simonw/datasette/issues/1229 | 780830464 | MDEyOklzc3VlQ29tbWVudDc4MDgzMDQ2NA== | 22429695 | 2021-02-17T20:24:30Z | 2021-03-29T00:17:21Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=h1) Report > Merging [#1229](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=desc) (a095248) into [main](https://codecov.io/gh/simonw/datasette/commit/8e18c7943181f228ce5ebcea48deb59ce50bee1f?el=desc) (8e18c79) will **not change** coverage. > The diff coverage is `100.00%`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1229/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1229 +/- ## ======================================= Coverage 91.51% 91.51% ======================================= Files 34 34 Lines 4255 4255 ======================================= Hits 3894 3894 Misses 361 361 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1229/diff?src=pr&el=tree#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `95.85% <100.00%> (ø)` | | ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=footer). Last update [8e18c79...a095248](https://codecov.io/gh/simonw/datasette/pull/1229?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
810507413 | |
https://github.com/simonw/datasette/pull/1229#issuecomment-808987304 | https://api.github.com/repos/simonw/datasette/issues/1229 | 808987304 | MDEyOklzc3VlQ29tbWVudDgwODk4NzMwNA== | 9599 | 2021-03-29T00:17:13Z | 2021-03-29T00:17:13Z | OWNER | Thanks for figuring this out! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
810507413 | |
https://github.com/simonw/datasette/pull/1252#issuecomment-808986495 | https://api.github.com/repos/simonw/datasette/issues/1252 | 808986495 | MDEyOklzc3VlQ29tbWVudDgwODk4NjQ5NQ== | 9599 | 2021-03-29T00:13:59Z | 2021-03-29T00:13:59Z | OWNER | Neat fix, thank you! | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0 } |
825217564 | |
https://github.com/simonw/datasette/pull/1279#issuecomment-808986036 | https://api.github.com/repos/simonw/datasette/issues/1279 | 808986036 | MDEyOklzc3VlQ29tbWVudDgwODk4NjAzNg== | 9599 | 2021-03-29T00:11:50Z | 2021-03-29T00:11:50Z | OWNER | Thanks for the fix. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842556944 | |
https://github.com/simonw/datasette/issues/1280#issuecomment-808983160 | https://api.github.com/repos/simonw/datasette/issues/1280 | 808983160 | MDEyOklzc3VlQ29tbWVudDgwODk4MzE2MA== | 9599 | 2021-03-28T23:59:28Z | 2021-03-29T00:10:05Z | OWNER | Might be easier to do this using https://github.com/coleifer/pysqlite3 rather than try to replace the system `sqlite3` on the Ubuntu GitHub Actions instances. These instructions should help: https://github.com/coleifer/pysqlite3#building-a-statically-linked-library | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842862708 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-808981968 | https://api.github.com/repos/simonw/datasette/issues/1276 | 808981968 | MDEyOklzc3VlQ29tbWVudDgwODk4MTk2OA== | 9599 | 2021-03-28T23:52:31Z | 2021-03-28T23:52:31Z | OWNER | Testing this on Glitch by adding `https://github.com/simonw/datasette/archive/48d5e0e6ac8975cfd869d4e8c69c64ca0c65e29e.zip` as a dependency. That fixed it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-808979608 | https://api.github.com/repos/simonw/datasette/issues/1276 | 808979608 | MDEyOklzc3VlQ29tbWVudDgwODk3OTYwOA== | 9599 | 2021-03-28T23:38:34Z | 2021-03-28T23:38:34Z | OWNER | Aha! https://www.sqlite.org/pragma.html says: > The table-valued functions for PRAGMA feature was added in SQLite version 3.16.0 (2017-01-02). Prior versions of SQLite cannot use this feature. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-808979218 | https://api.github.com/repos/simonw/datasette/issues/1276 | 808979218 | MDEyOklzc3VlQ29tbWVudDgwODk3OTIxOA== | 9599 | 2021-03-28T23:35:53Z | 2021-03-28T23:36:26Z | OWNER | Here's where I run that: https://github.com/simonw/datasette/blob/6f41c8a2bef309a66588b2875c3e24d26adb4850/datasette/database.py#L249-L253 That's from when I added the `--crossdb` option in #1232: https://github.com/simonw/datasette/commit/6f41c8a2bef309a66588b2875c3e24d26adb4850#diff-4e20309c969326a0008dc9237f6807f48d55783315fbfc1e7dfa480b550e16f9R249 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-808979049 | https://api.github.com/repos/simonw/datasette/issues/1276 | 808979049 | MDEyOklzc3VlQ29tbWVudDgwODk3OTA0OQ== | 9599 | 2021-03-28T23:34:38Z | 2021-03-28T23:34:38Z | OWNER | The Glitch server logs showed: > `ERROR: conn=<sqlite3.Connection object at 0x7f88ebadd8f0>, sql = 'select seq, name, file from pragma_database_list() where seq > 0', params = None: no such table: pragma_database_list` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-808978808 | https://api.github.com/repos/simonw/datasette/issues/1276 | 808978808 | MDEyOklzc3VlQ29tbWVudDgwODk3ODgwOA== | 9599 | 2021-03-28T23:32:58Z | 2021-03-28T23:33:58Z | OWNER | I just managed to replicate this bug on Glitch: https://nicar-2020.glitch.me/data > Invalid SQL > no such table: pragma_database_list https://nicar-2020.glitch.me/-/versions says: ```json { "python": { "version": "3.7.10", "full": "3.7.10 (default, Feb 20 2021, 21:21:24) \n[GCC 5.4.0 20160609]" }, "datasette": { "version": "0.55" }, "asgi": "3.0", "uvicorn": "0.13.4", "sqlite": { "version": "3.11.0", "fts_versions": [ "FTS4", "FTS3" ], "extensions": { "json1": null }, "compile_options": [ "ENABLE_COLUMN_METADATA", "ENABLE_DBSTAT_VTAB", "ENABLE_FTS3", "ENABLE_FTS3_PARENTHESIS", "ENABLE_JSON1", "ENABLE_LOAD_EXTENSION", "ENABLE_RTREE", "ENABLE_UNLOCK_NOTIFY", "ENABLE_UPDATE_DELETE_LIMIT", "HAVE_ISNAN", "LIKE_DOESNT_MATCH_BLOBS", "MAX_SCHEMA_RETRY=25", "OMIT_LOOKASIDE", "SECURE_DELETE", "SOUNDEX", "SYSTEM_MALLOC", "TEMP_STORE=1", "THREADSAFE=1" ] } } ``` That's [SQLite 3.11.0 from 2016-02-15](https://www.sqlite.org/releaselog/3_11_0.html) with no FTS5. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/pull/1279#issuecomment-808762613 | https://api.github.com/repos/simonw/datasette/issues/1279 | 808762613 | MDEyOklzc3VlQ29tbWVudDgwODc2MjYxMw== | 22429695 | 2021-03-27T17:03:37Z | 2021-03-27T17:03:37Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1279?src=pr&el=h1) Report > Merging [#1279](https://codecov.io/gh/simonw/datasette/pull/1279?src=pr&el=desc) (14d8977) into [main](https://codecov.io/gh/simonw/datasette/commit/3fcfc8513465339ac5f055296cbb67f5262af02b?el=desc) (3fcfc85) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1279/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1)](https://codecov.io/gh/simonw/datasette/pull/1279?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## main #1279 +/- ## ======================================= Coverage 91.51% 91.51% ======================================= Files 34 34 Lines 4255 4255 ======================================= Hits 3894 3894 Misses 361 361 ``` ------ [Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1279?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1279?src=pr&el=footer). Last update [3fcfc85...14d8977](https://codecov.io/gh/simonw/datasette/pull/1279?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842556944 | |
https://github.com/simonw/datasette/issues/1273#issuecomment-808759984 | https://api.github.com/repos/simonw/datasette/issues/1273 | 808759984 | MDEyOklzc3VlQ29tbWVudDgwODc1OTk4NA== | 9599 | 2021-03-27T16:43:17Z | 2021-03-27T16:43:17Z | OWNER | That rivers example in the tutorial would work a lot better with a live demo. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
838382890 | |
https://github.com/simonw/datasette/issues/1273#issuecomment-808756921 | https://api.github.com/repos/simonw/datasette/issues/1273 | 808756921 | MDEyOklzc3VlQ29tbWVudDgwODc1NjkyMQ== | 9599 | 2021-03-27T16:19:45Z | 2021-03-27T16:26:28Z | OWNER | I have a better recipe for using spatial indexes now on https://simonwillison.net/2021/Jan/24/drawing-shapes-spatialite/ ```sql select AsGeoJSON(geometry), * from CPAD_2020a_SuperUnits where PARK_NAME like '%mini%' and Intersects(GeomFromGeoJSON(:freedraw), geometry) = 1 and CPAD_2020a_SuperUnits.rowid in ( select rowid from SpatialIndex where f_table_name = 'CPAD_2020a_SuperUnits' and search_frame = GeomFromGeoJSON(:freedraw) ) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
838382890 | |
https://github.com/simonw/datasette/issues/1273#issuecomment-808757721 | https://api.github.com/repos/simonw/datasette/issues/1273 | 808757721 | MDEyOklzc3VlQ29tbWVudDgwODc1NzcyMQ== | 9599 | 2021-03-27T16:25:48Z | 2021-03-27T16:25:48Z | OWNER | > This will give you back an additional column of GeoJSON. You can copy and paste GeoJSON from this column into the debugging tool at geojson.io to visualize it on a map. That should promote `datasette-leaflet-geojson` instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
838382890 | |
https://github.com/simonw/datasette/issues/1090#issuecomment-808757659 | https://api.github.com/repos/simonw/datasette/issues/1090 | 808757659 | MDEyOklzc3VlQ29tbWVudDgwODc1NzY1OQ== | 9599 | 2021-03-27T16:25:25Z | 2021-03-27T16:25:25Z | OWNER | Related feature request: ability to set default values for canned queries: #1258 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
741862364 | |
https://github.com/simonw/datasette/issues/1090#issuecomment-808757155 | https://api.github.com/repos/simonw/datasette/issues/1090 | 808757155 | MDEyOklzc3VlQ29tbWVudDgwODc1NzE1NQ== | 9599 | 2021-03-27T16:21:43Z | 2021-03-27T16:21:43Z | OWNER | Idea for these: imitate https://django-sql-dashboard.readthedocs.io/en/latest/widgets.html#custom-widgets and drive them with templates. So a custom widget type of `textarea` would look for a template called `widgets/textarea.html` - which means users could define brand new custom widgets just by creating their own template files. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
741862364 | |
https://github.com/simonw/datasette/issues/1278#issuecomment-808756366 | https://api.github.com/repos/simonw/datasette/issues/1278 | 808756366 | MDEyOklzc3VlQ29tbWVudDgwODc1NjM2Ng== | 9599 | 2021-03-27T16:15:47Z | 2021-03-27T16:15:47Z | OWNER | https://timezones-api.datasette.io/ is now up and running on Cloud Run. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842416110 | |
https://github.com/simonw/datasette/issues/1278#issuecomment-808652008 | https://api.github.com/repos/simonw/datasette/issues/1278 | 808652008 | MDEyOklzc3VlQ29tbWVudDgwODY1MjAwOA== | 9599 | 2021-03-27T04:47:17Z | 2021-03-27T04:47:17Z | OWNER | https://github.com/simonw/timezones-api is that project, it's pretty old now. I'll try to get it running on Cloud Run. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842416110 | |
https://github.com/simonw/datasette/issues/1258#issuecomment-808651088 | https://api.github.com/repos/simonw/datasette/issues/1258 | 808651088 | MDEyOklzc3VlQ29tbWVudDgwODY1MTA4OA== | 9599 | 2021-03-27T04:41:52Z | 2021-03-27T04:42:14Z | OWNER | Right now they look like this: ```yaml databases: fixtures: queries: neighborhood_search: params: - text ``` In addition to being able to specify defaults, I'd also like to add other things in the future - most significantly the ability to specify a different input widget (e.g. textarea v.s. single-line input) So maybe this looks like: ```yaml params: - name: text default: "" - name: age widget: number ``` | { "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
828858421 | |
https://github.com/simonw/datasette/issues/1258#issuecomment-808650266 | https://api.github.com/repos/simonw/datasette/issues/1258 | 808650266 | MDEyOklzc3VlQ29tbWVudDgwODY1MDI2Ng== | 9599 | 2021-03-27T04:37:07Z | 2021-03-27T04:37:07Z | OWNER | I like that idea. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
828858421 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-808649480 | https://api.github.com/repos/simonw/datasette/issues/1249 | 808649480 | MDEyOklzc3VlQ29tbWVudDgwODY0OTQ4MA== | 9599 | 2021-03-27T04:32:10Z | 2021-03-27T04:32:10Z | OWNER | I'll close this issue after I ship Datasette 0.56 and confirm that the Dockerfile was correctly built and published to Docker Hub. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-808649322 | https://api.github.com/repos/simonw/datasette/issues/1249 | 808649322 | MDEyOklzc3VlQ29tbWVudDgwODY0OTMyMg== | 9599 | 2021-03-27T04:31:28Z | 2021-03-27T04:31:28Z | OWNER | One last test of that Dockerfile: ``` (datasette) datasette % docker build -f Dockerfile -t datasetteproject/datasette:0.55a --build-arg VERSION=0.55 . (datasette) datasette % docker run datasetteproject/datasette:0.55a datasette --get '/-/versions.json' | jq { "python": { "version": "3.9.2", "full": "3.9.2 (default, Feb 19 2021, 17:23:45) \n[GCC 8.3.0]" }, "datasette": { "version": "0.55" }, "asgi": "3.0", "uvicorn": "0.13.4", "sqlite": { "version": "3.27.2", "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "extensions": { "json1": null }, "compile_options": [ "COMPILER=gcc-8.3.0", "ENABLE_COLUMN_METADATA", "ENABLE_DBSTAT_VTAB", "ENABLE_FTS3", "ENABLE_FTS3_PARENTHESIS", "ENABLE_FTS3_TOKENIZER", "ENABLE_FTS4", "ENABLE_FTS5", "ENABLE_JSON1", "ENABLE_LOAD_EXTENSION", "ENABLE_PREUPDATE_HOOK", "ENABLE_RTREE", "ENABLE_SESSION", "ENABLE_STMTVTAB", "ENABLE_UNLOCK_NOTIFY", "ENABLE_UPDATE_DELETE_LIMIT", "HAVE_ISNAN", "LIKE_DOESNT_MATCH_BLOBS", "MAX_SCHEMA_RETRY=25", "MAX_VARIABLE_NUMBER=250000", "OMIT_LOOKASIDE", "SECURE_DELETE", "SOUNDEX", "TEMP_STORE=1", "THREADSAFE=1", "USE_URI" ] } } (datasette) datasette % docker run datasetteproject/datasette:0.55a datasette --get '/-/versions.json' --load-extension=spatialite | jq { "python": { "version": "3.9.2", "full": "3.9.2 (default, Feb 19 2021, 17:23:45) \n[GCC 8.3.0]" }, "datasette": { "version": "0.55" }, "asgi": "3.0", "uvicorn": "0.13.4", "sqlite": { "version": "3.27.2", "fts_versions": [ "FTS5", "FTS4", "FTS3" ], "extensions": { "json1": null, "spatialite": "5.0.1" }, "compile_options": [ "COMPILER=gcc-8.3.0", "ENABLE_COLUMN_METADATA", "ENABLE_DBSTAT_VTAB", "ENAB… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1272#issuecomment-808648974 | https://api.github.com/repos/simonw/datasette/issues/1272 | 808648974 | MDEyOklzc3VlQ29tbWVudDgwODY0ODk3NA== | 9599 | 2021-03-27T04:29:42Z | 2021-03-27T04:29:42Z | OWNER | I'm skipping this for the moment because the new Dockerfile shape introduced in https://github.com/simonw/datasette/issues/1249#issuecomment-804404544 isn't compatible with this technique, since it installs Datasette from PyPI rather than directly from the repo. Will need to change that if I want to do this unit tests thing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
838245338 | |
https://github.com/simonw/datasette/issues/1272#issuecomment-808647937 | https://api.github.com/repos/simonw/datasette/issues/1272 | 808647937 | MDEyOklzc3VlQ29tbWVudDgwODY0NzkzNw== | 9599 | 2021-03-27T04:23:19Z | 2021-03-27T04:23:36Z | OWNER | Part of the challenge here is only running if a Docker daemon is available. I think this pattern works, in `tests/test_dockerfile.py`: ```python import httpx import pathlib import pytest import subprocess root = pathlib.Path(__file__).parent.parent def docker_is_available(): try: client = httpx.Client( transport=httpx.HTTPTransport(uds="/var/run/docker.sock") ) client.get("http://docker/info") return True except httpx.ConnectError: return False @pytest.fixture def build_container(): assert (root / "Dockerfile").exists() subprocess.check_call([ "docker", "build", str(root), "-t", "datasette-dockerfile-test" ]) @pytest.mark.skipif(not docker_is_available(), reason="Docker is not available" ) def test_dockerfile(build_container): output = subprocess.check_output([ "docker", "run", "datasette-dockerfile-test", "datasette", "--get", "/_memory?sql=select+1&shape=_array" ]) assert False, "Implement better assertion here" ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
838245338 | |
https://github.com/simonw/datasette/issues/1276#issuecomment-808642405 | https://api.github.com/repos/simonw/datasette/issues/1276 | 808642405 | MDEyOklzc3VlQ29tbWVudDgwODY0MjQwNQ== | 9599 | 2021-03-27T03:53:18Z | 2021-03-27T03:53:18Z | OWNER | That's really odd. What version of SQLite are you using on the server? You can tell by visiting `https://your-site/-/versions` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841456306 | |
https://github.com/simonw/datasette/issues/1277#issuecomment-808641846 | https://api.github.com/repos/simonw/datasette/issues/1277 | 808641846 | MDEyOklzc3VlQ29tbWVudDgwODY0MTg0Ng== | 9599 | 2021-03-27T03:49:34Z | 2021-03-27T03:49:34Z | OWNER | I fixed this already, it's a duplicate of #1239 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842212586 | |
https://github.com/simonw/sqlite-utils/issues/252#issuecomment-808302971 | https://api.github.com/repos/simonw/sqlite-utils/issues/252 | 808302971 | MDEyOklzc3VlQ29tbWVudDgwODMwMjk3MQ== | 9599 | 2021-03-26T15:21:38Z | 2021-03-26T15:21:38Z | OWNER | Already got that! It's the `--nl` option - works for both importing and exporting data: https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-json-data | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
842062949 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-807647791 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 807647791 | MDEyOklzc3VlQ29tbWVudDgwNzY0Nzc5MQ== | 9599 | 2021-03-25T22:42:48Z | 2021-03-25T22:44:31Z | OWNER | Idea: enhance `lambda` to allow it to return a dictionary of values, which will then be used to populate new columns. Use a `--multicolumn` option to indicate this: sqlite-utils convert lambda mydb.db mytable mycolumn \ --code '{"first_name": value.split()[0], "last_name": value.split()[1]}' \ --multicolumn --drop The `--drop` means "drop the `mycolumn` column after making this change". Maybe `--multi` is a better name than `--multicolumn` here, since either way it's going to need additional explanation somewhere. Would this overlap with #239 at all? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-807642041 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 807642041 | MDEyOklzc3VlQ29tbWVudDgwNzY0MjA0MQ== | 9599 | 2021-03-25T22:39:22Z | 2021-03-25T22:39:22Z | OWNER | Here's the full current implementation of that tool: https://github.com/simonw/sqlite-transform/blob/0.5/sqlite_transform/cli.py My current plan is to make this functionality available as the following: sqlite-utils convert jsonsplit mydb.db mytable mycolumn sqlite-utils convert parsedatetime mydb.db mytable mycolumn sqlite-utils convert parsedate mydb.db mytable mycolumn sqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()' | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/datasette/issues/1258#issuecomment-807459633 | https://api.github.com/repos/simonw/datasette/issues/1258 | 807459633 | MDEyOklzc3VlQ29tbWVudDgwNzQ1OTYzMw== | 1385831 | 2021-03-25T20:48:33Z | 2021-03-25T20:49:34Z | NONE | What about allowing default parameters when defining the query in metadata.yml? Something like: ``` databases: fec: queries: search_by_name: params: - q default-param-values: q: "text to search" sql: |- SELECT... ``` For now, I'm using a custom database-<file>.html file that hardcodes a default param in the link, but I'd rather not customize the template just for that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
828858421 | |
https://github.com/simonw/datasette/issues/741#issuecomment-806166575 | https://api.github.com/repos/simonw/datasette/issues/741 | 806166575 | MDEyOklzc3VlQ29tbWVudDgwNjE2NjU3NQ== | 9599 | 2021-03-24T20:30:33Z | 2021-03-24T20:30:33Z | OWNER | `datasette package` is a mostly unmaintained feature at this point - it has a bit of test coverage but I've not made any improvements to it in a few years, and I don't use it for my own projects. I'll make this change to `package` at the same time as I land it for `publish` though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
607223136 | |
https://github.com/simonw/datasette/issues/741#issuecomment-806010960 | https://api.github.com/repos/simonw/datasette/issues/741 | 806010960 | MDEyOklzc3VlQ29tbWVudDgwNjAxMDk2MA== | 596279 | 2021-03-24T17:19:42Z | 2021-03-24T17:19:42Z | NONE | Ah, okay so `--extra-options` applies to both `datasette publish` and `datasette package`? There wren't any examples of it being used with `publish` in the docs, so this tripped me up for a bit. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
607223136 | |
https://github.com/simonw/datasette/issues/1274#issuecomment-805216038 | https://api.github.com/repos/simonw/datasette/issues/1274 | 805216038 | MDEyOklzc3VlQ29tbWVudDgwNTIxNjAzOA== | 9599 | 2021-03-23T20:14:53Z | 2021-03-23T20:14:53Z | OWNER | Yes this is one of the main reasons I'm planning to switch to encouraging YAML be default instead of JSON (while still supporting JSON) - YAML supports comments and multi-line strings. See #1153 for YAML by default in the documentation. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
839008371 | |
https://github.com/simonw/datasette/issues/1274#issuecomment-805214307 | https://api.github.com/repos/simonw/datasette/issues/1274 | 805214307 | MDEyOklzc3VlQ29tbWVudDgwNTIxNDMwNw== | 7476523 | 2021-03-23T20:12:29Z | 2021-03-23T20:12:29Z | CONTRIBUTOR | One issue I could see with adding first class support for metadata in hjson format is that this would require adding an additional dependency to handle this, for a feature that would be unused by many users. I wonder if this could fit in as a plugin instead; if a hook existed for loading metadata (maybe as part of https://github.com/simonw/datasette/issues/860) the metadata could then come from any source, as specified by plugins, e.g. hjson, toml, XML, a database table etc. Until/unless this exists, a few ideas for how you could add comments: - Using YAML as you suggest. - A common pattern is adding a `"comment"` key for comments to any object in JSON - I don't think including an unnecessary key like this would break anything in Datasette, but not certain. - You could use another tool as a preprocessor for your JSON metadata - e.g. hjson or Jsonnet. You'd write the metadata in that format, and then convert that into JSON to actually use as your final metadata. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
839008371 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805109341 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805109341 | MDEyOklzc3VlQ29tbWVudDgwNTEwOTM0MQ== | 9599 | 2021-03-23T17:55:48Z | 2021-03-23T18:41:57Z | OWNER | Beginnings of a UI element for switching between them: ```html <div style="border: 1px solid rgb(225, 228, 229); background-color: rgb(238, 255, 204); padding: 0.3em; position: relative; top: 3px; font-family: courier;"> <a href="#" style="display: inline-block; padding-left: 0px; padding-right: 2em;">JSON</a> <a href="#" style="display: inline-block;">YAML</a> </div> ``` <img width="646" alt="Metadata_—_Datasette_documentation" src="https://user-images.githubusercontent.com/9599/112194637-51f92500-8bc6-11eb-9662-3faa7ef37538.png"> That `<pre>` has a padding of 12px, so using 12px padding on the tab links should get them to line up better. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805047117 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805047117 | MDEyOklzc3VlQ29tbWVudDgwNTA0NzExNw== | 9599 | 2021-03-23T16:30:15Z | 2021-03-23T16:46:06Z | OWNER | https://cdnjs.cloudflare.com/ajax/libs/js-yaml/4.0.0/js-yaml.min.js is only 12.5KB zipped, 38KB total - so that's not a bad option. https://github.com/nodeca/js-yaml | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1270#issuecomment-805058241 | https://api.github.com/repos/simonw/datasette/issues/1270 | 805058241 | MDEyOklzc3VlQ29tbWVudDgwNTA1ODI0MQ== | 9599 | 2021-03-23T16:45:39Z | 2021-03-23T16:45:39Z | OWNER | I managed to build SpatiaLite such that this isn't necessary any more. I'm still interested in pursuing this further though - it feels like it could be a more robust way of implementing timeouts, but I need to prove to myself that it's better (maybe better performance, or handles more edge-cases?). Not sure how to prove that yet. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
837350092 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805056806 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805056806 | MDEyOklzc3VlQ29tbWVudDgwNTA1NjgwNg== | 9599 | 2021-03-23T16:43:38Z | 2021-03-23T16:43:38Z | OWNER | I used this code to get that: ```javascript var jsonVersion = JSON.stringify(window.jsyaml.load(document.querySelector('.highlight-yaml').textContent), null, 4); div.querySelector('.highlight pre').innerText = jsonVersion; div.querySelector('.highlight pre').style.whiteSpace = 'pre-wrap' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805055291 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805055291 | MDEyOklzc3VlQ29tbWVudDgwNTA1NTI5MQ== | 9599 | 2021-03-23T16:41:31Z | 2021-03-23T16:41:31Z | OWNER | One downside of doing this conversion in JavaScript: it's much harder to get the same JSON syntax highlighting as that provided by Sphinx: <img width="655" alt="Metadata_—_Datasette_documentation" src="https://user-images.githubusercontent.com/9599/112183909-ef9b2700-8bbb-11eb-882b-aeb387c29e61.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805050163 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805050163 | MDEyOklzc3VlQ29tbWVudDgwNTA1MDE2Mw== | 9599 | 2021-03-23T16:34:35Z | 2021-03-23T16:35:32Z | OWNER | https://docs.datasette.io/en/stable/metadata.html has this example: ```yaml title: Demonstrating Metadata from YAML description_html: |- <p>This description includes a long HTML string</p> <ul> <li>YAML is better for embedding HTML strings than JSON!</li> </ul> license: ODbL license_url: https://opendatacommons.org/licenses/odbl/ databases: fixtures: tables: no_primary_key: hidden: true queries: neighborhood_search: sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood; title: Search neighborhoods description_html: |- <p>This demonstrates <em>basic</em> LIKE search ``` I ran this in the browser dev tools: ```javascript var s = document.createElement('script') s.src = 'https://cdnjs.cloudflare.com/ajax/libs/js-yaml/4.0.0/js-yaml.min.js' document.head.appendChild(s) var yamlExample = document.querySelector('.highlight-yaml').textContent); console.log(JSON.stringify(window.jsyaml.load(yamlExample), null, 4)) ``` And got: ```json { "title": "Demonstrating Metadata from YAML", "description_html": "<p>This description includes a long HTML string</p>\n<ul>\n <li>YAML is better for embedding HTML strings than JSON!</li>\n</ul>", "license": "ODbL", "license_url": "https://opendatacommons.org/licenses/odbl/", "databases": { "fixtures": { "tables": { "no_primary_key": { "hidden": true } }, "queries": { "neighborhood_search": { "sql": "select neighborhood, facet_cities.name, state\nfrom facetable join facet_cities on facetable.city_id = facet_cities.id\nwhere neighborhood like '%' || :text || '%' order by neighborhood;", "title": "Search neighborhoods", … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805042880 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805042880 | MDEyOklzc3VlQ29tbWVudDgwNTA0Mjg4MA== | 9599 | 2021-03-23T16:24:32Z | 2021-03-23T16:24:32Z | OWNER | ... actually I think I would do that conversion in Python. The client-side YAML parsers all look a little bit heavy to me in terms of additional page weight. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-805041522 | https://api.github.com/repos/simonw/datasette/issues/1153 | 805041522 | MDEyOklzc3VlQ29tbWVudDgwNTA0MTUyMg== | 9599 | 2021-03-23T16:22:46Z | 2021-03-23T16:22:46Z | OWNER | That's a good idea. I could do that with JavaScript - loading YAML and converting it to JSON in JavaScript shouldn't be hard, and it's better than JSON-to-YAML because there's only one correct JSON representation of a YAML file whereas you can represent a JSON document in YAML in a bunch of different ways. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-805033155 | https://api.github.com/repos/simonw/datasette/issues/1249 | 805033155 | MDEyOklzc3VlQ29tbWVudDgwNTAzMzE1NQ== | 9599 | 2021-03-23T16:12:13Z | 2021-03-23T16:12:13Z | OWNER | Don't forget to update this bit of the docs: https://docs.datasette.io/en/0.55/spatialite.html#building-spatialite-from-source > The packaged versions of SpatiaLite usually provide SpatiaLite 4.3.0a. For an example of how to build the most recent unstable version, 4.4.0-RC0 (which includes the powerful [VirtualKNN module](https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN)), take a look at the [Datasette Dockerfile](https://github.com/simonw/datasette/blob/master/Dockerfile). See also #1273 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/pull/1159#issuecomment-804698315 | https://api.github.com/repos/simonw/datasette/issues/1159 | 804698315 | MDEyOklzc3VlQ29tbWVudDgwNDY5ODMxNQ== | 552629 | 2021-03-23T07:58:28Z | 2021-03-23T07:58:38Z | NONE | @mroswell Did you try it with more columns ? The display is flexible and columns get closer as new ones are added. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
774332247 | |
https://github.com/simonw/datasette/issues/1153#issuecomment-804640440 | https://api.github.com/repos/simonw/datasette/issues/1153 | 804640440 | MDEyOklzc3VlQ29tbWVudDgwNDY0MDQ0MA== | 192568 | 2021-03-23T05:58:20Z | 2021-03-23T05:58:20Z | CONTRIBUTOR | Could there be a little widget that offers conversion from one to the other? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771202454 | |
https://github.com/simonw/datasette/pull/1159#issuecomment-804639427 | https://api.github.com/repos/simonw/datasette/issues/1159 | 804639427 | MDEyOklzc3VlQ29tbWVudDgwNDYzOTQyNw== | 192568 | 2021-03-23T05:56:02Z | 2021-03-23T05:56:02Z | CONTRIBUTOR | With just three facets, I like it, but it does take more horizontal space. Would be nice to have a switch somewhere, enabling either original compact option or this proposed more-readable option. Also some control over word wrap (width setting) and facet spacing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
774332247 | |
https://github.com/simonw/datasette/issues/164#issuecomment-804541064 | https://api.github.com/repos/simonw/datasette/issues/164 | 804541064 | MDEyOklzc3VlQ29tbWVudDgwNDU0MTA2NA== | 192568 | 2021-03-23T02:45:12Z | 2021-03-23T02:45:12Z | CONTRIBUTOR | "datasette skeleton" feature removed #476 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
280013907 | |
https://github.com/simonw/datasette/issues/163#issuecomment-804540869 | https://api.github.com/repos/simonw/datasette/issues/163 | 804540869 | MDEyOklzc3VlQ29tbWVudDgwNDU0MDg2OQ== | 9599 | 2021-03-23T02:44:33Z | 2021-03-23T02:44:33Z | OWNER | Comments welcome! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
279547886 | |
https://github.com/simonw/datasette/issues/163#issuecomment-804539729 | https://api.github.com/repos/simonw/datasette/issues/163 | 804539729 | MDEyOklzc3VlQ29tbWVudDgwNDUzOTcyOQ== | 192568 | 2021-03-23T02:41:14Z | 2021-03-23T02:41:14Z | CONTRIBUTOR | I'm visiting old issues for context while learning datasette. Let me know if okay to make the occasional comment like this one. querystring argument now located at: https://docs.datasette.io/en/latest/settings.html#sql-time-limit-ms | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
279547886 | |
https://github.com/simonw/datasette/issues/88#issuecomment-804471733 | https://api.github.com/repos/simonw/datasette/issues/88 | 804471733 | MDEyOklzc3VlQ29tbWVudDgwNDQ3MTczMw== | 192568 | 2021-03-22T23:46:36Z | 2021-03-22T23:46:36Z | CONTRIBUTOR | Google Map API limits seem to prevent https://nhs-england-map.netlify.com from being a working demo. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273775212 | |
https://github.com/simonw/datasette/pull/1271#issuecomment-804299406 | https://api.github.com/repos/simonw/datasette/issues/1271 | 804299406 | MDEyOklzc3VlQ29tbWVudDgwNDI5OTQwNg== | 9599 | 2021-03-22T18:36:14Z | 2021-03-22T21:49:27Z | OWNER | This isn't actually working - the outer code attempts to send an `.interrupt()` call to the connection object via the `connections` thread-local, which doesn't work because it's a thread-local so the connection isn't visible to that code. Need to figure out how to communicate with that thread properly. Also a test that fails in this particular case would be a good idea! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
837956424 | |
https://github.com/simonw/datasette/issues/1149#issuecomment-804415619 | https://api.github.com/repos/simonw/datasette/issues/1149 | 804415619 | MDEyOklzc3VlQ29tbWVudDgwNDQxNTYxOQ== | 192568 | 2021-03-22T21:43:16Z | 2021-03-22T21:43:16Z | CONTRIBUTOR | Sounds like a good idea. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
769520939 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804406675 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804406675 | MDEyOklzc3VlQ29tbWVudDgwNDQwNjY3NQ== | 9599 | 2021-03-22T21:26:27Z | 2021-03-22T21:26:27Z | OWNER | (Without the `apt-get update ...` SpatiaLite line it's 125MB) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804404544 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804404544 | MDEyOklzc3VlQ29tbWVudDgwNDQwNDU0NA== | 9599 | 2021-03-22T21:22:56Z | 2021-03-22T21:24:24Z | OWNER | Final version of Dockerfile which installs the specified version from GitHub: docker build . -t datasette-spatialite --build-arg VERSION=0.55 ```dockerfile FROM python:3.9.2-slim-buster as build # Version of Datasette to install, e.g. 0.55 # docker build . -t datasette --build-arg VERSION=0.55 ARG VERSION # software-properties-common provides add-apt-repository # which we need in order to install a more recent release # of libsqlite3-mod-spatialite from the sid distribution RUN apt-get update && \ apt-get -y --no-install-recommends install software-properties-common && \ add-apt-repository "deb http://httpredir.debian.org/debian sid main" && \ apt-get update && \ apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite && \ apt-get remove -y software-properties-common && \ apt clean && \ rm -rf /var/lib/apt && \ rm -rf /var/lib/dpkg RUN pip install https://github.com/simonw/datasette/archive/refs/tags/${VERSION}.zip && \ find /usr/local/lib -name '__pycache__' | xargs rm -r && \ rm -rf /root/.cache/pip EXPOSE 8001 CMD ["datasette"] ``` Run against 0.55 this produces an image of 262MB | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804384196 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804384196 | MDEyOklzc3VlQ29tbWVudDgwNDM4NDE5Ng== | 9599 | 2021-03-22T20:48:46Z | 2021-03-22T20:48:46Z | OWNER | I think part of the reason it's smaller is that I ran `pip install datasette` instead of using `COPY . /datasette` followed by `pip install /datasette`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804380181 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804380181 | MDEyOklzc3VlQ29tbWVudDgwNDM4MDE4MQ== | 9599 | 2021-03-22T20:42:16Z | 2021-03-22T20:42:16Z | OWNER | Considering the image on Docker Hub is 383MB, I'm happy with getting that down to 262MB. I'm going to stop looking for new optimizations here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804379644 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804379644 | MDEyOklzc3VlQ29tbWVudDgwNDM3OTY0NA== | 9599 | 2021-03-22T20:41:23Z | 2021-03-22T20:41:23Z | OWNER | I tried adding `apt-get remove -y software-properties-common &&` to remove `software-properties-common` but it made no difference to the image size. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804372977 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804372977 | MDEyOklzc3VlQ29tbWVudDgwNDM3Mjk3Nw== | 9599 | 2021-03-22T20:30:37Z | 2021-03-22T20:30:37Z | OWNER | I tried copying just the `mod_spatialite.so` file into a second stage build but it failed. So I ran `bash` in a working image and used `ldd` to figure out what it was linked to: ``` root@39683f91e588:/usr/lib/x86_64-linux-gnu# ldd mod_spatialite.so linux-vdso.so.1 (0x00007ffd021f4000) libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f5c75412000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f5c753f0000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5c752ac000) libminizip.so.1 => /usr/lib/x86_64-linux-gnu/libminizip.so.1 (0x00007f5c750a0000) librttopo.so.1 => /usr/lib/x86_64-linux-gnu/librttopo.so.1 (0x00007f5c75028000) libfreexl.so.1 => /usr/lib/x86_64-linux-gnu/libfreexl.so.1 (0x00007f5c7501c000) libproj.so.19 => /usr/lib/x86_64-linux-gnu/libproj.so.19 (0x00007f5c74ca7000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f5c74a89000) libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f5c74967000) libgeos_c.so.1 => /usr/lib/x86_64-linux-gnu/libgeos_c.so.1 (0x00007f5c7492b000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5c74766000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5c74760000) libicuuc.so.67 => /usr/lib/x86_64-linux-gnu/libicuuc.so.67 (0x00007f5c74575000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f5c7454d000) /lib64/ld-linux-x86-64.so.2 (0x00007f5c75d49000) libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f5c744c7000) libcurl-gnutls.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007f5c74439000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f5c7426c000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f5c74250000) libgeos-3.9.0.so => /usr/lib/x86_64-linux-gnu/libgeos-3.9.0.so (0x00007f5c74040000) libicudata.so.67 => /usr/lib/x86_64-linux-gnu/libicudata.so.67 (0x00007f5c72527000) libwebp.so.6 => /usr/lib/x86_64-linux-gnu/libwebp.so.6 (0x00007f5c724bc000) libzstd.so.1 => /usr/lib/x86_64-linux-gnu/lib… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804368372 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804368372 | MDEyOklzc3VlQ29tbWVudDgwNDM2ODM3Mg== | 9599 | 2021-03-22T20:22:43Z | 2021-03-22T20:22:43Z | OWNER | ```dockerfile FROM python:3.9.2-slim-buster as build # software-properties-common provides add-apt-repository RUN apt-get update && \ apt-get -y --no-install-recommends install software-properties-common && \ add-apt-repository "deb http://httpredir.debian.org/debian sid main" && \ apt-get update && \ apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/apt && \ rm -rf /var/lib/dpkg RUN pip install datasette && \ find /usr/local/lib -name '__pycache__' | xargs rm -r && \ rm -rf /root/.cache/pip EXPOSE 8001 CMD ["datasette"] ``` 262 MB | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804363687 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804363687 | MDEyOklzc3VlQ29tbWVudDgwNDM2MzY4Nw== | 9599 | 2021-03-22T20:15:00Z | 2021-03-22T20:15:00Z | OWNER | ``` RUN pip install datasette && \ find /usr/local/lib -name '__pycache__' | xargs rm -r ``` That dropped it to 265MB. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804360701 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804360701 | MDEyOklzc3VlQ29tbWVudDgwNDM2MDcwMQ== | 9599 | 2021-03-22T20:10:07Z | 2021-03-22T20:10:07Z | OWNER | Adding `--no-install-recommends` dropped it to 275MB | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804347152 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804347152 | MDEyOklzc3VlQ29tbWVudDgwNDM0NzE1Mg== | 9599 | 2021-03-22T19:47:56Z | 2021-03-22T19:48:03Z | OWNER | I wrote a bunch of tips on creating smaller Docker images here: https://simonwillison.net/2018/Nov/19/smaller-python-docker-images/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804344553 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804344553 | MDEyOklzc3VlQ29tbWVudDgwNDM0NDU1Mw== | 9599 | 2021-03-22T19:43:25Z | 2021-03-22T19:43:25Z | OWNER | Does `--no-install-recommends` make a difference? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804338678 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804338678 | MDEyOklzc3VlQ29tbWVudDgwNDMzODY3OA== | 9599 | 2021-03-22T19:33:43Z | 2021-03-22T19:33:43Z | OWNER | Replacing `rm -rf /var/lib/{apt,dpkg,cache,log}/` with ``` rm -rf /var/lib/apt && \ rm -rf /var/lib/dpkg ``` Got the size down to 305MB. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804318314 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804318314 | MDEyOklzc3VlQ29tbWVudDgwNDMxODMxNA== | 9599 | 2021-03-22T19:04:30Z | 2021-03-22T19:04:30Z | OWNER | Considering the image on Docker Hub right now is `383MB` this is actually an improvement. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804317545 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804317545 | MDEyOklzc3VlQ29tbWVudDgwNDMxNzU0NQ== | 9599 | 2021-03-22T19:03:22Z | 2021-03-22T19:03:22Z | OWNER | This Dockerfile: ```dockerfile FROM python:3.9.2-slim-buster as build # software-properties-common provides add-apt-repository RUN apt-get update && \ apt-get -y install software-properties-common && \ add-apt-repository "deb http://httpredir.debian.org/debian sid main" && \ apt-get update && \ apt-get -t sid install -y libsqlite3-mod-spatialite && \ apt clean && \ rm -rf /var/lib/{apt,dpkg,cache,log}/ RUN pip install datasette EXPOSE 8001 CMD ["datasette"] ``` Produces a 344MB image that includes a working SpatiaLite 5.0 module. And weirdly... it doesn't exhibit the hanging bug! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804310353 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804310353 | MDEyOklzc3VlQ29tbWVudDgwNDMxMDM1Mw== | 9599 | 2021-03-22T18:52:12Z | 2021-03-22T18:52:12Z | OWNER | This Dockerfile: ```dockerfile FROM python:3.9.2-slim-buster as build # Setup build dependencies RUN apt update \ && apt install -y python3-dev build-essential wget libxml2-dev libproj-dev \ libminizip-dev libgeos-dev libsqlite3-dev zlib1g-dev pkg-config git \ && apt clean RUN wget "https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz" && tar xzf sqlite-autoconf-3340100.tar.gz \ && cd sqlite-autoconf-3340100 && ./configure --disable-static --enable-fts5 --enable-json1 \ CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1" \ && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/freexl-1.0.6.tar.gz" && tar zxf freexl-1.0.6.tar.gz \ && cd freexl-1.0.6 && ./configure && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1.tar.gz" && tar zxf libspatialite-5.0.1.tar.gz \ && cd libspatialite-5.0.1 && ./configure --disable-rttopo && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/readosm-sources/readosm-1.1.0.tar.gz" && tar zxf readosm-1.1.0.tar.gz && cd readosm-1.1.0 && ./configure && make && make install RUN wget "http://www.gaia-gis.it/gaia-sins/spatialite-tools-5.0.0.tar.gz" && tar zxf spatialite-tools-5.0.0.tar.gz \ && cd spatialite-tools-5.0.0 && ./configure --disable-rttopo && make && make install # Add local code to the image instead of fetching from pypi. #COPY . /datasette #RUN pip install /datasette RUN pip install datasette FROM python:3.9.2-slim-buster # Copy python dependencies and spatialite libraries COPY --from=build /usr/local/lib/ /usr/local/lib/ # Copy executables COPY --from=build /usr/local/bin /usr/local/bin # Copy spatial extensions COPY --from=build /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ENV LD_LIBRARY_PATH=/usr/local/lib EXPOSE 8001 CMD ["datasette"] ``` Produced a 448MB image. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804309510 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804309510 | MDEyOklzc3VlQ29tbWVudDgwNDMwOTUxMA== | 9599 | 2021-03-22T18:50:50Z | 2021-03-22T18:50:50Z | OWNER | Ideally I'd like to use the Debian stable `python:3.9.2-slim-buster` base image but install SpatiaLite from Debian unstable here: https://packages.debian.org/sid/libspatialite7 This pattern might let me do that: https://github.com/helmesjo/cpp_bash_utils/blob/f031e926249f8e2d7f260f22dc8974c6d5be11fe/docker/images/linux-gcc.dockerfile#L20-L24 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/pull/1271#issuecomment-804265042 | https://api.github.com/repos/simonw/datasette/issues/1271 | 804265042 | MDEyOklzc3VlQ29tbWVudDgwNDI2NTA0Mg== | 9599 | 2021-03-22T17:45:45Z | 2021-03-22T17:45:45Z | OWNER | I can remove this code too: https://github.com/simonw/datasette/blob/6f41c8a2bef309a66588b2875c3e24d26adb4850/datasette/database.py#L190-L192 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
837956424 | |
https://github.com/simonw/datasette/issues/1249#issuecomment-804263434 | https://api.github.com/repos/simonw/datasette/issues/1249 | 804263434 | MDEyOklzc3VlQ29tbWVudDgwNDI2MzQzNA== | 9599 | 2021-03-22T17:43:25Z | 2021-03-22T17:43:25Z | OWNER | I figured out the cause of the hang in #1268 - it was caused by `select count(*) from SpatialIndex` interacting badly with the `set_progress_handler()` mechanism I was using to implement query time limits. #1271 has a replacement for that using `asyncio.wait_for()` and `conn.interrupt()` which should resolve the SpatiaLite issue too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
824064069 | |
https://github.com/simonw/datasette/issues/1268#issuecomment-804261915 | https://api.github.com/repos/simonw/datasette/issues/1268 | 804261915 | MDEyOklzc3VlQ29tbWVudDgwNDI2MTkxNQ== | 9599 | 2021-03-22T17:41:12Z | 2021-03-22T17:41:12Z | OWNER | Closing this because I've figured out the root of the problem now, and I have a potential solution. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
837308703 |