issue_comments
10,495 rows sorted by updated_at descending
This data as json, CSV (advanced)
issue >30
- Show column metadata plus links for foreign keys on arbitrary query results 50
- Redesign default .json format 48
- Rethink how .ext formats (v.s. ?_format=) works before 1.0 48
- JavaScript plugin hooks mechanism similar to pluggy 47
- Updated Dockerfile with SpatiaLite version 5.0 45
- Complete refactor of TableView and table.html template 45
- Port Datasette to ASGI 42
- Authentication (and permissions) as a core concept 40
- Deploy a live instance of demos/apache-proxy 34
- await datasette.client.get(path) mechanism for executing internal requests 33
- Maintain an in-memory SQLite table of connected databases and their tables 32
- Ability to sort (and paginate) by column 31
- link_or_copy_directory() error - Invalid cross-device link 28
- Export to CSV 27
- base_url configuration setting 27
- Documentation with recommendations on running Datasette in production without using Docker 27
- Optimize all those calls to index_list and foreign_key_list 27
- Support cross-database joins 26
- Ability for a canned query to write to the database 26
- table.transform() method for advanced alter table 26
- New pattern for views that return either JSON or HTML, available for plugins 26
- Proof of concept for Datasette on AWS Lambda with EFS 25
- WIP: Add Gmail takeout mbox import 25
- Redesign register_output_renderer callback 24
- Make it easier to insert geometries, with documentation and maybe code 24
- "datasette insert" command and plugin hook 23
- Datasette Plugins 22
- .json and .csv exports fail to apply base_url 22
- Idea: import CSV to memory, run SQL, export in a single command 22
- Plugin hook for dynamic metadata 22
- …
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1095687566 | https://github.com/simonw/datasette/issues/1708#issuecomment-1095687566 | https://api.github.com/repos/simonw/datasette/issues/1708 | IC_kwDOBm6k_c5BTt2O | simonw 9599 | 2022-04-11T23:24:30Z | 2022-04-11T23:24:30Z | OWNER | Redesigned template contextWarning: if you use any custom templates with your Datasette instance they are likely to break when you upgrade to 1.0. The template context has been redesigned to be based on the documented JSON API. This means that the template context can be considered stable going forward, so any custom templates you implement should continue to work when you upgrade Datasette in the future. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette 1.0 alpha upcoming release notes 1200649124 | |
1095673947 | https://github.com/simonw/datasette/issues/1705#issuecomment-1095673947 | https://api.github.com/repos/simonw/datasette/issues/1705 | IC_kwDOBm6k_c5BTqhb | simonw 9599 | 2022-04-11T23:03:49Z | 2022-04-11T23:03:49Z | OWNER | I'll also encourage testing against both Datasette 0.x and Datasette 1.0 using a GitHub Actions matrix. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
How to upgrade your plugin for 1.0 documentation 1197926598 | |
1095673670 | https://github.com/simonw/datasette/issues/1710#issuecomment-1095673670 | https://api.github.com/repos/simonw/datasette/issues/1710 | IC_kwDOBm6k_c5BTqdG | simonw 9599 | 2022-04-11T23:03:25Z | 2022-04-11T23:03:25Z | OWNER | Dupe of: - #1705 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Guide for plugin authors to upgrade their plugins for 1.0 1200649889 | |
1095671940 | https://github.com/simonw/datasette/issues/1709#issuecomment-1095671940 | https://api.github.com/repos/simonw/datasette/issues/1709 | IC_kwDOBm6k_c5BTqCE | simonw 9599 | 2022-04-11T23:00:39Z | 2022-04-11T23:01:41Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Redesigned JSON API with ?_extra= parameters 1200649502 | |
1095672127 | https://github.com/simonw/datasette/issues/1711#issuecomment-1095672127 | https://api.github.com/repos/simonw/datasette/issues/1711 | IC_kwDOBm6k_c5BTqE_ | simonw 9599 | 2022-04-11T23:00:58Z | 2022-04-11T23:00:58Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Template context powered entirely by the JSON API format 1200650491 | |
1095277937 | https://github.com/simonw/datasette/issues/1707#issuecomment-1095277937 | https://api.github.com/repos/simonw/datasette/issues/1707 | IC_kwDOBm6k_c5BSJ1x | simonw 9599 | 2022-04-11T16:32:31Z | 2022-04-11T16:33:00Z | OWNER | That's a really interesting idea! That page is one of the least developed at the moment. There's plenty of room for it to grow new useful features. I like this suggestion because it feels like a good opportunity to introduce some unobtrusive JavaScript. Could use a details/summary element that uses Could even do something with the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[feature] expanded detail page 1200224939 | |
1094453751 | https://github.com/simonw/datasette/issues/1699#issuecomment-1094453751 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BPAn3 | eyeseast 25778 | 2022-04-11T01:32:12Z | 2022-04-11T01:32:12Z | CONTRIBUTOR | Was looking through old issues and realized a bunch of this got discussed in #1101 (including by me!), so sorry to rehash all this. Happy to help with whatever piece of it I can. Would be very excited to be able to use format plugins with exports. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1094152642 | https://github.com/simonw/datasette/issues/1706#issuecomment-1094152642 | https://api.github.com/repos/simonw/datasette/issues/1706 | IC_kwDOBm6k_c5BN3HC | simonw 9599 | 2022-04-10T01:11:54Z | 2022-04-10T01:11:54Z | OWNER | This relates to this much larger vision: - #417 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[feature] immutable mode for a directory, not just individual sqlite file 1198822563 | |
1094152173 | https://github.com/simonw/datasette/issues/1706#issuecomment-1094152173 | https://api.github.com/repos/simonw/datasette/issues/1706 | IC_kwDOBm6k_c5BN2_t | simonw 9599 | 2022-04-10T01:08:50Z | 2022-04-10T01:08:50Z | OWNER | This is a good idea - it matches the way |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[feature] immutable mode for a directory, not just individual sqlite file 1198822563 | |
1093454899 | https://github.com/simonw/datasette/pull/1693#issuecomment-1093454899 | https://api.github.com/repos/simonw/datasette/issues/1693 | IC_kwDOBm6k_c5BLMwz | simonw 9599 | 2022-04-08T23:07:04Z | 2022-04-08T23:07:04Z | OWNER | Tests failed here due to this issue: - https://github.com/psf/black/pull/2987 A future Black release should fix that. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump black from 22.1.0 to 22.3.0 1184850337 | |
1092850719 | https://github.com/simonw/datasette/pull/1703#issuecomment-1092850719 | https://api.github.com/repos/simonw/datasette/issues/1703 | IC_kwDOBm6k_c5BI5Qf | codecov[bot] 22429695 | 2022-04-08T13:18:04Z | 2022-04-08T13:18:04Z | NONE | Codecov Report
```diff @@ Coverage Diff @@ main #1703 +/-=======================================
Coverage 91.75% 91.75% Continue to review full report at Codecov.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update beautifulsoup4 requirement from <4.11.0,>=4.8.1 to >=4.8.1,<4.12.0 1197298420 | |
1092386254 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092386254 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHH3O | eyeseast 25778 | 2022-04-08T02:39:25Z | 2022-04-08T02:39:25Z | CONTRIBUTOR | And just to think this through a little more, here's what
Alternately, that could be an async generator, like this:
Not sure which makes more sense, but I think this pattern would open up a lot of possibility. If you had your stream_indented_json function, you could do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092370880 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092370880 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHEHA | eyeseast 25778 | 2022-04-08T02:07:40Z | 2022-04-08T02:07:40Z | CONTRIBUTOR | So maybe
And stream gets an iterator, instead of a list of rows, so it can efficiently handle large queries. Maybe it also gets passed a destination stream, or it returns an iterator. I'm not sure what makes more sense. Either way, that might cover both CLI exports and streaming responses. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092361727 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092361727 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHB3_ | simonw 9599 | 2022-04-08T01:47:43Z | 2022-04-08T01:47:43Z | OWNER | A render mode for that plugin hook that writes to a stream is exactly what I have in mind: - #1062 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092357672 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092357672 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHA4o | eyeseast 25778 | 2022-04-08T01:39:40Z | 2022-04-08T01:39:40Z | CONTRIBUTOR |
That's my thinking, too. It's really the thing I've been wanting since writing
I think this probably needs either a new plugin hook separate from |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092321966 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092321966 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BG4Ku | simonw 9599 | 2022-04-08T00:20:32Z | 2022-04-08T00:20:56Z | OWNER | If we do this I'm keen to have it be more than just an alternative to the existing My best thought on how to differentiate them so far is plugins: if Datasette plugins that provide alternative outputs - like One way that could work: a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1087428593 | https://github.com/simonw/datasette/issues/1549#issuecomment-1087428593 | https://api.github.com/repos/simonw/datasette/issues/1549 | IC_kwDOBm6k_c5A0Nfx | fgregg 536941 | 2022-04-04T11:17:13Z | 2022-04-04T11:17:13Z | CONTRIBUTOR | another way to get the behavior of downloading the file is to use the download attribute of the anchor tag https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a#attr-download |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Redesign CSV export to improve usability 1077620955 | |
1086784547 | https://github.com/simonw/datasette/issues/1698#issuecomment-1086784547 | https://api.github.com/repos/simonw/datasette/issues/1698 | IC_kwDOBm6k_c5AxwQj | simonw 9599 | 2022-04-03T06:10:24Z | 2022-04-03T06:10:24Z | OWNER | Warning added here: https://docs.datasette.io/en/latest/publish.html#publishing-to-google-cloud-run |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add a warning about bots and Cloud Run 1190828163 | |
1085323192 | https://github.com/simonw/datasette/issues/1697#issuecomment-1085323192 | https://api.github.com/repos/simonw/datasette/issues/1697 | IC_kwDOBm6k_c5AsLe4 | simonw 9599 | 2022-04-01T02:01:51Z | 2022-04-01T02:01:51Z | OWNER | Huh, turns out |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`Request.fake(..., url_vars={})` 1189113609 | |
1084216224 | https://github.com/simonw/datasette/pull/1574#issuecomment-1084216224 | https://api.github.com/repos/simonw/datasette/issues/1574 | IC_kwDOBm6k_c5An9Og | fs111 33631 | 2022-03-31T07:45:25Z | 2022-03-31T07:45:25Z | NONE | @simonw I like that you want to go "slim by default". Do you want another PR for that or should I just wait? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
introduce new option for datasette package to use a slim base image 1084193403 | |
1083351437 | https://github.com/simonw/datasette/issues/1696#issuecomment-1083351437 | https://api.github.com/repos/simonw/datasette/issues/1696 | IC_kwDOBm6k_c5AkqGN | simonw 9599 | 2022-03-30T16:20:49Z | 2022-03-30T16:21:02Z | OWNER | Maybe like this: ```html 283 rows where dcode = 3 (Human Related: Other)``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Show foreign key label when filtering 1186696202 | |
1082663746 | https://github.com/simonw/datasette/issues/1692#issuecomment-1082663746 | https://api.github.com/repos/simonw/datasette/issues/1692 | IC_kwDOBm6k_c5AiCNC | simonw 9599 | 2022-03-30T06:14:39Z | 2022-03-30T06:14:51Z | OWNER | I like your design, though I think it should be I think |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][feature request]: Support additional script tag attributes when loading custom JS 1182227211 | |
1082661795 | https://github.com/simonw/datasette/issues/1692#issuecomment-1082661795 | https://api.github.com/repos/simonw/datasette/issues/1692 | IC_kwDOBm6k_c5AiBuj | simonw 9599 | 2022-03-30T06:11:41Z | 2022-03-30T06:11:41Z | OWNER | This is a good idea. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][feature request]: Support additional script tag attributes when loading custom JS 1182227211 | |
1082617386 | https://github.com/simonw/datasette/issues/1695#issuecomment-1082617386 | https://api.github.com/repos/simonw/datasette/issues/1695 | IC_kwDOBm6k_c5Ah24q | simonw 9599 | 2022-03-30T04:46:18Z | 2022-03-30T04:46:18Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to un-filter facet not shown for `?col__exact=value` 1185868354 | |
1082617241 | https://github.com/simonw/datasette/issues/1695#issuecomment-1082617241 | https://api.github.com/repos/simonw/datasette/issues/1695 | IC_kwDOBm6k_c5Ah22Z | simonw 9599 | 2022-03-30T04:45:55Z | 2022-03-30T04:45:55Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to un-filter facet not shown for `?col__exact=value` 1185868354 | ||
1082476727 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1082476727 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AhUi3 | strada 770231 | 2022-03-29T23:52:38Z | 2022-03-29T23:52:38Z | NONE | @simonw Thanks for looking into it and documenting the solution! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1081860312 | https://github.com/simonw/datasette/pull/1694#issuecomment-1081860312 | https://api.github.com/repos/simonw/datasette/issues/1694 | IC_kwDOBm6k_c5Ae-DY | codecov[bot] 22429695 | 2022-03-29T13:17:30Z | 2022-03-29T13:17:30Z | NONE | Codecov Report
```diff @@ Coverage Diff @@ main #1694 +/-=======================================
Coverage 91.74% 91.74% Continue to review full report at Codecov.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update click requirement from <8.1.0,>=7.1.1 to >=7.1.1,<8.2.0 1184850675 | |
1081079506 | https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1081079506 | https://api.github.com/repos/simonw/sqlite-utils/issues/421 | IC_kwDOCGYnMM5Ab_bS | learning4life 24938923 | 2022-03-28T19:58:55Z | 2022-03-28T20:05:57Z | NONE | Sure, it is from the documentation example: Extracting columns into a separate table sqlite-utils insert global.db power_plants \ 'global_power_plant_database.csv?raw=true' --csv Extract those columns:sqlite-utils extract global.db power_plants country country_long \ --table countries \ --fk-column country_id \ --rename country_long name ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"Error: near "(": syntax error" when using sqlite-utils indexes CLI 1180427792 | |
1081047053 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1081047053 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5Ab3gN | simonw 9599 | 2022-03-28T19:22:37Z | 2022-03-28T19:22:37Z | OWNER | Wrote about this in my weeknotes: https://simonwillison.net/2022/Mar/28/datasette-auth0/#new-features-as-documentation |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1080141111 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1080141111 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AYaU3 | simonw 9599 | 2022-03-28T03:25:57Z | 2022-03-28T03:54:37Z | OWNER | So now this should solve your problem: ``` echo '[{"name": "notaword"}, {"name": "word"}] ' | python3 -m sqlite_utils insert listings.db listings - --convert ' import enchant d = enchant.Dict("en_US") def convert(row): global d row["is_dictionary_word"] = d.check(row["name"]) ' ``` |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1079806857 | https://github.com/simonw/datasette/issues/1688#issuecomment-1079806857 | https://api.github.com/repos/simonw/datasette/issues/1688 | IC_kwDOBm6k_c5AXIuJ | hydrosquall 9020979 | 2022-03-27T01:01:14Z | 2022-03-27T01:01:14Z | CONTRIBUTOR | Thank you! I went through the cookiecutter template, and published my first package here: https://github.com/hydrosquall/datasette-nteract-data-explorer |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? 1181432624 | |
1079788375 | https://github.com/simonw/datasette/issues/1690#issuecomment-1079788375 | https://api.github.com/repos/simonw/datasette/issues/1690 | IC_kwDOBm6k_c5AXENX | simonw 9599 | 2022-03-26T22:43:00Z | 2022-03-26T22:43:00Z | OWNER | Then I can update this section of the documentation which currently recommends the above pattern: https://docs.datasette.io/en/stable/authentication.html#the-ds-actor-cookie |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Idea: `datasette.set_actor_cookie(response, actor)` 1182141761 | |
1079788346 | https://github.com/simonw/datasette/issues/1690#issuecomment-1079788346 | https://api.github.com/repos/simonw/datasette/issues/1690 | IC_kwDOBm6k_c5AXEM6 | simonw 9599 | 2022-03-26T22:42:40Z | 2022-03-26T22:42:40Z | OWNER | I don't want to do a So |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Idea: `datasette.set_actor_cookie(response, actor)` 1182141761 | |
1079779040 | https://github.com/simonw/datasette/issues/1689#issuecomment-1079779040 | https://api.github.com/repos/simonw/datasette/issues/1689 | IC_kwDOBm6k_c5AXB7g | simonw 9599 | 2022-03-26T21:35:57Z | 2022-03-26T21:35:57Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette.add_message() documentation is incorrect 1182065616 | ||
1079550754 | https://github.com/simonw/datasette/issues/1688#issuecomment-1079550754 | https://api.github.com/repos/simonw/datasette/issues/1688 | IC_kwDOBm6k_c5AWKMi | hydrosquall 9020979 | 2022-03-26T01:27:27Z | 2022-03-26T03:16:29Z | CONTRIBUTOR |
As a workaround, I found I can serve my statics from a non-plugin specific folder using the --static CLI flag.
It's not ideal because it means I'll change the cache pattern path depending on how the plugin is running (via pip install or as a one off script), but it's usable as a workaround. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? 1181432624 | |
1079582485 | https://github.com/simonw/datasette/issues/1688#issuecomment-1079582485 | https://api.github.com/repos/simonw/datasette/issues/1688 | IC_kwDOBm6k_c5AWR8V | simonw 9599 | 2022-03-26T03:15:34Z | 2022-03-26T03:15:34Z | OWNER | Yup, you're right in what you figured out here: stand-alone plugins can't currently package static assets other then using the static folder. The You can run that yourself, or you can run it using this GitHub template repository: https://github.com/simonw/datasette-plugin-template-repository |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? 1181432624 | |
1079441621 | https://github.com/simonw/sqlite-utils/issues/417#issuecomment-1079441621 | https://api.github.com/repos/simonw/sqlite-utils/issues/417 | IC_kwDOCGYnMM5AVvjV | simonw 9599 | 2022-03-25T21:18:37Z | 2022-03-25T21:18:37Z | OWNER | Updated documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#inserting-newline-delimited-json |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
insert fails on JSONL with whitespace 1175744654 | |
1079407962 | https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1079407962 | https://api.github.com/repos/simonw/sqlite-utils/issues/421 | IC_kwDOCGYnMM5AVnVa | simonw 9599 | 2022-03-25T20:25:10Z | 2022-03-25T20:25:18Z | OWNER | Can you share either your whole My hunch is that you may have a table or column with a name that triggers the error. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"Error: near "(": syntax error" when using sqlite-utils indexes CLI 1180427792 | |
1079406708 | https://github.com/simonw/sqlite-utils/issues/422#issuecomment-1079406708 | https://api.github.com/repos/simonw/sqlite-utils/issues/422 | IC_kwDOCGYnMM5AVnB0 | simonw 9599 | 2022-03-25T20:23:21Z | 2022-03-25T20:23:21Z | OWNER | Fixing this would require a bump to 4.0 because it would break existing code. The alternative would be to introduce a new |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider not running convert functions against null values 1181236173 | |
1079404281 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079404281 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AVmb5 | simonw 9599 | 2022-03-25T20:19:50Z | 2022-03-25T20:19:50Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | ||
1079384771 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079384771 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AVhrD | simonw 9599 | 2022-03-25T19:51:34Z | 2022-03-25T19:53:01Z | OWNER | This works: ``` % sqlite-utils insert dogs.db dogs dogs.json --convert ' import random print("seeding") random.seed(10) print(random.random()) def convert(row):
global random
print(row)
row["random_score"] = random.random()
'
seeding
0.5714025946899135
{'id': 1, 'name': 'Cleo'}
{'id': 2, 'name': 'Pancakes'}
{'id': 3, 'name': 'New dog'}
(sqlite-utils) sqlite-utils % sqlite-utils rows dogs.db dogs
[{"id": 1, "name": "Cleo", "random_score": 0.4288890546751146},
{"id": 2, "name": "Pancakes", "random_score": 0.5780913011344704},
{"id": 3, "name": "New dog", "random_score": 0.20609823213950174}]
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1079376283 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079376283 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AVfmb | simonw 9599 | 2022-03-25T19:39:30Z | 2022-03-25T19:43:35Z | OWNER | Actually this doesn't work as I thought. This demo shows that the initialization code is run once per item, not a single time at the start of the run: ``` % sqlite-utils insert dogs.db dogs dogs.json --convert ' import random print("seeding") random.seed(10) print(random.random()) def convert(row):
print(row)
row["random_score"] = random.random()
'
seeding
0.5714025946899135
seeding
0.5714025946899135
seeding
0.5714025946899135
seeding
0.5714025946899135
... my mistake, that happened because I changed this line in order to try to get local imports to work:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1079243535 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1079243535 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AU_MP | simonw 9599 | 2022-03-25T17:25:12Z | 2022-03-25T17:25:12Z | OWNER | That documentation is split across a few places. This is the only bit that talks about But that's for Since both |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1079018557 | https://github.com/simonw/datasette/pull/1685#issuecomment-1079018557 | https://api.github.com/repos/simonw/datasette/issues/1685 | IC_kwDOBm6k_c5AUIQ9 | codecov[bot] 22429695 | 2022-03-25T13:16:48Z | 2022-03-25T13:16:48Z | NONE | Codecov Report
```diff @@ Coverage Diff @@ main #1685 +/-=======================================
Coverage 91.74% 91.74% Continue to review full report at Codecov.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0 1180778860 | |
1078343231 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078343231 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5ARjY_ | simonw 9599 | 2022-03-24T21:16:10Z | 2022-03-24T21:17:20Z | OWNER | Aha! This may be possible already: https://github.com/simonw/sqlite-utils/blob/396f80fcc60da8dd844577114f7920830a2e5403/sqlite_utils/utils.py#L311-L316 And yes, this does indeed work - you can do something like this: ``` echo '{"name": "harry"}' | sqlite-utils insert db.db people - --convert ' import time Simulate something expensivetime.sleep(1) def convert(row):
row["upper"] = row["name"].upper()
'
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1078328774 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078328774 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5ARf3G | simonw 9599 | 2022-03-24T21:12:33Z | 2022-03-24T21:12:33Z | OWNER | Here's how the At the end it does this:
The question is if there's a sensible way to allow people to further customize that function by executing some code first, in a way that's easy to explain. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1078322301 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078322301 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5AReR9 | simonw 9599 | 2022-03-24T21:10:52Z | 2022-03-24T21:10:52Z | OWNER | I can think of three ways forward:
I quite like the idea of that third option - I'm going to prototype it and see if I can work something out. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1078315922 | https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078315922 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | IC_kwDOCGYnMM5ARcuS | simonw 9599 | 2022-03-24T21:09:27Z | 2022-03-24T21:09:27Z | OWNER | Yeah, this is WAY harder than it should be. There's a clumsy workaround you could use which looks something like this: create a file ```python import enchant d = enchant.Dict("en_US") def check(word):
return d.check(word)
So yeah, this is definitely a big feature gap. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use a `--convert` function that runs initialization code first 1178546862 | |
1078126065 | https://github.com/simonw/datasette/issues/1684#issuecomment-1078126065 | https://api.github.com/repos/simonw/datasette/issues/1684 | IC_kwDOBm6k_c5AQuXx | fgregg 536941 | 2022-03-24T20:08:56Z | 2022-03-24T20:13:19Z | CONTRIBUTOR | would be nice if the behavior was
This is based on my assumption that what determines autofaceting is the rarity of unique values. Which may not be true! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Mechanism for disabling faceting on large tables only 1179998071 | |
1077671779 | https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1077671779 | https://api.github.com/repos/simonw/sqlite-utils/issues/399 | IC_kwDOCGYnMM5AO_dj | eyeseast 25778 | 2022-03-24T14:11:33Z | 2022-03-24T14:11:43Z | CONTRIBUTOR | Coming back to this. I was about to add a utility function to datasette-geojson to convert lat/lng columns to geometries. Thankfully I googled first. There's a SpatiaLite function for this: MakePoint.
I'm not sure if that would work with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make it easier to insert geometries, with documentation and maybe code 1124731464 | |
1077047295 | https://github.com/simonw/datasette/issues/1581#issuecomment-1077047295 | https://api.github.com/repos/simonw/datasette/issues/1581 | IC_kwDOBm6k_c5AMm__ | fgregg 536941 | 2022-03-24T04:08:18Z | 2022-03-24T04:08:18Z | CONTRIBUTOR | this has been addressed by the datasette-hashed-urls plugin |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
when hashed urls are turned on, the _memory db has improperly long-lived cache expiry 1089529555 | |
1077047152 | https://github.com/simonw/datasette/pull/1582#issuecomment-1077047152 | https://api.github.com/repos/simonw/datasette/issues/1582 | IC_kwDOBm6k_c5AMm9w | fgregg 536941 | 2022-03-24T04:07:58Z | 2022-03-24T04:07:58Z | CONTRIBUTOR | this has been obviated by the datasette-hashed-urls plugin |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
don't set far expiry if hash is '000' 1090055810 | |
1076696791 | https://github.com/simonw/datasette/issues/1682#issuecomment-1076696791 | https://api.github.com/repos/simonw/datasette/issues/1682 | IC_kwDOBm6k_c5ALRbX | simonw 9599 | 2022-03-23T18:45:49Z | 2022-03-23T18:45:49Z | OWNER | The problem is here in It should be resolving |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
SQL queries against databases with different routes are broken 1178521513 | |
1076683297 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076683297 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALOIh | simonw 9599 | 2022-03-23T18:32:32Z | 2022-03-23T18:32:32Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | ||
1076666293 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076666293 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALJ-1 | simonw 9599 | 2022-03-23T18:16:29Z | 2022-03-23T18:16:29Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | ||
1076665837 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076665837 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALJ3t | simonw 9599 | 2022-03-23T18:16:01Z | 2022-03-23T18:16:01Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | ||
1076662556 | https://github.com/simonw/sqlite-utils/pull/419#issuecomment-1076662556 | https://api.github.com/repos/simonw/sqlite-utils/issues/419 | IC_kwDOCGYnMM5ALJEc | codecov[bot] 22429695 | 2022-03-23T18:12:47Z | 2022-03-23T18:12:47Z | NONE | Codecov Report
```diff @@ Coverage Diff @@ main #419 +/-=======================================
Coverage 96.55% 96.55% Continue to review full report at Codecov.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ignore common generated files 1178484369 | |
1076652046 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076652046 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALGgO | simonw 9599 | 2022-03-23T18:02:30Z | 2022-03-23T18:02:30Z | OWNER | Two new things to add to the release notes from https://github.com/simonw/datasette/compare/0.61a0...main - https://github.com/simonw/datasette/issues/1678 - https://github.com/simonw/datasette/issues/1675 (now also a documented API) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | |
1076647495 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076647495 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALFZH | simonw 9599 | 2022-03-23T17:58:16Z | 2022-03-23T17:58:16Z | OWNER | I think the release notes are fine, but they need an opening paragraph highlighting the changes that are most likely to break backwards compatibility. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | |
1076645636 | https://github.com/simonw/datasette/pull/1574#issuecomment-1076645636 | https://api.github.com/repos/simonw/datasette/issues/1574 | IC_kwDOBm6k_c5ALE8E | simonw 9599 | 2022-03-23T17:56:35Z | 2022-03-23T17:56:35Z | OWNER | I'd actually like to switch to slim as the default - I think Datasette should ship the smallest possible container that can still support extra packages being installed using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
introduce new option for datasette package to use a slim base image 1084193403 | |
1076644362 | https://github.com/simonw/datasette/pull/1665#issuecomment-1076644362 | https://api.github.com/repos/simonw/datasette/issues/1665 | IC_kwDOBm6k_c5ALEoK | simonw 9599 | 2022-03-23T17:55:39Z | 2022-03-23T17:55:39Z | OWNER | Thanks for the PR - I spotted an error about this and went through and fixed this in all of my repos the other day: https://github.com/search?o=desc&q=user%3Asimonw+google-github-actions%2Fsetup-gcloud%40v0&s=committer-date&type=Commits |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Pin setup-gcloud to v0 instead of master 1173828092 | |
1076638278 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076638278 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALDJG | simonw 9599 | 2022-03-23T17:50:55Z | 2022-03-23T17:50:55Z | OWNER | Release notes are mostly written for the alpha, just need to clean them up a bit https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/docs/changelog.rst#061a0-2022-03-19 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | |
1075438684 | https://github.com/simonw/datasette/issues/1681#issuecomment-1075438684 | https://api.github.com/repos/simonw/datasette/issues/1681 | IC_kwDOBm6k_c5AGeRc | simonw 9599 | 2022-03-22T17:45:50Z | 2022-03-22T17:49:09Z | OWNER | I would expect this to break against SQL views that include calculated columns though - something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Potential bug in numeric handling where_clause for filters 1177101697 | |
1075437598 | https://github.com/simonw/datasette/issues/1681#issuecomment-1075437598 | https://api.github.com/repos/simonw/datasette/issues/1681 | IC_kwDOBm6k_c5AGeAe | simonw 9599 | 2022-03-22T17:44:42Z | 2022-03-22T17:45:04Z | OWNER | My hunch is that this mechanism doesn't actually do anything useful at all, because of the type conversion that automatically happens for data from tables based on the column type affinities, see: - #1671 So either remove the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Potential bug in numeric handling where_clause for filters 1177101697 | |
1075432283 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGctb | simonw 9599 | 2022-03-22T17:39:04Z | 2022-03-22T17:43:12Z | OWNER | Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like Though... it looks like there's a bug in that? It doesn't account for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1075435185 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075435185 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGdax | simonw 9599 | 2022-03-22T17:42:09Z | 2022-03-22T17:42:09Z | OWNER | Also made me realize that this query:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1075428030 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075428030 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGbq- | simonw 9599 | 2022-03-22T17:34:30Z | 2022-03-22T17:34:30Z | OWNER | No, I think I need to use Even the details from the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1075425513 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075425513 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGbDp | simonw 9599 | 2022-03-22T17:31:53Z | 2022-03-22T17:31:53Z | OWNER | The alternative to using This feels a bit neater to me, but I still then need to solve the problem of how to identify the "type" of a column that I want to use in a query. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074479932 | https://github.com/simonw/datasette/issues/339#issuecomment-1074479932 | https://api.github.com/repos/simonw/datasette/issues/339 | IC_kwDOBm6k_c5AC0M8 | simonw 9599 | 2022-03-21T22:22:34Z | 2022-03-21T22:22:34Z | OWNER | Closing this as obsolete since Datasette no longer uses Sanic. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way 340396247 | |
1074479768 | https://github.com/simonw/datasette/issues/276#issuecomment-1074479768 | https://api.github.com/repos/simonw/datasette/issues/276 | IC_kwDOBm6k_c5AC0KY | simonw 9599 | 2022-03-21T22:22:20Z | 2022-03-21T22:22:20Z | OWNER | I'm closing this issue because this is now solved by a number of neat plugins:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle spatialite geometry columns better 324835838 | |
1074478299 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074478299 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACzzb | simonw 9599 | 2022-03-21T22:20:26Z | 2022-03-21T22:20:26Z | OWNER | Thinking about options for fixing this... The following query works fine:
If someone clicks on "View and edit SQL" from a filtered table page I don't want them to have to wonder why that But... for querying views, the So one fix would be to get the SQL generating logic to use casts like this any time it is operating against a view. An even better fix would be to detect which columns in a view come from a table and which ones might not, and only use casts for the columns that aren't definitely from a table. The trick I was exploring here might be able to help with that: - #1293 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074470568 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074470568 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACx6o | simonw 9599 | 2022-03-21T22:11:14Z | 2022-03-21T22:12:49Z | OWNER | I wonder if this will be a problem with generated columns, or with SQLite strict tables? My hunch is that strict tables will continue to work without any changes, because https://www.sqlite.org/stricttables.html says nothing about their impact on comparison operations. I should test this to make absolutely sure though. Generated columns have a type, so my hunch is they will continue to work fine too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074468450 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074468450 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACxZi | simonw 9599 | 2022-03-21T22:08:35Z | 2022-03-21T22:10:00Z | OWNER | Relevant section of the SQLite documentation: 3.2. Affinity Of Expressions:
In your example, Then 4.2. Type Conversions Prior To Comparison fills in the rest:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074465536 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074465536 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACwsA | simonw 9599 | 2022-03-21T22:04:31Z | 2022-03-21T22:04:31Z | OWNER | Oh this is fascinating! I replicated the bug (thanks for the steps to reproduce) and it looks like this is down to the following: Against views, This doesn't happen against tables because of SQLite's type affinity mechanism, which handles the type conversion automatically. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074459746 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074459746 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACvRi | simonw 9599 | 2022-03-21T21:55:45Z | 2022-03-21T21:55:45Z | OWNER | I'm going to change the original logic to set n=1 for times that are |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074458506 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074458506 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACu-K | simonw 9599 | 2022-03-21T21:53:47Z | 2022-03-21T21:53:47Z | OWNER | Oh interesting, it turns out there is ONE place in the code that sets the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074454687 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074454687 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACuCf | simonw 9599 | 2022-03-21T21:48:02Z | 2022-03-21T21:48:02Z | OWNER | Here's another microbenchmark that measures how many nanoseconds it takes to run 1,000 vmops: ```python import sqlite3 import time db = sqlite3.connect(":memory:") i = 0 out = [] def count(): global i i += 1000 out.append(((i, time.perf_counter_ns()))) db.set_progress_handler(count, 1000) print("Start:", time.perf_counter_ns()) all = db.execute(""" with recursive counter(x) as ( select 0 union select x + 1 from counter ) select * from counter limit 10000; """).fetchall() print("End:", time.perf_counter_ns()) print() print("So how long does it take to execute 1000 ops?") prev_time_ns = None
for i, time_ns in out:
if prev_time_ns is not None:
print(time_ns - prev_time_ns, "ns")
prev_time_ns = time_ns
So how long does it take to execute 1000 ops? 47290 ns 49573 ns 48226 ns 45674 ns 53238 ns 47313 ns 52346 ns 48689 ns 47092 ns 87596 ns 69999 ns 52522 ns 52809 ns 53259 ns 52478 ns 53478 ns 65812 ns ``` 87596ns is 0.087596ms - so even a measure rate of every 1000 ops is easily finely grained enough to capture differences of less than 0.1ms. If anything I could bump that default 1000 up - and I can definitely eliminate the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074446576 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074446576 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACsDw | simonw 9599 | 2022-03-21T21:38:27Z | 2022-03-21T21:38:27Z | OWNER | OK here's a microbenchmark script: ```python import sqlite3 import timeit db = sqlite3.connect(":memory:") db_with_progress_handler_1 = sqlite3.connect(":memory:") db_with_progress_handler_1000 = sqlite3.connect(":memory:") db_with_progress_handler_1.set_progress_handler(lambda: None, 1) db_with_progress_handler_1000.set_progress_handler(lambda: None, 1000) def execute_query(db): cursor = db.execute(""" with recursive counter(x) as ( select 0 union select x + 1 from counter ) select * from counter limit 10000; """) list(cursor.fetchall()) print("Without progress_handler") print(timeit.timeit(lambda: execute_query(db), number=100)) print("progress_handler every 1000 ops") print(timeit.timeit(lambda: execute_query(db_with_progress_handler_1000), number=100)) print("progress_handler every 1 op")
print(timeit.timeit(lambda: execute_query(db_with_progress_handler_1), number=100))
So running every 1000 ops makes almost no difference at all, but running every single op is a 3.2x performance degradation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074439309 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074439309 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACqSN | simonw 9599 | 2022-03-21T21:28:58Z | 2022-03-21T21:28:58Z | OWNER | David Raymond solved it there: https://sqlite.org/forum/forumpost/330c8532d8a88bcd
Sure enough, adding that gets the VM steps number up to 190,007 which is close enough that I'm happy. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074378472 | https://github.com/simonw/datasette/issues/1676#issuecomment-1074378472 | https://api.github.com/repos/simonw/datasette/issues/1676 | IC_kwDOBm6k_c5ACbbo | simonw 9599 | 2022-03-21T20:18:10Z | 2022-03-21T20:18:10Z | OWNER | Maybe there is a better name for this method that helps emphasize its cascading nature. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider ensure_permissions() logic, can it be less confusing? 1175690070 | |
1074347023 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074347023 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACTwP | simonw 9599 | 2022-03-21T19:48:59Z | 2022-03-21T19:48:59Z | OWNER | Posed a question about that here: https://sqlite.org/forum/forumpost/de9ff10fa7 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074341924 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074341924 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACSgk | simonw 9599 | 2022-03-21T19:42:08Z | 2022-03-21T19:42:08Z | OWNER | Here's the Python-C implementation of It calls https://www.sqlite.org/c3ref/progress_handler.html says:
So maybe VM-steps and virtual machine instructions are different things? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074337997 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074337997 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACRjN | simonw 9599 | 2022-03-21T19:37:08Z | 2022-03-21T19:37:08Z | OWNER | This is weird: ```python import sqlite3 db = sqlite3.connect(":memory:") i = 0 def count(): global i i += 1 db.set_progress_handler(count, 1) db.execute(""" with recursive counter(x) as ( select 0 union select x + 1 from counter ) select * from counter limit 10000; """) print(i)
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074332718 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074332718 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACQQu | simonw 9599 | 2022-03-21T19:31:10Z | 2022-03-21T19:31:10Z | OWNER | How long does it take for SQLite to execute 1000 opcodes anyway? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074332325 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074332325 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACQKl | simonw 9599 | 2022-03-21T19:30:44Z | 2022-03-21T19:30:44Z | OWNER | So it looks like even for facet suggestion |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074331743 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074331743 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACQBf | simonw 9599 | 2022-03-21T19:30:05Z | 2022-03-21T19:30:05Z | OWNER | https://github.com/simonw/datasette/blob/1a7750eb29fd15dd2eea3b9f6e33028ce441b143/datasette/app.py#L118-L122 sets it to 50ms for facet suggestion but that's not going to pass
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074321862 | https://github.com/simonw/datasette/issues/1660#issuecomment-1074321862 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c5ACNnG | simonw 9599 | 2022-03-21T19:19:01Z | 2022-03-21T19:19:01Z | OWNER | I've simplified this a ton now. I'm going to keep working on this in the long-term but I think this issue can be closed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1074302559 | https://github.com/simonw/datasette/issues/1678#issuecomment-1074302559 | https://api.github.com/repos/simonw/datasette/issues/1678 | IC_kwDOBm6k_c5ACI5f | simonw 9599 | 2022-03-21T19:04:03Z | 2022-03-21T19:04:03Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make `check_visibility()` a documented API 1175715988 | ||
1074287177 | https://github.com/simonw/datasette/issues/1660#issuecomment-1074287177 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c5ACFJJ | simonw 9599 | 2022-03-21T18:51:42Z | 2022-03-21T18:51:42Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1074256603 | https://github.com/simonw/sqlite-utils/issues/417#issuecomment-1074256603 | https://api.github.com/repos/simonw/sqlite-utils/issues/417 | IC_kwDOCGYnMM5AB9rb | blaine 9954 | 2022-03-21T18:19:41Z | 2022-03-21T18:19:41Z | NONE | That makes sense; just a little hint that points folks towards doing the right thing might be helpful! fwiw, the reason I was using jq in the first place was just a quick way to extract one attribute from an actual JSON array. When I initially imported it, I got a table with a bunch of embedded JSON values, rather than a native table, because each array entry had two attributes, one with the data I actually wanted. Not sure how common a use-case this is, though (and easily fixed, aside from the jq weirdness!) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
insert fails on JSONL with whitespace 1175744654 | |
1074243540 | https://github.com/simonw/sqlite-utils/issues/417#issuecomment-1074243540 | https://api.github.com/repos/simonw/sqlite-utils/issues/417 | IC_kwDOCGYnMM5AB6fU | simonw 9599 | 2022-03-21T18:08:03Z | 2022-03-21T18:08:03Z | OWNER | I've not really thought about standards as much here as I should. It looks like there are two competing specs for newline-delimited JSON! http://ndjson.org/ is the one I've been using in
https://jsonlines.org/ is the other one. It is slightly less clear, but it does say this:
My interpretation of both of these is that newlines in the middle of a JSON object shouldn't be allowed. So what's The The thing I like about newline-delimited JSON is that it's really trivial to parse - loop through each line, run it through Unless someone has written a robust Python implementation of a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
insert fails on JSONL with whitespace 1175744654 | |
1074184240 | https://github.com/simonw/datasette/issues/1677#issuecomment-1074184240 | https://api.github.com/repos/simonw/datasette/issues/1677 | IC_kwDOBm6k_c5ABsAw | simonw 9599 | 2022-03-21T17:20:17Z | 2022-03-21T17:20:17Z | OWNER | This is weirdly different from how |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove `check_permission()` from `BaseView` 1175694248 | |
1074180312 | https://github.com/simonw/datasette/issues/1676#issuecomment-1074180312 | https://api.github.com/repos/simonw/datasette/issues/1676 | IC_kwDOBm6k_c5ABrDY | simonw 9599 | 2022-03-21T17:16:45Z | 2022-03-21T17:16:45Z | OWNER | When looking at this code earlier I assumed that the following would check each permission in turn and fail if any of them failed:
If that is indeed the right abstraction, I need to work to make the documentation as clear as possible. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider ensure_permissions() logic, can it be less confusing? 1175690070 | |
1074178865 | https://github.com/simonw/datasette/issues/1676#issuecomment-1074178865 | https://api.github.com/repos/simonw/datasette/issues/1676 | IC_kwDOBm6k_c5ABqsx | simonw 9599 | 2022-03-21T17:15:27Z | 2022-03-21T17:15:27Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider ensure_permissions() logic, can it be less confusing? 1175690070 | ||
1074177827 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074177827 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABqcj | simonw 9599 | 2022-03-21T17:14:31Z | 2022-03-21T17:14:31Z | OWNER | Updated documentation: https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/docs/internals.rst#await-ensure_permissionsactor-permissions
That's pretty hard to understand! I'm going to open a separate issue to reconsider if this is a useful enough abstraction given how confusing it is. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074161523 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074161523 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABmdz | simonw 9599 | 2022-03-21T16:59:55Z | 2022-03-21T17:00:03Z | OWNER | Also calling that function |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074158890 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074158890 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABl0q | simonw 9599 | 2022-03-21T16:57:15Z | 2022-03-21T16:57:15Z | OWNER | Idea: A new |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074156779 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074156779 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABlTr | simonw 9599 | 2022-03-21T16:55:08Z | 2022-03-21T16:56:02Z | OWNER | One benefit of the current design of I could return an object which evaluates to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074143209 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074143209 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABh_p | simonw 9599 | 2022-03-21T16:46:05Z | 2022-03-21T16:46:05Z | OWNER | The other difference though is that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074142617 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074142617 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABh2Z | simonw 9599 | 2022-03-21T16:45:27Z | 2022-03-21T16:45:27Z | OWNER | Though at that point So maybe
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user >30