github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1318777114 | https://api.github.com/repos/simonw/sqlite-utils/issues/510 | 1318777114 | IC_kwDOCGYnMM5OmvEa | 7908073 | 2022-11-17T15:09:47Z | 2022-11-17T15:09:47Z | CONTRIBUTOR | why close? is the only problem that the _config table that incorrectly says 4 for fts5? if so, that's still something that should be fixed | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1434911255 | |
https://github.com/simonw/sqlite-utils/issues/510#issuecomment-1318431389 | https://api.github.com/repos/simonw/sqlite-utils/issues/510 | 1318431389 | IC_kwDOCGYnMM5Olaqd | 1176293 | 2022-11-17T10:36:28Z | 2022-11-17T10:36:28Z | NONE | The virtual table's _config `version: 4` seems to indicate FTS5. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1434911255 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1317889323 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1317889323 | IC_kwDOBm6k_c5OjWUr | 536941 | 2022-11-17T00:47:36Z | 2022-11-17T00:47:36Z | CONTRIBUTOR | amazing! thanks @simonw | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1899#issuecomment-1317873458 | https://api.github.com/repos/simonw/datasette/issues/1899 | 1317873458 | IC_kwDOBm6k_c5OjScy | 95570 | 2022-11-17T00:31:07Z | 2022-11-17T00:31:07Z | CONTRIBUTOR | This is one way to fix it ```patch r.html diff --git a/datasette/static/cm-editor-6.0.1.js b/datasette/static/cm-editor-6.0.1.js index c1fd2ab..68cf398 100644 --- a/datasette/static/cm-editor-6.0.1.js +++ b/datasette/static/cm-editor-6.0.1.js @@ -22,7 +22,14 @@ export function editorFromTextArea(textarea, conf = {}) { // https://github.com/codemirror/lang-sql#user-content-sqlconfig.tables let view = new EditorView({ doc: textarea.value, + extensions: [ + EditorView.theme({ + ".cm-content": { + // Height on cm-content ensures the editor is focusable by clicking beyond the height of the text + minHeight: "70px", + }, + }), keymap.of([ { key: "Shift-Enter", diff --git a/datasette/templates/_codemirror.html b/datasette/templates/_codemirror.html index dea4710..c4629ae 100644 --- a/datasette/templates/_codemirror.html +++ b/datasette/templates/_codemirror.html @@ -4,7 +4,6 @@ .cm-editor { resize: both; overflow: hidden; - min-height: 70px; width: 80%; border: 1px solid #ddd; } ``` I don't love it but it seems to work for the default case. You can still retrigger the bug by resizing the editor to be > 70px high. The other approach would be to listen for a click on that empty region and move focus to the editor, or something | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1452495049 | |
https://github.com/simonw/datasette/pull/1898#issuecomment-1317870550 | https://api.github.com/repos/simonw/datasette/issues/1898 | 1317870550 | IC_kwDOBm6k_c5OjRvW | 22429695 | 2022-11-17T00:27:55Z | 2022-11-17T00:27:55Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1898?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.55**% // Head: **92.55**% // No change to project coverage :thumbsup: > Coverage data is based on head [(`5eb4ea4`)](https://codecov.io/gh/simonw/datasette/pull/1898?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`00e233d`)](https://codecov.io/gh/simonw/datasette/commit/00e233d7a7f6443cb95fb5227c23580c48551cad?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch has no changes to coverable lines. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #1898 +/- ## ======================================= Coverage 92.55% 92.55% ======================================= Files 35 35 Lines 4432 4432 ======================================= Hits 4102 4102 Misses 330 330 ``` Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1898?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1452485922 | |
https://github.com/simonw/datasette/issues/1897#issuecomment-1317840727 | https://api.github.com/repos/simonw/datasette/issues/1897 | 1317840727 | IC_kwDOBm6k_c5OjKdX | 9599 | 2022-11-16T23:57:52Z | 2022-11-16T23:57:52Z | OWNER | In terms of permissions: if you have `execute-sql` permission for a database then it's OK for you to see the table columns for that database. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1452457263 | |
https://github.com/simonw/datasette/issues/1897#issuecomment-1317839781 | https://api.github.com/repos/simonw/datasette/issues/1897 | 1317839781 | IC_kwDOBm6k_c5OjKOl | 9599 | 2022-11-16T23:56:47Z | 2022-11-16T23:56:47Z | OWNER | I'm going to call this `table_columns` in the template context (because `schema` might mean `CREATE TABLE ...`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1452457263 | |
https://github.com/simonw/datasette/issues/1897#issuecomment-1317838892 | https://api.github.com/repos/simonw/datasette/issues/1897 | 1317838892 | IC_kwDOBm6k_c5OjKAs | 9599 | 2022-11-16T23:55:42Z | 2022-11-16T23:55:42Z | OWNER | Here's where the schema is hard-coded at the moment: https://github.com/simonw/datasette/blob/00e233d7a7f6443cb95fb5227c23580c48551cad/datasette/templates/_codemirror_foot.html#L2-L7 I figured out how to extract that data from the `_internal` table in this comment: https://github.com/simonw/datasette/pull/1893#issuecomment-1317475720 Although that used JSON functions which may (in a real edge-case) not be available in the version of SQLite that Datasette is running on, so probably going to use a regular SQL query and then assemble the JSON separately. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1452457263 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317837416 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317837416 | IC_kwDOBm6k_c5OjJpo | 9599 | 2022-11-16T23:54:02Z | 2022-11-16T23:54:02Z | OWNER | I'm going to tackle #1897 in the next few minutes. Tests failed due to Prettier check, just pushed a fix so it would ignore `.bundle.js` too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317834838 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317834838 | IC_kwDOBm6k_c5OjJBW | 95570 | 2022-11-16T23:50:58Z | 2022-11-16T23:50:58Z | CONTRIBUTOR | Should we empty out the fixture schema to avoid fixture autocomplete showing up on live databases in the interim, or are you planning to tackle #1897 shortly? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317831555 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317831555 | IC_kwDOBm6k_c5OjIOD | 9599 | 2022-11-16T23:47:13Z | 2022-11-16T23:47:13Z | OWNER | I'll open a follow-up issue to fix the schema. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317831425 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317831425 | IC_kwDOBm6k_c5OjIMB | 9599 | 2022-11-16T23:47:05Z | 2022-11-16T23:47:05Z | OWNER | OK, let's do it! Thanks so much for this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317829214 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317829214 | IC_kwDOBm6k_c5OjHpe | 9599 | 2022-11-16T23:44:36Z | 2022-11-16T23:44:36Z | OWNER | Deployed that to https://datasette-pr-1893.vercel.app/fixtures - looks good to me! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316297347 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316297347 | IC_kwDOBm6k_c5OdRqD | 22429695 | 2022-11-16T04:05:12Z | 2022-11-16T23:27:45Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1893?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.55**% // Head: **92.55**% // No change to project coverage :thumbsup: > Coverage data is based on head [(`f254be4`)](https://codecov.io/gh/simonw/datasette/pull/1893?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`6f610e1`)](https://codecov.io/gh/simonw/datasette/commit/6f610e1d94b7b8ec605b5b7fcb01537f6adf9c5b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch has no changes to coverable lines. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #1893 +/- ## ======================================= Coverage 92.55% 92.55% ======================================= Files 35 35 Lines 4432 4432 ======================================= Hits 4102 4102 Misses 330 330 ``` Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1893?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317805482 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317805482 | IC_kwDOBm6k_c5OjB2q | 95570 | 2022-11-16T23:18:17Z | 2022-11-16T23:18:17Z | CONTRIBUTOR | Alright with https://github.com/simonw/datasette/pull/1893/commits/f254be4b38936e95e7a7f25866e7c6b0520db96f we should be getting autocomplete on fixture data. Give that a test and see what you think | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317797044 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317797044 | IC_kwDOBm6k_c5Oi_y0 | 9599 | 2022-11-16T23:08:34Z | 2022-11-16T23:08:34Z | OWNER | > I can push up a commit that uses the static fixtures schema for testing, but given that the query used to generate it is authed we would still need some work to make that work on live data, right? Yeah, push that up. I'm happy to wire in the query right after we land this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317789308 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317789308 | IC_kwDOBm6k_c5Oi958 | 95570 | 2022-11-16T22:59:57Z | 2022-11-16T22:59:57Z | CONTRIBUTOR | I can push up a commit that uses the static fixtures schema for testing, but given that the query used to generate it is authed we would still need some work to make that work on live data, right? Ideally it could come down to db and query views directly to avoid waiting on an extra xhr and managing that state change.On Nov 16, 2022, at 2:16 PM, Simon Willison ***@***.***> wrote: Honestly I'm not too bothered if table names with weird characters don't work correctly here - I care about those in the Datasette fixtures.db database because Datasette aims to support ANY valid SQLite database, so I need stuff in the test suite that includes weird edge cases like this. But I would hope very few people actually create tables with spaces in their names, so it's not a huge concern to me if autocompletion doesn't work properly for those. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1896#issuecomment-1317757112 | https://api.github.com/repos/simonw/datasette/issues/1896 | 1317757112 | IC_kwDOBm6k_c5Oi2C4 | 9599 | 2022-11-16T22:26:52Z | 2022-11-16T22:33:13Z | OWNER | Some ideas from walking the dog: Challenge: standard URL routing of request to database/table/row Standardize on the named components of the URL patterns - `database`, `table`, `pks` Async function that takes the request and the Datasette instance and returns a Resolved instance with: ``` .level - database or table or row (better name?) .database - the name of the database .db - the database object .table - the name of the table (or view) .is_view perhaps? .pk_values if it's a row ``` Should this attempt to resolve names queries too? ``` .where_sql - the where fragment you use .where_params - accompanying dictionary await datasette.resolve_request(request) ``` Or even better three methods: ```python datasette.resolve_database(request) datasette.resolve_table(request) datasette.resolve_row(request) ``` These can be typed correctly Methods raise `NotFound` if not found | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1452364777 | |
https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263 | https://api.github.com/repos/simonw/datasette/issues/1863 | 1317755263 | IC_kwDOBm6k_c5Oi1l_ | 9599 | 2022-11-16T22:24:59Z | 2022-11-16T22:24:59Z | OWNER | In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table). Since this is so common, I think it's worth extracting the logic out first. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425029242 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317746206 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317746206 | IC_kwDOBm6k_c5OizYe | 9599 | 2022-11-16T22:17:24Z | 2022-11-16T22:17:24Z | OWNER | Deployed 0a649e8f78c23e8db6869442eeb0dfe36a5443da: https://datasette-pr-1893.vercel.app/fixtures | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317744563 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317744563 | IC_kwDOBm6k_c5Oiy-z | 9599 | 2022-11-16T22:16:03Z | 2022-11-16T22:16:03Z | OWNER | Honestly I'm not too bothered if table names with weird characters don't work correctly here - I care about those in the Datasette `fixtures.db` database because Datasette aims to support ANY valid SQLite database, so I need stuff in the test suite that includes weird edge cases like this. But I would hope very few people actually create tables with spaces in their names, so it's not a huge concern to me if autocompletion doesn't work properly for those. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317715580 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317715580 | IC_kwDOBm6k_c5Oir58 | 95570 | 2022-11-16T21:49:51Z | 2022-11-16T21:49:51Z | CONTRIBUTOR | I think the table completion still has some quirks to work out. Something like ``` schema: { "[123_starts_with_digits]": ["content"], } ``` Seems to work alright, although it will append it after any other numbers you've started typing - so you end up with `select * from 12[123_starts_with_digits]` if you typed "12" to get the completion to appear. This might just be an issue with numeric names, I haven't tested it in a lot of detail. You can do ``` searchable: [ { label: "name with . and spaces", apply: "[name with . and spaces]", }, "pk", "text1", "text2", ], ``` Which is pretty neat and will show the non-escaped string but complete to the escaped one. You can't easily do that with the table names themselves (you can pass a `tables` array like so https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/src/sql.ts#L121 but it will overwrite the columns from the schema ). It's buggy enough (bad output for these unusual table names) that I'd suggest that work gets moved into a follow up to the upgrade to 6. That would give space to sort out how to deliver that to the view directly, figure out where name escaping should happen, and have overall testing to uncover bugs and fix papercuts before enabling it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317681193 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317681193 | IC_kwDOBm6k_c5Oijgp | 95570 | 2022-11-16T21:19:13Z | 2022-11-16T21:19:13Z | CONTRIBUTOR | Alright, added Cmd+Enter to submit (Ctrl+Enter on Windows as well bc of using Meta-Enter on codemirror). We can make that MacOS only by changing the combo to Cmd+Enter specifically but I think it's probably fine to have both. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317522323 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317522323 | IC_kwDOBm6k_c5Oh8uT | 95570 | 2022-11-16T18:59:49Z | 2022-11-16T18:59:49Z | CONTRIBUTOR | Or I guess you could return only the escaped table name and then we could derive the unescaped from the client side (removing the outer `[]` when present) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317520304 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317520304 | IC_kwDOBm6k_c5Oh8Ow | 95570 | 2022-11-16T18:58:43Z | 2022-11-16T18:58:43Z | CONTRIBUTOR | Nice. And is it possible to include another field which is an escaped table name (only when necessary) - i.e. `[123_starts_with_digits]`. Or is that easy enough to derive on the client? I'm thinking we'd map those to Completion objects so that CM would show the non escaped text but complete to escaped. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317475720 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317475720 | IC_kwDOBm6k_c5OhxWI | 9599 | 2022-11-16T18:25:16Z | 2022-11-16T18:25:16Z | OWNER | Here's a query that returns the exact JSON we need to pass to the schema: https://latest.datasette.io/_internal?sql=with+inner+as+%28%0D%0A++select%0D%0A++++table_name%2C%0D%0A++++json_group_array%28name%29+as+table_columns%0D%0A++from%0D%0A++++columns%0D%0A++where%0D%0A++++database_name+%3D+%3Adatabase%0D%0A++group+by%0D%0A++++table_name%0D%0A%29%0D%0Aselect%0D%0A++json_group_object%28table_name%2C+table_columns%29%0D%0Afrom%0D%0A++inner&database=fixtures ```sql with inner as ( select table_name, json_group_array(name) as table_columns from columns where database_name = :database group by table_name ) select json_group_object(table_name, table_columns) from inner ``` Returns (after pretty-printing): ```json { "123_starts_with_digits": [ "content" ], "Table With Space In Name": [ "content", "pk" ], "attraction_characteristic": [ "name", "pk" ], "binary_data": [ "data" ], "complex_foreign_keys": [ "f1", "f2", "f3", "pk" ], "compound_primary_key": [ "content", "pk1", "pk2" ], "compound_three_primary_keys": [ "content", "pk1", "pk2", "pk3" ], "custom_foreign_key_label": [ "foreign_key_with_custom_label", "pk" ], "facet_cities": [ "id", "name" ], "facetable": [ "_city_id", "_neighborhood", "complex_array", "created", "distinct_some_null", "n", "on_earth", "pk", "planet_int", "state", "tags" ], "foreign_key_references": [ "foreign_key_compound_pk1", "foreign_key_compound_pk2", "foreign_key_with_blank_label", "foreign_key_with_label", "foreign_key_with_no_label", "pk" ], "infinity": [ "value" ], … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317465874 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317465874 | IC_kwDOBm6k_c5Ohu8S | 9599 | 2022-11-16T18:21:17Z | 2022-11-16T18:21:54Z | OWNER | I was worrying about the server-side overhead of collecting together all of the tables and column names for databases that might have hundreds of tables... but then I remember that I built the `_internal` table precisely for this kind of thing - so gathering all of that data should still only be a single SQL query against an in-memory database. https://latest.datasette.io/login-as-root and then visit this page for an example query: https://latest.datasette.io/_internal?sql=select%0D%0A++database_name%2C%0D%0A++table_name%2C%0D%0A++json_group_array%28name%29%0D%0Afrom%0D%0A++columns%0D%0Awhere%0D%0A++database_name+%21%3D+%27_internal%27%0D%0Agroup+by%0D%0A++database_name%2C%0D%0A++table_name ```sql select database_name, table_name, json_group_array(name) from columns where database_name != '_internal' group by database_name, table_name ``` database_name | table_name | json_group_array(name) -- | -- | -- extra_database | searchable | ["pk","text1","text2"] extra_database | searchable_fts | ["__langid","content","docid","searchable_fts","text1","text2"] extra_database | searchable_fts_content | ["c0text1","c1text2","c2content","docid"] extra_database | searchable_fts_segdir | ["end_block","idx","leaves_end_block","level","root","start_block"] extra_database | searchable_fts_segments | ["block","blockid"] fixtures | 123_starts_with_digits | ["content"] fixtures | Table With Space In Name | ["content","pk"] fixtures | attraction_characteristic | ["name","pk"] fixtures | binary_data | ["data"] fixtures | complex_foreign_keys | ["f1","f2","f3","pk"] fixtures | compound_primary_key | ["content","pk1","pk2"] | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317456909 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317456909 | IC_kwDOBm6k_c5OhswN | 9599 | 2022-11-16T18:17:39Z | 2022-11-16T18:17:39Z | OWNER | Tiny feature request (since you're in this code already) - I keep hitting Command+Enter on my macOS keyboard to submit the query, but the correct shortcut is Shift+Enter. Would be great if both worked! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317452541 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317452541 | IC_kwDOBm6k_c5Ohrr9 | 9599 | 2022-11-16T18:15:52Z | 2022-11-16T18:15:52Z | OWNER | Deployed latest copy with: ``` datasette publish vercel fixtures.db \ --project datasette-pr-1893 \ --about 'PR 1893' \ --about_url https://github.com/simonw/datasette/pull/1893 \ --scope datasette \ --branch eccb1c6c781d69d8ec3c542ef65c78a4a0927a7c ``` https://datasette-pr-1893.vercel.app/fixtures | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317449610 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317449610 | IC_kwDOBm6k_c5Ohq-K | 9599 | 2022-11-16T18:14:28Z | 2022-11-16T18:14:28Z | OWNER | > I'm thinking of also adding `count` to the list since that's a common thing people would want to autocomplete. I notice BQ console highlights `count` in the same manner as other keywords like `select` as well. Huh, yeah we should definitely have `count` - surprised it's not on the list on https://www.sqlite.org/lang_keywords.html which is why we didn't get it from the GPT-3 generated schema. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1880#issuecomment-1317420812 | https://api.github.com/repos/simonw/datasette/issues/1880 | 1317420812 | IC_kwDOBm6k_c5Ohj8M | 525934 | 2022-11-16T17:50:29Z | 2022-11-16T17:50:29Z | NONE | I appreciate your response @simonw - thanks! I'll clarify what we need further - let's imagine we have 2000 SQLLite databases (for 2000 tenants), but we only want to run _one_ datasette instance for each of those tenants to query/use datasette against their _own_ database only. This means the "connection" between datasette and the SQLLite database would be dynamic, based on the tenantID that's required on an incoming request. Is there any specific config or other considerations in this use case, to minimize memory use on a single, efficient VM and serve queries to all these tenants? cc @muadham | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1433576351 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317329157 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317329157 | IC_kwDOBm6k_c5OhNkF | 95570 | 2022-11-16T16:46:52Z | 2022-11-16T16:46:52Z | CONTRIBUTOR | > <img alt="Screenshot 2022-11-15 at 8 27 17 PM" width="276" src="https://user-images.githubusercontent.com/95570/202083682-dab271f7-cb7b-44dd-8266-70b1eba265ee.png"> > > UI issue I see on the autocomplete popup with overlapping icon & text. Screenshot's from Firefox, it seems even a little more pronounced on Safari I checked and if I empty out app.css the bug goes away, so there's some kind of inheritance issue there. It's hard to debug bc the autocomplete popup goes away on blur (i.e. when trying to inspect it in devtools), but at least it's narrowed down a bit. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317326406 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317326406 | IC_kwDOBm6k_c5OhM5G | 95570 | 2022-11-16T16:45:09Z | 2022-11-16T16:45:09Z | CONTRIBUTOR | For escaped table names it looks like we could pass a Completion object (https://codemirror.net/docs/ref/#autocomplete) instead of a string which would allow the non escaped name to be a label and then the escaped name to actually complete in the editor, which might help with some of the funkiness I was seeing w/ completion | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317314064 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317314064 | IC_kwDOBm6k_c5OhJ4Q | 95570 | 2022-11-16T16:36:46Z | 2022-11-16T16:36:46Z | CONTRIBUTOR | With ```patch diff --git a/datasette/templates/_codemirror_foot.html b/datasette/templates/_codemirror_foot.html index ed709b3..74fe18e 100644 --- a/datasette/templates/_codemirror_foot.html +++ b/datasette/templates/_codemirror_foot.html @@ -7,7 +7,11 @@ sqlFormat.hidden = false; } if (sqlInput) { - var editor = (window.editor = cm.editorFromTextArea(sqlInput)); + var editor = (window.editor = cm.editorFromTextArea(sqlInput, { + schema: { + compound_three_primary_keys: ["pk1", "pk2", "pk3", "content"], + }, + })); ``` we get table autocompletion and column completion if you name the table in the query (see screencast). I do see bugs with escaped table names like `"'123_starts_with_digits'": ["col1", "col2"]` or `"[123_starts_with_digits]": ["col1", "col2"]` where it doesn't seem to pick up the column names though. I think it needs some further testing and debugging. https://user-images.githubusercontent.com/95570/202238521-e613b4e2-ba92-4418-9068-fc022edaee93.mp4 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1317281292 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1317281292 | IC_kwDOBm6k_c5OhB4M | 95570 | 2022-11-16T16:19:16Z | 2022-11-16T16:19:16Z | CONTRIBUTOR | Ha, nice idea! Updating the dialect with that list. I'm thinking of also adding `count` to the list since that's a common thing people would want to autocomplete. I notice BQ console highlights `count` in the same manner as other keywords like `select` as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/sqlite-utils/issues/512#issuecomment-1316530539 | https://api.github.com/repos/simonw/sqlite-utils/issues/512 | 1316530539 | IC_kwDOCGYnMM5OeKlr | 9599 | 2022-11-16T07:49:50Z | 2022-11-16T07:49:50Z | OWNER | Tests passed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450952393 | |
https://github.com/simonw/sqlite-utils/issues/512#issuecomment-1316447182 | https://api.github.com/repos/simonw/sqlite-utils/issues/512 | 1316447182 | IC_kwDOCGYnMM5Od2PO | 9599 | 2022-11-16T06:32:31Z | 2022-11-16T06:32:31Z | OWNER | Test failed again: https://github.com/simonw/sqlite-utils/actions/runs/3476950474/jobs/5812663096 `E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/universe/s/spatialite/libsqlite3-mod-spatialite_4.3.0a-6build1_amd64.deb Unable to connect to azure.archive.ubuntu.com:http:` That looks like an intermittent error. I'll try running it again in the morning. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450952393 | |
https://github.com/simonw/sqlite-utils/issues/512#issuecomment-1316437748 | https://api.github.com/repos/simonw/sqlite-utils/issues/512 | 1316437748 | IC_kwDOCGYnMM5Odz70 | 9599 | 2022-11-16T06:24:31Z | 2022-11-16T06:24:31Z | OWNER | ``` sqlite-utils % pipx run no_implicit_optional . Calculating full-repo metadata... Executing codemod... 11.43s 98% complete, 0.24s estimated for 5 files to go... ``` Then: ``` Finished codemodding 239 files! - Transformed 239 files successfully. - Skipped 0 files. - Failed to codemod 0 files. - 0 warnings were generated. ``` Here's the diff: ```diff diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index a06f4b7..e819d17 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -297,12 +297,12 @@ class Database: def __init__( self, - filename_or_conn: Union[str, pathlib.Path, sqlite3.Connection] = None, + filename_or_conn: Optional[Union[str, pathlib.Path, sqlite3.Connection]] = None, memory: bool = False, - memory_name: str = None, + memory_name: Optional[str] = None, recreate: bool = False, recursive_triggers: bool = True, - tracer: Callable = None, + tracer: Optional[Callable] = None, use_counts_table: bool = False, ): assert (filename_or_conn is not None and (not memory and not memory_name)) or ( @@ -341,7 +341,7 @@ class Database: self.conn.close() @contextlib.contextmanager - def tracer(self, tracer: Callable = None): + def tracer(self, tracer: Optional[Callable] = None): """ Context manager to temporarily set a tracer function - all executed SQL queries will be passed to this. @@ -378,7 +378,7 @@ class Database: def register_function( self, - fn: Callable = None, + fn: Optional[Callable] = None, deterministic: bool = False, replace: bool = False, name: Optional[str] = None, @@ -879,7 +879,7 @@ class Database: pk: Optional[Any] = None, foreign_keys: Optional[ForeignKeysType] = None, column_order: Optional[List[str]] = None, - not_null: Iterable[str] = None, + not_null: Optional[Iterable[… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450952393 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316412234 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316412234 | IC_kwDOBm6k_c5OdttK | 9599 | 2022-11-16T06:00:39Z | 2022-11-16T06:01:36Z | OWNER | Should note though that this is a classic example of GPT-3 making stuff up in places. > current: Returns the current date, time, or timestamp `select current` throws an error for me: https://latest.datasette.io/_memory?sql=select+current `select current_date, current_time, current_timestamp` works though: https://latest.datasette.io/_memory?sql=select+current_date%2C+current_time%2C+current_timestamp So let's drop `current` from the list. I'm OK with it though, I think it's likely good enough for the first attempt at this. We should drop `temp` and `temporary` too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316401895 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316401895 | IC_kwDOBm6k_c5OdrLn | 9599 | 2022-11-16T05:50:40Z | 2022-11-16T05:50:40Z | OWNER | So I think our dialect (at least to start with) should be: ``` keywords: "and as asc between by case cast current current_date current_time current_timestamp desc distinct each else escape except exists explain filter first for from full generated group having if in index inner intersect into isnull join last left like limit not null or order outer over pragma primary query raise range regexp right rollback row select set table temp temporary then to union unique using values view virtual when where", // https://www.sqlite.org/datatype3.html types: "null integer real text blob", builtin: "" ``` I left `builtin` blank here because I don't think we need any of those things at all. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316400688 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316400688 | IC_kwDOBm6k_c5Odq4w | 9599 | 2022-11-16T05:49:31Z | 2022-11-16T05:49:31Z | OWNER | OK, I decided to play around with GPT-3 for this. https://beta.openai.com/playground I prompted: ``` For each keyword say it it can be used in a SQLite select statement: and as asc between by case cast cross current current_date current_time current_timestamp desc detach distinct each else end escape except exclude exclusive exists explain fail filter first following for foreign from full generated glob group groups having if ignore immediate in index indexed initially inner insert instead intersect into is isnull join key last left like limit match materialized natural no not nothing notnull null nulls of offset on or order others outer over partition plan pragma preceding primary query raise range recursive references regexp reindex release rename replace restrict returning right rollback row rows savepoint select set table temp temporary then ties to transaction trigger unbounded union unique update using vacuum values view virtual when where window with without - and: yes - as: yes - asc: yes - between: yes - by: yes - case: yes - cast: yes - current: yes - current_date: yes - current_time: yes - current_timestamp: yes - desc: yes - detach: no - distinct: yes - each: yes - else: yes - end: yes - escape: yes - except: no - exclude: yes - exclusive: no ``` Having given it these examples I ran it a few times: ``` - exists: yes - explain: no - fail: no - filter: yes - first: yes - following: no - for: yes - foreign: no - from: yes - full: no - generated: no - glob: yes - group: yes - groups: no - having: yes - if: yes - ignore: no - immediate: no - in: yes - index: yes - indexed: no - initially: no - inner: no - insert: no - instead: no - intersect: no - into: yes - is: yes - isnull: yes - join: yes - key: no - last: yes - left: yes - like: yes - limit: yes - match: no - materialized: no -natural: yes - no: no - not: no - nothing: no - notnull: yes - null: yes - nulls: yes - of: no - offset: yes - on: yes - or: yes -… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316387382 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316387382 | IC_kwDOBm6k_c5Odno2 | 95570 | 2022-11-16T05:33:55Z | 2022-11-16T05:33:55Z | CONTRIBUTOR | I added a commit to make our own dialect at https://github.com/simonw/datasette/pull/1893/commits/e273fc8ed5341bdf0b622e722d761bd2acc30a90. Pulled in the full list of keywords from https://www.sqlite.org/lang_keywords.html but haven't gone through and pruned it to only include common select keywords. @simonw you'll have better knowledge than me on that - do you want to take a first shot at narrowing that down to the set that people will be using in the editor? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316340865 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316340865 | IC_kwDOBm6k_c5OdcSB | 9599 | 2022-11-16T04:49:30Z | 2022-11-16T04:49:43Z | OWNER | > The main issue is that we don't pass the relevant table data down to QueryView. If you can come up with a static example JSON data structure example that does the right thing, I'm happy to refactor QueryView to make that available to the template - or even have a separate `fetch()` that grabs just the data needed for the autocomplete as a separate hit when the page loads (whichever has better performance implications). I'm working a fair amount in the view classes at the moment so adding this to that work would make sense. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316339035 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316339035 | IC_kwDOBm6k_c5Odb1b | 95570 | 2022-11-16T04:47:11Z | 2022-11-16T04:47:11Z | CONTRIBUTOR | > Have you ever seen CodeMirror correctly auto-completing columns? I'm not entirely sure I believe that the feature works anywhere else. I was thinking of the BigQuery console, like <img width="516" alt="Screenshot 2022-11-15 at 8 31 10 PM" src="https://user-images.githubusercontent.com/95570/202084210-4d23d916-6862-4fd2-8f41-087d6355921a.png"> But they must be doing something pretty custom & appears to be using Monaco anyway. I suspect some kind of lower level autocomplete integration could make this work, but if the table completion is a good-enough starting point I think it's not too hard. The main issue is that we don't pass the relevant table data down to QueryView. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316320521 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316320521 | IC_kwDOBm6k_c5OdXUJ | 95570 | 2022-11-16T04:29:23Z | 2022-11-16T04:29:23Z | CONTRIBUTOR | <img width="276" alt="Screenshot 2022-11-15 at 8 27 17 PM" src="https://user-images.githubusercontent.com/95570/202083682-dab271f7-cb7b-44dd-8266-70b1eba265ee.png"> UI issue I see on the autocomplete popup with overlapping icon & text. Screenshot's from Firefox, it seems even a little more pronounced on Safari | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316318961 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316318961 | IC_kwDOBm6k_c5OdW7x | 95570 | 2022-11-16T04:27:51Z | 2022-11-16T04:27:51Z | CONTRIBUTOR | > The resize handle doesn't appear on Mobile Safari on iPhone - I don't think that particularly matters though. > > The textarea does get a weird border around it when focused on iPhone though. The default focus styles appear to be ``` .c1.cm-editor.cm-focused { outline: 1px dotted #212121; } ``` Which I also see on desktop. Would be nice to changed to whatever the default UA textarea styles are to blend in better but I wouldn't recommend removing it entirely - just to keep the visual indication that the element is focused. Maybe followup material to have a theming pass | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316294156 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316294156 | IC_kwDOBm6k_c5OdQ4M | 9599 | 2022-11-16T04:00:12Z | 2022-11-16T04:00:12Z | OWNER | Have you ever seen CodeMirror correctly auto-completing columns? I'm not entirely sure I believe that the feature works anywhere else. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316293353 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316293353 | IC_kwDOBm6k_c5OdQrp | 9599 | 2022-11-16T03:59:03Z | 2022-11-16T03:59:03Z | OWNER | Deployed a fresh copy: ``` datasette publish vercel fixtures.db \ --branch b7b2942b13f9ea09cfa9f8c73e2869b9bd2349ae \ --project datasette-pr-1893 \ --about 'PR 1893' \ --about_url https://github.com/simonw/datasette/pull/1893 \ --scope datasette ``` https://datasette-pr-1893.vercel.app/fixtures | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1886#issuecomment-1316289392 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1316289392 | IC_kwDOBm6k_c5OdPtw | 45195 | 2022-11-16T03:54:17Z | 2022-11-16T03:58:56Z | NONE | Happy Birthday Datasette! Thanks Simon!! I use datasette on everything most notably [my flickr metadata SQLite DB](https://www.dropbox.com/s/6j10e2vohp2j5kf/roland2019-2020.db?dl=0) to make art. Datasette lite on my 2019 flickr metadata is super helpful too: https://lite.datasette.io/?csv=https%3A%2F%2Fraw.githubusercontent.com%2Frtanglao%2Frt-flickr-sqlite-csv%2Fmain%2F2019-roland-flickr-metadata.csv Even better datasette lite on all firefox support questions from 2021: https://lite.datasette.io/?url=https%3A%2F%2Fraw.githubusercontent.com%2Frtanglao%2Frt-kits-api3%2Fmain%2FYEARLY_CSV_FILES%2F2021-firefox-sumo-questions.db Thanks again Simon! So great! What a gift to the world!!!!!! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1316262169 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1316262169 | IC_kwDOBm6k_c5OdJEZ | 9599 | 2022-11-16T03:22:40Z | 2022-11-16T03:22:40Z | OWNER | Actually this works as it should in desktop Safari: ![autocomplete-safari](https://user-images.githubusercontent.com/9599/202075764-fbc4b4c8-c92f-4f69-81fd-84002de5aea7.gif) I'm going to just put up with the weird behaviour in Mobile Safari. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316256386 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316256386 | IC_kwDOBm6k_c5OdHqC | 95570 | 2022-11-16T03:18:06Z | 2022-11-16T03:18:06Z | CONTRIBUTOR | > If you can get a version of this working with table and column autocompletion just using a static JavaScript object in the source code with the right tables and columns, I'm happy to take on the work of turning that static object into something that Datasette includes in the page itself with all of the correct values. This version "sort of" works when on the main database page where the template passes the relevant data https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25 by doing this and passing that into the `schema` object: ``` let TABLES_DATA = []; {% if tables is defined %} TABLES_DATA = {{ tables | tojson(indent=2) }}; {% endif %} // Turn into an object, shaped like https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/test/test-complete.ts#L27. const TABLES_SCHEMA = Object.fromEntries( new Map( TABLES_DATA.map((table) => { return [table.name, table.columns]; }) ).entries() ); ``` But there are a number of papercuts with it - it's not escaping table names with spaces (likely be fixable from the data being passed into the view) but mainly it doesn't seem to autocomplete columns. I think it might only want to do it when you first type the table name from my read of https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/test/test-complete.ts#L37. It's possible I'm just passing something wrong, but it may end up being something that needs feature work upstream. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316253186 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316253186 | IC_kwDOBm6k_c5OdG4C | 9599 | 2022-11-16T03:16:36Z | 2022-11-16T03:16:36Z | OWNER | Yeah I haven't written this down anywhere but Datasette definitely has an undocumented preference for lower-case SQL. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1316242752 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1316242752 | IC_kwDOBm6k_c5OdEVA | 9599 | 2022-11-16T03:10:52Z | 2022-11-16T03:12:47Z | OWNER | https://bugs.webkit.org/show_bug.cgi?id=201768 - " Datalist option's label not used" - marked as RESOLVED FIXED on March 31st 2020. The commit: https://trac.webkit.org/changeset/259330/webkit And here's the test mirrored on GitHub: https://cs.github.com/qtwebkit/webkit-mirror/blob/cc3fcd0b4bad1f7cf77c26e34aa01d16618d6d5e/LayoutTests/fast/forms/datalist/datalist-option-labels.html?q=datalist-option-labels.html | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316243602 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316243602 | IC_kwDOBm6k_c5OdEiS | 95570 | 2022-11-16T03:11:46Z | 2022-11-16T03:11:46Z | CONTRIBUTOR | Was just reviewing the SQL options and there's an [upperCaseKeywords](https://github.com/codemirror/lang-sql#user-content-sqlconfig.uppercasekeywords) if we'd rather have SELECT vs select. Datasette seems to prefer lowercase so probably best to keep it as-is | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1316240839 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1316240839 | IC_kwDOBm6k_c5OdD3H | 9599 | 2022-11-16T03:09:11Z | 2022-11-16T03:09:11Z | OWNER | Here's a polyfill for `<datalist>`: https://github.com/mfranzke/datalist-polyfill It shouldn't be necessary now that Safari has shipped support (apparently added in https://developer.apple.com/documentation/safari-release-notes/safari-12_1-release-notes#3130314 Safari 12.1 in March 2019). But it does look like Safari doesn't support differing `label` and `value` attributes, though documentation about this is hard to come by. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316236448 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316236448 | IC_kwDOBm6k_c5OdCyg | 9599 | 2022-11-16T03:04:57Z | 2022-11-16T03:04:57Z | OWNER | If you rebase from `main` you should get the fix for that test failure. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1316233532 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1316233532 | IC_kwDOBm6k_c5OdCE8 | 9599 | 2022-11-16T03:00:58Z | 2022-11-16T03:00:58Z | OWNER | Oops, introduced a test failure: ``` def test_table_html_foreign_key_facets(app_client): response = app_client.get( "/fixtures/foreign_key_references?_facet=foreign_key_with_blank_label" ) assert response.status == 200 > assert ( '<li><a href="http://localhost/fixtures/foreign_key_references?_facet=foreign_key_with_blank_label&foreign_key_with_blank_label=3">' "-</a> 1</li>" ) in response.text E assert '<li><a href="http://localhost/fixtures/foreign_key_references?_facet=foreign_key_with_blank_label&foreign_key_with_blank_label=3">-</a> 1</li>' in '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: foreign_key_references: 2 rows</title>\n <link rel="styleshe.../script>\n\n\n<!-- Templates considered: table-fixtures-foreign_key_references.html, *table.html -->\n</body>\n</html>' E + where '<!DOCTYPE html>\n<html>\n<head>\n <title>fixtures: foreign_key_references: 2 rows</title>\n <link rel="styleshe.../script>\n\n\n<!-- Templates considered: table-fixtures-foreign_key_references.html, *table.html -->\n</body>\n</html>' = <datasette.utils.testing.TestResponse object at 0x7fd1b0080640>.text ``` Need to fix this test: https://github.com/simonw/datasette/blob/eac028d3f77aa5473a5fcf59240635a1bca80f7d/tests/test_table_html.py#L616-L624 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316232588 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316232588 | IC_kwDOBm6k_c5OdB2M | 9599 | 2022-11-16T03:00:04Z | 2022-11-16T03:00:04Z | OWNER | Oops, the tests are failing because of a test failure I introduced here: - #1890 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316231560 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316231560 | IC_kwDOBm6k_c5OdBmI | 9599 | 2022-11-16T02:59:00Z | 2022-11-16T02:59:00Z | OWNER | The resize handle doesn't appear on Mobile Safari on iPhone - I don't think that particularly matters though. The textarea does get a weird border around it when focused on iPhone though. Focused: ![BF34E8FB-E35C-4CAB-9BFB-8EEF7E29B16C_1_201_a](https://user-images.githubusercontent.com/9599/202072748-c85bab94-a039-4ed6-8185-3cac25c78ed3.jpeg) Not focused: ![31A5CF38-D540-4A1A-8A7D-E29453D150F4_1_201_a](https://user-images.githubusercontent.com/9599/202072744-d9f0ea62-13b7-46ff-afe1-6d88d7fb8b53.jpeg) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316227073 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316227073 | IC_kwDOBm6k_c5OdAgB | 9599 | 2022-11-16T02:54:22Z | 2022-11-16T02:54:32Z | OWNER | If you can get a version of this working with table and column autocompletion just using a static JavaScript object in the source code with the right tables and columns, I'm happy to take on the work of turning that static object into something that Datasette includes in the page itself with all of the correct values. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316141764 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316141764 | IC_kwDOBm6k_c5OcrrE | 9599 | 2022-11-16T01:26:59Z | 2022-11-16T01:26:59Z | OWNER | Resizing works great for me - and the page automatically sizes the editor to fit an existing query, e.g. on https://datasette-pr-1893.vercel.app/fixtures?sql=select+id%2C+content%2C+content2%0D%0A++from+primary_key_multiple_columns_explicit_label%0D%0A++order+by+id%0D%0A++limit+101 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316137982 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316137982 | IC_kwDOBm6k_c5Ocqv- | 9599 | 2022-11-16T01:23:47Z | 2022-11-16T01:23:47Z | OWNER | Autocomplete here looks promising (I've wanted that to work for years!), but it does currently show a whole bunch of suggestions which aren't part of the SQLite SQL dialect: ![autocomplete](https://user-images.githubusercontent.com/9599/202060211-51ec9f45-bc52-459a-a729-27fc2faadff9.gif) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316135244 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316135244 | IC_kwDOBm6k_c5OcqFM | 9599 | 2022-11-16T01:21:41Z | 2022-11-16T01:21:41Z | OWNER | I just deployed a demo instance like this (using the commit hash from this PR): ```bash datasette publish vercel fixtures.db \ --branch 544f7025900b78f63c34b9985522271ba5fd9c0f \ --project datasette-pr-1893 \ --scope datasette \ --about 'PR 1893' \ --about_url https://github.com/simonw/datasette/pull/1893 ``` Here's the result: https://datasette-pr-1893.vercel.app/fixtures | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1316041828 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1316041828 | IC_kwDOBm6k_c5OcTRk | 95570 | 2022-11-15T23:51:35Z | 2022-11-15T23:51:35Z | CONTRIBUTOR | I experimented with autocompleting the actual schema in https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25, but it would need some work (current problems with it listed in the commit message there) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1315869946 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1315869946 | IC_kwDOBm6k_c5ObpT6 | 95570 | 2022-11-15T21:12:38Z | 2022-11-15T21:12:38Z | CONTRIBUTOR | https://github.com/Sphinxxxx/cm-resize isn't compatible with 6. There's a suggestion to try using CSS resize in https://discuss.codemirror.net/t/resizing-codemirror-6/3265/2 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1315869040 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1315869040 | IC_kwDOBm6k_c5ObpFw | 95570 | 2022-11-15T21:11:42Z | 2022-11-15T21:11:42Z | CONTRIBUTOR | extraKeys is done - Shift+Enter is added in the helper function, and it appears that the Tab behavior now defaults to what the `Tab: false` setting was doing (allowing it to escape to the form) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/pull/1893#issuecomment-1315853097 | https://api.github.com/repos/simonw/datasette/issues/1893 | 1315853097 | IC_kwDOBm6k_c5OblMp | 95570 | 2022-11-15T20:55:40Z | 2022-11-15T20:55:40Z | CONTRIBUTOR | Should also minify the bundled output | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450363982 | |
https://github.com/simonw/datasette/issues/1892#issuecomment-1315814786 | https://api.github.com/repos/simonw/datasette/issues/1892 | 1315814786 | IC_kwDOBm6k_c5Obb2C | 4399499 | 2022-11-15T20:14:38Z | 2022-11-15T20:14:38Z | NONE | I have no particular point, I just want to say being around for the 1.0 release of Datasette seems historic and legendary to witness. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450312343 | |
https://github.com/simonw/datasette/issues/1863#issuecomment-1315812212 | https://api.github.com/repos/simonw/datasette/issues/1863 | 1315812212 | IC_kwDOBm6k_c5ObbN0 | 9599 | 2022-11-15T20:12:02Z | 2022-11-15T20:12:02Z | OWNER | If the update succeeds it will return `{"ok": true}`. For consistency with `/db/table/-/insert` you can pass `"return": true` and it will return a `"row"` key with the now-updated full row. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425029242 | |
https://github.com/simonw/datasette/issues/1863#issuecomment-1315809867 | https://api.github.com/repos/simonw/datasette/issues/1863 | 1315809867 | IC_kwDOBm6k_c5ObapL | 9599 | 2022-11-15T20:09:44Z | 2022-11-15T20:09:44Z | OWNER | I'm also not going to implement `"alter": true` yet (which would add any missing columns based on the update) - I'll hold that off for a later feature. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425029242 | |
https://github.com/simonw/datasette/issues/1863#issuecomment-1315809260 | https://api.github.com/repos/simonw/datasette/issues/1863 | 1315809260 | IC_kwDOBm6k_c5Obafs | 9599 | 2022-11-15T20:09:11Z | 2022-11-15T20:09:11Z | OWNER | I'm going to use the error format I've been experimenting with here: - #1875 ```json { "type": "https://example.net/validation-error", "title": "Your request is not valid.", "errors": [ { "detail": "must be a positive integer", "pointer": "#/age" }, { "detail": "must be 'green', 'red' or 'blue'", "pointer": "#/profile/color" } ] } ``` I'm not quite ready to commit to a `type` URL though, so I'll leave that to be solved later should I fully embrace that RFC. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425029242 | |
https://github.com/simonw/datasette/issues/1863#issuecomment-1315808062 | https://api.github.com/repos/simonw/datasette/issues/1863 | 1315808062 | IC_kwDOBm6k_c5ObaM- | 9599 | 2022-11-15T20:08:04Z | 2022-11-15T20:08:04Z | OWNER | The initial design I'm going to implement will look like this: ``` POST /db/table/1/-/update Authorization: Bearer xxx Content-Type: application/json ``` ```json { "update": { "name": "New name" } } ``` Any fields that are not yet columns will return an error. Should it enforce types, in as much as an integer column should have a JSON integer passed to it, or should it allow strings containing valid integers? I'm going to allow strings, mainly as a workaround for the fact that JavaScript integers have a maximum size. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425029242 | |
https://github.com/simonw/datasette/issues/1892#issuecomment-1315805498 | https://api.github.com/repos/simonw/datasette/issues/1892 | 1315805498 | IC_kwDOBm6k_c5ObZk6 | 9599 | 2022-11-15T20:05:30Z | 2022-11-15T20:05:30Z | OWNER | One slight concern: https://latest.datasette.io/ will increasingly reflect a version that isn't the most recent production release. I might setup https://stable.datasette.io/ as a demo instance of the most recent non-alpha release to compensate for that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450312343 | |
https://github.com/simonw/datasette/issues/1892#issuecomment-1315804535 | https://api.github.com/repos/simonw/datasette/issues/1892 | 1315804535 | IC_kwDOBm6k_c5ObZV3 | 9599 | 2022-11-15T20:04:38Z | 2022-11-15T20:04:38Z | OWNER | I'll do this after the 1.0a0 release: - #1708 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1450312343 | |
https://github.com/simonw/datasette/issues/1708#issuecomment-1095675839 | https://api.github.com/repos/simonw/datasette/issues/1708 | 1095675839 | IC_kwDOBm6k_c5BTq-_ | 9599 | 2022-04-11T23:06:30Z | 2022-11-15T19:57:53Z | OWNER | # Datasette 1.0 alpha 1 This alpha release is a preview of Datasette 1.0. Datasette 1.0 marks a significant milestone in the project: it is the point from which various aspects of Datasette can be considered "stable", in that code developed against them should expect not to be broken by future releases in the 1.x series. This will hold true until the next major version release, Datasette 2.0 - which we hope to hold off releasing for as long as possible. The following Datasette components should be be considered stable after 1.0: - The plugin API. Plugins developed against 1.0 should continue to work unmodified throughout the 1.x series. - The JSON API. Code written that interacts with Datasette's default JSON web API should continue to work. - The template context. If you build custom templates against Datasette your custom pages should continue to work. Note that none of these components will cease to introduce new features. New plugin hooks, new JSON APIs and new template context variables can be introduced without breaking existing code. Since this alpha release previews features that will be frozen for 1.0, please test this thoroughly against your existing Datasette projects. You can install the alpha using: pip install datasette==1.0a0 ## JSON API changes The most significant changes introduced in this new alpha concern Datasette's JSON API. The default JSON returned by the `/database/table.json` endpoint has changed. It now returns an object with two keys: `rows` - which contains a list of objects representing the rows in the table or query, and `more` containing a `boolean` that shows if there are more rows or if this object contains them all. ```json { "rows": [{ "id": 1, "name": "Name 1" }, { "id": 2, "name": "Name 2" }], "more": false } ``` [ Initially I thought about going with `next_url`, which would be `null` if you have reached the last page of records. Maybe that would be better? But since `next_url` cannot be provided… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1200649124 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314891228 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314891228 | IC_kwDOBm6k_c5OX6Xc | 9599 | 2022-11-15T07:23:01Z | 2022-11-15T07:23:01Z | OWNER | Annoying: Mobile Safari doesn't seem to support separate labels and values. I should probably disable this feature on that browser, at least for foreign key facets (for the moment). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314856513 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314856513 | IC_kwDOBm6k_c5OXx5B | 9599 | 2022-11-15T06:56:29Z | 2022-11-15T06:56:29Z | OWNER | Looks like I can fix that like so: ```html <datalist id="datalist-_city_id"> <option label="San Francisco" value="1"></option> <option label="Los Angeles" value="2"></option> <option label="Detroit" value="3"></option> <option label="Memnonia" value="4"></option> </datalist> ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314850524 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314850524 | IC_kwDOBm6k_c5OXwbc | 9599 | 2022-11-15T06:48:37Z | 2022-11-15T06:48:37Z | OWNER | Spotted a bug with this on https://latest.datasette.io/fixtures/facetable?_facet=_city_id - the `_city_id` column is a foreign key, so you need to type `1` or `2` - but the autocomplete list shows the full text names for the cities. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314849867 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314849867 | IC_kwDOBm6k_c5OXwRL | 9599 | 2022-11-15T06:47:51Z | 2022-11-15T06:47:51Z | OWNER | Demo now live here: https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=party - select `party` and start typing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314848432 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314848432 | IC_kwDOBm6k_c5OXv6w | 9599 | 2022-11-15T06:46:08Z | 2022-11-15T06:46:08Z | OWNER | Wrote a TIL about `<datalist>`: https://til.simonwillison.net/html/datalist | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1862#issuecomment-1314845667 | https://api.github.com/repos/simonw/datasette/issues/1862 | 1314845667 | IC_kwDOBm6k_c5OXvPj | 9599 | 2022-11-15T06:42:09Z | 2022-11-15T06:42:32Z | OWNER | I implemented this as part of `/db/-/create`. https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425011030 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314835740 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314835740 | IC_kwDOBm6k_c5OXs0c | 9599 | 2022-11-15T06:30:26Z | 2022-11-15T06:30:26Z | OWNER | That prototype actually works really well! I'm going to add that to `table.js`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314833881 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314833881 | IC_kwDOBm6k_c5OXsXZ | 9599 | 2022-11-15T06:27:21Z | 2022-11-15T06:27:21Z | OWNER | Here's a prototype: ```javascript function createDataLists() { var facetResults = document.querySelectorAll(".facet-results [data-column]"); Array.from(facetResults).forEach(function (facetResult) { // Use link text from all links in the facet result var linkTexts = Array.from( facetResult.querySelectorAll("li:not(.facet-truncated) a") ).map(function (link) { return link.textContent; }); // Create a datalist element var datalist = document.createElement("datalist"); datalist.id = "datalist-" + facetResult.dataset.column; // Create an option element for each link text linkTexts.forEach(function (linkText) { var option = document.createElement("option"); option.value = linkText; datalist.appendChild(option); }); // Add the datalist to the facet result facetResult.appendChild(datalist); }); } createDataLists(); // When any select with name=_filter_column changes, update the datalist document.body.addEventListener("change", function (event) { if (event.target.name === "_filter_column") { event.target .closest(".filter-row") .querySelector(".filter-value") .setAttribute("list", "datalist-" + event.target.value); } }); ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314829751 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314829751 | IC_kwDOBm6k_c5OXrW3 | 9599 | 2022-11-15T06:20:50Z | 2022-11-15T06:20:50Z | OWNER | This finds the right links on the page: document.querySelectorAll('.facet-results [data-column] li:not(.facet-truncated) a') | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314825019 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314825019 | IC_kwDOBm6k_c5OXqM7 | 9599 | 2022-11-15T06:13:36Z | 2022-11-15T06:13:36Z | OWNER | This could start out as a purely JavaScript enhancement for pages that already figured out the available values through faceting, like you suggested. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314823752 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314823752 | IC_kwDOBm6k_c5OXp5I | 9599 | 2022-11-15T06:11:49Z | 2022-11-15T06:11:49Z | OWNER | I tried this out on https://congress-legislators.datasettes.com/legislators/legislator_terms for the `party` column - here's the demo: ![datalist](https://user-images.githubusercontent.com/9599/201839812-db887ce0-c4b9-432c-8620-5ac73f222a63.gif) I made this work by dropping the following HTML into the page in the browser DevTools: ```html <datalist id="party"> <option value="Anti-Administration"> <option value="Pro-Administration"> <option value="Republican"> <option value="Federalist"> <option value="Democratic Republican"> <option value="Pro-administration"> <option value="Anti-administration"> <option value="Unknown"> <option value="Adams"> <option value="Jackson"> <option value="Jackson Republican"> <option value="Crawford Republican"> <option value="Whig"> <option value="Jacksonian Republican"> <option value="Jacksonian"> <option value="Anti-Jacksonian"> <option value="Adams Democrat"> <option value="Nullifier"> <option value="Anti Mason"> <option value="Anti Masonic"> <option value="Anti Jacksonian"> <option value="Democrat"> <option value="Anti Jackson"> <option value="Union Democrat"> <option value="Conservative"> <option value="Ind. Democrat"> <option value="Independent"> <option value="Law and Order"> <option value="American"> <option value="Liberty"> <option value="Free Soil"> <option value="Ind. Republican-Democrat"> <option value="Ind. Whig"> <option value="Unionist"> <option value="States Rights"> <option value="Anti-Lecompton Democrat"> <option value="Constitutional Unionist"> <option value="Independent Democrat"> <option value="Unconditional Unionist"> <option value="Conservative Republican"> <option value="Ind. Republican"> <option value="Liberal Republican"> <option value="National Greenbacker"> <option value="Readjuster Democrat"> <option value="Readjuster"> <option value="Union"> <option value="Union Labor"> <option value="Populist"> <option value="Silver Republican"> <option value="Free Silver"> <option value="Silver"> <option value=… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1890#issuecomment-1314821337 | https://api.github.com/repos/simonw/datasette/issues/1890 | 1314821337 | IC_kwDOBm6k_c5OXpTZ | 9599 | 2022-11-15T06:08:19Z | 2022-11-15T06:08:19Z | OWNER | Oh interesting... this doesn't even need to be attached to the visible faceting feature, necessarily: Datasette could try to detect when a column has a limited number of options (which the faceting code handles already) and could turn those into an auto-complete interface. There's actually a native HTML element for this these days: the `<datalist>` https://developer.mozilla.org/en-US/docs/Web/HTML/Element/datalist | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1448143294 | |
https://github.com/simonw/datasette/issues/1882#issuecomment-1314813205 | https://api.github.com/repos/simonw/datasette/issues/1882 | 1314813205 | IC_kwDOBm6k_c5OXnUV | 9599 | 2022-11-15T06:00:41Z | 2022-11-15T06:00:41Z | OWNER | Documentation: - https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table - https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data Wrote a TIL about how I wrote some of those tests with Copilot: https://til.simonwillison.net/gpt3/writing-test-with-copilot | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1435294468 | |
https://github.com/simonw/datasette/issues/1886#issuecomment-1314627077 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1314627077 | IC_kwDOBm6k_c5OW54F | 11788561 | 2022-11-15T01:19:54Z | 2022-11-15T01:19:54Z | NONE | Datasette usage comments for its 5th anniversary celebration: I use Datasette and related tools for a Cosmology Researcher Talks database app project, which is [described in the github Readme](https://github.com/jrdmb/cosmotalks-datasette#readme) The app hosted on the Google Cloud Run service also uses other Datasette-related tools developed by Simon - datasette-render-markdown, csvs-to-sqlite, datasette-template-sql, and datasette-block-robots. This is one of two apps used for querying the talks database, each has it pros/cons as described in the github Readme. At present, over 170 different sites that host cosmology talks are scraped to collect new talks for import into the sqlite database. The shot-scraper and sqlite-utils tools are a major help for this. I also use the Mastodon API to get my favorites, toots, and boosts into a local database so I can do searches on the data. This was done on Twitter and was then extended to the Mastodon data. Again, sqlite-utils is an important tool for this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/datasette/issues/1875#issuecomment-1314620086 | https://api.github.com/repos/simonw/datasette/issues/1875 | 1314620086 | IC_kwDOBm6k_c5OW4K2 | 9599 | 2022-11-15T01:09:56Z | 2022-11-15T01:09:56Z | OWNER | Rough initial prototype: ```diff diff --git a/datasette/views/table.py b/datasette/views/table.py index 8b987221..518ac578 100644 --- a/datasette/views/table.py +++ b/datasette/views/table.py @@ -1103,19 +1103,30 @@ class TableInsertView(BaseView): except json.JSONDecodeError as e: return _errors(["Invalid JSON: {}".format(e)]) if not isinstance(data, dict): - return _errors(["JSON must be a dictionary"]) + return _errors([{"detail": "JSON must be a dictionary", "pointer": "#/"}]) keys = data.keys() # keys must contain "row" or "rows" if "row" not in keys and "rows" not in keys: return _errors(['JSON must have one or other of "row" or "rows"']) rows = [] + was_single_row = False if "row" in keys: if "rows" in keys: - return _errors(['Cannot use "row" and "rows" at the same time']) + return _errors( + [ + { + "detail": 'Cannot use "row" and "rows" at the same time', + "pointer": "#/row", + } + ] + ) + was_single_row = True row = data["row"] if not isinstance(row, dict): - return _errors(['"row" must be a dictionary']) + return _errors( + [{"detail": '"row" must be a dictionary', "pointer": "#/row"}] + ) rows = [row] data["return"] = True else: @@ -1152,9 +1163,12 @@ class TableInsertView(BaseView): invalid_columns = set(row.keys()) - columns if invalid_columns: errors.append( - "Row {} has invalid columns: {}".format( - i, ", ".join(sorted(invalid_columns)) - ) + { + "detail": "In… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430797211 | |
https://github.com/simonw/datasette/issues/1875#issuecomment-1314615592 | https://api.github.com/repos/simonw/datasette/issues/1875 | 1314615592 | IC_kwDOBm6k_c5OW3Eo | 9599 | 2022-11-15T01:04:28Z | 2022-11-15T01:04:28Z | OWNER | Worth noting this bit in RFC 7807: > The fictional problem type here defines the "errors" extension, an > array that describes the details of each validation error. Each > member is an object containing "detail" to describe the issue, and > "pointer" to locate the problem within the request's content using a > JSON Pointer [JSON-POINTER]. So the list of `"errors"` with JSON Pointer isn't technically part of the spec, it's an imaginary extension. It fits what I need to do though, so I'm inclined to stick with it anyway. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430797211 | |
https://github.com/simonw/datasette/issues/1875#issuecomment-1314545407 | https://api.github.com/repos/simonw/datasette/issues/1875 | 1314545407 | IC_kwDOBm6k_c5OWl7_ | 9599 | 2022-11-14T23:30:34Z | 2022-11-14T23:30:34Z | OWNER | TIL: https://til.simonwillison.net/json/json-pointer | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430797211 | |
https://github.com/simonw/datasette/issues/1875#issuecomment-1314491884 | https://api.github.com/repos/simonw/datasette/issues/1875 | 1314491884 | IC_kwDOBm6k_c5OWY3s | 9599 | 2022-11-14T22:26:11Z | 2022-11-14T22:26:54Z | OWNER | Spec looks pretty simple: > A JSON Pointer is a Unicode string (see [RFC4627], Section 3) > containing a sequence of zero or more reference tokens, each prefixed > by a `/` (%x2F) character. > > Because the characters `~` (%x7E) and `/` (%x2F) have special > meanings in JSON Pointer, `~` needs to be encoded as `~0` and `/` > needs to be encoded as `~1` when these characters appear in a > reference token. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430797211 | |
https://github.com/simonw/datasette/issues/1875#issuecomment-1314491150 | https://api.github.com/repos/simonw/datasette/issues/1875 | 1314491150 | IC_kwDOBm6k_c5OWYsO | 9599 | 2022-11-14T22:25:20Z | 2022-11-14T22:25:20Z | OWNER | That's using JSON Pointer: https://www.rfc-editor.org/rfc/rfc6901 There's a Python library for that here https://github.com/stefankoegl/python-json-pointer/blob/master/jsonpointer.py - which looks simple and clean and well maintained and documented, but it only handles the "what is at this pointer within this JSON object" case - I need to generate the correct JSON pointer to explain where my error is. So I think I'll end up hand-rolling this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430797211 | |
https://github.com/simonw/datasette/issues/1875#issuecomment-1314488010 | https://api.github.com/repos/simonw/datasette/issues/1875 | 1314488010 | IC_kwDOBm6k_c5OWX7K | 9599 | 2022-11-14T22:21:43Z | 2022-11-14T22:21:43Z | OWNER | Here's the most relevant example from the RFC spec: ``` POST /details HTTP/1.1 Host: account.example.com Accept: application/json ``` ```json { "age": 42.3, "profile": { "color": "yellow" } } ``` ``` HTTP/1.1 400 Bad Request Content-Type: application/problem+json Content-Language: en ``` ```json { "type": "https://example.net/validation-error", "title": "Your request is not valid.", "errors": [ { "detail": "must be a positive integer", "pointer": "#/age" }, { "detail": "must be 'green', 'red' or 'blue'", "pointer": "#/profile/color" } ] } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430797211 | |
https://github.com/simonw/datasette/issues/1886#issuecomment-1314455003 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1314455003 | IC_kwDOBm6k_c5OWP3b | 17053189 | 2022-11-14T21:51:11Z | 2022-11-14T21:51:11Z | NONE | Happy Birthday Datasette! I am a librarian at the Université du Québec à Montréal (UQAM) and I've been using Datasette to publish excerpts of our library data. There are several use cases I'm working with as a proof of concept : 1. New titles list : based on reports of recent acquisitions by subject, discipline, etc. 2. List of all UQAM theses and dissertations : based on an extract of bibliographic records 3. List of all publications by UQAM Authors : based on an extract of bibliographic records See our prototype under construction here : https://datasette-bib.uqam.ca/ (some bits and pieces have been translated into French) Datasette is amazing, there is so much potential here for libraries. Thanks to Simon and all the contributors for this outstanding effort. Also sqlite-utils deserves special mention as incredibly handy and useful. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/datasette/issues/1886#issuecomment-1314241058 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1314241058 | IC_kwDOBm6k_c5OVboi | 25778 | 2022-11-14T19:06:35Z | 2022-11-14T19:06:35Z | CONTRIBUTOR | This probably counts as a case study: https://github.com/eyeseast/spatial-data-cooking-show. Even has video. Seriously, though, this workflow has become integral to my work with reporters and editors across USA TODAY Network. Very often, I get sent a folder of data in mixed formats, with a vague ask of how we should communicate some part of it to users. Datasette and its constellation of tools makes it easy to get a quick look at that data, run exploratory queries, map it and ask questions to figure out what's important to show. And then I export a version of the data that's exactly what I need for display. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/datasette/issues/1886#issuecomment-1314223118 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1314223118 | IC_kwDOBm6k_c5OVXQO | 639730 | 2022-11-14T18:51:20Z | 2022-11-14T18:51:20Z | NONE | I use Datasette to analyze blocklists by using csv-to-sqlite to pull their contents into a database and Datasette to look around through them. I also use its REST API to query said database as part of filtering out garbage from domains found in those blocklists. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/datasette/issues/1884#issuecomment-1314066229 | https://api.github.com/repos/simonw/datasette/issues/1884 | 1314066229 | IC_kwDOBm6k_c5OUw81 | 25778 | 2022-11-14T16:48:35Z | 2022-11-14T16:48:35Z | CONTRIBUTOR | I'm realizing I don't know if a virtual table will ever return a count. Maybe it depends on the implementation. For these three, just checking now, it'll always return zero. That said, I'm not sure there's any downside to having them return zero and caching that. (They're hidden, too.) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1439009231 | |
https://github.com/simonw/datasette/issues/1884#issuecomment-1314054300 | https://api.github.com/repos/simonw/datasette/issues/1884 | 1314054300 | IC_kwDOBm6k_c5OUuCc | 9599 | 2022-11-14T16:40:06Z | 2022-11-14T16:40:06Z | OWNER | I wonder if there are any reasons that inspect SHOULD try to count virtual tables? Like are there any likely uses for a cirial table where the count is both interesting and likely to be accessed often enough that it's worth caching? I have an issue open to add a setting to disable table counts entirely: - #1818 Maybe that should be expanded to automatically disable row counts for virtual tables entirely? Which would mean no count would be shown for them in the UI. If you desperately wanted a count you would then have to run a count(*) query against them explicitly. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1439009231 | |
https://github.com/simonw/datasette/issues/1884#issuecomment-1313962183 | https://api.github.com/repos/simonw/datasette/issues/1884 | 1313962183 | IC_kwDOBm6k_c5OUXjH | 25778 | 2022-11-14T15:46:32Z | 2022-11-14T15:46:32Z | CONTRIBUTOR | It does work, though I think it's probably still worth excluding virtual tables that will always be zero. Here's the same inspection as before, now with `--load-extension spatialite`: ```json { "alltheplaces": { "hash": "0843cfe414439ab903c22d1121b7ddbc643418c35c7f0edbcec82ef1452411df", "size": 963375104, "file": "alltheplaces.db", "tables": { "spatial_ref_sys": { "count": 6215 }, "spatialite_history": { "count": 18 }, "sqlite_sequence": { "count": 2 }, "geometry_columns": { "count": 3 }, "spatial_ref_sys_aux": { "count": 6164 }, "views_geometry_columns": { "count": 0 }, "virts_geometry_columns": { "count": 0 }, "geometry_columns_statistics": { "count": 3 }, "views_geometry_columns_statistics": { "count": 0 }, "virts_geometry_columns_statistics": { "count": 0 }, "geometry_columns_field_infos": { "count": 0 }, "views_geometry_columns_field_infos": { "count": 0 }, "virts_geometry_columns_field_infos": { "count": 0 }, "geometry_columns_time": { "count": 3 }, "geometry_columns_auth": { "count": 3 }, "views_geometry_columns_auth": { "count": 0 }, "virts_geometry_columns_auth": { "count": 0 }, "data_licenses": { "count": 10 }, "sql_statements_log": { "count": 0 }, "states": { "count": 56 }, "counties": { "count": 3234 }, "idx_states_geometry_rowid": { "count": 56 }, "idx_states_geometry_node": { "count": 3 }, "idx_states_geometry_parent": { "count": 2 }, "idx_counties_geometry_rowid": { "count": 3234 }, "idx_counties_geom… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1439009231 |