github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1425#issuecomment-894881016 | https://api.github.com/repos/simonw/datasette/issues/1425 | 894881016 | IC_kwDOBm6k_c41Vsz4 | 9599 | 2021-08-09T00:21:53Z | 2021-08-09T00:21:53Z | OWNER | Still one test failure: ``` def test_hook_render_cell_link_from_json(app_client): sql = """ select '{"href": "http://example.com/", "label":"Example"}' """.strip() path = "/fixtures?" + urllib.parse.urlencode({"sql": sql}) response = app_client.get(path) td = Soup(response.body, "html.parser").find("table").find("tbody").find("td") a = td.find("a") > assert a is not None, str(a) E AssertionError: None E assert None is not None ``` The weird thing about this one is that I can't replicate it on my laptop - but it happens in CI every time, including when I shell in and try to run that single test. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963528457 | |
https://github.com/simonw/datasette/issues/1425#issuecomment-894869692 | https://api.github.com/repos/simonw/datasette/issues/1425 | 894869692 | IC_kwDOBm6k_c41VqC8 | 9599 | 2021-08-08T23:08:29Z | 2021-08-08T23:08:29Z | OWNER | Updated documentation: https://docs.datasette.io/en/latest/plugin_hooks.html#render-cell-value-column-table-database-datasette | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963528457 | |
https://github.com/simonw/datasette/issues/1425#issuecomment-894865323 | https://api.github.com/repos/simonw/datasette/issues/1425 | 894865323 | IC_kwDOBm6k_c41Vo-r | 9599 | 2021-08-08T22:33:19Z | 2021-08-08T22:33:19Z | OWNER | I can do this with the `await_me_maybe()` function, as seen here: https://github.com/simonw/datasette/blob/a21853c9dade240734abc6b4f750fae09a3e840a/datasette/app.py#L864-L873 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963528457 | |
https://github.com/simonw/datasette/issues/1424#issuecomment-894864744 | https://api.github.com/repos/simonw/datasette/issues/1424 | 894864744 | IC_kwDOBm6k_c41Vo1o | 9599 | 2021-08-08T22:27:31Z | 2021-08-08T22:27:31Z | OWNER | https://docs.python.org/3/library/sqlite3.html#exceptions is useful - it looks like `sqlite3.DatabaseError` is the super-class of all of the other exceptions that we might see. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963527045 | |
https://github.com/simonw/datasette/issues/1424#issuecomment-894864682 | https://api.github.com/repos/simonw/datasette/issues/1424 | 894864682 | IC_kwDOBm6k_c41Vo0q | 9599 | 2021-08-08T22:26:46Z | 2021-08-08T22:26:46Z | OWNER | Note that the `sqlite3` exceptions are in `sqlite3` if using the Python standard library but are in `pysqlite3` if that module is being used instead. So maybe encourage people to use them from `datasette.sqlite.sqlite3` instead, which will point to the correct package. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963527045 | |
https://github.com/simonw/datasette/issues/1424#issuecomment-894864616 | https://api.github.com/repos/simonw/datasette/issues/1424 | 894864616 | IC_kwDOBm6k_c41Vozo | 9599 | 2021-08-08T22:26:08Z | 2021-08-08T22:26:08Z | OWNER | - `datasette.database.QueryInterrupted` for queries that were interrupted - `sqlite3.OperationalError` - `sqlite3.DatabaseError` and more | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963527045 | |
https://github.com/simonw/datasette/issues/1424#issuecomment-894864404 | https://api.github.com/repos/simonw/datasette/issues/1424 | 894864404 | IC_kwDOBm6k_c41VowU | 9599 | 2021-08-08T22:24:06Z | 2021-08-08T22:24:06Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/de5ce2e56339ad8966f417a4758f7c210c017dec/datasette/database.py#L176-L200 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
963527045 | |
https://github.com/simonw/datasette/issues/1422#issuecomment-894607989 | https://api.github.com/repos/simonw/datasette/issues/1422 | 894607989 | IC_kwDOBm6k_c41UqJ1 | 9599 | 2021-08-07T05:31:57Z | 2021-08-07T05:31:57Z | OWNER | Demo: https://latest.datasette.io/fixtures/neighborhood_search Documentation: https://docs.datasette.io/en/latest/sql_queries.html#additional-canned-query-options | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
961367843 | |
https://github.com/simonw/datasette/issues/1421#issuecomment-894606843 | https://api.github.com/repos/simonw/datasette/issues/1421 | 894606843 | IC_kwDOBm6k_c41Up37 | 9599 | 2021-08-07T05:17:12Z | 2021-08-07T05:17:12Z | OWNER | Marking this blocked because I don't have a way around the needing-a-SQLite-SQL-parser problem at the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959999095 | |
https://github.com/simonw/datasette/issues/1421#issuecomment-894606796 | https://api.github.com/repos/simonw/datasette/issues/1421 | 894606796 | IC_kwDOBm6k_c41Up3M | 9599 | 2021-08-07T05:16:39Z | 2021-08-07T05:16:39Z | OWNER | Urgh, yeah I've seen this one before. Fixing it pretty much requires writing a full SQLite SQL syntax parser in Python, which is frustratingly complicated for solving this issue! You can work around this for a canned query by using the optional `params:` argument documented here: https://docs.datasette.io/en/stable/sql_queries.html#canned-query-parameters | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959999095 | |
https://github.com/simonw/datasette/issues/1422#issuecomment-894589140 | https://api.github.com/repos/simonw/datasette/issues/1422 | 894589140 | IC_kwDOBm6k_c41UljU | 9599 | 2021-08-07T01:58:16Z | 2021-08-07T01:58:24Z | OWNER | Also need to consider this hidden field - it should pass the `_hide_sql` or `_show_sql` parameters depending on the same logic: https://github.com/simonw/datasette/blob/acc22436622ff8476c30acf45ed60f54b4aaa5d9/datasette/templates/query.html#L47-L49 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
961367843 | |
https://github.com/simonw/datasette/issues/1423#issuecomment-894454644 | https://api.github.com/repos/simonw/datasette/issues/1423 | 894454644 | IC_kwDOBm6k_c41UEt0 | 9599 | 2021-08-06T18:52:49Z | 2021-08-06T18:52:49Z | OWNER | This means that the counts would be unavailable to users who cannot see tooltips (e.g. mobile users) on pages that did not have any facets that broke the 30 limit and hence displayed that "..." link. I think I'm OK with that, for the moment. May revisit in the future. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
962391325 | |
https://github.com/simonw/datasette/issues/1423#issuecomment-894454087 | https://api.github.com/repos/simonw/datasette/issues/1423 | 894454087 | IC_kwDOBm6k_c41UElH | 9599 | 2021-08-06T18:51:42Z | 2021-08-06T18:51:42Z | OWNER | The invisible tooltip could say "Showing 30 items, more available" (helping save you from counting up to 20 if you know about the secret feature). The numbers could then be fully displayed on the "..." page. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
962391325 | |
https://github.com/simonw/datasette/issues/1423#issuecomment-894453520 | https://api.github.com/repos/simonw/datasette/issues/1423 | 894453520 | IC_kwDOBm6k_c41UEcQ | 9599 | 2021-08-06T18:50:40Z | 2021-08-06T18:50:40Z | OWNER | Point of confusion: if only 30 options are shown, but there's a `...` at the end, what would the number be? It can't be the total number of facets because we haven't counted them all - but if it's just the number of displayed facets that's like to be confusing. So the original idea of showing the counts only if `_facet_size=max` is a good one. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
962391325 | |
https://github.com/simonw/datasette/issues/1423#issuecomment-894452990 | https://api.github.com/repos/simonw/datasette/issues/1423 | 894452990 | IC_kwDOBm6k_c41UET- | 9599 | 2021-08-06T18:49:37Z | 2021-08-06T18:49:37Z | OWNER | Could display them always, like this: <img width="1299" alt="fixtures__compound_three_primary_keys__1_001_rows_and_School_enrollment_analysis_Colorado_-_Google_Docs" src="https://user-images.githubusercontent.com/9599/128557997-6bedb52b-45f1-4ab1-9a79-ddf6c01f6036.png"> That's with `<span style="font-size: 0.8em;color: #666;">23</span>` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
962391325 | |
https://github.com/simonw/datasette/issues/1423#issuecomment-893996604 | https://api.github.com/repos/simonw/datasette/issues/1423 | 893996604 | IC_kwDOBm6k_c41SU48 | 9599 | 2021-08-06T04:43:07Z | 2021-08-06T04:43:37Z | OWNER | Problem: on a page which doesn't have quite enough facet values to trigger the display of the "..." link that links to `?_facet_size=max` the user would still have to manually count the values - up to 30 by default. So maybe the count should always be shown, perhaps as a non-bold light colored number? I could even hide it in a non-discoverable tooltip. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
962391325 | |
https://github.com/simonw/datasette/issues/1419#issuecomment-893133496 | https://api.github.com/repos/simonw/datasette/issues/1419 | 893133496 | IC_kwDOBm6k_c41PCK4 | 9599 | 2021-08-05T03:22:44Z | 2021-08-05T03:22:44Z | OWNER | I ran into this exact same problem today! I only just learned how to use filter on aggregates: https://til.simonwillison.net/sqlite/sqlite-aggregate-filter-clauses A workaround I used is to add this to the deploy command: datasette publish cloudrun ... --install=pysqlite3-binary This will install the https://pypi.org/project/pysqlite3-binary for package which bundles a more recent SQLite version. | { "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959710008 | |
https://github.com/simonw/datasette/issues/1422#issuecomment-893131703 | https://api.github.com/repos/simonw/datasette/issues/1422 | 893131703 | IC_kwDOBm6k_c41PBu3 | 9599 | 2021-08-05T03:16:46Z | 2021-08-05T03:16:46Z | OWNER | The logic for this is a little bit fiddly, due to the need to switch to using `?_show_sql=1` on the link depending on the context. - If metadata says hide and there's no query string, hide and link to `?_show_sql=1` - If metadata says hide but query string says `?_show_sql=1`, show and have hide link linking to URL without `?_show_sql=1` - Otherwise, show and link to `?_hide_sql=1` - ... or if that query string is there then hide and link to URL without `?_hide_sql=1` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
961367843 | |
https://github.com/simonw/datasette/issues/1422#issuecomment-893122356 | https://api.github.com/repos/simonw/datasette/issues/1422 | 893122356 | IC_kwDOBm6k_c41O_c0 | 9599 | 2021-08-05T02:52:31Z | 2021-08-05T02:52:44Z | OWNER | If you do this it should still be possible to view the SQL - which means we need a new parameter. I propose `?_show_sql=1` to over-ride the hidden default. I think the configuration should use `hide_sql: true` - looking like this: ```yaml databases: fixtures: queries: neighborhood_search: hide_sql: true sql: |- select neighborhood, facet_cities.name, state from facetable join facet_cities on facetable.city_id = facet_cities.id where neighborhood like '%' || :text || '%' order by neighborhood title: Search neighborhoods ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
961367843 | |
https://github.com/simonw/datasette/issues/1420#issuecomment-893079520 | https://api.github.com/repos/simonw/datasette/issues/1420 | 893079520 | IC_kwDOBm6k_c41O0_g | 9599 | 2021-08-05T00:54:59Z | 2021-08-05T00:54:59Z | OWNER | Just saw this error: `ERROR: (gcloud.run.deploy) The `--cpu` flag is not supported on the fully managed version of Cloud Run. Specify `--platform gke` or run `gcloud config set run/platform gke` to work with Cloud Run for Anthos deployed on Google Cloud.` Which is weird because I managed to run this successfully the other day. Maybe a region difference thing? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959898166 | |
https://github.com/simonw/sqlite-utils/issues/308#issuecomment-892988609 | https://api.github.com/repos/simonw/sqlite-utils/issues/308 | 892988609 | IC_kwDOCGYnMM41OezB | 9599 | 2021-08-04T21:30:59Z | 2021-08-04T21:30:59Z | OWNER | OK, this works great as a proof-of-concept (though maybe I should remove the output from the cells so people have to run it themselves in the Binder environment?) Next challenge: how extensive should it be? What features should I cover? Should I do a basic tutorial and then several advanced tutorials? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
961008507 | |
https://github.com/simonw/sqlite-utils/issues/308#issuecomment-892957281 | https://api.github.com/repos/simonw/sqlite-utils/issues/308 | 892957281 | IC_kwDOCGYnMM41OXJh | 9599 | 2021-08-04T20:37:00Z | 2021-08-04T20:37:00Z | OWNER | https://mybinder.org/v2/gh/simonw/sqlite-utils/main?filepath=docs%2Ftutorial.ipynb | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
961008507 | |
https://github.com/simonw/datasette/issues/1420#issuecomment-892376353 | https://api.github.com/repos/simonw/datasette/issues/1420 | 892376353 | IC_kwDOBm6k_c41MJUh | 9599 | 2021-08-04T05:33:12Z | 2021-08-04T05:33:12Z | OWNER | In the Cloud Run console (before I deleted these services) when I click "Edit and deploy new revision" I see this for the default one: <img width="551" alt="Deploy_new_revision_–_Cloud_Run_–_datasette_–_Google_Cloud_Platform" src="https://user-images.githubusercontent.com/9599/128127309-7ec2064f-b107-4d99-b630-ee58734c4a72.png"> And this for the big one: <img width="546" alt="Deploy_new_revision_–_Cloud_Run_–_datasette_–_Google_Cloud_Platform" src="https://user-images.githubusercontent.com/9599/128127366-7cdcb245-2e83-4b88-9d4f-0c595a486139.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959898166 | |
https://github.com/simonw/datasette/issues/1420#issuecomment-892374253 | https://api.github.com/repos/simonw/datasette/issues/1420 | 892374253 | IC_kwDOBm6k_c41MIzt | 9599 | 2021-08-04T05:27:21Z | 2021-08-04T05:29:59Z | OWNER | I'll delete these services shortly afterwards. Right now: https://fixtures-over-provisioned-issue-1420-j7hipcg4aq-uc.a.run.app/-/psutil (deployed with `--memory 8G --cpu 4`) returns: ``` process.memory_info() pmem(rss=60456960, vms=518930432, shared=0, text=0, lib=0, data=0, dirty=0) ... psutil.cpu_times(True) scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0) scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0) scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0) scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0) psutil.virtual_memory() svmem(total=2147483648, available=2092531712, percent=2.6, used=33103872, free=2092531712, active=44130304, inactive=10792960, buffers=0, cached=21848064, shared=262144, slab=0) ``` https://fixtures-default-issue-1420-j7hipcg4aq-uc.a.run.app/-/psutil returns: ``` process.memory_info() pmem(rss=49324032, vms=140595200, shared=0, text=0, lib=0, data=0, dirty=0) ... psutil.cpu_times(True) scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0) scputimes(user=0.0, nice=0.0, system=0.0, idle=0.0, iowait=0.0, irq=0.0, softirq=0.0, steal=0.0, guest=0.0, guest_nice=0.0) psutil.virtual_memory() svmem(total=2147483648, available=2091188224, percent=2.6, used=40071168, free=2091188224, active=41586688, inactive=7983104, buffers=0, cached=16224256, shared=262144, slab=0) ``` These numbers are different enough that I assume this works as advertised. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959898166 | |
https://github.com/simonw/datasette/issues/1420#issuecomment-892372509 | https://api.github.com/repos/simonw/datasette/issues/1420 | 892372509 | IC_kwDOBm6k_c41MIYd | 9599 | 2021-08-04T05:22:29Z | 2021-08-04T05:22:29Z | OWNER | Testing this manually with: datasette publish cloudrun fixtures.db --memory 8G --cpu 4 \ --service fixtures-over-provisioned-issue-1420 --install datasette-psutil And for comparison: datasette publish cloudrun fixtures.db --service fixtures-default-issue-1420 \ --install datasette-psutil | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959898166 | |
https://github.com/simonw/datasette/issues/1420#issuecomment-892365639 | https://api.github.com/repos/simonw/datasette/issues/1420 | 892365639 | IC_kwDOBm6k_c41MGtH | 9599 | 2021-08-04T05:05:07Z | 2021-08-04T05:05:07Z | OWNER | https://github.com/simonw/datasette/blob/cd8b7bee8fb5c1cdce7c8dbfeb0166011abc72c6/datasette/publish/cloudrun.py#L153-L158 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959898166 | |
https://github.com/simonw/datasette/pull/1418#issuecomment-891984359 | https://api.github.com/repos/simonw/datasette/issues/1418 | 891984359 | IC_kwDOBm6k_c41Kpnn | 9599 | 2021-08-03T16:21:56Z | 2021-08-03T16:21:56Z | OWNER | Failed with: ``` docs/authentication.rst:63: perfom ==> perform docs/authentication.rst:76: perfom ==> perform docs/changelog.rst:429: repsonse ==> response docs/changelog.rst:503: permissons ==> permissions docs/changelog.rst:717: compatibilty ==> compatibility docs/changelog.rst:1172: browseable ==> browsable docs/deploying.rst:191: similiar ==> similar docs/internals.rst:434: Respons ==> Response, respond docs/internals.rst:440: Respons ==> Response, respond docs/internals.rst:717: tha ==> than, that, the docs/performance.rst:42: databse ==> database docs/plugin_hooks.rst:667: utilites ==> utilities docs/publish.rst:168: countainer ==> container docs/settings.rst:352: inalid ==> invalid docs/sql_queries.rst:406: preceeded ==> preceded, proceeded ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959284434 | |
https://github.com/simonw/datasette/issues/1417#issuecomment-891979858 | https://api.github.com/repos/simonw/datasette/issues/1417 | 891979858 | IC_kwDOBm6k_c41KohS | 9599 | 2021-08-03T16:15:30Z | 2021-08-03T16:15:30Z | OWNER | Docs: https://pypi.org/project/codespell/ There's a `codespell --ignore-words=FILE` option for ignoring words. I don't have any that need ignoring yet but I'm going to add that file anyway, that way I can have codespell be a failing test but still provide a way to work around it if it incorrectly flags a correct word. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
959278472 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-891380382 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 891380382 | IC_kwDOCGYnMM41IWKe | 9599 | 2021-08-02T22:39:46Z | 2021-08-02T22:39:46Z | OWNER | Documentation: https://sqlite-utils.datasette.io/en/stable/cli.html#converting-data-in-columns | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/298#issuecomment-891359751 | https://api.github.com/repos/simonw/sqlite-utils/issues/298 | 891359751 | IC_kwDOCGYnMM41IRIH | 9599 | 2021-08-02T21:55:16Z | 2021-08-02T21:55:16Z | OWNER | This is a feature already! You can do this: sqlite-utils insert nl-demo.db mytable data.ndjson --nl See https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-newline-delimited-json | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
951581763 | |
https://github.com/simonw/sqlite-utils/issues/300#issuecomment-891356906 | https://api.github.com/repos/simonw/sqlite-utils/issues/300 | 891356906 | IC_kwDOCGYnMM41IQbq | 9599 | 2021-08-02T21:49:10Z | 2021-08-02T21:49:10Z | OWNER | I'm nervous to set this by default in the Python library because it's a global setting, so I can't isolate it to just code that is being called by a `sqlite_utils` method. The new `sqlite-utils convert` command from #251 DOES turn it on - I think having it on by default for a CLI tool that uses user-defined functions makes sense: https://github.com/simonw/sqlite-utils/blob/e83aef951bd3e8c179511faddb607239a5fa8682/sqlite_utils/cli.py#L2006 I like the suggestion to include this in the documentation. I'll do that! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956832836 | |
https://github.com/simonw/sqlite-utils/issues/306#issuecomment-891352587 | https://api.github.com/repos/simonw/sqlite-utils/issues/306 | 891352587 | IC_kwDOCGYnMM41IPYL | 9599 | 2021-08-02T21:39:34Z | 2021-08-02T21:39:34Z | OWNER | This older TIL would have saved me some time! https://til.simonwillison.net/sphinx/sphinx-ext-extlinks | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
958516743 | |
https://github.com/simonw/datasette/issues/1227#issuecomment-891352132 | https://api.github.com/repos/simonw/datasette/issues/1227 | 891352132 | IC_kwDOBm6k_c41IPRE | 9599 | 2021-08-02T21:38:39Z | 2021-08-02T21:38:39Z | OWNER | Relevant TIL: https://til.simonwillison.net/vscode/vs-code-regular-expressions | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
810394616 | |
https://github.com/simonw/sqlite-utils/issues/306#issuecomment-891352073 | https://api.github.com/repos/simonw/sqlite-utils/issues/306 | 891352073 | IC_kwDOCGYnMM41IPQJ | 9599 | 2021-08-02T21:38:32Z | 2021-08-02T21:38:32Z | OWNER | TIL about this: https://til.simonwillison.net/vscode/vs-code-regular-expressions | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
958516743 | |
https://github.com/simonw/sqlite-utils/issues/304#issuecomment-891257730 | https://api.github.com/repos/simonw/sqlite-utils/issues/304 | 891257730 | IC_kwDOCGYnMM41H4OC | 9599 | 2021-08-02T19:00:00Z | 2021-08-02T19:00:00Z | OWNER | Documented here: - https://sqlite-utils.datasette.io/en/latest/python-api.html#converting-data-in-columns - https://sqlite-utils.datasette.io/en/latest/cli.html#converting-data-in-columns | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957731178 | |
https://github.com/simonw/sqlite-utils/issues/305#issuecomment-890713185 | https://api.github.com/repos/simonw/sqlite-utils/issues/305 | 890713185 | IC_kwDOCGYnMM41FzRh | 9599 | 2021-08-02T04:53:53Z | 2021-08-02T04:53:53Z | OWNER | I'm going to rename `.execute_count()` to `.count_where()` - but I'll keep `.execute_count()` in the codebase since it's a documented API at https://sqlite-utils.datasette.io/en/3.13/python-api.html#count and I don't want to do a major version bump. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957741820 | |
https://github.com/simonw/sqlite-utils/issues/304#issuecomment-890711924 | https://api.github.com/repos/simonw/sqlite-utils/issues/304 | 890711924 | IC_kwDOCGYnMM41Fy90 | 9599 | 2021-08-02T04:50:13Z | 2021-08-02T04:50:13Z | OWNER | For the Python function, the args will be `where=None, where_args=None`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957731178 | |
https://github.com/simonw/sqlite-utils/issues/304#issuecomment-890704624 | https://api.github.com/repos/simonw/sqlite-utils/issues/304 | 890704624 | IC_kwDOCGYnMM41FxLw | 9599 | 2021-08-02T04:28:42Z | 2021-08-02T04:28:42Z | OWNER | For the command-line version this can duplicate the `--param` option to allow named parameters in the where clause: https://github.com/simonw/sqlite-utils/blob/c7e8d72be9fe8fe0811f685a18eebc637662d41b/sqlite_utils/cli.py#L1096-L1102 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957731178 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890553783 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890553783 | IC_kwDOCGYnMM41FMW3 | 9599 | 2021-08-01T16:59:09Z | 2021-08-01T16:59:09Z | OWNER | I'm going with `recipes.jsonsplit()` rather than `recipe.jsonsplit()` because the Python module containing the recipes will be called `recipes`. I'll set up a `r.jsonsplit()` shortcut too as a convenience. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890552827 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890552827 | IC_kwDOCGYnMM41FMH7 | 9599 | 2021-08-01T16:52:00Z | 2021-08-01T16:52:00Z | OWNER | I'll finish the work on this in a PR. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/302#issuecomment-890548009 | https://api.github.com/repos/simonw/sqlite-utils/issues/302 | 890548009 | IC_kwDOCGYnMM41FK8p | 9599 | 2021-08-01T16:18:13Z | 2021-08-01T16:18:13Z | OWNER | Basic API design: db[table].convert("headline", lambda v: v.upper()) You can pass a list of columns instead of a single game column string to apply it to multiple columns. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957529248 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448623 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890448623 | IC_kwDOCGYnMM41Eyrv | 9599 | 2021-08-01T04:33:30Z | 2021-08-01T04:33:30Z | OWNER | I've started an implementation in the `convert` branch - no documentation yet, and I've not implemented the recipes. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448119 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890448119 | IC_kwDOCGYnMM41Eyj3 | 9599 | 2021-08-01T04:28:05Z | 2021-08-01T04:30:28Z | OWNER | In which case I think `--code` should be a positional argument instead: ``` sqlite-utils convert mydb.db mytable col 'recipe.parsedatetime(value, dayfirst=True)' sqlite-utils convert mydb.db mytable col 'recipe.jsonsplit(value, delimiter=":")' sqlite-utils convert mydb.db mytable col 'recipe.jsonsplit(value, delimiter=":")' sqlite-utils convert mydb.db mytable col '{"lower": value.lower(), "upper": value.upper()}' --multi ``` One problem with this: we already accept one or more columns. I think that's OK though since the code is now a required argument, so it means we have to treat everything between the table and the final code argument as a column. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890447102 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890447102 | IC_kwDOCGYnMM41EyT- | 9599 | 2021-08-01T04:20:18Z | 2021-08-01T04:29:26Z | OWNER | I could stick them in a `recipe` namespace so you do this: ``` sqlite-utils convert mydb.db mytable col --code 'recipe.parsedatetime(value, dayfirst=True)' sqlite-utils convert mydb.db mytable col --code 'recipe.jsonsplit(value, delimiter=":")' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890448190 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890448190 | IC_kwDOCGYnMM41Eyk- | 9599 | 2021-08-01T04:28:49Z | 2021-08-01T04:28:49Z | OWNER | Would make sense to accept code from standard input too: echo 'value.upper()' | sqlite-utils convert my.db mytable col - | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446808 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890446808 | IC_kwDOCGYnMM41EyPY | 9599 | 2021-08-01T04:18:18Z | 2021-08-01T04:28:18Z | OWNER | Or.... how about making the `parsedate()` and `parsedatetime()` and `jsonsplit()` functions available within the namespace that is configured for the `--code` block? Then you could do something like this: ``` sqlite-utils convert mydb.db mytable col --code 'parsedatetime(value, dayfirst=True)' sqlite-utils convert mydb.db mytable col --code 'jsonsplit(value, delimiter=":")' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446943 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890446943 | IC_kwDOCGYnMM41EyRf | 9599 | 2021-08-01T04:19:09Z | 2021-08-01T04:19:09Z | OWNER | That's a pretty neat fix, though it's a bit more challenging on the documentation front - maybe the help text for `sqlite-utils convert --help` gets a fair bit longer? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446506 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890446506 | IC_kwDOCGYnMM41EyKq | 9599 | 2021-08-01T04:16:36Z | 2021-08-01T04:16:36Z | OWNER | Back to the design board then. One way to handle this would be the long-form: ``` sqlite-utils convert jsonsplit mydb.db mytable mycolumn sqlite-utils convert parsedatetime mydb.db mytable mycolumn sqlite-utils convert parsedate mydb.db mytable mycolumn sqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()' ``` I like the idea that `lambda` is the default action, but in this form it's required that the second argument (the word after `convert`) be the name of the recipe that is being applied to avoid any potential confusion with the database filename. An ugly solution would be to make all four of those options available on `sqlite-utils convert` - and return an error if you try and use one of those without specifying the accompanying recipe. That's a bit gross though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890446166 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890446166 | IC_kwDOCGYnMM41EyFW | 9599 | 2021-08-01T04:14:26Z | 2021-08-01T04:14:26Z | OWNER | Problem with the `-r/--recipe` idea: the `parsedate` and `parsedatetime` and `jsonsplit` recipes in the current `sqlite-transform` tool all take additional options. For `sqlite-transform parsedate` and `parsedatetime`: ```python @click.option( "--dayfirst", is_flag=True, help="Assume day comes first in ambiguous dates, e.g. 03/04/05", ) @click.option( "--yearfirst", is_flag=True, help="Assume year comes first in ambiguous dates, e.g. 03/04/05", ) ``` For `jsonsplit`: ```python @click.option("--delimiter", default=",", help="Delimiter to split on") @click.option( "--type", type=click.Choice(("int", "float")), help="Type to use for values - int or float (defaults to string)", ) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-890443079 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 890443079 | IC_kwDOCGYnMM41ExVH | 9599 | 2021-08-01T03:46:43Z | 2021-08-01T03:46:43Z | OWNER | Note that there's already a concept of `conversions` which might be confused with `convert`? https://sqlite-utils.datasette.io/en/stable/python-api.html#converting-column-values-using-sql-functions ```python db["example"].insert({ "name": "The Bigfoot Discovery Museum" }, conversions={"name": "upper(?)"}) ``` I think that's OK though - that's a Python library feature, `sqlite-utils convert` is a CLI thing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/datasette/issues/1411#issuecomment-890441844 | https://api.github.com/repos/simonw/datasette/issues/1411 | 890441844 | IC_kwDOBm6k_c41ExB0 | 9599 | 2021-08-01T03:27:30Z | 2021-08-01T03:27:30Z | OWNER | Confirmed: https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 no longer exhibits the bug. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957345476 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890400425 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890400425 | IC_kwDOBm6k_c41Em6p | 9599 | 2021-07-31T20:25:16Z | 2021-07-31T20:26:25Z | OWNER | If I was prone to over-thinking (which I am) I'd note that `allow_facet` and `allow_download` and `allow_csv_stream` are all settings that do NOT have an equivalent in the newer permissions system, which is itself a little weird and inconsistent. So maybe there's a future task where I introduce those as both permissions and metadata `"allow_x"` blocks, then rename the settings themselves to be called `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`. If I was going to do that I should get it in before Datasette 1.0. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890400121 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890400121 | IC_kwDOBm6k_c41Em15 | 9599 | 2021-07-31T20:22:21Z | 2021-07-31T20:23:34Z | OWNER | I think `default_allow_sql` is more consistent with the current naming conventions, because both `allow` and `default` are used as prefixes at the moment but neither of them are ever used as a suffix. Plus `default_allow_sql off` makes sense to me but `allow_default_sql off` does not - what is "default SQL"? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890400059 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890400059 | IC_kwDOBm6k_c41Em07 | 9599 | 2021-07-31T20:21:51Z | 2021-07-31T20:21:51Z | OWNER | One of these two options: - `--setting default_allow_sql off` - `--setting allow_sql_default off` Existing settings from https://docs.datasette.io/en/0.58.1/settings.html with similar names that I need to be consistent with: - `default_page_size` - `allow_facet` - `default_facet_size` - `allow_download` - `default_cache_ttl` - `default_cache_ttl_hashed` - `allow_csv_stream` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890399806 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890399806 | IC_kwDOBm6k_c41Emw- | 9599 | 2021-07-31T20:18:46Z | 2021-07-31T20:18:46Z | OWNER | My rationale for removing it: https://github.com/simonw/datasette/issues/813#issuecomment-640916290 > Naming problem: Datasette already has a config option with this name: > > $ datasette serve data.db --config allow_sql:1 > > https://datasette.readthedocs.io/en/stable/config.html#allow-sql > > It's confusing to have two things called `allow_sql` that do slightly different things. > > I could retire the `--config allow_sql:0` option entirely, since the new `metadata.json` mechanism can be used to achieve the exact same thing. > > I'm going to do that. This is true. The `"allow_sql"` permissions block in `metadata.json` does indeed have a name that is easily confused with `--setting allow_sql off`. So I definitely need to pick a different name from the setting. `--setting default_allow_sql off` is a good option here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890397753 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890397753 | IC_kwDOBm6k_c41EmQ5 | 9599 | 2021-07-31T19:57:56Z | 2021-07-31T19:57:56Z | OWNER | I think the correct solution is for the default permissions logic to take the `allow_sql` setting into account, and to return `False` if that setting is set to `off` AND the current actor fails the `actor_matches_allow` checks. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890397652 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890397652 | IC_kwDOBm6k_c41EmPU | 9599 | 2021-07-31T19:56:48Z | 2021-07-31T19:56:48Z | OWNER | The other option would be to use the setting to pick the `default=` argument when calling `self.ds.permission_allowed( request.actor, "execute-sql", resource=database, default=True)`. The problem with that is that there are actually a few different places which perform that check, so changing all of them raises the risk of missing one in the future: https://github.com/simonw/datasette/blob/a6c8e7fa4cffdeff84e9e755dcff4788fd6154b8/datasette/views/table.py#L436-L444 https://github.com/simonw/datasette/blob/a6c8e7fa4cffdeff84e9e755dcff4788fd6154b8/datasette/views/table.py#L964-L966 https://github.com/simonw/datasette/blob/d23a2671386187f61872b9f6b58e0f80ac61f8fe/datasette/views/database.py#L220-L221 https://github.com/simonw/datasette/blob/d23a2671386187f61872b9f6b58e0f80ac61f8fe/datasette/views/database.py#L343-L345 https://github.com/simonw/datasette/blob/d23a2671386187f61872b9f6b58e0f80ac61f8fe/datasette/views/database.py#L134-L136 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890397261 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890397261 | IC_kwDOBm6k_c41EmJN | 9599 | 2021-07-31T19:52:25Z | 2021-07-31T19:52:25Z | OWNER | I think I can make this modification by teaching the default permissions code here to take the `allow_sql` setting into account: https://github.com/simonw/datasette/blob/ff253f5242e4b0b5d85d29d38b8461feb5ea997a/datasette/default_permissions.py#L38-L45 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890397169 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890397169 | IC_kwDOBm6k_c41EmHx | 9599 | 2021-07-31T19:51:35Z | 2021-07-31T19:51:35Z | OWNER | I'm going to stick with `--setting allow_sql off`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-890397124 | https://api.github.com/repos/simonw/datasette/issues/1409 | 890397124 | IC_kwDOBm6k_c41EmHE | 9599 | 2021-07-31T19:51:10Z | 2021-07-31T19:51:10Z | OWNER | I think I may like `disable_sql` better. Some options: - `--setting allow_sql off` (consistent with `allow_facet` and `allow_download` and `allow_csv_stream` - all which default to `on` already) - `--setting disable_sql on` - `--setting disable_custom_sql on` The existence of three `allow_*` settings does make a strong argument for staying consistent with that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/issues/1408#issuecomment-890390845 | https://api.github.com/repos/simonw/datasette/issues/1408 | 890390845 | IC_kwDOBm6k_c41Ekk9 | 9599 | 2021-07-31T19:00:32Z | 2021-07-31T19:00:32Z | OWNER | When I revisit this I can also look at dropping the `@pytest.mark.serial` hack, and maybe the `restore_working_directory()` fixture hack too: https://github.com/simonw/datasette/blob/ff253f5242e4b0b5d85d29d38b8461feb5ea997a/pytest.ini#L9-L10 https://github.com/simonw/datasette/blob/ff253f5242e4b0b5d85d29d38b8461feb5ea997a/tests/conftest.py#L62-L75 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957302085 | |
https://github.com/simonw/datasette/issues/1408#issuecomment-890390495 | https://api.github.com/repos/simonw/datasette/issues/1408 | 890390495 | IC_kwDOBm6k_c41Ekff | 9599 | 2021-07-31T18:57:39Z | 2021-07-31T18:57:39Z | OWNER | Opening this issue as an optional follow-up to the work I did in #1406. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957302085 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-890390342 | https://api.github.com/repos/simonw/datasette/issues/1406 | 890390342 | IC_kwDOBm6k_c41EkdG | 9599 | 2021-07-31T18:56:35Z | 2021-07-31T18:56:35Z | OWNER | But... I've lost enough time to this already, and removing `runner.isolated_filesystem()` has the tests passing again. So I'm not going to work on this any more. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-890390198 | https://api.github.com/repos/simonw/datasette/issues/1406 | 890390198 | IC_kwDOBm6k_c41Eka2 | 9599 | 2021-07-31T18:55:33Z | 2021-07-31T18:55:33Z | OWNER | To clarify: the core problem here is that an error is thrown any time you call `os.getcwd()` but the directory you are currently in has been deleted. `runner.isolated_filesystem()` assumes that the current directory in has not been deleted. But the various temporary directory utilities in `pytest` work by creating directories and then deleting them. Maybe there's a larger problem here that I play a bit fast and loose with `os.chdir()` in both the test suite and in various lines of code in Datasette itself (in particular in the publish commands)? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1407#issuecomment-890388656 | https://api.github.com/repos/simonw/datasette/issues/1407 | 890388656 | IC_kwDOBm6k_c41EkCw | 9599 | 2021-07-31T18:42:41Z | 2021-07-31T18:42:41Z | OWNER | I'll try `tempfile.gettempdir()` - on macOS it returns something like `'/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T'` which is still long but hopefully not too long. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957298475 | |
https://github.com/simonw/datasette/issues/1407#issuecomment-890388200 | https://api.github.com/repos/simonw/datasette/issues/1407 | 890388200 | IC_kwDOBm6k_c41Ej7o | 9599 | 2021-07-31T18:38:41Z | 2021-07-31T18:38:41Z | OWNER | The `path` variable there looked like this: `/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/pytest-of-simon/pytest-696/popen-gw0/uds0/datasette.sock` I think what's happening here is that `pytest-xdist` causes `tmp_path_factory.mktemp("uds")` to create significantly longer paths, which in this case is breaking some limit. So for this code to work with `pytest-xdist` I need to make sure the random path to `datasette.sock` is shorter. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957298475 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-890259755 | https://api.github.com/repos/simonw/datasette/issues/1406 | 890259755 | IC_kwDOBm6k_c41EEkr | 9599 | 2021-07-31T00:04:54Z | 2021-07-31T00:04:54Z | OWNER | STILL failing. I'm going to try removing all instances of `isolated_filesystem()` in favour of a different pattern using pytest temporary files, then see if I can get that to work without the serial hack. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-889555977 | https://api.github.com/repos/simonw/datasette/issues/1406 | 889555977 | IC_kwDOBm6k_c41BYwJ | 9599 | 2021-07-30T01:06:57Z | 2021-07-30T01:06:57Z | OWNER | Looking at the source code in Click for `isolated_filesystem()`: https://github.com/pallets/click/blob/9da166957f5848b641231d485467f6140bca2bc0/src/click/testing.py#L450-L468 ```python @contextlib.contextmanager def isolated_filesystem( self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None ) -> t.Iterator[str]: """A context manager that creates a temporary directory and changes the current working directory to it. This isolates tests that affect the contents of the CWD to prevent them from interfering with each other. :param temp_dir: Create the temporary directory under this directory. If given, the created directory is not removed when exiting. .. versionchanged:: 8.0 Added the ``temp_dir`` parameter. """ cwd = os.getcwd() t = tempfile.mkdtemp(dir=temp_dir) os.chdir(t) ``` How about if I pass in that optional `temp_dir` as a temp directory created using the `pytest-xdist` aware pytest mechanisms: https://docs.pytest.org/en/6.2.x/tmpdir.html | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-889553052 | https://api.github.com/repos/simonw/datasette/issues/1406 | 889553052 | IC_kwDOBm6k_c41BYCc | 9599 | 2021-07-30T00:58:43Z | 2021-07-30T00:58:43Z | OWNER | Tests are still failing in the job that calculates coverage. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-889550391 | https://api.github.com/repos/simonw/datasette/issues/1406 | 889550391 | IC_kwDOBm6k_c41BXY3 | 9599 | 2021-07-30T00:49:31Z | 2021-07-30T00:49:31Z | OWNER | That fixed it. My hunch is that Click's `runner.isolated_filesystem()` mechanism doesn't play well with `pytest-xdist`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-889548536 | https://api.github.com/repos/simonw/datasette/issues/1406 | 889548536 | IC_kwDOBm6k_c41BW74 | 9599 | 2021-07-30T00:43:47Z | 2021-07-30T00:43:47Z | OWNER | Still couldn't replicate on my laptop. On a hunch, I'm going to add `@pytest.mark.serial` to every test that uses `runner.isolated_filesystem()`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1406#issuecomment-889547142 | https://api.github.com/repos/simonw/datasette/issues/1406 | 889547142 | IC_kwDOBm6k_c41BWmG | 9599 | 2021-07-30T00:39:49Z | 2021-07-30T00:39:49Z | OWNER | It happens in CI but not on my laptop. I think I need to run the tests on my laptop like this: https://github.com/simonw/datasette/blob/121e10c29c5b412fddf0326939f1fe46c3ad9d4a/.github/workflows/test.yml#L27-L30 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
956303470 | |
https://github.com/simonw/datasette/issues/1241#issuecomment-889539227 | https://api.github.com/repos/simonw/datasette/issues/1241 | 889539227 | IC_kwDOBm6k_c41BUqb | 9599 | 2021-07-30T00:15:26Z | 2021-07-30T00:15:26Z | OWNER | One possible treatment: <img width="442" alt="fixtures__sortable__201_rows" src="https://user-images.githubusercontent.com/9599/127581105-dca1193c-1bdd-491e-89d1-ac3136c05c3b.png"> ```html <style> a.action-button { display: inline-block; border: 1.5px solid #666; border-radius: 0.5em; background-color: #ffffff; color: #666; padding: 0.1em 0.8em; text-decoration: none; margin-right: 0.3em; font-size: 0.85em; } </style> <p> {% if query.sql and allow_execute_sql %} <a class="action-button" title="{{ query.sql }}" href="{{ urls.database(database) }}?{{ {'sql': query.sql}|urlencode|safe }}{% if query.params %}&{{ query.params|urlencode|safe }}{% endif %}"> View and edit SQL </a> {% endif %} <a href="#" class="action-button">Copy and share link</a> </p> ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
814595021 | |
https://github.com/simonw/datasette/issues/1405#issuecomment-889525741 | https://api.github.com/repos/simonw/datasette/issues/1405 | 889525741 | IC_kwDOBm6k_c41BRXt | 9599 | 2021-07-29T23:33:30Z | 2021-07-29T23:33:30Z | OWNER | New documentation section for `datasette.utils` is here: https://docs.datasette.io/en/latest/internals.html#the-datasette-utils-module | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
955316250 | |
https://github.com/simonw/datasette/issues/1405#issuecomment-888694261 | https://api.github.com/repos/simonw/datasette/issues/1405 | 888694261 | IC_kwDOBm6k_c40-GX1 | 9599 | 2021-07-28T23:52:21Z | 2021-07-28T23:52:21Z | OWNER | Document that it can raise a `BadMetadataError` exception. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
955316250 | |
https://github.com/simonw/datasette/issues/1405#issuecomment-888694144 | https://api.github.com/repos/simonw/datasette/issues/1405 | 888694144 | IC_kwDOBm6k_c40-GWA | 9599 | 2021-07-28T23:51:59Z | 2021-07-28T23:51:59Z | OWNER | https://github.com/simonw/datasette/blob/eccfeb0871dd4bc27870faf64f80ac68e5b6bc0d/datasette/utils/__init__.py#L918-L926 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
955316250 | |
https://github.com/simonw/datasette/issues/1404#issuecomment-887095569 | https://api.github.com/repos/simonw/datasette/issues/1404 | 887095569 | IC_kwDOBm6k_c404AER | 9599 | 2021-07-26T23:27:07Z | 2021-07-26T23:27:07Z | OWNER | Updated documentation: https://github.com/simonw/datasette/blob/eccfeb0871dd4bc27870faf64f80ac68e5b6bc0d/docs/plugin_hooks.rst#register-routes-datasette | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
953352015 | |
https://github.com/simonw/datasette/issues/1402#issuecomment-886969541 | https://api.github.com/repos/simonw/datasette/issues/1402 | 886969541 | IC_kwDOBm6k_c403hTF | 9599 | 2021-07-26T19:31:40Z | 2021-07-26T19:31:40Z | OWNER | Datasette could do a pretty good job of this by default, using `twitter:card` and `og:url` tags - like on https://til.simonwillison.net/jq/extracting-objects-recursively I could also provide a mechanism to customize these - in particular to add images of some sort. It feels like something that should tie in to the metadata mechanism. | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
951185411 | |
https://github.com/simonw/datasette/issues/1402#issuecomment-886968648 | https://api.github.com/repos/simonw/datasette/issues/1402 | 886968648 | IC_kwDOBm6k_c403hFI | 9599 | 2021-07-26T19:30:14Z | 2021-07-26T19:30:14Z | OWNER | I really like this idea. I was thinking it might make a good plugin, but there's not a great mechanism for plugins to inject extra `<head>` content at the moment - plus this actually feels like a reasonable feature for Datasette core itself. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
951185411 | |
https://github.com/simonw/sqlite-utils/issues/251#issuecomment-886122696 | https://api.github.com/repos/simonw/sqlite-utils/issues/251 | 886122696 | IC_kwDOCGYnMM400SjI | 9599 | 2021-07-24T23:21:32Z | 2021-07-24T23:21:32Z | OWNER | > ``` > sqlite-utils convert jsonsplit mydb.db mytable mycolumn > sqlite-utils convert parsedatetime mydb.db mytable mycolumn > sqlite-utils convert parsedate mydb.db mytable mycolumn > sqlite-utils convert lambda mydb.db mytable mycolumn --code='str(value).upper()' > ``` This is a bit verbose - and having added `--multi` and `--output` the `lambda` command keeps getting more and more flexible compared to the others. New idea: ditch the sub-sub-commands and move the `jsonsplit` and `parsedate` recipes to be options of `convert` - maybe like this: sqlite-utils convert my.db mytable col1 --jsonsplit or: sqlite-utils convert my.db mytable col1 --recipe jsonsplit or `-r jsonsplit` for short. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
841377702 | |
https://github.com/simonw/sqlite-utils/issues/299#issuecomment-886117120 | https://api.github.com/repos/simonw/sqlite-utils/issues/299 | 886117120 | IC_kwDOCGYnMM400RMA | 9599 | 2021-07-24T22:12:01Z | 2021-07-24T22:12:01Z | OWNER | Documentation here: https://sqlite-utils.datasette.io/en/latest/cli.html#showing-the-schema | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
952154468 | |
https://github.com/simonw/datasette/issues/123#issuecomment-882138084 | https://api.github.com/repos/simonw/datasette/issues/123 | 882138084 | IC_kwDOBm6k_c40lFvk | 9599 | 2021-07-19T00:04:31Z | 2021-07-19T00:04:31Z | OWNER | I've been thinking more about this one today too. An extension of this (touched on in #417, Datasette Library) would be to support pointing Datasette at a directory and having it automatically load any CSV files it finds anywhere in that folder or its descendants - either loading them fully, or providing a UI that allows users to select a file to open it in Datasette. For larger files I think the right thing to do is import them into an on-disk SQLite database, which is limited only by available disk space. For smaller files loading them into an in-memory database should work fine. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
275125561 | |
https://github.com/simonw/sqlite-utils/issues/297#issuecomment-882052852 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | 882052852 | IC_kwDOCGYnMM40kw70 | 9599 | 2021-07-18T12:59:20Z | 2021-07-18T12:59:20Z | OWNER | I'm not too worried about `sqlite-utils memory` because if your data is large enough that you can benefit from this optimization you probably should use a real file as opposed to a disposable memory database when analyzing it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944846776 | |
https://github.com/simonw/datasette/issues/1199#issuecomment-881932880 | https://api.github.com/repos/simonw/datasette/issues/1199 | 881932880 | IC_kwDOBm6k_c40kTpQ | 9599 | 2021-07-17T17:39:17Z | 2021-07-17T17:39:17Z | OWNER | I asked about optimizing performance on the SQLite forum and this came up as a suggestion: https://sqlite.org/forum/forumpost/9a6b9ae8e2048c8b?t=c I can start by trying this: PRAGMA mmap_size=268435456; | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
792652391 | |
https://github.com/simonw/datasette/issues/1396#issuecomment-881686662 | https://api.github.com/repos/simonw/datasette/issues/1396 | 881686662 | IC_kwDOBm6k_c40jXiG | 9599 | 2021-07-16T20:02:44Z | 2021-07-16T20:02:44Z | OWNER | Confirmed fixed: 0.58.1 was successfully published to Docker Hub in https://github.com/simonw/datasette/runs/3089447346 and the `latest` tag on https://hub.docker.com/r/datasetteproject/datasette/tags was updated. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944903881 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881677620 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881677620 | IC_kwDOBm6k_c40jVU0 | 9599 | 2021-07-16T19:44:12Z | 2021-07-16T19:44:12Z | OWNER | That fixed the race condition in the `datasette-graphql` tests, which is the only place that I've been able to successfully replicate this. I'm going to land this change. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881674857 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881674857 | IC_kwDOBm6k_c40jUpp | 9599 | 2021-07-16T19:38:39Z | 2021-07-16T19:38:39Z | OWNER | I can't replicate the race condition locally with or without this patch. I'm going to push the commit and then test the CI run from `datasette-graphql` that was failing against it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881671706 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881671706 | IC_kwDOBm6k_c40jT4a | 9599 | 2021-07-16T19:32:05Z | 2021-07-16T19:32:05Z | OWNER | The test suite passes with that change. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881668759 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881668759 | IC_kwDOBm6k_c40jTKX | 9599 | 2021-07-16T19:27:46Z | 2021-07-16T19:27:46Z | OWNER | Second attempt at this: ```diff diff --git a/datasette/app.py b/datasette/app.py index 5976d8b..5f348cb 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -224,6 +224,7 @@ class Datasette: self.inspect_data = inspect_data self.immutables = set(immutables or []) self.databases = collections.OrderedDict() + self._refresh_schemas_lock = asyncio.Lock() self.crossdb = crossdb if memory or crossdb or not self.files: self.add_database(Database(self, is_memory=True), name="_memory") @@ -332,6 +333,12 @@ class Datasette: self.client = DatasetteClient(self) async def refresh_schemas(self): + if self._refresh_schemas_lock.locked(): + return + async with self._refresh_schemas_lock: + await self._refresh_schemas() + + async def _refresh_schemas(self): internal_db = self.databases["_internal"] if not self.internal_db_created: await init_internal_db(internal_db) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881665383 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881665383 | IC_kwDOBm6k_c40jSVn | 9599 | 2021-07-16T19:21:35Z | 2021-07-16T19:21:35Z | OWNER | https://stackoverflow.com/a/25799871/6083 has a good example of using `asyncio.Lock()`: ```python stuff_lock = asyncio.Lock() async def get_stuff(url): async with stuff_lock: if url in cache: return cache[url] stuff = await aiohttp.request('GET', url) cache[url] = stuff return stuff ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881664408 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881664408 | IC_kwDOBm6k_c40jSGY | 9599 | 2021-07-16T19:19:35Z | 2021-07-16T19:19:35Z | OWNER | The only place that calls `refresh_schemas()` is here: https://github.com/simonw/datasette/blob/dd5ee8e66882c94343cd3f71920878c6cfd0da41/datasette/views/base.py#L120-L124 Ideally only one call to `refresh_schemas()` would be running at any one time. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881663968 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881663968 | IC_kwDOBm6k_c40jR_g | 9599 | 2021-07-16T19:18:42Z | 2021-07-16T19:18:42Z | OWNER | The race condition happens inside this method - initially with the call to `await init_internal_db()`: https://github.com/simonw/datasette/blob/dd5ee8e66882c94343cd3f71920878c6cfd0da41/datasette/app.py#L334-L359 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881204782 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881204782 | IC_kwDOBm6k_c40hh4u | 9599 | 2021-07-16T06:14:12Z | 2021-07-16T06:14:12Z | OWNER | Here's the traceback I got from `datasette-graphql` (annoyingly only running the tests in GitHub Actions CI - I've not been able to replicate on my laptop yet): ``` tests/test_utils.py . [100%] =================================== FAILURES =================================== _________________________ test_graphql_examples[path0] _________________________ ds = <datasette.app.Datasette object at 0x7f6b8b6f8fd0> path = PosixPath('/home/runner/work/datasette-graphql/datasette-graphql/examples/filters.md') @pytest.mark.asyncio @pytest.mark.parametrize( "path", (pathlib.Path(__file__).parent.parent / "examples").glob("*.md") ) async def test_graphql_examples(ds, path): content = path.read_text() query = graphql_re.search(content)[1] try: variables = variables_re.search(content)[1] except TypeError: variables = "{}" expected = json.loads(json_re.search(content)[1]) response = await ds.client.post( "/graphql", json={ "query": query, "variables": json.loads(variables), }, ) > assert response.status_code == 200, response.json() E AssertionError: {'data': {'repos_arraycontains': None, 'users_contains': None, 'users_date': None, 'users_endswith': None, ...}, 'erro...", 'path': ['users_gt']}, {'locations': [{'column': 5, 'line': 34}], 'message': "'rows'", 'path': ['users_gte']}, ...]} E assert 500 == 200 E + where 500 = <Response [500 Internal Server Error]>.status_code tests/test_graphql.py:142: AssertionError ----------------------------- Captured stderr call ----------------------------- table databases already exists table databases already exists table databases already exists table databases already exists table databases already exists table databases already exists table databases already exists table databases already exists … | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1231#issuecomment-881204343 | https://api.github.com/repos/simonw/datasette/issues/1231 | 881204343 | IC_kwDOBm6k_c40hhx3 | 9599 | 2021-07-16T06:13:11Z | 2021-07-16T06:13:11Z | OWNER | This just broke the `datasette-graphql` test suite: https://github.com/simonw/datasette-graphql/issues/77 - I need to figure out a solution here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
811367257 | |
https://github.com/simonw/datasette/issues/1394#issuecomment-881129149 | https://api.github.com/repos/simonw/datasette/issues/1394 | 881129149 | IC_kwDOBm6k_c40hPa9 | 9599 | 2021-07-16T02:23:32Z | 2021-07-16T02:23:32Z | OWNER | Wrote about this in the annotated release notes for 0.58: https://simonwillison.net/2021/Jul/16/datasette-058/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944870799 | |
https://github.com/simonw/datasette/issues/759#issuecomment-881125124 | https://api.github.com/repos/simonw/datasette/issues/759 | 881125124 | IC_kwDOBm6k_c40hOcE | 9599 | 2021-07-16T02:11:48Z | 2021-07-16T02:11:54Z | OWNER | I added `"searchmode": "raw"` as a supported option for table metadata in #1389 and released that in Datasette 0.58. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612673948 | |
https://github.com/simonw/datasette/issues/1396#issuecomment-880967052 | https://api.github.com/repos/simonw/datasette/issues/1396 | 880967052 | MDEyOklzc3VlQ29tbWVudDg4MDk2NzA1Mg== | 9599 | 2021-07-15T19:47:25Z | 2021-07-15T19:47:25Z | OWNER | Actually I'm going to close this now and re-open it if the problem occurs again in the future. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944903881 | |
https://github.com/simonw/datasette/issues/1394#issuecomment-880900534 | https://api.github.com/repos/simonw/datasette/issues/1394 | 880900534 | MDEyOklzc3VlQ29tbWVudDg4MDkwMDUzNA== | 9599 | 2021-07-15T17:58:03Z | 2021-07-15T17:58:03Z | OWNER | Started a conversation about this on the SQLite forum: https://sqlite.org/forum/forumpost/2d76f2bcf65d256a?t=h | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944870799 | |
https://github.com/simonw/datasette/issues/1396#issuecomment-880374156 | https://api.github.com/repos/simonw/datasette/issues/1396 | 880374156 | MDEyOklzc3VlQ29tbWVudDg4MDM3NDE1Ng== | 9599 | 2021-07-15T04:03:18Z | 2021-07-15T04:03:18Z | OWNER | I fixed `datasette:latest` by running the following on my laptop: ``` docker pull datasetteproject/datasette:0.58 docker tag datasetteproject/datasette:0.58 datasetteproject/datasette:latest docker login -u datasetteproject -p ... docker push datasetteproject/datasette:latest ``` Confirmed on https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated that `datasette:latest` and `datasette:0.58` both now have the same digest of `3b5ba478040e`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944903881 | |
https://github.com/simonw/datasette/issues/1396#issuecomment-880372149 | https://api.github.com/repos/simonw/datasette/issues/1396 | 880372149 | MDEyOklzc3VlQ29tbWVudDg4MDM3MjE0OQ== | 9599 | 2021-07-15T03:56:49Z | 2021-07-15T03:56:49Z | OWNER | I'm going to leave this open until I next successfully publish a new version. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944903881 | |
https://github.com/simonw/datasette/issues/1396#issuecomment-880326049 | https://api.github.com/repos/simonw/datasette/issues/1396 | 880326049 | MDEyOklzc3VlQ29tbWVudDg4MDMyNjA0OQ== | 9599 | 2021-07-15T01:50:05Z | 2021-07-15T01:50:05Z | OWNER | I think I made a mistake in this commit: https://github.com/simonw/datasette/commit/0486303b60ce2784fd2e2ecdbecf304b7d6e6659 <img width="770" alt="Explicitly_push_version_tag__refs__1281_·_simonw_datasette_0486303" src="https://user-images.githubusercontent.com/9599/125715241-dc96d147-191a-4a69-bdf0-1cb092d97330.png"> It looks like I copied `$VERSION_TAG` from here - but it's not available in the `publish.yml` flow: https://github.com/simonw/datasette/blob/0486303b60ce2784fd2e2ecdbecf304b7d6e6659/.github/workflows/push_docker_tag.yml#L18-L25 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
944903881 |