github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/262#issuecomment-1397942113 | https://api.github.com/repos/simonw/datasette/issues/262 | 1397942113 | IC_kwDOBm6k_c5TUudh | 9599 | 2023-01-20T05:33:00Z | 2023-01-20T05:33:00Z | OWNER | I'm going to write code which parses `?_extra=` in the comma separated or multiple parameter format and then looks up functions in a dictionary. It will return an error if you ask for an extra that does not exist. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/262#issuecomment-1385807684 | https://api.github.com/repos/simonw/datasette/issues/262 | 1385807684 | IC_kwDOBm6k_c5Smb9E | 9599 | 2023-01-17T17:51:54Z | 2023-01-19T23:20:59Z | OWNER | In most cases, the `?_extra=xxx` name exactly corresponds to the additional key that is added to the JSON. `?_facet=...` is one example of a query string argument that causes an extra key - `"facet_results"` - to be added to the JSON even though it wasn't requested by name in a `?_extra=`. Am I OK with that? I think so. Related issue: - #1558 Actually there's an edge-case here that's worth considering: it's possible to use metadata to set default facets for a table. If you do this for a table, then `.json` for that table will always calculate and return those facets - which may be an expensive and unnecessary operation. So maybe we don't include `facet_results` in the JSON unless explicitly asked for in that case, but have a rule that `?_facet` implies `?_extra=facet_results`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/1989#issuecomment-1397734424 | https://api.github.com/repos/simonw/datasette/issues/1989 | 1397734424 | IC_kwDOBm6k_c5TT7wY | 9599 | 2023-01-19T23:18:45Z | 2023-01-19T23:18:45Z | OWNER | There's a `?_nocol=x` argument you can use to hide a column when you link to a table, but that won't help you if you need to hide the column for privacy reasons: https://docs.datasette.io/en/latest/json_api.html#special-table-arguments One solution right now is to define a SQL view for the things that you DO want people to be able to see, and then use Datasette's permission system to hide the tables (`'"allow": false` in metadata for each table) but show the views. If you want to redact specific columns there's a plugin for doing that: https://datasette.io/plugins/datasette-mask-columns This does make sense as more of a core Datasette feature though - tagging it as a feature suggestion. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1531991339 | |
https://github.com/simonw/datasette/pull/1996#issuecomment-1396933348 | https://api.github.com/repos/simonw/datasette/issues/1996 | 1396933348 | IC_kwDOBm6k_c5TQ4Lk | 22429695 | 2023-01-19T12:55:57Z | 2023-01-19T12:55:57Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.11**% // Head: **92.11**% // Increases project coverage by **`+0.00%`** :tada: > Coverage data is based on head [(`96b8b7f`)](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`6a352e9`)](https://codecov.io/gh/simonw/datasette/commit/6a352e99ab988dbf8fd22a100049caa6ad33f1ec?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch coverage: 100.00% of modified lines in pull request are covered. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #1996 +/- ## ======================================= Coverage 92.11% 92.11% ======================================= Files 38 38 Lines 5555 5556 +1 ======================================= + Hits 5117 5118 +1 Misses 438 438 ``` | [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/utils/\_\_init\_\_.py](https://codecov.io/gh/simonw/datasette/pull/1996?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.87% <100.00%> (+<0.01%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1538342965 | |
https://github.com/simonw/datasette/issues/1497#issuecomment-1387433455 | https://api.github.com/repos/simonw/datasette/issues/1497 | 1387433455 | IC_kwDOBm6k_c5Sso3v | 270255 | 2023-01-18T17:13:45Z | 2023-01-18T17:13:45Z | NONE | You may have just been talking to yourself here, but I found your documentation of the process incredibly useful when I hit the same error myself. Thanks! 🌈 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1034535001 | |
https://github.com/simonw/datasette/issues/262#issuecomment-1385805702 | https://api.github.com/repos/simonw/datasette/issues/262 | 1385805702 | IC_kwDOBm6k_c5SmbeG | 9599 | 2023-01-17T17:50:17Z | 2023-01-17T17:50:17Z | OWNER | Or maybe have a `permissions` extra which includes `allow_execute_sql` and `private`? Could anything else go in there? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/262#issuecomment-1384752452 | https://api.github.com/repos/simonw/datasette/issues/262 | 1384752452 | IC_kwDOBm6k_c5SiaVE | 9599 | 2023-01-17T02:14:41Z | 2023-01-17T02:15:58Z | OWNER | Thinking about `?_extra=` values just for the table JSON. The default shape will look like this: ```json { "ok": true, "rows": [{"id": 1, "name": "Name"}], "next": null, } ``` The table extras could be: - `count` - adds a `"count"` field with a full `count(*)` for that filtered table - `next_url` - the full URL to the next page - `columns` - adds `"columns": ["id", "name"]` - `expandable_columns` - a list of columns that can be expanded (note that `"expanded_columns": [...]` shows up automatically if the user passes any `?_label=` options, like on https://latest.datasette.io/fixtures/facetable.json?_label=_city_id ) - I'm tempted to rename this to `label_columns` and have it add both `label_columns` and `label_columns_selected` or similar. - `primary_keys` - a list of primary keys e.g. `["id"]` - not sure what to do about `rowid` columns here - `query` - a `{"sql": "select ...", "params": {"p0": "1"}}` object - `units` - the units feature - `suggested_facets` - suggested facets - `metadata` - a `{"metadata": {"source_url": "..."}}` etc block - differs from current in that it would be nested in `"metadata": {...}`. Stuff currently in https://latest.datasette.io/fixtures/facetable.json that is not yet covered by the above: ``` "database": "fixtures", "table": "facetable", "is_view": false, "human_description_en": "where id = 1", "private": false, "allow_execute_sql": true, "query_ms": 16.749476999393664, ``` I'm tempted to bundle `database`, `table`, `is_view` and `human_description_en` into one (not sure what to call it though, perhaps `display_details`?) - and then drop `allow_execute_sql` entirely and have `private` and `query_ms` as their own named extras. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/782#issuecomment-1368269732 | https://api.github.com/repos/simonw/datasette/issues/782 | 1368269732 | IC_kwDOBm6k_c5RjiOk | 9599 | 2022-12-31T19:32:33Z | 2023-01-17T02:05:45Z | OWNER | New thinking on the trimmed-down default. Previously I was going to use `"row"` and `"next_url"` - I now want to do this instead: ```json { "ok": true, "rows": [ { "pk1": "a", "pk2": "a", "pk3": "a", "content": "a-a-a" }, { "pk1": "a", "pk2": "a", "pk3": "b", "content": "a-a-b" } ], "next": "a,a,b" } ``` If there isn't a next page it will return `"next": null`. This is even more succinct. I'm OK with people having to request `next_url` if they don't want to construct the new URL themselves. The `"ok": true` is there so it can be `false` for errors, consistently. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/262#issuecomment-1384743243 | https://api.github.com/repos/simonw/datasette/issues/262 | 1384743243 | IC_kwDOBm6k_c5SiYFL | 9599 | 2023-01-17T02:01:26Z | 2023-01-17T02:01:26Z | OWNER | I'm tempted NOT to document the JSON for the `/.json` page, simply because I'm not at all convinced that the current homepage design is the best possible use of that space - and I'd like to reserve the opportunity to redesign that in e.g. Datasette 1.1 without it being a breaking change to the documented JSON API. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/262#issuecomment-1384742385 | https://api.github.com/repos/simonw/datasette/issues/262 | 1384742385 | IC_kwDOBm6k_c5SiX3x | 9599 | 2023-01-17T02:00:23Z | 2023-01-17T02:00:38Z | OWNER | I'm not actually too happy about how `/fixtures.json` currently entirely changes shape based on whether or not you pass a `?sql=` argument to it. Maybe I can fix that disparity with extras too? The list of tables you see on `/fixtures.json` without the `?sql=` could become another extra. The HTML version of that page could know to request that extra by default. This would also support running a SQL query but also returning a list of tables - which can be useful for building a SQL editor interface which hints at the tables that are available to the user - or even for generating the configuration needed by the CodeMirror editor's SQL completion, added in: - #1893 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/262#issuecomment-1384741055 | https://api.github.com/repos/simonw/datasette/issues/262 | 1384741055 | IC_kwDOBm6k_c5SiXi_ | 9599 | 2023-01-17T01:58:24Z | 2023-01-17T01:58:24Z | OWNER | As suggested in this issue: - #1721 There are three parts of the Datasette API that need to support extras: - Table, e.g. https://latest.datasette.io/fixtures/facetable.json - Row, e.g. https://latest.datasette.io/fixtures/facetable/1.json - Query, e.g. https://latest.datasette.io/fixtures/neighborhood_search.json or https://latest.datasette.io/fixtures.json?sql=%0Aselect+_neighborhood%2C+facet_cities.name%2C+state%0Afrom+facetable%0A++++join+facet_cities%0A++++++++on+facetable._city_id+%3D+facet_cities.id%0Awhere+_neighborhood+like+%27%25%27+||+%3Atext+||+%27%25%27%0Aorder+by+_neighborhood%3B%0A&text= There are two other pages I should consider though: - https://latest.datasette.io/.json - the JSON version of the https://latest.datasette.io/ homepage - https://latest.datasette.io/fixtures.json - note that this is different from the same URL with `?sql=...` appended to it. This is the index of tables in a specific database | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/262#issuecomment-712988146 | https://api.github.com/repos/simonw/datasette/issues/262 | 712988146 | MDEyOklzc3VlQ29tbWVudDcxMjk4ODE0Ng== | 9599 | 2020-10-20T16:32:02Z | 2023-01-17T01:54:13Z | OWNER | Just realized I added an undocumented `?_extras=` option to the row view years ago and forgot about it - it's not even documented. Added in a30c5b220c15360d575e94b0e67f3255e120b916 - https://latest.datasette.io/fixtures/attraction_characteristic/2.json?_extras=foreign_key_tables That will need to be made consistent with the new mechanism. I think `?_extra=a&_extra=b` is more consistent with other Datasette features (like `?_facet=col1&_facet=col2`) but potentially quite verbose. So I could support `?_extra=a,b,c` as an alternative allowed syntax, or I could allow `?_extra=single` and `?_extras=comma,separated`. I think I prefer allowing commas in `?_extra=`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1382655354 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31 | 1382655354 | IC_kwDOD079W85SaaV6 | 2704860 | 2023-01-14T04:08:36Z | 2023-01-14T04:08:36Z | NONE | I just tried this branch and saw some errors. I installed this PR locally with: ```bash pip install https://github.com/RhetTbull/dogsheep-photos/archive/update_for_bigsur.zip ``` <details> <summary>System Details</summary> **OS:** MacOS Monterey **Python Version:** Python 3.10.8 </details> <details> <summary>Stacktrace</summary> ```python Traceback (most recent call last): File "/Users/df/.venvs/photo-experiments/bin/dogsheep-photos", line 8, in <module> sys.exit(cli()) File "/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/Users/df/.venvs/photo-experiments/lib/python3.10/site-packages/dogsheep_photos/cli.py", line 254, in apple_photos sha256 = calculate_hash(pathlib.Path(photo.path)) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 960, in __new__ self = cls._from_parts(args) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 594, in _from_parts drv, root, parts = self._parse_args(args) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/pathlib.py", line 578, in _parse_args a = os.fspath(a) TypeError: expected str, bytes or os.PathLike object, not NoneType ``` </details> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771511344 | |
https://github.com/simonw/datasette/issues/1988#issuecomment-1379502161 | https://api.github.com/repos/simonw/datasette/issues/1988 | 1379502161 | IC_kwDOBm6k_c5SOYhR | 9599 | 2023-01-11T21:24:24Z | 2023-01-11T21:25:05Z | OWNER | Renaming `sql()` to `template_sql()` in https://github.com/simonw/datasette-template-sql and shipping a 2.0 release of that plugin (it is currently at 1.0.2) would be a sensible short-term fix too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529707837 | |
https://github.com/simonw/datasette/issues/1988#issuecomment-1379500131 | https://api.github.com/repos/simonw/datasette/issues/1988 | 1379500131 | IC_kwDOBm6k_c5SOYBj | 9599 | 2023-01-11T21:22:31Z | 2023-01-11T21:22:31Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/6a352e99ab988dbf8fd22a100049caa6ad33f1ec/datasette/app.py#L1138-L1149 https://github.com/simonw/datasette/blob/6a352e99ab988dbf8fd22a100049caa6ad33f1ec/datasette/app.py#L1193-L1195 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529707837 | |
https://github.com/simonw/datasette/issues/1988#issuecomment-1379492499 | https://api.github.com/repos/simonw/datasette/issues/1988 | 1379492499 | IC_kwDOBm6k_c5SOWKT | 9599 | 2023-01-11T21:15:01Z | 2023-01-11T21:20:14Z | OWNER | This is tricky. `datasette-template-sql` registers a custom `sql()` function, but @cldellow spotted a situation where this would interfere with an existing `{{ sql }}` variable in a template context! https://github.com/simonw/datasette-template-sql/blob/70483fd94bb0349761830e37c9ab928e8966841a/datasette_template_sql/__init__.py ```python @hookimpl def extra_template_vars(datasette): async def execute_sql(sql, args=None, database=None): db = datasette.get_database(database) return (await db.execute(sql, args)).rows return {"sql": execute_sql} ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529707837 | |
https://github.com/simonw/datasette/issues/1988#issuecomment-1379495504 | https://api.github.com/repos/simonw/datasette/issues/1988 | 1379495504 | IC_kwDOBm6k_c5SOW5Q | 9599 | 2023-01-11T21:18:00Z | 2023-01-11T21:18:25Z | OWNER | A few options: - Ensure that the explicit template context overrides anything that plugins might do - so in this case the `sql(...)` function would not be available on that page. This would break people who have custom templates that use that function though. - Encourage naming conventions where functions and variables from plugins are less likely to interfere with existing functionality. - Completely change how templates work, so you never have a variable called `{{ sql }}` - it is always accessed via some parent object instead, such as `{{ page.sql }}`. That last option actually fits quite well with my efforts to unify template rendering with JSON (and JSON extras) so it might be the best way to address this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529707837 | |
https://github.com/simonw/datasette/issues/1987#issuecomment-1379313564 | https://api.github.com/repos/simonw/datasette/issues/1987 | 1379313564 | IC_kwDOBm6k_c5SNqec | 9599 | 2023-01-11T18:29:25Z | 2023-01-11T18:29:25Z | OWNER | Fixed: - https://docs.datasette.io/en/latest/installation.html#using-pip - https://docs.datasette.io/en/stable/installation.html#using-pip - https://docs.datasette.io/en/0.64.1/installation.html#using-pip | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529452371 | |
https://github.com/simonw/datasette/issues/1987#issuecomment-1379311498 | https://api.github.com/repos/simonw/datasette/issues/1987 | 1379311498 | IC_kwDOBm6k_c5SNp-K | 9599 | 2023-01-11T18:27:30Z | 2023-01-11T18:27:46Z | OWNER | I'm going to leave the bad link in the old versions, because I can't figure out a way to update them using ReadTheDocs - shipping 0.64.1 will at least mean that both `/latest/` and `/stable/` have a good link. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529452371 | |
https://github.com/simonw/datasette/issues/1987#issuecomment-1379282756 | https://api.github.com/repos/simonw/datasette/issues/1987 | 1379282756 | IC_kwDOBm6k_c5SNi9E | 9599 | 2023-01-11T18:04:22Z | 2023-01-11T18:04:22Z | OWNER | That change is now live on https://docs.datasette.io/en/latest/installation.html#using-pip I may need to ship a new stable dot-release to update the `/en/stable/` page though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529452371 | |
https://github.com/simonw/datasette/issues/1987#issuecomment-1379276484 | https://api.github.com/repos/simonw/datasette/issues/1987 | 1379276484 | IC_kwDOBm6k_c5SNhbE | 9599 | 2023-01-11T18:00:35Z | 2023-01-11T18:00:35Z | OWNER | Next challenge: how to best deploy this. Ideally I'd like to fix all of the older versions of the docs too - pages like https://docs.datasette.io/en/0.59.4/installation.html#using-pip Not sure if ReadTheDocs has a mechanism for that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1529452371 | |
https://github.com/simonw/datasette/issues/1985#issuecomment-1378253346 | https://api.github.com/repos/simonw/datasette/issues/1985 | 1378253346 | IC_kwDOBm6k_c5SJnoi | 9599 | 2023-01-11T05:18:05Z | 2023-01-11T05:18:05Z | OWNER | `Datasette(path)` should either work or it should return an easily debugged error message. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1528448642 | |
https://github.com/simonw/datasette/pull/1982#issuecomment-1376620851 | https://api.github.com/repos/simonw/datasette/issues/1982 | 1376620851 | IC_kwDOBm6k_c5SDZEz | 49699333 | 2023-01-10T02:03:18Z | 2023-01-10T02:03:18Z | CONTRIBUTOR | Looks like sphinx is up-to-date now, so this is no longer needed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1525560504 | |
https://github.com/simonw/datasette/pull/1984#issuecomment-1376619691 | https://api.github.com/repos/simonw/datasette/issues/1984 | 1376619691 | IC_kwDOBm6k_c5SDYyr | 9599 | 2023-01-10T02:02:16Z | 2023-01-10T02:02:16Z | OWNER | Preview at https://datasette--1984.org.readthedocs.build/en/1984/ looks good to me. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1526635374 | |
https://github.com/simonw/datasette/issues/1983#issuecomment-1375810027 | https://api.github.com/repos/simonw/datasette/issues/1983 | 1375810027 | IC_kwDOBm6k_c5SATHr | 25778 | 2023-01-09T15:35:58Z | 2023-01-09T15:35:58Z | CONTRIBUTOR | Yes please, and thank you. I realized I was maybe getting myself in trouble using that, but I think it's a good way to standardize JSON handling. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1525815985 | |
https://github.com/simonw/datasette/issues/979#issuecomment-1375801928 | https://api.github.com/repos/simonw/datasette/issues/979 | 1375801928 | IC_kwDOBm6k_c5SARJI | 9599 | 2023-01-09T15:32:22Z | 2023-01-09T15:32:22Z | OWNER | This will not be part of the new default JSON design. It will be handled by an ?_extra instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
710650633 | |
https://github.com/simonw/datasette/issues/1558#issuecomment-1375800361 | https://api.github.com/repos/simonw/datasette/issues/1558 | 1375800361 | IC_kwDOBm6k_c5SAQwp | 9599 | 2023-01-09T15:31:17Z | 2023-01-09T15:31:17Z | OWNER | Related: - #782 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1082584499 | |
https://github.com/simonw/datasette/issues/1983#issuecomment-1375797856 | https://api.github.com/repos/simonw/datasette/issues/1983 | 1375797856 | IC_kwDOBm6k_c5SAQJg | 9599 | 2023-01-09T15:29:34Z | 2023-01-09T15:29:34Z | OWNER | I had forgotten this had special handling for cursors! The default representation of a row as a tuple rather than a dictionary might not be the right thing to do based on: - #782 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1525815985 | |
https://github.com/simonw/datasette/issues/1983#issuecomment-1375795441 | https://api.github.com/repos/simonw/datasette/issues/1983 | 1375795441 | IC_kwDOBm6k_c5SAPjx | 9599 | 2023-01-09T15:27:53Z | 2023-01-09T15:27:53Z | OWNER | https://github.com/simonw/datasette/blob/7b48664d751e59222c01c97a2739d53d6363bdda/datasette/utils/__init__.py#L163-L178 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1525815985 | |
https://github.com/simonw/datasette/issues/1978#issuecomment-1375708725 | https://api.github.com/repos/simonw/datasette/issues/1978 | 1375708725 | IC_kwDOBm6k_c5R_6Y1 | 25778 | 2023-01-09T14:30:00Z | 2023-01-09T14:30:00Z | CONTRIBUTOR | Totally missed that issue. I can close this as a duplicate. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1522778923 | |
https://github.com/simonw/datasette/pull/1977#issuecomment-1375596856 | https://api.github.com/repos/simonw/datasette/issues/1977 | 1375596856 | IC_kwDOBm6k_c5R_fE4 | 49699333 | 2023-01-09T13:06:14Z | 2023-01-09T13:06:14Z | CONTRIBUTOR | Superseded by #1982. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1522552817 | |
https://github.com/simonw/datasette/issues/1981#issuecomment-1375142205 | https://api.github.com/repos/simonw/datasette/issues/1981 | 1375142205 | IC_kwDOBm6k_c5R9wE9 | 9599 | 2023-01-09T06:05:44Z | 2023-01-09T06:05:44Z | OWNER | I want the labels to take as much space as they need, but line up with each other. If they are more than half the width of the page then the form fields should break to another line. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524983536 | |
https://github.com/simonw/datasette/issues/1980#issuecomment-1375062405 | https://api.github.com/repos/simonw/datasette/issues/1980 | 1375062405 | IC_kwDOBm6k_c5R9cmF | 9599 | 2023-01-09T03:23:53Z | 2023-01-09T03:23:53Z | OWNER | Some potential solutions: - Make it so the primary key is _always_ sortable - not a bad idea, the `sortable_columns` thing was designed to protect against sorting GIANT tables by a non-indexed column, and the primary key always has an index - Make sure that `<select>` menu never includes keys that cannot be sorted - this would be a good fix anyway. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524867951 | |
https://github.com/simonw/datasette/issues/1980#issuecomment-1375061747 | https://api.github.com/repos/simonw/datasette/issues/1980 | 1375061747 | IC_kwDOBm6k_c5R9cbz | 9599 | 2023-01-09T03:22:36Z | 2023-01-09T03:22:36Z | OWNER | The problem here is that the HTML ended up with this HTML in it: ```html <select name="_sort" id="sort_by"> <option value="">Sort...</option> <option value="id" selected>Sort by id</option> ``` But `id` is not a valid sort option any more. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524867951 | |
https://github.com/simonw/datasette/issues/1978#issuecomment-1374916593 | https://api.github.com/repos/simonw/datasette/issues/1978 | 1374916593 | IC_kwDOBm6k_c5R84_x | 9599 | 2023-01-08T20:06:16Z | 2023-01-08T20:06:16Z | OWNER | Related issue - I'm not 100% settled on the design for these yet: - https://github.com/simonw/datasette/issues/1048#issuecomment-1179757912 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1522778923 | |
https://github.com/simonw/datasette/issues/1979#issuecomment-1374659874 | https://api.github.com/repos/simonw/datasette/issues/1979 | 1374659874 | IC_kwDOBm6k_c5R76Ui | 9599 | 2023-01-07T23:59:11Z | 2023-01-07T23:59:11Z | OWNER | I back-ported this fix to `0.63.x` as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524076587 | |
https://github.com/simonw/datasette/issues/1979#issuecomment-1374659233 | https://api.github.com/repos/simonw/datasette/issues/1979 | 1374659233 | IC_kwDOBm6k_c5R76Kh | 9599 | 2023-01-07T23:56:25Z | 2023-01-07T23:56:25Z | OWNER | ``` % datasette --load-extension foo Error: Your Python installation does not have the ability to load SQLite extensions. More information: https://datasette.io/help/extensions ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524076587 | |
https://github.com/simonw/datasette/issues/1979#issuecomment-1374658468 | https://api.github.com/repos/simonw/datasette/issues/1979 | 1374658468 | IC_kwDOBm6k_c5R75-k | 9599 | 2023-01-07T23:53:41Z | 2023-01-07T23:53:41Z | OWNER | https://datasette.io/help/extensions now redirects to this new section of documentation: https://docs.datasette.io/en/latest/installation.html#a-note-about-extensions | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524076587 | |
https://github.com/simonw/datasette/issues/1979#issuecomment-1374657204 | https://api.github.com/repos/simonw/datasette/issues/1979 | 1374657204 | IC_kwDOBm6k_c5R75q0 | 9599 | 2023-01-07T23:48:42Z | 2023-01-07T23:48:42Z | OWNER | Once I ship the next release I should change that `/help/extensions` link to go to `/stable/` and not `/latest/`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524076587 | |
https://github.com/simonw/datasette/issues/1979#issuecomment-1374582375 | https://api.github.com/repos/simonw/datasette/issues/1979 | 1374582375 | IC_kwDOBm6k_c5R7nZn | 9599 | 2023-01-07T19:22:39Z | 2023-01-07T19:22:39Z | OWNER | This helps: ```diff diff --git a/datasette/cli.py b/datasette/cli.py index 2b61292b..ea98879c 100644 --- a/datasette/cli.py +++ b/datasette/cli.py @@ -4,13 +4,15 @@ import click from click import formatting from click.types import CompositeParamType from click_default_group import DefaultGroup +import functools import json import os import pathlib import shutil from subprocess import call -import sys from runpy import run_module +import sys +import textwrap import webbrowser from .app import ( OBSOLETE_SETTINGS, @@ -126,7 +128,7 @@ class Setting(CompositeParamType): def sqlite_extensions(fn): - return click.option( + fn = click.option( "sqlite_extensions", "--load-extension", type=LoadExtension(), @@ -134,6 +136,25 @@ def sqlite_extensions(fn): multiple=True, help="Path to a SQLite extension to load, and optional entrypoint", )(fn) + # Wrap it in a custom error handler + @functools.wraps(fn) + def wrapped(*args, **kwargs): + try: + return fn(*args, **kwargs) + except AttributeError as e: + if "enable_load_extension" in str(e): + raise click.ClickException( + textwrap.dedent( + """ + Your Python installation does not have the ability to load SQLite extensions. + + More information: https://docs.datasette.io/en/stable/installation.html#extensions + """ + ).strip() + ) + raise + + return wrapped @click.group(cls=DefaultGroup, default="serve", default_if_no_args=True) ``` Need to write help for that to link to. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1524076587 | |
https://github.com/simonw/datasette/issues/1409#issuecomment-1374555507 | https://api.github.com/repos/simonw/datasette/issues/1409 | 1374555507 | IC_kwDOBm6k_c5R7g1z | 9599 | 2023-01-07T18:06:01Z | 2023-01-07T18:06:01Z | OWNER | Documentation: - https://docs.datasette.io/en/latest/authentication.html#controlling-the-ability-to-execute-arbitrary-sql - https://docs.datasette.io/en/latest/settings.html#setting-default-allow-sql | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
957310278 | |
https://github.com/simonw/datasette/pull/1977#issuecomment-1373600402 | https://api.github.com/repos/simonw/datasette/issues/1977 | 1373600402 | IC_kwDOBm6k_c5R33qS | 22429695 | 2023-01-06T13:09:51Z | 2023-01-06T13:09:51Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1977?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.15**% // Head: **92.15**% // No change to project coverage :thumbsup: > Coverage data is based on head [(`a4eebee`)](https://codecov.io/gh/simonw/datasette/pull/1977?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`c41278b`)](https://codecov.io/gh/simonw/datasette/commit/c41278b46f7936b4b1a8a14bf285bed82c81c609?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch has no changes to coverable lines. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #1977 +/- ## ======================================= Coverage 92.15% 92.15% ======================================= Files 38 38 Lines 5542 5542 ======================================= Hits 5107 5107 Misses 435 435 ``` Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1977?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1522552817 | |
https://github.com/simonw/datasette/pull/1976#issuecomment-1373592231 | https://api.github.com/repos/simonw/datasette/issues/1976 | 1373592231 | IC_kwDOBm6k_c5R31qn | 49699333 | 2023-01-06T13:02:15Z | 2023-01-06T13:02:15Z | CONTRIBUTOR | Superseded by #1977. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1520712722 | |
https://github.com/simonw/datasette/pull/1976#issuecomment-1372203113 | https://api.github.com/repos/simonw/datasette/issues/1976 | 1372203113 | IC_kwDOBm6k_c5Ryihp | 22429695 | 2023-01-05T13:15:04Z | 2023-01-05T13:15:04Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1976?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.15**% // Head: **92.15**% // No change to project coverage :thumbsup: > Coverage data is based on head [(`af76abc`)](https://codecov.io/gh/simonw/datasette/pull/1976?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`c41278b`)](https://codecov.io/gh/simonw/datasette/commit/c41278b46f7936b4b1a8a14bf285bed82c81c609?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch has no changes to coverable lines. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #1976 +/- ## ======================================= Coverage 92.15% 92.15% ======================================= Files 38 38 Lines 5542 5542 ======================================= Hits 5107 5107 Misses 435 435 ``` Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1976?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1520712722 | |
https://github.com/simonw/datasette/pull/1974#issuecomment-1372188571 | https://api.github.com/repos/simonw/datasette/issues/1974 | 1372188571 | IC_kwDOBm6k_c5Rye-b | 49699333 | 2023-01-05T13:02:40Z | 2023-01-05T13:02:40Z | CONTRIBUTOR | Superseded by #1976. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1516376583 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-1370786026 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54 | 1370786026 | IC_kwDODEm0Qs5RtIjq | 6764957 | 2023-01-04T11:06:44Z | 2023-01-04T11:06:44Z | NONE | as of 2023 it appears that `tweets: not yet implemented` is happening.. pretty important for a twitter export functionality! this seems an impossible task with twitter liable to switch this around every other day | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779088071 | |
https://github.com/simonw/sqlite-utils/issues/520#issuecomment-1369333759 | https://api.github.com/repos/simonw/sqlite-utils/issues/520 | 1369333759 | IC_kwDOCGYnMM5Rnl__ | 9599 | 2023-01-03T02:23:43Z | 2023-01-03T02:23:43Z | OWNER | The documentation here does at least say the following: https://sqlite-utils.datasette.io/en/3.30/python-api.html#reading-rows-from-a-file > - **fp** (*BinaryIO*) -- a file-like object containing binary data | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1516644980 | |
https://github.com/simonw/datasette/issues/1973#issuecomment-1369044959 | https://api.github.com/repos/simonw/datasette/issues/1973 | 1369044959 | IC_kwDOBm6k_c5Rmfff | 193185 | 2023-01-02T15:41:40Z | 2023-01-02T15:41:40Z | CONTRIBUTOR | Thanks for the response! Yes, it does seem like a pretty nice developer experience--both the automagical labelling of fkeys, and the ability to index the row by column name in addition to column index. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515815014 | |
https://github.com/simonw/datasette/issues/1973#issuecomment-1368936146 | https://api.github.com/repos/simonw/datasette/issues/1973 | 1368936146 | IC_kwDOBm6k_c5RmE7S | 9599 | 2023-01-02T13:13:03Z | 2023-01-02T13:13:03Z | OWNER | It's quite handy that the label for foreign keys is made available to the custom render function here - if that was properly documented (and maybe had a slightly more intuitive API) I think it might be a good solution here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515815014 | |
https://github.com/simonw/datasette/pull/1974#issuecomment-1368935738 | https://api.github.com/repos/simonw/datasette/issues/1974 | 1368935738 | IC_kwDOBm6k_c5RmE06 | 22429695 | 2023-01-02T13:12:30Z | 2023-01-02T13:12:30Z | NONE | # [Codecov](https://codecov.io/gh/simonw/datasette/pull/1974?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Base: **92.10**% // Head: **92.10**% // No change to project coverage :thumbsup: > Coverage data is based on head [(`1e213a3`)](https://codecov.io/gh/simonw/datasette/pull/1974?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) compared to base [(`572bdb5`)](https://codecov.io/gh/simonw/datasette/commit/572bdb5b8095bbdc5ec2aa95400055e49ea028cb?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). > Patch has no changes to coverable lines. <details><summary>Additional details and impacted files</summary> ```diff @@ Coverage Diff @@ ## main #1974 +/- ## ======================================= Coverage 92.10% 92.10% ======================================= Files 38 38 Lines 5538 5538 ======================================= Hits 5101 5101 Misses 437 437 ``` Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) </details> [:umbrella: View full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1974?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1516376583 | |
https://github.com/simonw/datasette/issues/1973#issuecomment-1368935320 | https://api.github.com/repos/simonw/datasette/issues/1973 | 1368935320 | IC_kwDOBm6k_c5RmEuY | 9599 | 2023-01-02T13:12:00Z | 2023-01-02T13:12:00Z | OWNER | Thanks for catching this! Definitely a bug. This might be a good opportunity to reconsider how his should work though. It's possible that Datasette will eventually support alternative database backends - PostgreSQL, DuckDB and ClickHouse are all on my radar here. If that happens, this plugin hook returning a SQLite Row wouldn't make sense. Some kind of thing abstraction layer - maybe CustomRow, maybe something a little bit better - might be a good idea. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515815014 | |
https://github.com/simonw/datasette/issues/782#issuecomment-1368285442 | https://api.github.com/repos/simonw/datasette/issues/782 | 1368285442 | IC_kwDOBm6k_c5RjmEC | 9599 | 2022-12-31T22:02:16Z | 2022-12-31T22:02:16Z | OWNER | https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_size=2 now returns `count`: ```json { "database": "fixtures", "table": "compound_three_primary_keys", "is_view": false, "human_description_en": "", "rows": [ { "pk1": "a", "pk2": "a", "pk3": "a", "content": "a-a-a" }, { "pk1": "a", "pk2": "a", "pk3": "b", "content": "a-a-b" } ], "truncated": false, "count": 1001, ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-1368278278 | https://api.github.com/repos/simonw/datasette/issues/782 | 1368278278 | IC_kwDOBm6k_c5RjkUG | 9599 | 2022-12-31T20:49:38Z | 2022-12-31T20:49:38Z | OWNER | I'm going to rename `filtered_table_rows_count` to `count` - to match the SQL `count(*)` function. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-1368269811 | https://api.github.com/repos/simonw/datasette/issues/782 | 1368269811 | IC_kwDOBm6k_c5RjiPz | 9599 | 2022-12-31T19:33:09Z | 2022-12-31T19:33:09Z | OWNER | Here's the so-far updated documentation for this change: https://github.com/simonw/datasette/blob/a2dca62360ad4a961d4c46f68eae41b7d5c7b2c9/docs/json_api.rst#different-shapes | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-1368269283 | https://api.github.com/repos/simonw/datasette/issues/782 | 1368269283 | IC_kwDOBm6k_c5RjiHj | 9599 | 2022-12-31T19:29:45Z | 2022-12-31T19:29:45Z | OWNER | https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_size=2 now shows the new default: ```json { "database": "fixtures", "table": "compound_three_primary_keys", "is_view": false, "human_description_en": "", "rows": [ { "pk1": "a", "pk2": "a", "pk3": "a", "content": "a-a-a" }, { "pk1": "a", "pk2": "a", "pk3": "b", "content": "a-a-b" } ], ``` The old format can be had like this: https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_size=2&_shape=arrays | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/1970#issuecomment-1368268233 | https://api.github.com/repos/simonw/datasette/issues/1970 | 1368268233 | IC_kwDOBm6k_c5Rjh3J | 9599 | 2022-12-31T19:22:58Z | 2022-12-31T19:22:58Z | OWNER | https://latest.datasette.io/_internal/databases shows the fix now: <img width="542" alt="image" src="https://user-images.githubusercontent.com/9599/210153822-11a94a8e-9049-42e9-89b5-c709cde63467.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515182998 | |
https://github.com/simonw/datasette/issues/782#issuecomment-1368268148 | https://api.github.com/repos/simonw/datasette/issues/782 | 1368268148 | IC_kwDOBm6k_c5Rjh10 | 9599 | 2022-12-31T19:22:07Z | 2022-12-31T19:22:07Z | OWNER | It turned out the most significant part of this change - switching from an array of arrays to an array of objects for the `"rows"` key - was really easy: Datasette already had a `?_shape=arrays` v.s. `?_shape=objects` mechanism, so I switched which one was the default in https://github.com/simonw/datasette/commit/234230e59574ccb8d8a24c45ccd325f725812377 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/pull/1967#issuecomment-1368267484 | https://api.github.com/repos/simonw/datasette/issues/1967 | 1368267484 | IC_kwDOBm6k_c5Rjhrc | 9599 | 2022-12-31T19:15:50Z | 2022-12-31T19:15:50Z | OWNER | My Firefox tab before: <img width="169" alt="image" src="https://user-images.githubusercontent.com/9599/210153361-42241d3b-7681-42ed-a491-c0d513ef41fe.png"> And after: <img width="169" alt="image" src="https://user-images.githubusercontent.com/9599/210153655-f74ef94f-7223-468b-9ca0-cf944393d993.png"> | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
1503010009 | |
https://github.com/simonw/datasette/issues/1966#issuecomment-1368267406 | https://api.github.com/repos/simonw/datasette/issues/1966 | 1368267406 | IC_kwDOBm6k_c5RjhqO | 9599 | 2022-12-31T19:15:10Z | 2022-12-31T19:15:10Z | OWNER | It's working again now - not sure what happened there, I was paying for Heroku for that already. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1501900064 | |
https://github.com/simonw/datasette/issues/1971#issuecomment-1368266904 | https://api.github.com/repos/simonw/datasette/issues/1971 | 1368266904 | IC_kwDOBm6k_c5RjhiY | 9599 | 2022-12-31T19:09:48Z | 2022-12-31T19:09:48Z | OWNER | For the moment I'm going to pin to Sphinx 5.3.0 Note that I got this warning there: ``` [sphinx-autobuild] > sphinx-build -b html /Users/simon/Dropbox/Development/datasette/docs /Users/simon/Dropbox/Development/datasette/docs/_build Running Sphinx v5.3.0 loading pickled environment... done WARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'. ``` So I should probably fix that in advance of the 6.0 upgrade. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515185383 | |
https://github.com/simonw/datasette/issues/1971#issuecomment-1368266658 | https://api.github.com/repos/simonw/datasette/issues/1971 | 1368266658 | IC_kwDOBm6k_c5Rjhei | 9599 | 2022-12-31T19:07:19Z | 2022-12-31T19:07:19Z | OWNER | So it looks like Furo doesn't work with Sphinx 6.0 yet - there' s a fix on `main` but it hasn't been released: - https://github.com/pradyunsg/furo/discussions/569 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515185383 | |
https://github.com/simonw/datasette/issues/1971#issuecomment-1368266457 | https://api.github.com/repos/simonw/datasette/issues/1971 | 1368266457 | IC_kwDOBm6k_c5RjhbZ | 9599 | 2022-12-31T19:05:26Z | 2022-12-31T19:05:26Z | OWNER | Ran this locally: pip install -U Sphinx Got these warnings: ``` Installing collected packages: Sphinx Attempting uninstall: Sphinx Found existing installation: Sphinx 5.1.1 Uninstalling Sphinx-5.1.1: Successfully uninstalled Sphinx-5.1.1 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. sphinx-basic-ng 0.0.1a12 requires sphinx<6.0,>=4.0, but you have sphinx 6.0.0 which is incompatible. furo 2022.6.21 requires sphinx<6.0,>=4.0, but you have sphinx 6.0.0 which is incompatible. ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515185383 | |
https://github.com/simonw/datasette/pull/1967#issuecomment-1368265354 | https://api.github.com/repos/simonw/datasette/issues/1967 | 1368265354 | IC_kwDOBm6k_c5RjhKK | 9599 | 2022-12-31T18:56:35Z | 2022-12-31T18:56:35Z | OWNER | Thanks for this! Datasette has a favicon in this file: https://github.com/simonw/datasette/blob/2b4a8aa0833068da25072b6247bf349949ca00ee/datasette/static/favicon.png - so I'll copy that over and use it here as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1503010009 | |
https://github.com/simonw/datasette/issues/1970#issuecomment-1368264882 | https://api.github.com/repos/simonw/datasette/issues/1970 | 1368264882 | IC_kwDOBm6k_c5RjhCy | 9599 | 2022-12-31T18:52:01Z | 2022-12-31T18:52:01Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/e03aed00026cc2e59c09ca41f69a247e1a85cc89/datasette/app.py#L418-L424 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1515182998 | |
https://github.com/simonw/datasette/issues/1614#issuecomment-1364345119 | https://api.github.com/repos/simonw/datasette/issues/1614 | 1364345119 | IC_kwDOBm6k_c5RUkEf | 536941 | 2022-12-23T21:27:10Z | 2022-12-23T21:27:10Z | CONTRIBUTOR | is this issue closed by #1893? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1115435536 | |
https://github.com/simonw/datasette/issues/1796#issuecomment-1364345071 | https://api.github.com/repos/simonw/datasette/issues/1796 | 1364345071 | IC_kwDOBm6k_c5RUkDv | 536941 | 2022-12-23T21:27:02Z | 2022-12-23T21:27:02Z | CONTRIBUTOR | @simonw is this issue closed by #1893? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1355148385 | |
https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1364141224 | https://api.github.com/repos/simonw/sqlite-utils/issues/488 | 1364141224 | IC_kwDOCGYnMM5RTySo | 1176293 | 2022-12-23T17:38:55Z | 2022-12-23T17:38:55Z | NONE | > text columns containing empty strings should not be rewritten to null. I would actually appreciate an option to do just that for text columns as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1373224657 | |
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-1359468823 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46 | 1359468823 | IC_kwDODFdgUs5RB9kX | 1839645 | 2022-12-20T14:39:39Z | 2022-12-20T14:40:15Z | NONE | Just a quick +1 to this one from me - I would like to do a better job of tracking who is reviewing one another's pull requests in repositories, since this is a specific kind of maintenance work that I think often goes unrewarded. I can't seem to figure this out just by looking at the `pull_request` or `issue_comments` tables, so I think it would be helpful to support PR reviews natively (even if just for summary statistics). Alternatively if there is a way in the API to tell if an issue comment is part of a review, then perhaps you could quickly calculate the number of unique reviews that an author performed. But that was beyond my SQL-foo :-) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
664485022 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1357084279 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1357084279 | IC_kwDOBm6k_c5Q43Z3 | 178162 | 2022-12-19T04:34:16Z | 2022-12-19T04:34:16Z | NONE | You were super-close on the python version of the test here, changing `http` to `https` on 8b73fc6b47dffd8836f5c58aae1e57c1f66a5754 is enough to pass the test: ```diff diff --git a/tests/conftest.py b/tests/conftest.py index 69dee68b4a3f..ba07a11d37f6 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -207,7 +207,7 @@ def ds_localhost_https_server(tmp_path_factory): stderr=subprocess.STDOUT, cwd=tempfile.gettempdir(), ) - wait_until_responds("http://localhost:8042/", verify=client_cert) + wait_until_responds("https://localhost:8042/", verify=client_cert) # Check it started successfully assert not ds_proc.poll(), ds_proc.stdout.read().decode("utf-8") yield ds_proc, client_cert ``` My speculation about what was happening here: when `wait_until_responds()` would time out due to SSL connection problems, because `.terminate()` isn’t in a `finally`, the datasette process wouldn’t get killed. That could (1) hang CI and (2) cause all your future local test runs to mysteriously fail because they’d be secretly talking to that old datasette process still hanging around from a past test run with an old temporary server certificate, and that old server cert wouldn’t validate against your newly-created ca cert. A `finally` for `.terminate()` would help; a fancier version could be a context manager for running the external `datasette` process that could: - ensure the process always exited when no longer needed - if you want to be fancy, call `terminate()`, `wait()` for a short timeout for the process to exit, then try `kill()` and `wait()` again; raise an exception complaining about the seemingly-unkillable process if all that fails - raise an error if the process exited with a non-zero error code; here it’s likely that some `datasette` executions were secretly failing with `[Errno 48] error while attempting to bind on address ('127.0.0.1', 8042): address already in use` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1886#issuecomment-1356842576 | https://api.github.com/repos/simonw/datasette/issues/1886 | 1356842576 | IC_kwDOBm6k_c5Q38ZQ | 18738650 | 2022-12-18T17:34:20Z | 2022-12-18T17:34:20Z | NONE | A bit late to this, but I have made an app to publish air quality data in Bristol, UK. [air quality data in Bristol, UK.](https://brisaq-wfzqhmj43q-ew.a.run.app/) Next step to see if I can make a streamlit app based on this to produce some nice charts. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1447050738 | |
https://github.com/simonw/datasette/pull/1965#issuecomment-1356827218 | https://api.github.com/repos/simonw/datasette/issues/1965 | 1356827218 | IC_kwDOBm6k_c5Q34pS | 9599 | 2022-12-18T16:01:36Z | 2022-12-18T16:01:36Z | OWNER | Will link to this from my TIL shortly. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1501843596 | |
https://github.com/simonw/datasette/pull/1965#issuecomment-1356827167 | https://api.github.com/repos/simonw/datasette/issues/1965 | 1356827167 | IC_kwDOBm6k_c5Q34of | 9599 | 2022-12-18T16:01:22Z | 2022-12-18T16:01:22Z | OWNER | This is great, thank you! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1501843596 | |
https://github.com/simonw/datasette/issues/1964#issuecomment-1356697705 | https://api.github.com/repos/simonw/datasette/issues/1964 | 1356697705 | IC_kwDOBm6k_c5Q3ZBp | 9599 | 2022-12-18T06:37:23Z | 2022-12-18T06:37:23Z | OWNER | I'm certain the two other cog menus (the app menu on the right of the nav bar and the column action menus) have the same problem. Would be great to figure out the right ARIA attributes for these too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1501778647 | |
https://github.com/simonw/datasette/issues/1771#issuecomment-1356694671 | https://api.github.com/repos/simonw/datasette/issues/1771 | 1356694671 | IC_kwDOBm6k_c5Q3YSP | 9599 | 2022-12-18T06:34:20Z | 2022-12-18T06:34:20Z | OWNER | Now live on https://latest.datasette.io/fixtures/attraction_characteristic | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1306984363 | |
https://github.com/simonw/datasette/issues/1771#issuecomment-1356680769 | https://api.github.com/repos/simonw/datasette/issues/1771 | 1356680769 | IC_kwDOBm6k_c5Q3U5B | 9599 | 2022-12-18T05:56:05Z | 2022-12-18T05:56:05Z | OWNER | This does the trick: ```css div.select-wrapper:focus-within { border: 1px solid black; } ``` ![tab-select-border-fix](https://user-images.githubusercontent.com/9599/208283826-de48212f-a213-40fc-9b37-9d66f0858f21.gif) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1306984363 | |
https://github.com/simonw/datasette/issues/1771#issuecomment-1356657451 | https://api.github.com/repos/simonw/datasette/issues/1771 | 1356657451 | IC_kwDOBm6k_c5Q3PMr | 1473102 | 2022-12-18T04:04:32Z | 2022-12-18T04:04:32Z | NONE | the problem is: ``` .select-wrapper select:focus { outline: none; } ``` I sometimes add this js: ``` window.addEventListener('keydown', function check_tab(e) { if (e.key === 'Tab') { document.documentElement.classList.add('user-is-tabbing') window.removeEventListener('keydown', check_tab) } }) ``` and then in the css, using a `html.user-is-tabbing` selector undo any outlines I removed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1306984363 | |
https://github.com/simonw/datasette/issues/1771#issuecomment-1356655630 | https://api.github.com/repos/simonw/datasette/issues/1771 | 1356655630 | IC_kwDOBm6k_c5Q3OwO | 9599 | 2022-12-18T03:43:12Z | 2022-12-18T03:43:12Z | OWNER | The border is actually on the div that wraps the select box: <img width="1171" alt="image" src="https://user-images.githubusercontent.com/9599/208280249-9d9848b5-6e33-46c3-ad99-bc1948e08f25.png"> I tried adding a `border: 1px dotted black` to `select:focus` but it's not quite right - it jumps around a bit like this: ![Tabbing to the selects shows a 1px border but the element expands in size by one pixel, causing a visual jump](https://user-images.githubusercontent.com/9599/208280271-41a07f68-b8b1-4908-a4e2-aac4304d6c09.gif) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1306984363 | |
https://github.com/simonw/datasette/issues/1771#issuecomment-1356655217 | https://api.github.com/repos/simonw/datasette/issues/1771 | 1356655217 | IC_kwDOBm6k_c5Q3Opx | 9599 | 2022-12-18T03:38:16Z | 2022-12-18T03:38:16Z | OWNER | OK I see what you mean: https://latest.datasette.io/fixtures/attraction_characteristic ![Animated GIF of the table page hitting tab a bunch - the cog icon highlights and so does the text input but the two select boxes do not](https://user-images.githubusercontent.com/9599/208280176-1e2de671-fe69-43e8-8d62-bf7aa8f4d36e.gif) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1306984363 | |
https://github.com/simonw/datasette/issues/1963#issuecomment-1356652057 | https://api.github.com/repos/simonw/datasette/issues/1963 | 1356652057 | IC_kwDOBm6k_c5Q3N4Z | 9599 | 2022-12-18T03:23:22Z | 2022-12-18T03:23:22Z | OWNER | https://pypi.org/project/datasette/0.63.3/ is released. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1501713288 | |
https://github.com/simonw/datasette/issues/1963#issuecomment-1356651943 | https://api.github.com/repos/simonw/datasette/issues/1963 | 1356651943 | IC_kwDOBm6k_c5Q3N2n | 9599 | 2022-12-18T03:23:03Z | 2022-12-18T03:23:03Z | OWNER | Oh that's annoying... every step in publish succeeded except the static docs one: <img width="1305" alt="image" src="https://user-images.githubusercontent.com/9599/208279782-8582c130-820c-48c5-a4ca-a77438570b02.png"> https://github.com/simonw/datasette/actions/runs/3723015082/jobs/6314292722 This means the documentation database used to update the search engine on https://datasette.io/ won't reflect the very latest changelog. I'm OK with that - I'll fix this workflow so that next time I publish a release this will work correctly. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1501713288 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356640463 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356640463 | IC_kwDOBm6k_c5Q3LDP | 9599 | 2022-12-18T02:45:18Z | 2022-12-18T02:45:18Z | OWNER | ... and with this change, the following now works correctly: ``` % datasette install datasette-gunicorn % datasette gunicorn fixtures.db -p 8855 [2022-12-17 18:44:29 -0800] [7651] [INFO] Starting gunicorn 20.1.0 [2022-12-17 18:44:29 -0800] [7651] [INFO] Listening at: http://127.0.0.1:8855 (7651) [2022-12-17 18:44:29 -0800] [7651] [INFO] Using worker: uvicorn.workers.UvicornWorker [2022-12-17 18:44:29 -0800] [7653] [INFO] Booting worker with pid: 7653 [2022-12-17 18:44:29 -0800] [7653] [INFO] Started server process [7653] [2022-12-17 18:44:29 -0800] [7653] [INFO] Waiting for application startup. [2022-12-17 18:44:29 -0800] [7653] [INFO] Application startup complete. ``` So this issue is now fixed! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356640266 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356640266 | IC_kwDOBm6k_c5Q3LAK | 9599 | 2022-12-18T02:43:00Z | 2022-12-18T02:43:00Z | OWNER | https://github.com/simonw/datasette/actions/runs/3722908296/jobs/6314093163 shows that new test passing in CI: ``` Generated a certificate for 'localhost', '127.0.0.1', '::1' Configure your server to use the following files: cert=/home/runner/work/datasette/datasette/server.pem key=/home/runner/work/datasette/datasette/server.key Configure your client to use the following files: cert=/home/runner/work/datasette/datasette/client.pem INFO: Started server process [4036] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on https://127.0.0.1:8152/ (Press CTRL+C to quit) % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed INFO: 127.0.0.1:56726 - "GET /_memory.json HTTP/1.1" 200 OK 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 213 0 213 0 0 11542 0 --:--:-- --:--:-- --:--:-- 11833 INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [4036] {"database": "_memory", "private": false, "path": "/_memory", "size": 0, "tables": [], "hidden_count": 0, "views": [], "queries": [], "allow_execute_sql": true, "table_columns": {}, "query_ms": 1.4545189999921604}0 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1221#issuecomment-1356639873 | https://api.github.com/repos/simonw/datasette/issues/1221 | 1356639873 | IC_kwDOBm6k_c5Q3K6B | 9599 | 2022-12-18T02:39:04Z | 2022-12-18T02:39:04Z | OWNER | I ended up moving this test out of Python and into a `bash` script here: https://github.com/simonw/datasette/commit/d1d369456a7319b9de39175605568cbc9b852478 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
806849424 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356633937 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356633937 | IC_kwDOBm6k_c5Q3JdR | 9599 | 2022-12-18T02:28:22Z | 2022-12-18T02:28:22Z | OWNER | This `bash` script does the job: ```bash #!/bin/bash # Generate certificates python -m trustme # This creates server.pem, server.key, client.pem # Start the server in the background datasette --memory \ --ssl-keyfile=server.key \ --ssl-certfile=server.pem \ -p 8152 & # Store the background process ID in a variable server_pid=$! # Wait for the server to start sleep 2 # Make a test request using curl curl -f --cacert client.pem 'https://localhost:8152/_memory.json' # Save curl's exit code (-f option causes it to return one on HTTP errors) curl_exit_code=$? # Shut down the server kill $server_pid sleep 1 # Clean up the certificates rm server.pem server.key client.pem echo $curl_exit_code exit $curl_exit_code ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356630092 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356630092 | IC_kwDOBm6k_c5Q3IhM | 9599 | 2022-12-18T02:20:01Z | 2022-12-18T02:20:01Z | OWNER | Asked ChatGPT: > Write a bash script which starts a server in the background using "datasette -p 8002", then uses curl to make a test request against it, then shuts the server down again at the end It gave me: ```bash #!/bin/bash # Start the server in the background datasette -p 8002 & # Store the background process ID in a variable server_pid=$! # Make a test request using curl curl http://localhost:8002 # Shut down the server kill $server_pid ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356629783 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356629783 | IC_kwDOBm6k_c5Q3IcX | 9599 | 2022-12-18T02:18:43Z | 2022-12-18T02:18:43Z | OWNER | Various attempts at a fix which didn't work: ```diff diff --git a/tests/conftest.py b/tests/conftest.py index 69dee68b..899d36fd 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,4 +1,3 @@ -import asyncio import httpx import os import pathlib @@ -6,6 +5,7 @@ import pytest import pytest_asyncio import re import subprocess +import sys import tempfile import time import trustme @@ -27,13 +27,23 @@ UNDOCUMENTED_PERMISSIONS = { _ds_client = None -def wait_until_responds(url, timeout=5.0, client=httpx, **kwargs): +def wait_until_responds(url, timeout=5.0, client=None, **kwargs): + client = client or httpx.Client(**kwargs) start = time.time() while time.time() - start < timeout: try: - client.get(url, **kwargs) + if "verify" in kwargs: + print(kwargs["verify"]) + print( + "Contents of verify file: {}".format( + open(kwargs.get("verify")).read() + ) + ) + print("client = {}, kwargs = {}".format(client, kwargs)) + client.get(url) return - except httpx.ConnectError: + except (httpx.ConnectError, httpx.RemoteProtocolError) as ex: + print(ex) time.sleep(0.1) raise AssertionError("Timed out waiting for {} to respond".format(url)) @@ -166,7 +176,7 @@ def check_permission_actions_are_documented(): @pytest.fixture(scope="session") def ds_localhost_http_server(): ds_proc = subprocess.Popen( - ["datasette", "--memory", "-p", "8041"], + [sys.executable, "-m", "datasette", "--memory", "-p", "8041"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, # Avoid FileNotFoundError: [Errno 2] No such file or directory: @@ -180,7 +190,7 @@ def ds_localhost_http_server(): ds_proc.terminate() -@pytest.fixture(scope="session") +@pytest.fixture def ds_localhost_https_serve… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356627931 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356627931 | IC_kwDOBm6k_c5Q3H_b | 9599 | 2022-12-18T02:13:01Z | 2022-12-18T02:13:01Z | OWNER | Rather than continue to bang my head against this, I'm tempted to rewrite this test to happen outside of Python world - in a bash script run by GitHub Actions, for example. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356627331 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356627331 | IC_kwDOBm6k_c5Q3H2D | 9599 | 2022-12-18T02:11:17Z | 2022-12-18T02:11:17Z | OWNER | This issue might be relevant, but I tried the suggested fix in there (`Connection: close` on the incoming requests) and it didn't fix my problem: - https://github.com/encode/httpx/discussions/2056 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356626334 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356626334 | IC_kwDOBm6k_c5Q3Hme | 9599 | 2022-12-18T02:04:01Z | 2022-12-18T02:04:07Z | OWNER | I used the steps to test manually from this comment: https://github.com/simonw/datasette/issues/1221#issuecomment-777901052 In one terminal: ``` cd /tmp python -m trustme datasette --memory --ssl-keyfile=/tmp/server.key --ssl-certfile=/tmp/server.pem -p 8003 ``` Then in another terminal: ``` curl --cacert /tmp/client.pem 'https://localhost:8003/_memory.json' ``` This worked correctly, outputting the expected JSON. So the feature still works, it's just the test that is broken for some reason. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356625642 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356625642 | IC_kwDOBm6k_c5Q3Hbq | 9599 | 2022-12-18T02:00:57Z | 2022-12-18T02:00:57Z | OWNER | I added the TLS support here: - https://github.com/simonw/datasette/issues/1221 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356625556 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356625556 | IC_kwDOBm6k_c5Q3HaU | 9599 | 2022-12-18T02:00:18Z | 2022-12-18T02:00:18Z | OWNER | Maybe the reason the ASGI lifespan stuff broke was this line: https://github.com/simonw/datasette/blob/8b73fc6b47dffd8836f5c58aae1e57c1f66a5754/datasette/cli.py#L630-L632 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356620233 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356620233 | IC_kwDOBm6k_c5Q3GHJ | 9599 | 2022-12-18T01:31:10Z | 2022-12-18T01:31:10Z | OWNER | During the polling loop it constantly raises: `httpx.RemoteProtocolError`: Server disconnected without sending a response | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356618913 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356618913 | IC_kwDOBm6k_c5Q3Fyh | 9599 | 2022-12-18T01:29:05Z | 2022-12-18T01:29:05Z | OWNER | Now the only failure is in the `https` test - which fails like this (in CI and on my laptop): ``` message = str(exc) > raise mapped_exc(message) from exc E httpx.RemoteProtocolError: Server disconnected without sending a response. /opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/httpx/_transports/default.py:77: RemoteProtocolError =========================== short test summary info ============================ ERROR tests/test_cli_serve_server.py::test_serve_localhost_https - httpx.RemoteProtocolError: Server disconnected without sending a response. ================= 30 passed, 1264 deselected, 1 error in 6.15s ================= ``` That's this test: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/test_cli_serve_server.py#L16-L24 And this fixture: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/conftest.py#L178-L215 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356610089 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356610089 | IC_kwDOBm6k_c5Q3Dop | 9599 | 2022-12-18T01:12:39Z | 2022-12-18T01:12:39Z | OWNER | ... and it turns out those tests saved me. Because I forgot to check if `datasette` would actually start a server correctly! ``` % datasette fixtures.db -p 8852 INFO: Started server process [3538] INFO: Waiting for application startup. ERROR: Exception in 'lifespan' protocol Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/uvicorn/lifespan/on.py", line 86, in main await app(scope, self.receive, self.send) File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__ return await self.app(scope, receive, send) File "/Users/simon/Dropbox/Development/datasette/datasette/utils/asgi.py", line 437, in __call__ return await self.asgi(scope, receive, send) File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/asgi_csrf.py", line 39, in app_wrapped_with_csrf await app(scope, receive, send) File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 1457, in __call__ path = scope["path"] KeyError: 'path' ERROR: Application startup failed. Exiting. ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356609095 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356609095 | IC_kwDOBm6k_c5Q3DZH | 9599 | 2022-12-18T01:10:43Z | 2022-12-18T01:10:43Z | OWNER | Improved version of that fixture: ```diff diff --git a/tests/conftest.py b/tests/conftest.py index 44c44f87..69dee68b 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -27,6 +27,17 @@ UNDOCUMENTED_PERMISSIONS = { _ds_client = None +def wait_until_responds(url, timeout=5.0, client=httpx, **kwargs): + start = time.time() + while time.time() - start < timeout: + try: + client.get(url, **kwargs) + return + except httpx.ConnectError: + time.sleep(0.1) + raise AssertionError("Timed out waiting for {} to respond".format(url)) + + @pytest_asyncio.fixture async def ds_client(): from datasette.app import Datasette @@ -161,13 +172,7 @@ def ds_localhost_http_server(): # Avoid FileNotFoundError: [Errno 2] No such file or directory: cwd=tempfile.gettempdir(), ) - # Loop until port 8041 serves traffic - while True: - try: - httpx.get("http://localhost:8041/") - break - except httpx.ConnectError: - time.sleep(0.1) + wait_until_responds("http://localhost:8041/") # Check it started successfully assert not ds_proc.poll(), ds_proc.stdout.read().decode("utf-8") yield ds_proc @@ -202,12 +207,7 @@ def ds_localhost_https_server(tmp_path_factory): stderr=subprocess.STDOUT, cwd=tempfile.gettempdir(), ) - while True: - try: - httpx.get("https://localhost:8042/", verify=client_cert) - break - except httpx.ConnectError: - time.sleep(0.1) + wait_until_responds("http://localhost:8042/", verify=client_cert) # Check it started successfully assert not ds_proc.poll(), ds_proc.stdout.read().decode("utf-8") yield ds_proc, client_cert @@ -231,12 +231,7 @@ def ds_unix_domain_socket_server(tmp_path_factory): # Poll until available transport = httpx.HTTPTransport(uds=uds) client = httpx.Client(transport=transport) - while T… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356600917 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356600917 | IC_kwDOBm6k_c5Q3BZV | 9599 | 2022-12-18T01:02:26Z | 2022-12-18T01:02:26Z | OWNER | This bit here looks like it could hang! ```python # Loop until port 8041 serves traffic while True: try: httpx.get("http://localhost:8041/") break except httpx.ConnectError: time.sleep(0.1) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356599930 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356599930 | IC_kwDOBm6k_c5Q3BJ6 | 9599 | 2022-12-18T01:01:47Z | 2022-12-18T01:01:47Z | OWNER | I think that's this test: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/test_cli_serve_server.py#L6-L13 Using this fixture: https://github.com/simonw/datasette/blob/63fb750f39cac6f49b451387fdff659ecd9edc5c/tests/conftest.py#L155-L175 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356596740 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356596740 | IC_kwDOBm6k_c5Q3AYE | 9599 | 2022-12-18T00:59:47Z | 2022-12-18T00:59:47Z | OWNER | Hitting `Ctrl+C` while using `--full-trace` gave me more clues: ``` % pytest -m serial tests/test_cli_serve_server.py --full-trace ======================================================= test session starts ======================================================== platform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0 SQLite: 3.39.4 rootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0 asyncio: mode=strict collected 3 items tests/test_cli_serve_server.py ^C^C ====================================================== no tests ran in 3.49s ======================================================= Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/httpcore/_exceptions.py", line 8, in map_exceptions yield File "/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/httpcore/backends/sync.py", line 86, in connect_tcp sock = socket.create_connection( File "/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/socket.py", line 845, in create_connection raise err File "/Users/simon/.pyenv/versions/3.10.3/lib/python3.10/socket.py", line 833, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 61] Connection refused [...] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356595665 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356595665 | IC_kwDOBm6k_c5Q3AHR | 9599 | 2022-12-18T00:58:16Z | 2022-12-18T00:58:16Z | OWNER | `pytest -m serial` on my Mac laptop also freezes: ``` (datasette) datasette % pytest -m serial ======================================================= test session starts ======================================================== platform darwin -- Python 3.10.3, pytest-7.1.3, pluggy-1.0.0 SQLite: 3.39.4 rootdir: /Users/simon/Dropbox/Development/datasette, configfile: pytest.ini plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, asyncio-0.19.0, timeout-2.1.0, profiling-1.7.0 asyncio: mode=strict collected 1295 items / 1264 deselected / 31 selected tests/test_package.py . [ 3%] tests/test_cli_serve_server.py ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356489200 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356489200 | IC_kwDOBm6k_c5Q2mHw | 9599 | 2022-12-17T22:29:51Z | 2022-12-17T22:29:51Z | OWNER | No, it still causes the tests to hang (I let them run for 12 minutes): <img width="664" alt="image" src="https://user-images.githubusercontent.com/9599/208268223-7fb40e3f-833b-4204-9acb-079adce6452f.png"> Interesting that the regular tests passed an then the `pytest -m serial` ones seem to have failed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 | |
https://github.com/simonw/datasette/issues/1955#issuecomment-1356487139 | https://api.github.com/repos/simonw/datasette/issues/1955 | 1356487139 | IC_kwDOBm6k_c5Q2lnj | 9599 | 2022-12-17T22:16:52Z | 2022-12-17T22:16:52Z | OWNER | I'm trying this fix again, after a bunch of work on the test suite in: - #1959 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1496652622 |