github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-695113871 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 695113871 | MDEyOklzc3VlQ29tbWVudDY5NTExMzg3MQ== | 9599 | 2020-09-18T22:30:17Z | 2020-09-18T22:30:17Z | MEMBER | I think I know what's going on here: https://github.com/dogsheep/dogsheep-beta/blob/0f1b951c5131d16f3c8559a8e4d79ed5c559e3cb/dogsheep_beta/__init__.py#L166-L171 This is a logic bug - the `compiled` variable could be the template from the previous loop! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695109140 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25 | 695109140 | MDEyOklzc3VlQ29tbWVudDY5NTEwOTE0MA== | 9599 | 2020-09-18T22:12:20Z | 2020-09-18T22:12:20Z | MEMBER | Documented here: https://github.com/dogsheep/dogsheep-beta/blob/534fc9689227eba70e69a45da0cee5820bbda9e1/README.md#datasette-plugin | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
704685890 | |
https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695108895 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25 | 695108895 | MDEyOklzc3VlQ29tbWVudDY5NTEwODg5NQ== | 9599 | 2020-09-18T22:11:32Z | 2020-09-18T22:11:32Z | MEMBER | I'm going to make this a new plugin configuration setting, `template_debug`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
704685890 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694557425 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694557425 | MDEyOklzc3VlQ29tbWVudDY5NDU1NzQyNQ== | 9599 | 2020-09-17T23:41:01Z | 2020-09-17T23:41:01Z | MEMBER | I removed all of the `json.loads()` calls and I'm still getting that `Undefined` error. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694554584 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694554584 | MDEyOklzc3VlQ29tbWVudDY5NDU1NDU4NA== | 9599 | 2020-09-17T23:31:25Z | 2020-09-17T23:31:25Z | MEMBER | I'd prefer it if errors in these template fragments were displayed as errors inline where the fragment should have been inserted, rather than 500ing the whole page - especially since the template fragments are user-provided and could have all kinds of odd errors in them which should be as easy to debug as possible. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694553579 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694553579 | MDEyOklzc3VlQ29tbWVudDY5NDU1MzU3OQ== | 9599 | 2020-09-17T23:28:37Z | 2020-09-17T23:28:37Z | MEMBER | More investigation in pdb: ``` (dogsheep-beta) dogsheep-beta % datasette . --get '/-/beta?q=pycon&sort=oldest' --pdb > /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads() -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' (Pdb) list 336 if s.startswith('\ufeff'): 337 raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)", 338 s, 0) 339 else: 340 if not isinstance(s, (bytes, bytearray)): 341 -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' 342 f'not {s.__class__.__name__}') 343 s = s.decode(detect_encoding(s), 'surrogatepass') 344 345 if "encoding" in kw: 346 import warnings (Pdb) bytes <class 'bytes'> (Pdb) locals()['s'] Undefined (Pdb) type(locals()['s']) <class 'jinja2.runtime.Undefined'> ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694552681 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694552681 | MDEyOklzc3VlQ29tbWVudDY5NDU1MjY4MQ== | 9599 | 2020-09-17T23:25:54Z | 2020-09-17T23:25:54Z | MEMBER | This is the template fragment it's rendering: ```html+jinja <div style="overflow: hidden;"> <p>Tweet by <a href="https://twitter.com/{{ display.screen_name }}">@{{ display.screen_name }}</a> ({{ display.user_name }}, {{ "{:,}".format(display.followers_count or 0) }} followers) on <a href="https://twitter.com/{{ display.screen_name }}/status/{{ display.tweet_id }}">{{ display.created_at }}</a></p> </p> <blockquote>{{ display.full_text }}</blockquote> {% if display.media_urls and json.loads(display.media_urls) %} {% for url in json.loads(display.media_urls) %} <img src="{{ url }}" style="height: 200px;"> {% endfor %} {% endif %} </div> ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694552393 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694552393 | MDEyOklzc3VlQ29tbWVudDY5NDU1MjM5Mw== | 9599 | 2020-09-17T23:25:01Z | 2020-09-17T23:25:17Z | MEMBER | Ran `locals()` In the debugger: `{'range': <class 'range'>, 'dict': <class 'dict'>, 'lipsum': <function generate_lorem_ipsum at 0x10aeff430>, 'cycler': <class 'jinja2.utils.Cycler'>, 'joiner': <class 'jinja2.utils.Joiner'>, 'namespace': <class 'jinja2.utils.Namespace'>, 'rank': -9.383801886431414, 'rowid': 14297, 'type': 'twitter.db/tweets', 'key': '312658917933076480', 'title': 'Tweet by @chrisstreeter', 'category': 2, 'timestamp': '2013-03-15T20:17:49+00:00', 'search_1': '@simonw are you at pycon? Would love to meet you.', 'display': {'avatar_url': 'https://pbs.twimg.com/profile_images/806275088597204993/38yLHfJi_normal.jpg', 'user_name': 'Chris Streeter', 'screen_name': 'chrisstreeter', 'followers_count': 280, 'tweet_id': 312658917933076480, 'created_at': '2013-03-15T20:17:49+00:00', 'full_text': '@simonw are you at pycon? Would love to meet you.', 'media_urls_2': '[]', 'media_urls': '[]'}, 'json': <module 'json' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>}` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694551646 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694551646 | MDEyOklzc3VlQ29tbWVudDY5NDU1MTY0Ng== | 9599 | 2020-09-17T23:22:48Z | 2020-09-17T23:22:48Z | MEMBER | Looks like its happening in a Jinja fragment template for one of the results: ``` /Users/simon/Dropbox/Development/dogsheep-beta/dogsheep_beta/__init__.py(169)process_results() -> output = compiled.render({**result, **{"json": json}}) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/asyncsupport.py(71)render() -> return original_render(self, *args, **kwargs) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(1090)render() -> self.environment.handle_exception() /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(832)handle_exception() -> reraise(*rewrite_traceback_stack(source=source)) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/_compat.py(28)reraise() -> raise value.with_traceback(tb) <template>(5)top-level template code() > /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads() -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694551406 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694551406 | MDEyOklzc3VlQ29tbWVudDY5NDU1MTQwNg== | 9599 | 2020-09-17T23:22:07Z | 2020-09-17T23:22:07Z | MEMBER | Neat, I can debug this with the new `--pdb` option: datasette . --get '/-/beta?q=pycon&sort=oldest' --pdb | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-beta/issues/16#issuecomment-694548909 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16 | 694548909 | MDEyOklzc3VlQ29tbWVudDY5NDU0ODkwOQ== | 9599 | 2020-09-17T23:15:09Z | 2020-09-17T23:15:09Z | MEMBER | I have sort by date now, #21. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
694493566 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693794700 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693794700 | MDEyOklzc3VlQ29tbWVudDY5Mzc5NDcwMA== | 9599 | 2020-09-17T04:02:39Z | 2020-09-17T04:02:39Z | MEMBER | It would be useful if you could pass an `--accept` option to this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693789129 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693789129 | MDEyOklzc3VlQ29tbWVudDY5Mzc4OTEyOQ== | 9599 | 2020-09-17T03:40:01Z | 2020-09-17T03:40:01Z | MEMBER | Bug with endpoints that return dictionaries rather than arrays: ``` github-to-sqlite get /users/simonw [ "login", "id", "node_id", "avatar_url", "gravatar_id", "url", "html_url", "followers_url", "following_url", "gists_url", "starred_url", "subscriptions_url", "organizations_url", "repos_url", "events_url", "received_events_url", "type", "site_admin", "name", "company", "blog", "location", "email", "hireable", "bio", "twitter_username", "public_repos", "public_gists", "followers", "following", "created_at", "updated_at" ] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693788387 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693788387 | MDEyOklzc3VlQ29tbWVudDY5Mzc4ODM4Nw== | 9599 | 2020-09-17T03:36:47Z | 2020-09-17T03:36:58Z | MEMBER | Fun demo of the `--nl` option: github-to-sqlite get /users/simonw/repos --paginate --nl | sqlite-utils insert simonw.db repos - --nl | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693788032 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693788032 | MDEyOklzc3VlQ29tbWVudDY5Mzc4ODAzMg== | 9599 | 2020-09-17T03:35:22Z | 2020-09-17T03:35:22Z | MEMBER | Documentation: https://github.com/dogsheep/github-to-sqlite/blob/b02bf135485c0a7a3768868967f45a6b5e515289/README.md#making-authenticated-api-calls | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693775622 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693775622 | MDEyOklzc3VlQ29tbWVudDY5Mzc3NTYyMg== | 9599 | 2020-09-17T02:48:34Z | 2020-09-17T02:48:34Z | MEMBER | I'd like a `--paginate` option that does the same thing as https://github.com/simonw/paginate-json | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/simonw/datasette/issues/507#issuecomment-693775390 | https://api.github.com/repos/simonw/datasette/issues/507 | 693775390 | MDEyOklzc3VlQ29tbWVudDY5Mzc3NTM5MA== | 9599 | 2020-09-17T02:47:35Z | 2020-09-17T02:47:35Z | OWNER | I have a pattern for creating screenshots using Puppeteer running in a GitHub Action now, see https://simonwillison.net/2020/Sep/3/weeknotes-airtable-screenshots-dogsheep/#weeknotes-2020-09-03-social-media-cards-tils | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
455852801 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693773191 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693773191 | MDEyOklzc3VlQ29tbWVudDY5Mzc3MzE5MQ== | 9599 | 2020-09-17T02:39:26Z | 2020-09-17T02:39:26Z | MEMBER | I'm going to start with `github-to-sqlite get` and `github-to-sqlite post` - I may add `put` and suchlike later on. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693695177 | https://api.github.com/repos/simonw/sqlite-utils/issues/159 | 693695177 | MDEyOklzc3VlQ29tbWVudDY5MzY5NTE3Nw== | 9599 | 2020-09-16T22:17:53Z | 2020-09-16T22:17:53Z | OWNER | @spdkils can you share a minimal code example that exhibits the behavior you're seeing? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702386948 | |
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693694968 | https://api.github.com/repos/simonw/sqlite-utils/issues/159 | 693694968 | MDEyOklzc3VlQ29tbWVudDY5MzY5NDk2OA== | 9599 | 2020-09-16T22:17:19Z | 2020-09-16T22:17:19Z | OWNER | That's strange... this test here doesn't manually commit a transaction and passes: https://github.com/simonw/sqlite-utils/blob/7805d53bcf11199bd1f2b07e05ae90151f9d0eb0/tests/test_delete.py#L17-L23 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702386948 | |
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693694343 | https://api.github.com/repos/simonw/sqlite-utils/issues/159 | 693694343 | MDEyOklzc3VlQ29tbWVudDY5MzY5NDM0Mw== | 9599 | 2020-09-16T22:15:39Z | 2020-09-16T22:15:39Z | OWNER | Independent of the transaction changes in #121 I may be able to check `self.conn.in_transaction` to see if a transaction is active and, if one is NOT active, execute the delete inside of one. https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.in_transaction | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702386948 | |
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693589321 | https://api.github.com/repos/simonw/sqlite-utils/issues/159 | 693589321 | MDEyOklzc3VlQ29tbWVudDY5MzU4OTMyMQ== | 9599 | 2020-09-16T18:41:42Z | 2020-09-16T18:41:42Z | OWNER | Yeah I'm going to class this as a bug - that's definitely confusing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702386948 | |
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693486183 | https://api.github.com/repos/simonw/sqlite-utils/issues/159 | 693486183 | MDEyOklzc3VlQ29tbWVudDY5MzQ4NjE4Mw== | 11712349 | 2020-09-16T15:34:13Z | 2020-09-16T15:34:13Z | NONE | I appreciate the response, it's just unexpected. If I insert, it commits, if I update it commits, if I upsert it commits... if I delete.. it doesn't??? Confused me... I did just db commit it... But it's confusing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702386948 | |
https://github.com/simonw/sqlite-utils/pull/158#issuecomment-693199392 | https://api.github.com/repos/simonw/sqlite-utils/issues/158 | 693199392 | MDEyOklzc3VlQ29tbWVudDY5MzE5OTM5Mg== | 9599 | 2020-09-16T06:21:29Z | 2020-09-16T06:21:29Z | OWNER | Thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
697203800 | |
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-693199049 | https://api.github.com/repos/simonw/sqlite-utils/issues/159 | 693199049 | MDEyOklzc3VlQ29tbWVudDY5MzE5OTA0OQ== | 9599 | 2020-09-16T06:20:26Z | 2020-09-16T06:20:26Z | OWNER | See #121 - I need to think harder about how this all interacts with transactions. You can do this: ```python with db.conn: db["mytable"].delete_where() ``` But that should be documented and maybe rethought. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702386948 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693010291 | https://api.github.com/repos/simonw/datasette/issues/943 | 693010291 | MDEyOklzc3VlQ29tbWVudDY5MzAxMDI5MQ== | 9599 | 2020-09-15T22:20:55Z | 2020-09-15T22:20:55Z | OWNER | Should I instantiate a single `Client` and reuse it for all internal requests, or can I instantiate a new `Client` for each request? https://www.python-httpx.org/advanced/#why-use-a-client says that the main benefit of a Client instance is HTTP connection pooling - which isn't an issue for these internal requests since they won't be using the HTTP protocol at all, they'll be calling the ASGI application directly. So I'm leaning towards instantiating a fresh client for every internal request. I'll run a microbenchmark to check that this doesn't have any unpleasant performance implications. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693008540 | https://api.github.com/repos/simonw/datasette/issues/943 | 693008540 | MDEyOklzc3VlQ29tbWVudDY5MzAwODU0MA== | 9599 | 2020-09-15T22:16:07Z | 2020-09-15T22:16:07Z | OWNER | I think I can use `async with httpx.AsyncClient(base_url="http://localhost/") as client:` to ensure I don't need to use `http://localhost/` on every call. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693007512 | https://api.github.com/repos/simonw/datasette/issues/943 | 693007512 | MDEyOklzc3VlQ29tbWVudDY5MzAwNzUxMg== | 9599 | 2020-09-15T22:13:30Z | 2020-09-15T22:13:30Z | OWNER | I could solve streaming using something like this: ```python async with datasette.stream("GET", "/fixtures/compound_three_primary_keys.csv?_stream=on&_size=max") as response: async for chunk in response.aiter_bytes(): print(chunk) ``` Which would be a wrapper around `AsyncClient.stream(method, url, ...)` from https://www.python-httpx.org/async/#streaming-responses | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693005033 | https://api.github.com/repos/simonw/datasette/issues/943 | 693005033 | MDEyOklzc3VlQ29tbWVudDY5MzAwNTAzMw== | 9599 | 2020-09-15T22:06:58Z | 2020-09-15T22:10:58Z | OWNER | What if `datasette.get()` was an alias for `httpx.get()`, pre-configured to route to the correct application? And with some sugar that added `http://localhost/` to the beginning of the path if it was missing? This would make `httpx` a dependency of core Datasette, which I think is OK. It would also solve the return type problem: I would return whatever `httpx` returns. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693004770 | https://api.github.com/repos/simonw/datasette/issues/943 | 693004770 | MDEyOklzc3VlQ29tbWVudDY5MzAwNDc3MA== | 9599 | 2020-09-15T22:06:13Z | 2020-09-15T22:06:13Z | OWNER | I'm tempted to create a `await datasette.request()` method which can take any HTTP verb - then have `datasette.get()` and `datasette.post()` as thin wrappers around it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693004572 | https://api.github.com/repos/simonw/datasette/issues/943 | 693004572 | MDEyOklzc3VlQ29tbWVudDY5MzAwNDU3Mg== | 9599 | 2020-09-15T22:05:39Z | 2020-09-15T22:05:39Z | OWNER | Maybe these methods become the way most Datasette tests are written, replacing the existing `TestClient` mechanism? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693004296 | https://api.github.com/repos/simonw/datasette/issues/943 | 693004296 | MDEyOklzc3VlQ29tbWVudDY5MzAwNDI5Ng== | 9599 | 2020-09-15T22:04:54Z | 2020-09-15T22:04:54Z | OWNER | So what should I do about streaming responses? I could deliberately ignore them - through an exception if you attempt to run `await datasette.get(...)` against a streaming URL. I could load the entire response into memory and return it as a wrapped object. I could support some kind of asynchronous iterator mechanism. This would be pretty elegant if I could decide the right syntax for it - it would allow plugins to take advantage of other internal URLs that return streaming content without needing to load that content entirely into memory in order to process it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/943#issuecomment-693003652 | https://api.github.com/repos/simonw/datasette/issues/943 | 693003652 | MDEyOklzc3VlQ29tbWVudDY5MzAwMzY1Mg== | 9599 | 2020-09-15T22:03:08Z | 2020-09-15T22:03:08Z | OWNER | I'm not going to mess around with formats - you'll get back the exact response that a web client would receive. Question: what should the response object look like? e.g. if you do: response = await datasette.get("/db/table.json") What should `response` be? I could reuse the Datasette `Response` class from `datasette.utils.asgi`. This would work well for regular responses which just have a status code, some headers and a response body. It wouldn't be great for streaming responses though such as you get back from `?_stream=1` CSV exports. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681375466 | |
https://github.com/simonw/datasette/issues/891#issuecomment-693001937 | https://api.github.com/repos/simonw/datasette/issues/891 | 693001937 | MDEyOklzc3VlQ29tbWVudDY5MzAwMTkzNw== | 9599 | 2020-09-15T21:58:56Z | 2020-09-15T21:58:56Z | OWNER | Here's what that looks like: ``` Traceback (most recent call last): File "/Users/simon/Dropbox/Development/datasette/plugins/sql_error.py", line 5, in oh_no_error return 100 / 0 ZeroDivisionError: division by zero ERROR: conn=<sqlite3.Connection object at 0x10bce0030>, sql = 'select oh_no_error()', params = {}: user-defined function raised exception INFO: 127.0.0.1:54066 - "GET /data?sql=select+oh_no_error%28%29 HTTP/1.1" 400 Bad Request ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
653529088 | |
https://github.com/simonw/datasette/issues/891#issuecomment-693000522 | https://api.github.com/repos/simonw/datasette/issues/891 | 693000522 | MDEyOklzc3VlQ29tbWVudDY5MzAwMDUyMg== | 9599 | 2020-09-15T21:55:11Z | 2020-09-15T21:55:11Z | OWNER | I'm going to turn this on. If people complain about it I can turn it off again (or make it a configuration setting). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
653529088 | |
https://github.com/simonw/datasette/issues/891#issuecomment-692999893 | https://api.github.com/repos/simonw/datasette/issues/891 | 692999893 | MDEyOklzc3VlQ29tbWVudDY5Mjk5OTg5Mw== | 9599 | 2020-09-15T21:53:36Z | 2020-09-15T21:53:36Z | OWNER | Here's the commit (from 15 years ago) where `enable_callback_tracebacks` was first added: https://github.com/ghaering/pysqlite/commit/1e8bd36be93b7d7425910642b72e4152c77b0dfd > - Exceptions in callbacks lead to the query being aborted now instead of silently leading to generating values. > - Exceptions in callbacks can be echoed to stderr if you call the module level function enable_callback_tracebacks: enable_callback_tracebacks(1). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
653529088 | |
https://github.com/simonw/datasette/issues/891#issuecomment-692998061 | https://api.github.com/repos/simonw/datasette/issues/891 | 692998061 | MDEyOklzc3VlQ29tbWVudDY5Mjk5ODA2MQ== | 9599 | 2020-09-15T21:49:03Z | 2020-09-15T21:49:03Z | OWNER | I've been trying to figure out why this is an optional setting that defaults to off. I think it's because it writes directly to `stderr`, so the maintainers of `sqlite3` reasonably decided that people should be able to opt in to that rather than having weird stuff show up on `stderr` that they weren't expecting. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
653529088 | |
https://github.com/simonw/datasette/issues/891#issuecomment-692968792 | https://api.github.com/repos/simonw/datasette/issues/891 | 692968792 | MDEyOklzc3VlQ29tbWVudDY5Mjk2ODc5Mg== | 9599 | 2020-09-15T20:44:15Z | 2020-09-15T20:44:15Z | OWNER | https://github.com/peter-wangxu/persist-queue/issues/74 warns that this might not work with PyPy. I could solve that with: ```python if hasattr(sqlite3, "enable_callback_tracebacks"): sqlite3.enable_callback_tracebacks(True) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
653529088 | |
https://github.com/simonw/datasette/issues/877#issuecomment-692967733 | https://api.github.com/repos/simonw/datasette/issues/877 | 692967733 | MDEyOklzc3VlQ29tbWVudDY5Mjk2NzczMw== | 9599 | 2020-09-15T20:42:04Z | 2020-09-15T20:42:04Z | OWNER | I'm not going to drop CSRF protection - it's still needed for older browsers - but I have relaxed the circumstances under which it is applied. It only applies to requests that include cookies for example, so API clients that don't send cookies don't need to worry about it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648421105 | |
https://github.com/simonw/datasette/issues/889#issuecomment-692967123 | https://api.github.com/repos/simonw/datasette/issues/889 | 692967123 | MDEyOklzc3VlQ29tbWVudDY5Mjk2NzEyMw== | 9599 | 2020-09-15T20:40:52Z | 2020-09-15T20:40:52Z | OWNER | Thanks - I've fixed this in `datasette-media` and the other plugins that use that hook now I think. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
649907676 | |
https://github.com/simonw/datasette/issues/888#issuecomment-692966625 | https://api.github.com/repos/simonw/datasette/issues/888 | 692966625 | MDEyOklzc3VlQ29tbWVudDY5Mjk2NjYyNQ== | 9599 | 2020-09-15T20:39:49Z | 2020-09-15T20:39:49Z | OWNER | Thanks, I've fixed that now. It only affected the GitHub release notes - the ones at https://docs.datasette.io/en/stable/changelog.html#v0-45 had the correct links. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
649702801 | |
https://github.com/simonw/datasette/issues/634#issuecomment-692965761 | https://api.github.com/repos/simonw/datasette/issues/634 | 692965761 | MDEyOklzc3VlQ29tbWVudDY5Mjk2NTc2MQ== | 9599 | 2020-09-15T20:37:58Z | 2020-09-15T20:37:58Z | OWNER | I fixed this in 5e0b72247ecab4ce0fcec599b77a83d73a480872 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
522352520 | |
https://github.com/simonw/datasette/issues/849#issuecomment-692965391 | https://api.github.com/repos/simonw/datasette/issues/849 | 692965391 | MDEyOklzc3VlQ29tbWVudDY5Mjk2NTM5MQ== | 9599 | 2020-09-15T20:37:14Z | 2020-09-15T20:37:14Z | OWNER | I've been running on `main` for a while now with no issues. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
639072811 | |
https://github.com/simonw/datasette/issues/956#issuecomment-692965022 | https://api.github.com/repos/simonw/datasette/issues/956 | 692965022 | MDEyOklzc3VlQ29tbWVudDY5Mjk2NTAyMg== | 9599 | 2020-09-15T20:36:34Z | 2020-09-15T20:36:34Z | OWNER | https://hub.docker.com/r/datasetteproject/datasette/tags - 0.49.1 was successfully pushed to Docker Hub by https://github.com/simonw/datasette/runs/1119815175?check_suite_focus=true | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688427751 | |
https://github.com/simonw/datasette/issues/956#issuecomment-692955850 | https://api.github.com/repos/simonw/datasette/issues/956 | 692955850 | MDEyOklzc3VlQ29tbWVudDY5Mjk1NTg1MA== | 9599 | 2020-09-15T20:17:49Z | 2020-09-15T20:17:49Z | OWNER | I think I've fixed this with recent changes I made as part of #941 - but I won't know until I release the next version. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
688427751 | |
https://github.com/simonw/datasette/issues/946#issuecomment-692955379 | https://api.github.com/repos/simonw/datasette/issues/946 | 692955379 | MDEyOklzc3VlQ29tbWVudDY5Mjk1NTM3OQ== | 9599 | 2020-09-15T20:16:50Z | 2020-09-15T20:16:50Z | OWNER | Can't reproduce this bug now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
682184050 | |
https://github.com/simonw/datasette/issues/492#issuecomment-692953174 | https://api.github.com/repos/simonw/datasette/issues/492 | 692953174 | MDEyOklzc3VlQ29tbWVudDY5Mjk1MzE3NA== | 9599 | 2020-09-15T20:12:29Z | 2020-09-15T20:12:29Z | OWNER | I fixed this in ea340cf320a2566d24517fb4a0c9852c5059e771 for #963 (a duplicate of this issue). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
449854604 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692951144 | https://api.github.com/repos/simonw/datasette/issues/967 | 692951144 | MDEyOklzc3VlQ29tbWVudDY5Mjk1MTE0NA== | 9599 | 2020-09-15T20:08:12Z | 2020-09-15T20:08:12Z | OWNER | I think the easiest fix is for me to ensure that calls to `__len__` on the `MagicParameters` class always return at least 1. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692946616 | https://api.github.com/repos/simonw/datasette/issues/967 | 692946616 | MDEyOklzc3VlQ29tbWVudDY5Mjk0NjYxNg== | 9599 | 2020-09-15T19:59:21Z | 2020-09-15T19:59:21Z | OWNER | I wish I could call https://www.sqlite.org/c3ref/bind_parameter_count.html and https://www.sqlite.org/c3ref/bind_parameter_name.html from Python. Might be possible to do that using `ctypes` - see this example code: https://mail.python.org/pipermail//pypy-commit/2013-February/071372.html ```python param_count = lib.sqlite3_bind_parameter_count(self.statement) for idx in range(1, param_count + 1): param_name = lib.sqlite3_bind_parameter_name(self.statement, idx) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692945504 | https://api.github.com/repos/simonw/datasette/issues/967 | 692945504 | MDEyOklzc3VlQ29tbWVudDY5Mjk0NTUwNA== | 9599 | 2020-09-15T19:57:10Z | 2020-09-15T19:57:10Z | OWNER | So the problem actually occurs when the `MagicParameters` class wraps an empty dictionary. Relevant code: https://github.com/simonw/datasette/blob/853c5fc37011a7bc09ca3a1af287102f00827c82/datasette/views/database.py#L228-L236 And: https://github.com/simonw/datasette/blob/853c5fc37011a7bc09ca3a1af287102f00827c82/datasette/views/database.py#L364-L383 I'm passing a special magic parameters dictionary for the Python `sqlite3` module to look up parameters in. When that dictionary is `{}` a `__len__` check is performed on that dictionary, the result comes back as 0 and as a result it assumes there are no parameters. I tracked down the relevant C code: https://github.com/python/cpython/blob/81715808716198471fbca0a3db42ac408468dbc5/Modules/_sqlite/statement.c#L218-L237 ```c Py_BEGIN_ALLOW_THREADS num_params_needed = sqlite3_bind_parameter_count(self->st); Py_END_ALLOW_THREADS if (PyTuple_CheckExact(parameters) || PyList_CheckExact(parameters) || (!PyDict_Check(parameters) && PySequence_Check(parameters))) { /* parameters passed as sequence */ if (PyTuple_CheckExact(parameters)) { num_params = PyTuple_GET_SIZE(parameters); } else if (PyList_CheckExact(parameters)) { num_params = PyList_GET_SIZE(parameters); } else { num_params = PySequence_Size(parameters); } if (num_params != num_params_needed) { PyErr_Format(pysqlite_ProgrammingError, "Incorrect number of bindings supplied. The current " "statement uses %d, and there are %zd supplied.", num_params_needed, num_params); return; } ``` It looks to me like this should fail if the number of keys known to be in the dictionary differs from the number of named parameters in the query. But if those numbers fail to match it still works as far as I can tell - it's only dictionary length of 0 that is causing the problems. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692940375 | https://api.github.com/repos/simonw/datasette/issues/967 | 692940375 | MDEyOklzc3VlQ29tbWVudDY5Mjk0MDM3NQ== | 9599 | 2020-09-15T19:47:09Z | 2020-09-15T19:47:09Z | OWNER | Yes! The tests all pass if I update the test function to do this: ```python response = magic_parameters_client.post( "/data/runme_post{}".format(qs), {"ignore_me": "1"}, csrftoken_from=use_csrf or None, allow_redirects=False, ) ``` So the bug only occurs if the POST body is completely empty. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692938935 | https://api.github.com/repos/simonw/datasette/issues/967 | 692938935 | MDEyOklzc3VlQ29tbWVudDY5MjkzODkzNQ== | 9599 | 2020-09-15T19:44:21Z | 2020-09-15T19:44:41Z | OWNER | While I'm running the above test, in the rounds that work the `receive()` awaitable returns `{'type': 'http.request', 'body': b'csrftoken=IlpwUGlSMFVVa3Z3ZlVoamQi.uY2U1tF4i0M-5M6x34vnBCmJgr0'}` In the rounds that fails it returns `{'type': 'http.request'}` So it looks like the `csrftoken_from=True` parameter may be helping just by ensuring the `body` key is present and not missing. I wonder if it would work if a body of `b''` was present there? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692937150 | https://api.github.com/repos/simonw/datasette/issues/967 | 692937150 | MDEyOklzc3VlQ29tbWVudDY5MjkzNzE1MA== | 9599 | 2020-09-15T19:42:57Z | 2020-09-15T19:42:57Z | OWNER | New (failing) test: ```python @pytest.mark.parametrize("use_csrf", [True, False]) @pytest.mark.parametrize("return_json", [True, False]) def test_magic_parameters_csrf_json(magic_parameters_client, use_csrf, return_json): magic_parameters_client.ds._metadata["databases"]["data"]["queries"]["runme_post"][ "sql" ] = "insert into logs (line) values (:_header_host)" qs = "" if return_json: qs = "?_json=1" response = magic_parameters_client.post( "/data/runme_post{}".format(qs), {}, csrftoken_from=use_csrf or None, allow_redirects=False, ) if return_json: assert response.status == 200 assert response.json["ok"], response.json else: assert response.status == 302 messages = magic_parameters_client.ds.unsign( response.cookies["ds_messages"], "messages" ) assert [["Query executed, 1 row affected", 1]] == messages post_actual = magic_parameters_client.get( "/data/logs.json?_sort_desc=rowid&_shape=array" ).json[0]["line"] assert post_actual == "localhost" ``` It passes twice, fails twice - failures are for the ones where `use_csrf` is `False`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692927867 | https://api.github.com/repos/simonw/datasette/issues/967 | 692927867 | MDEyOklzc3VlQ29tbWVudDY5MjkyNzg2Nw== | 9599 | 2020-09-15T19:25:23Z | 2020-09-15T19:25:23Z | OWNER | Hunch: I think the `asgi-csrf` middleware may be consuming the request body and failing to restore it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692835066 | https://api.github.com/repos/simonw/datasette/issues/967 | 692835066 | MDEyOklzc3VlQ29tbWVudDY5MjgzNTA2Ng== | 9599 | 2020-09-15T16:40:12Z | 2020-09-15T16:40:12Z | OWNER | Is the bug here that magic parameters are incompatible with CSRF-exempt requests (e.g. request with no cookies)? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692834670 | https://api.github.com/repos/simonw/datasette/issues/967 | 692834670 | MDEyOklzc3VlQ29tbWVudDY5MjgzNDY3MA== | 9599 | 2020-09-15T16:39:29Z | 2020-09-15T16:39:29Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/853c5fc37011a7bc09ca3a1af287102f00827c82/datasette/views/database.py#L222-L236 This issue may not be about `_json=1` interacting with magic parameters after all. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692834064 | https://api.github.com/repos/simonw/datasette/issues/967 | 692834064 | MDEyOklzc3VlQ29tbWVudDY5MjgzNDA2NA== | 9599 | 2020-09-15T16:38:21Z | 2020-09-15T16:38:21Z | OWNER | So the mystery here is why does omitting `csrftoken_from=True` break the `MagicParameters` mechanism? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/967#issuecomment-692832113 | https://api.github.com/repos/simonw/datasette/issues/967 | 692832113 | MDEyOklzc3VlQ29tbWVudDY5MjgzMjExMw== | 9599 | 2020-09-15T16:34:53Z | 2020-09-15T16:37:43Z | OWNER | This is so weird. In the test I wrote for this the following passed: response = magic_parameters_client.post("/data/runme_post?_json=1", {}, csrftoken_from=True) But without the `csrftoken_from=True` parameter it failed with the bindings error: response = magic_parameters_client.post("/data/runme_post?_json=1", {}) Here's the test I wrote: ```python def test_magic_parameters_json_body(magic_parameters_client): magic_parameters_client.ds._metadata["databases"]["data"]["queries"]["runme_post"][ "sql" ] = "insert into logs (line) values (:_header_host)" response = magic_parameters_client.post("/data/runme_post?_json=1", {}, csrftoken_from=True) assert response.status == 200 assert response.json["ok"], response.json post_actual = magic_parameters_client.get( "/data/logs.json?_sort_desc=rowid&_shape=array" ).json[0]["line"] ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
702069429 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692340275 | https://api.github.com/repos/simonw/datasette/issues/940 | 692340275 | MDEyOklzc3VlQ29tbWVudDY5MjM0MDI3NQ== | 9599 | 2020-09-14T22:09:35Z | 2020-09-14T22:09:35Z | OWNER | I'm going to cross my fingers and hope that this works - I don't want to leave this issue open until Datasette 0.50. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692339645 | https://api.github.com/repos/simonw/datasette/issues/940 | 692339645 | MDEyOklzc3VlQ29tbWVudDY5MjMzOTY0NQ== | 9599 | 2020-09-14T22:07:58Z | 2020-09-14T22:07:58Z | OWNER | I shipped the Docker build manually by running the following in a tmate session: docker login # Typed my username and password interactively export REPO=datasetteproject/datasette docker build -f Dockerfile -t $REPO:0.49 . docker tag $REPO:0.49 $REPO:latest docker push $REPO | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692337397 | https://api.github.com/repos/simonw/datasette/issues/940 | 692337397 | MDEyOklzc3VlQ29tbWVudDY5MjMzNzM5Nw== | 9599 | 2020-09-14T22:01:56Z | 2020-09-14T22:01:56Z | OWNER | I'm going to switch to using this logic to decide if I should ship to Docker: https://github.community/t/release-prerelease-action-triggers/17275/2 if: "!github.event.release.prerelease" | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692336564 | https://api.github.com/repos/simonw/datasette/issues/940 | 692336564 | MDEyOklzc3VlQ29tbWVudDY5MjMzNjU2NA== | 9599 | 2020-09-14T21:59:40Z | 2020-09-14T21:59:40Z | OWNER | Using https://github.com/marketplace/actions/debugging-with-tmate to manually submit a new build from within an interactive GitHub Actions session. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692332430 | https://api.github.com/repos/simonw/datasette/issues/940 | 692332430 | MDEyOklzc3VlQ29tbWVudDY5MjMzMjQzMA== | 9599 | 2020-09-14T21:48:59Z | 2020-09-14T21:48:59Z | OWNER | So now I've released Datasette 0.49 but failed to push a new Docker image. This is bad, and I need to fix it. I'd like to push to Docker from GitHub Actions, so I think I'm going to create a one-off workflow task for doing that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692331919 | https://api.github.com/repos/simonw/datasette/issues/940 | 692331919 | MDEyOklzc3VlQ29tbWVudDY5MjMzMTkxOQ== | 9599 | 2020-09-14T21:47:39Z | 2020-09-14T21:47:39Z | OWNER | I bet that's because the `github.ref` actually looks like this: `${GITHUB_REF#refs/tags/}` And the `refs/tags/` part has an `a` in it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-692331349 | https://api.github.com/repos/simonw/datasette/issues/940 | 692331349 | MDEyOklzc3VlQ29tbWVudDY5MjMzMTM0OQ== | 9599 | 2020-09-14T21:46:11Z | 2020-09-14T21:46:11Z | OWNER | Just release Datasette 0.49 - which shipped to PyPI just fine but skipped the Docker step for some reason! https://github.com/simonw/datasette/runs/1114585275?check_suite_focus=true <img width="929" alt="Release_0_49_·_simonw_datasette_c024952" src="https://user-images.githubusercontent.com/9599/93141431-0571e180-f699-11ea-93c3-acaa68bd1272.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/880#issuecomment-692324230 | https://api.github.com/repos/simonw/datasette/issues/880 | 692324230 | MDEyOklzc3VlQ29tbWVudDY5MjMyNDIzMA== | 9599 | 2020-09-14T21:28:15Z | 2020-09-14T21:28:21Z | OWNER | Documentation here: https://docs.datasette.io/en/latest/sql_queries.html#json-api-for-writable-canned-queries | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/880#issuecomment-692299770 | https://api.github.com/repos/simonw/datasette/issues/880 | 692299770 | MDEyOklzc3VlQ29tbWVudDY5MjI5OTc3MA== | 9599 | 2020-09-14T20:36:40Z | 2020-09-14T20:36:40Z | OWNER | The JSON response will look like this: ```json { "ok": true, "message": "A message", "redirect": "/blah" } ``` `"ok"` will be `true` if everything went right and `false` if there was an error. The `"message"` and `"redirect"` will be whatever was configured using the on_success_message - the message shown `on_success_message`, `on_success_redirect`, `on_error_message` and `on_error_redirect` settings, see https://docs.datasette.io/en/stable/sql_queries.html#writable-canned-queries | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/880#issuecomment-692298011 | https://api.github.com/repos/simonw/datasette/issues/880 | 692298011 | MDEyOklzc3VlQ29tbWVudDY5MjI5ODAxMQ== | 9599 | 2020-09-14T20:33:13Z | 2020-09-14T20:33:13Z | OWNER | I'm going to support several ways of indicating that you would like a JSON response instead of getting a HTTP redirect from your writable canned query submission: - Use the `Accept: application/json` request header - Include `?_json=1` in the request query string - Include `"_json": 1` in the form submission (or the JSON body submission) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/880#issuecomment-692272860 | https://api.github.com/repos/simonw/datasette/issues/880 | 692272860 | MDEyOklzc3VlQ29tbWVudDY5MjI3Mjg2MA== | 9599 | 2020-09-14T19:43:47Z | 2020-09-14T19:43:47Z | OWNER | I'm going to add support for POST content that is sent as a JSON document, in addition to the existing support for key=value encoded POST bodies. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/880#issuecomment-692271804 | https://api.github.com/repos/simonw/datasette/issues/880 | 692271804 | MDEyOklzc3VlQ29tbWVudDY5MjI3MTgwNA== | 9599 | 2020-09-14T19:41:37Z | 2020-09-14T19:41:37Z | OWNER | Relevant code section: https://github.com/simonw/datasette/blob/1552ac931e4d2cf516caac3ceeab4fd24da1510a/datasette/views/database.py#L209-L232 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/965#issuecomment-692244252 | https://api.github.com/repos/simonw/datasette/issues/965 | 692244252 | MDEyOklzc3VlQ29tbWVudDY5MjI0NDI1Mg== | 9599 | 2020-09-14T18:49:48Z | 2020-09-14T18:49:48Z | OWNER | Documented here: https://docs.datasette.io/en/latest/custom_templates.html#custom-error-pages | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
701294727 | |
https://github.com/simonw/datasette/issues/965#issuecomment-692231257 | https://api.github.com/repos/simonw/datasette/issues/965 | 692231257 | MDEyOklzc3VlQ29tbWVudDY5MjIzMTI1Nw== | 9599 | 2020-09-14T18:25:04Z | 2020-09-14T18:25:04Z | OWNER | In documenting this I realized that it's confusing that the default `500.html` template is often used for non-500 errors (404 for example). I think I'll rename that default template to `error.html` instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
701294727 | |
https://github.com/simonw/datasette/issues/964#issuecomment-692212641 | https://api.github.com/repos/simonw/datasette/issues/964 | 692212641 | MDEyOklzc3VlQ29tbWVudDY5MjIxMjY0MQ== | 9599 | 2020-09-14T17:49:44Z | 2020-09-14T17:49:44Z | OWNER | Documentation: https://docs.datasette.io/en/latest/custom_templates.html#returning-404s | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
700728217 | |
https://github.com/simonw/datasette/issues/965#issuecomment-692207341 | https://api.github.com/repos/simonw/datasette/issues/965 | 692207341 | MDEyOklzc3VlQ29tbWVudDY5MjIwNzM0MQ== | 9599 | 2020-09-14T17:40:05Z | 2020-09-14T17:40:05Z | OWNER | Also link to these from the docs added in #964. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
701294727 | |
https://github.com/simonw/datasette/issues/944#issuecomment-691788478 | https://api.github.com/repos/simonw/datasette/issues/944 | 691788478 | MDEyOklzc3VlQ29tbWVudDY5MTc4ODQ3OA== | 9599 | 2020-09-14T03:21:45Z | 2020-09-14T03:21:45Z | OWNER | Having tried this out I think it does need a `raise_404()` mechanism - which needs to be smart enough to trigger the default 404 handler without accidentally going into an infinite loop. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681516976 | |
https://github.com/simonw/datasette/issues/880#issuecomment-691785692 | https://api.github.com/repos/simonw/datasette/issues/880 | 691785692 | MDEyOklzc3VlQ29tbWVudDY5MTc4NTY5Mg== | 9599 | 2020-09-14T03:10:11Z | 2020-09-14T03:10:11Z | OWNER | Answer: no, it's [not safe](https://twitter.com/glenathan/status/1305081266065244162) to skip CSRF if there's an `Accept: application/json` header because of a nasty old `crossdomain.xml` Flash vulnerability: https://blog.appsecco.com/exploiting-csrf-on-json-endpoints-with-flash-and-redirects-681d4ad6b31b?gi=a5ee3d7a8235 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/940#issuecomment-691781345 | https://api.github.com/repos/simonw/datasette/issues/940 | 691781345 | MDEyOklzc3VlQ29tbWVudDY5MTc4MTM0NQ== | 9599 | 2020-09-14T02:53:25Z | 2020-09-14T02:53:49Z | OWNER | That worked: https://github.com/simonw/datasette/runs/1110040212?check_suite_focus=true ran and deployed https://pypi.org/project/datasette/0.49a1/ to PyPI but it skipped the push to Docker step because there was an "a" in the tag. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-691779693 | https://api.github.com/repos/simonw/datasette/issues/940 | 691779693 | MDEyOklzc3VlQ29tbWVudDY5MTc3OTY5Mw== | 9599 | 2020-09-14T02:46:39Z | 2020-09-14T02:46:39Z | OWNER | I think those should be single quoted. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-691779510 | https://api.github.com/repos/simonw/datasette/issues/940 | 691779510 | MDEyOklzc3VlQ29tbWVudDY5MTc3OTUxMA== | 9599 | 2020-09-14T02:45:53Z | 2020-09-14T02:45:53Z | OWNER | This bit here: https://github.com/simonw/datasette/blob/c18117cf08ad67c704dab29e3cb3b88f1de4026b/.github/workflows/publish.yml#L58-L62 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/940#issuecomment-691779361 | https://api.github.com/repos/simonw/datasette/issues/940 | 691779361 | MDEyOklzc3VlQ29tbWVudDY5MTc3OTM2MQ== | 9599 | 2020-09-14T02:45:04Z | 2020-09-14T02:45:04Z | OWNER | Package deploys are still broken, just got this error trying to ship 0.49a1: https://github.com/simonw/datasette/actions/runs/253099665 > The workflow is not valid. .github/workflows/publish.yml (Line: 61, Col: 9): Unexpected symbol: '"a"'. Located at position 24 within expression: !(contains(github.ref, "a") || contains(github.ref, "b")) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
679808124 | |
https://github.com/simonw/datasette/issues/944#issuecomment-691774262 | https://api.github.com/repos/simonw/datasette/issues/944 | 691774262 | MDEyOklzc3VlQ29tbWVudDY5MTc3NDI2Mg== | 9599 | 2020-09-14T02:24:08Z | 2020-09-14T02:24:08Z | OWNER | Actually don't need `{{ raise_404("Museum not found") }}` because we already have `{{ custom_status(404) }}`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681516976 | |
https://github.com/simonw/datasette/issues/944#issuecomment-691769222 | https://api.github.com/repos/simonw/datasette/issues/944 | 691769222 | MDEyOklzc3VlQ29tbWVudDY5MTc2OTIyMg== | 9599 | 2020-09-14T02:01:33Z | 2020-09-14T02:01:33Z | OWNER | I'm going to cache the `list_templates()` result in memory. If you want to add a new template-defined route you will need to restart the server. I think that's acceptable. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
681516976 | |
https://github.com/simonw/datasette/issues/519#issuecomment-691566247 | https://api.github.com/repos/simonw/datasette/issues/519 | 691566247 | MDEyOklzc3VlQ29tbWVudDY5MTU2NjI0Nw== | 9599 | 2020-09-12T22:48:53Z | 2020-09-12T22:48:53Z | OWNER | I think I've figured out what to do about stability of the HTML and the default templates with respect to semantic versioning. I'm going to announce that the JSON API - including the variables made available to templates - should be considered stable according to semver. I will only break backwards compatibility at that level in a major version release. The template HTML (and default CSS) will not be considered a stable interface. They won't change on bug fix releases but they may change (albeit described in the release notes) on minor version bumps. Since the template inputs are stable, you can run your own copy of the previous version's templates if something breaks. This means users (and plugin authors) who make changes to the default Datasette UI will have to test their changes against every minor release. I think that's OK. If you write plugins that don't affect the Datasette HTML UI you will be able to expect stability across minor version releases. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
459590021 | |
https://github.com/simonw/datasette/issues/880#issuecomment-691558387 | https://api.github.com/repos/simonw/datasette/issues/880 | 691558387 | MDEyOklzc3VlQ29tbWVudDY5MTU1ODM4Nw== | 9599 | 2020-09-12T22:04:48Z | 2020-09-12T22:04:48Z | OWNER | Is it safe to skip CSRF checks if the incoming request has `Accept: application/json` on it? I'm not sure that matters since `asgi-csrf` already won't reject requests that either have no cookies or are using a `Authorization: Bearer ...` header. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/880#issuecomment-691557675 | https://api.github.com/repos/simonw/datasette/issues/880 | 691557675 | MDEyOklzc3VlQ29tbWVudDY5MTU1NzY3NQ== | 9599 | 2020-09-12T22:01:02Z | 2020-09-12T22:01:11Z | OWNER | Maybe POST to `.json` doesn't actually make sense. I could instead support `POST /db/queryname` with an optional mechanism for requesting that the response to that POST be in a JSON format. Could be a `Accept: application/json` header with an option of including `"_accept": "json"` as a POST parameter instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/880#issuecomment-691557429 | https://api.github.com/repos/simonw/datasette/issues/880 | 691557429 | MDEyOklzc3VlQ29tbWVudDY5MTU1NzQyOQ== | 9599 | 2020-09-12T21:59:39Z | 2020-09-12T21:59:39Z | OWNER | What should happen when something does a POST to an extension that was registered by a plugin, e.g. `POST /db/table.atom` ? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648637666 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691554088 | https://api.github.com/repos/simonw/datasette/issues/782 | 691554088 | MDEyOklzc3VlQ29tbWVudDY5MTU1NDA4OA== | 9599 | 2020-09-12T21:39:03Z | 2020-09-12T21:39:03Z | OWNER | Plan: release a new release of Datasette (probably 0.49) with the new JSON API design, but provide a plugin called something like `datasette-api-0-48` which runs as ASGI wrapping middleware and internally rewrites incoming requests to e.g. `/db/table.json` to behave if they have the `?_extra=` params on them necessary to produce the 0.48 version of the JSON. Anyone who has built applications against 0.48 can install that plugin. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/262#issuecomment-691526975 | https://api.github.com/repos/simonw/datasette/issues/262 | 691526975 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjk3NQ== | 9599 | 2020-09-12T18:22:44Z | 2020-09-12T18:22:44Z | OWNER | Are there any interesting use-cases for a plugin hook that allows plugins to define their own `?_extra=` blocks? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691526878 | https://api.github.com/repos/simonw/datasette/issues/782 | 691526878 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjg3OA== | 9599 | 2020-09-12T18:21:41Z | 2020-09-12T18:22:20Z | OWNER | Would it be so bad if the default format had a `"rows"` key containing the array of rows? Maybe it wouldn't. The reason I always use `?_shape=array` is because I want an array of objects, rather than an array of arrays that I have to match up again with their columns. A default format that's an object rather than array also gives something for the `?_extra=` parameter to add its extras to. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691526762 | https://api.github.com/repos/simonw/datasette/issues/782 | 691526762 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjc2Mg== | 9599 | 2020-09-12T18:20:19Z | 2020-09-12T18:20:19Z | OWNER | I'd like to revisit the idea of using `?_extra=x` to opt-in to extra blocks of JSON, from #262 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/262#issuecomment-691526719 | https://api.github.com/repos/simonw/datasette/issues/262 | 691526719 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjcxOQ== | 9599 | 2020-09-12T18:19:50Z | 2020-09-12T18:19:50Z | OWNER | > Idea: `?_extra=sqllog` could output a lot of every individual SQL statement that was executed in order to generate the page - useful for seeing how foreign key expansion and faceting actually works. I built a version of that a while ago as the `?_trace=1` argument. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/262#issuecomment-389702480 | https://api.github.com/repos/simonw/datasette/issues/262 | 389702480 | MDEyOklzc3VlQ29tbWVudDM4OTcwMjQ4MA== | 9599 | 2018-05-17T00:00:39Z | 2020-09-12T18:19:30Z | OWNER | Idea: `?_extra=sqllog` could output a lot of every individual SQL statement that was executed in order to generate the page - useful for seeing how foreign key expansion and faceting actually works. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
323658641 | |
https://github.com/simonw/datasette/issues/680#issuecomment-691526635 | https://api.github.com/repos/simonw/datasette/issues/680 | 691526635 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjYzNQ== | 9599 | 2020-09-12T18:18:50Z | 2020-09-12T18:18:50Z | OWNER | I'm happy with the not-quite-automated way I'm doing this, so I'm going to close this issue. That's documented here https://docs.datasette.io/en/0.48/contributing.html#release-process - I use https://euangoddard.github.io/clipboard2markdown/ to create the GitHub releases markdown version. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
569275763 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691526489 | https://api.github.com/repos/simonw/datasette/issues/782 | 691526489 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjQ4OQ== | 9599 | 2020-09-12T18:17:16Z | 2020-09-12T18:17:16Z | OWNER | (I think I may have been over-thinking the details of this is for a couple of years now.) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691526416 | https://api.github.com/repos/simonw/datasette/issues/782 | 691526416 | MDEyOklzc3VlQ29tbWVudDY5MTUyNjQxNg== | 9599 | 2020-09-12T18:16:36Z | 2020-09-12T18:16:36Z | OWNER | I'm going to hack together a preview of this in a branch and deploy it somewhere so people can see what I've got planned. Much easier to evaluate a working prototype than static examples. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/50#issuecomment-691501132 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/50 | 691501132 | MDEyOklzc3VlQ29tbWVudDY5MTUwMTEzMg== | 706257 | 2020-09-12T14:48:10Z | 2020-09-12T14:48:10Z | NONE | This seems to be an issue even with larger values of `--stop_after`: ``` $ twitter-to-sqlite favorites twitter.db --stop_after=2000 Importing favorites [####################################] 198 $ ``` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
698791218 | |
https://github.com/simonw/datasette/issues/963#issuecomment-691379980 | https://api.github.com/repos/simonw/datasette/issues/963 | 691379980 | MDEyOklzc3VlQ29tbWVudDY5MTM3OTk4MA== | 9599 | 2020-09-12T01:50:56Z | 2020-09-12T01:50:56Z | OWNER | Good bug - looks like a problem with the hidden form fields. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
699947574 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691323302 | https://api.github.com/repos/simonw/datasette/issues/782 | 691323302 | MDEyOklzc3VlQ29tbWVudDY5MTMyMzMwMg== | 9599 | 2020-09-11T21:38:27Z | 2020-09-11T21:40:04Z | OWNER | Another idea: the default output could be the list of dicts: ```json [ { "pk1": "a", "pk2": "a", "pk3": "a", "content": "a-a-a" }, ... ] ``` BUT... I could include pagination information in the HTTP headers - as seen in the WordPress REST API or the GitHub API: ``` ~ % curl -s -i 'https://api.github.com/repos/simonw/datasette/commits' | head -n 40 HTTP/1.1 200 OK server: GitHub.com date: Fri, 11 Sep 2020 21:37:46 GMT content-type: application/json; charset=utf-8 status: 200 OK cache-control: public, max-age=60, s-maxage=60 vary: Accept, Accept-Encoding, Accept, X-Requested-With etag: W/"71c99379743513394e880c6306b66bf9" last-modified: Fri, 11 Sep 2020 21:32:54 GMT x-github-media-type: github.v3; format=json link: <https://api.github.com/repositories/107914493/commits?page=2>; rel="next", <https://api.github.com/repositories/107914493/commits?page=44>; rel="last" access-control-expose-headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset access-control-allow-origin: * strict-transport-security: max-age=31536000; includeSubdomains; preload x-frame-options: deny x-content-type-options: nosniff x-xss-protection: 1; mode=block referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin content-security-policy: default-src 'none' X-Ratelimit-Limit: 60 X-Ratelimit-Remaining: 55 X-Ratelimit-Reset: 1599863850 X-Ratelimit-Used: 5 Accept-Ranges: bytes Content-Length: 118240 X-GitHub-Request-Id: EC76:0EAD:313F40:5291A4:5F5BEE37 [ { "sha": "d02f6151dae073135a22d0123e8abdc6cbef7c50", "node_id": "MDY6Q29tbWl0MTA3OTE0NDkzOmQwMmY2MTUxZGFlMDczMTM1YTIyZDAxMjNlOGFiZGM2Y2JlZjdjNTA=", "commit": { ``` Alternative shapes would provide the pagination information (and other extensions) in the JSON, e.g.: `/squirrels/squirrels.json?_shape=paginated` ```json… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/947#issuecomment-691318133 | https://api.github.com/repos/simonw/datasette/issues/947 | 691318133 | MDEyOklzc3VlQ29tbWVudDY5MTMxODEzMw== | 9599 | 2020-09-11T21:23:40Z | 2020-09-11T21:23:40Z | OWNER | I'm going to use exit code 1 for any errors, be they 500 or 404. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
684111953 | |
https://github.com/simonw/datasette/issues/962#issuecomment-691250299 | https://api.github.com/repos/simonw/datasette/issues/962 | 691250299 | MDEyOklzc3VlQ29tbWVudDY5MTI1MDI5OQ== | 9599 | 2020-09-11T18:33:50Z | 2020-09-11T18:33:50Z | OWNER | Since this is purely a debugging option I'm going to allow myself not to write a unit test for it! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
699622046 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/50#issuecomment-690860653 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/50 | 690860653 | MDEyOklzc3VlQ29tbWVudDY5MDg2MDY1Mw== | 370930 | 2020-09-11T04:04:08Z | 2020-09-11T04:04:08Z | CONTRIBUTOR | There's probably a nicer way of doing (hence this is a comment rather than a PR), but this appears to fix it: ```diff --- a/twitter_to_sqlite/utils.py +++ b/twitter_to_sqlite/utils.py @@ -181,6 +181,7 @@ def fetch_timeline( args["tweet_mode"] = "extended" min_seen_id = None num_rate_limit_errors = 0 + seen_count = 0 while True: if min_seen_id is not None: args["max_id"] = min_seen_id - 1 @@ -208,6 +209,7 @@ def fetch_timeline( yield tweet min_seen_id = min(t["id"] for t in tweets) max_seen_id = max(t["id"] for t in tweets) + seen_count += len(tweets) if last_since_id is not None: max_seen_id = max((last_since_id, max_seen_id)) last_since_id = max_seen_id @@ -217,7 +219,9 @@ def fetch_timeline( replace=True, ) if stop_after is not None: - break + if seen_count >= stop_after: + break + args["count"] = min(args["count"], stop_after - seen_count) time.sleep(sleep) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
698791218 |